2024-06-13T06:31:53.353 INFO:root:teuthology version: 0.0.1.dev316+gc1b06c2 2024-06-13T06:31:53.354 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} --archive /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750 --name teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/orig.config.yaml 2024-06-13T06:31:53.408 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-13T06:31:53.547 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7752750' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 1373963c202323287783ccb099bbae1ec8f14a5d owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8815 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 250/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 1373963c202323287783ccb099bbae1ec8f14a5d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCMd31Njvca0ltm/7v3W3UbvA78QJmAtvsVhUYkB4xXSqi+AOascFH/m/XjSmz9x4uZxMpkEdVS6erXjB/Bb/Bc= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOgOLh7nHExVbVHCnISEMzElwt/TcSgBopZSjTEg6TtOeo2k7tVAfo8sRbG1AI9sRcHsyHARoxVm13M1IRIQELg= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLGSFsyFiYQ8gVRd/Fn/djHTon/RMy0lUjA7A0SVKZq2d7pbLJOVB2seWF+IIBbF7dLPaL3n2jdK3NAPaa2ZEmg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: c1b06c2b5d3a02a3583265a115f09e052855bcf2 timestamp: 2024-06-12_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-13T06:31:53.547 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa; will attempt to use it 2024-06-13T06:31:53.548 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks 2024-06-13T06:31:53.548 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-13T06:31:53.549 INFO:teuthology.task.internal:Checking packages... 2024-06-13T06:31:53.569 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-13T06:31:53.570 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-13T06:31:53.570 INFO:teuthology.packaging:ref: None 2024-06-13T06:31:53.570 INFO:teuthology.packaging:tag: None 2024-06-13T06:31:53.570 INFO:teuthology.packaging:branch: squid 2024-06-13T06:31:53.570 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:31:53.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-13T06:31:53.820 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2998.g1373963c 2024-06-13T06:31:53.821 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-13T06:31:53.828 INFO:teuthology.task.internal:no buildpackages task found 2024-06-13T06:31:53.828 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-13T06:31:53.835 INFO:teuthology.task.internal:Saving configuration 2024-06-13T06:31:53.853 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-13T06:31:53.860 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-13T06:31:53.884 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-13 06:26:53.592722', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCMd31Njvca0ltm/7v3W3UbvA78QJmAtvsVhUYkB4xXSqi+AOascFH/m/XjSmz9x4uZxMpkEdVS6erXjB/Bb/Bc='} 2024-06-13T06:31:53.906 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-13 06:26:53.591327', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOgOLh7nHExVbVHCnISEMzElwt/TcSgBopZSjTEg6TtOeo2k7tVAfo8sRbG1AI9sRcHsyHARoxVm13M1IRIQELg='} 2024-06-13T06:31:53.923 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-13 06:26:53.593943', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLGSFsyFiYQ8gVRd/Fn/djHTon/RMy0lUjA7A0SVKZq2d7pbLJOVB2seWF+IIBbF7dLPaL3n2jdK3NAPaa2ZEmg='} 2024-06-13T06:31:53.924 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-13T06:31:53.930 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-13T06:31:53.930 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-13T06:31:53.930 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-13T06:31:53.930 INFO:teuthology.run_tasks:Running task console_log... 2024-06-13T06:31:54.003 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7887e97be0>, signals=[15]) 2024-06-13T06:31:54.003 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-13T06:31:54.009 INFO:teuthology.task.internal:Opening connections... 2024-06-13T06:31:54.009 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T06:31:54.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:31:54.086 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-06-13T06:31:54.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:31:54.159 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-06-13T06:31:54.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:31:54.234 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-13T06:31:54.240 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-06-13T06:31:54.259 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-06-13T06:31:54.259 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-06-13T06:31:54.316 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-06-13T06:31:54.316 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-06-13T06:31:54.316 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-13T06:31:54.317 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-13T06:31:54.318 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-06-13T06:31:54.339 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-06-13T06:31:54.357 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-06-13T06:31:54.357 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-06-13T06:31:54.414 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-06-13T06:31:54.414 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-06-13T06:31:54.414 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-13T06:31:54.415 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-13T06:31:54.415 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-06-13T06:31:54.440 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-06-13T06:31:54.458 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-06-13T06:31:54.458 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-06-13T06:31:54.515 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-06-13T06:31:54.516 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-13T06:31:54.516 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-06-13T06:31:54.516 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-13T06:31:54.516 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-13T06:31:54.516 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-13T06:31:54.516 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-06-13T06:31:54.547 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-13T06:31:54.557 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-13T06:31:54.563 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-13T06:31:54.563 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-06-13T06:31:54.565 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-06-13T06:31:54.567 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-06-13T06:31:54.583 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-13T06:31:54.589 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-13T06:31:54.589 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-06-13T06:31:54.622 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-06-13T06:31:54.625 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-06-13T06:31:54.655 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-13T06:31:54.716 INFO:teuthology.run_tasks:Running task kernel... 2024-06-13T06:31:54.730 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-13T06:31:54.730 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-06-13T06:31:54.730 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-06-13T06:31:54.731 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-13T06:31:54.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:31:54.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-06-13T06:31:54.988 DEBUG:teuthology.task.kernel:sha1 for client.0 is ff88c41504f3e4c4268de18904a6f347f1b3f2d6 2024-06-13T06:31:54.988 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "ff88c41504f3e4c4268de18904a6f347f1b3f2d6"... 2024-06-13T06:31:54.988 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-06-13T06:31:55.006 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-437.el9.x86_64 2024-06-13T06:31:55.006 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs ff88c41504f3e4c4268de18904a6f347f1b3f2d6 2024-06-13T06:31:55.006 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-06-13T06:31:55.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum upgrade -y linux-firmware 2024-06-13T06:31:59.861 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.1 MB 00:03 2024-06-13T06:32:01.786 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-06-13T06:32:06.136 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.3 MB 00:00 2024-06-13T06:32:07.712 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 17 kB 00:00 2024-06-13T06:32:08.931 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-13T06:32:14.312 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-13T06:32:16.486 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout: linux-firmware noarch 20240603-144.el9 baseos 380 M 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 1 Package 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:32:16.491 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 380 M 2024-06-13T06:32:16.492 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-06-13T06:32:16.799 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] linux-firmware-20240603-144.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/linux-firmware-20240603-144.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-13T06:32:36.194 INFO:teuthology.orchestra.run.smithi026.stdout:linux-firmware-20240603-144.el9.noarch.rpm 20 MB/s | 380 MB 00:19 2024-06-13T06:32:36.195 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:32:36.196 INFO:teuthology.orchestra.run.smithi026.stdout:Total 19 MB/s | 380 MB 00:19 2024-06-13T06:32:39.437 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-06-13T06:32:39.515 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-06-13T06:32:39.515 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-06-13T06:32:42.900 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-06-13T06:32:42.900 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-06-13T06:32:52.516 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-06-13T06:32:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : linux-firmware-20240603-144.el9.noarch 1/2 2024-06-13T06:32:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-06-13T06:32:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-06-13T06:32:53.538 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : linux-firmware-20240603-144.el9.noarch 1/2 2024-06-13T06:32:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-06-13T06:32:54.249 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:32:54.250 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-06-13T06:32:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: linux-firmware-20240603-144.el9.noarch 2024-06-13T06:32:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:32:54.250 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-06-13T06:32:54.554 INFO:teuthology.task.kernel:Downloading kernel ff88c41504f3e4c4268de18904a6f347f1b3f2d6 on client.0... 2024-06-13T06:32:54.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ff88c41504f3e4c4268de18904a6f347f1b3f2d6 2024-06-13T06:32:54.773 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/ff88c41504f3e4c4268de18904a6f347f1b3f2d6/centos/9/flavors/default/x86_64/ 2024-06-13T06:32:54.773 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gff88c41504f3-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/ff88c41504f3e4c4268de18904a6f347f1b3f2d6/centos/9/flavors/default/x86_64/ --input-file=- 2024-06-13T06:32:54.776 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi026.front.sepia.ceph.com to complete... 2024-06-13T06:33:07.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13 06:33:07 URL:https://3.chacra.ceph.com/r/kernel/testing/ff88c41504f3e4c4268de18904a6f347f1b3f2d6/centos/9/flavors/default/x86_64/kernel-6.9.0_gff88c41504f3-1.x86_64.rpm [356881057/356881057] -> "/tmp/kernel.x86_64.rpm" [1] 2024-06-13T06:33:07.111 INFO:teuthology.orchestra.run.smithi026.stderr:FINISHED --2024-06-13 06:33:07-- 2024-06-13T06:33:07.111 INFO:teuthology.orchestra.run.smithi026.stderr:Total wall clock time: 12s 2024-06-13T06:33:07.111 INFO:teuthology.orchestra.run.smithi026.stderr:Downloaded: 1 files, 340M in 12s (28.3 MB/s) 2024-06-13T06:33:07.114 INFO:teuthology.task.kernel:Installing kernel ff88c41504f3e4c4268de18904a6f347f1b3f2d6 on client.0... 2024-06-13T06:33:07.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-06-13T06:33:09.263 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying... ######################################## 2024-06-13T06:33:09.321 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing... ######################################## 2024-06-13T06:33:09.374 INFO:teuthology.orchestra.run.smithi026.stdout:Updating / installing... 2024-06-13T06:33:17.714 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-6.9.0_gff88c41504f3-1 ######################################## 2024-06-13T06:33:58.218 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-06-13T06:33:58.219 DEBUG:teuthology.orchestra.run.smithi026:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-06-13T06:33:58.305 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/System.map-6.9.0-gff88c41504f3 2024-06-13T06:33:58.305 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/config-6.9.0-gff88c41504f3 2024-06-13T06:33:58.305 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/vmlinuz-6.9.0-gff88c41504f3 2024-06-13T06:33:58.305 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/System.map 2024-06-13T06:33:58.305 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/config 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/crc32c-intel.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-06-13T06:33:58.306 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/serpent-avx2.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kernel 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kernel/cpu 2024-06-13T06:33:58.307 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kernel/cpu/mce 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kvm 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kvm/kvm-amd.ko 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kvm/kvm-intel.ko 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kvm/kvm.ko 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/ansi_cprng.ko 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/anubis.ko 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/arc4.ko 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_memcpy.ko 2024-06-13T06:33:58.308 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_pq.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_raid6_recov.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_tx.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_xor.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/raid6test.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/authenc.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/authencesn.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/blake2b_generic.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/blowfish_common.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/blowfish_generic.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/camellia_generic.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cast5_generic.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cast6_generic.ko 2024-06-13T06:33:58.309 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cast_common.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/ccm.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cmac.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/crc32_generic.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/crypto_engine.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/crypto_user.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cts.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/des_generic.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/dh_generic.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/echainiv.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/essiv.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/fcrypt.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/khazad.ko 2024-06-13T06:33:58.310 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/lz4.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/lz4hc.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/md4.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/michael_mic.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/pcbc.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/pcrypt.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/rmd160.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/seed.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/serpent_generic.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/tcrypt.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/tea.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/twofish_common.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/twofish_generic.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/vmac.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/wp512.ko 2024-06-13T06:33:58.311 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/xcbc.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/xor.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/xxhash_generic.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/acpi_ipmi.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/acpi_pad.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/ec_sys.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/platform_profile.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/sbs.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/sbshc.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/video.ko 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata 2024-06-13T06:33:58.312 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/acard-ahci.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/ahci_platform.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/ata_generic.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/libahci_platform.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_acpi.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_ali.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_amd.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_artop.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_atiixp.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_atp867x.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_cmd640.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_cmd64x.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_cypress.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_efar.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_hpt366.ko 2024-06-13T06:33:58.313 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_hpt37x.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_hpt3x2n.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_hpt3x3.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_it8213.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_it821x.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_jmicron.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_marvell.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_mpiix.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_netcell.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_ninja32.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_ns87410.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_ns87415.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_oldpiix.ko 2024-06-13T06:33:58.314 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_opti.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_optidma.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_pcmcia.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_pdc2027x.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_piccolo.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_rdc.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_sch.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_serverworks.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_sil680.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_sis.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_sl82c105.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_triflex.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_via.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pdc_adma.ko 2024-06-13T06:33:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_inic162x.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_mv.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_nv.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_promise.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_qstor.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_sil.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_sil24.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_sis.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_svw.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_sx4.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_uli.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_via.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_vsc.ko 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/auxdisplay 2024-06-13T06:33:58.316 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/auxdisplay/ks0108.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/base 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/base/regmap 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/base/regmap/regmap-i2c.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/base/regmap/regmap-mmio.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/bcma 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/bcma/bcma.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/aoe 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/aoe/aoe.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/brd.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/drbd 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/drbd/drbd.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/floppy.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/loop.ko 2024-06-13T06:33:58.317 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/mtip32xx 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/nbd.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/null_blk 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/null_blk/null_blk.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/pktcdvd.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/rbd.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/virtio_blk.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/xen-blkback 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/xen-blkfront.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/zram 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/zram/zram.ko 2024-06-13T06:33:58.318 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hangcheck-timer.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/amd-rng.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/intel-rng.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/via-rng.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/virtio-rng.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_si.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-06-13T06:33:58.319 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/lp.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/mwave 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/mwave/mwave.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ppdev.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tlclk.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_atmel.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_infineon.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_nsc.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_tis.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/uv_mmtimer.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/virtio_console.ko 2024-06-13T06:33:58.320 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/p4-clockmod.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/powernow-k8.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/speedstep-lib.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/ccp 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/ccp/ccp.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat/qat_common 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-06-13T06:33:58.321 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/padlock-aes.ko 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/padlock-sha.ko 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/virtio 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dca 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dca/dca.ko 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/devfreq 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma/dw 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma/dw/dw_dmac.ko 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma/ioat 2024-06-13T06:33:58.322 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma/ioat/ioatdma.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/amd64_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/e752x_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/edac_mce_amd.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i3000_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i3200_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i5100_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i5400_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i7300_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i7core_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i82975x_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/ie31200_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/sb_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/x38_edac.ko 2024-06-13T06:33:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/firmware 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/firmware/edd.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/firmware/iscsi_ibft.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/gpio 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/gpio/gpio-ich.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/gpio/gpio-viperboard.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-appleir.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-aureal.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-axff.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-dr.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-elecom.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-elo.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-emsff.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-gaff.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-gt683r.ko 2024-06-13T06:33:58.324 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-gyration.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-holtek-kbd.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-holtek-mouse.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-holtekff.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-hyperv.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-icade.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-keytouch.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-kye.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-lcpower.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-led.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-lenovo.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-logitech-dj.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-multitouch.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-ortek.ko 2024-06-13T06:33:58.325 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-penmount.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-petalynx.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-picolcd.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-pl.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-plantronics.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-primax.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-rmi.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-arvo.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-common.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-isku.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-kone.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-konepure.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-06-13T06:33:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-lua.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-pyra.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-ryos.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-savu.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-saitek.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-samsung.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-sensor-hub.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-sjoy.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-sony.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-speedlink.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-steelseries.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-sunplus.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-thrustmaster.ko 2024-06-13T06:33:58.327 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-tivo.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-tmff.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-topseed.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-twinhan.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-uclogic.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-waltop.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-wiimote.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-xinmo.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-zpff.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-zydacron.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/uhid.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/wacom.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hv 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hv/hv_balloon.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hv/hv_utils.ko 2024-06-13T06:33:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hv/hv_vmbus.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/abituguru.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/abituguru3.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/acpi_power_meter.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ad7414.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ad7418.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adc128d818.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm1025.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm1026.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm1029.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm1031.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm9240.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ads7828.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7410.ko 2024-06-13T06:33:58.329 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7411.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7462.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7470.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7475.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7x10.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/amc6821.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/applesmc.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/asb100.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/asc7621.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/asus_atk0110.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/atxp1.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/coretemp.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/dme1737.ko 2024-06-13T06:33:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ds1621.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ds620.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/emc1403.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/emc6w201.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/f71805f.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/f71882fg.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/f75375s.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/fam15h_power.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/fschmd.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/g760a.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/g762.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/gl518sm.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/gl520sm.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/hwmon-vid.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/i5500_temp.ko 2024-06-13T06:33:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/i5k_amb.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ibmaem.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ibmpex.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ina209.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ina2xx.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/it87.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/k10temp.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/k8temp.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lineage-pem.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm63.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm73.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm75.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm77.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm78.ko 2024-06-13T06:33:58.332 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm80.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm83.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm85.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm87.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm90.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm92.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm93.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm95234.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm95241.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm95245.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc2945.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4151.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4215.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4222.ko 2024-06-13T06:33:58.333 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4245.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4260.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4261.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max16065.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max1619.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max1668.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max197.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max6639.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max6650.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max6697.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/mcp3021.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/nct6683.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/nct6775-core.ko 2024-06-13T06:33:58.334 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/nct6775.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/nct7802.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pc87360.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pc87427.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pcf8591.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/max16064.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/max34440.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/max8688.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-06-13T06:33:58.335 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/powr1220.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sch5627.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sch5636.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sch56xx-common.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sht15.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sht21.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/shtc1.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sis5595.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/smsc47b397.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/smsc47m1.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/smsc47m192.ko 2024-06-13T06:33:58.336 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/thmc50.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/tmp102.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/tmp103.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/tmp401.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/tmp421.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/via-cputemp.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/via686a.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/vt1211.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/vt8231.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83627ehf.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83627hf.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83781d.ko 2024-06-13T06:33:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83791d.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83792d.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83793.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83795.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83l785ts.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83l786ng.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/algos 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-06-13T06:33:58.338 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-i801.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-isch.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-parport.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-06-13T06:33:58.339 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-via.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-dev.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-mux.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-smbus.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-stub.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/ib_cm.ko 2024-06-13T06:33:58.340 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/ib_core.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/ib_umad.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/iw_cm.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/rdma_cm.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/cxgb4 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mlx4 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mlx5 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mthca 2024-06-13T06:33:58.341 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/ipoib 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/iser 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/isert 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/srp 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/srpt 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input 2024-06-13T06:33:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/gameport 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/gameport/fm801-gp.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/gameport/gameport.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/joydev.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/keyboard 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/keyboard/gpio_keys.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/appletouch.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/bcm5974.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/cyapatp.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/elan_i2c.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/sermouse.ko 2024-06-13T06:33:58.343 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/synaptics_usb.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/vsxxxaa.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/rmi4 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/rmi4/rmi_core.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio/altera_ps2.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio/arc_ps2.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio/serio_raw.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/sparse-keymap.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/leds-blinkm.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/leds-lm3530.ko 2024-06-13T06:33:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/leds-lp3944.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/leds-ss4200.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/bcache 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/bcache/bcache.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-bio-prison.ko 2024-06-13T06:33:58.345 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-cache-smq.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-cache.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-crypt.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-delay.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-flakey.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-log-userspace.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-multipath.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-queue-length.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-raid.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-round-robin.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-service-time.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-switch.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-thin-pool.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-verity.ko 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/persistent-data 2024-06-13T06:33:58.346 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/raid0.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/raid1.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/raid10.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/raid456.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptbase.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptctl.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptfc.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptlan.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptsas.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptscsih.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptspi.ko 2024-06-13T06:33:58.347 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/lpc_ich.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/lpc_sch.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/mfd-core.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/sm501.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/viperboard.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/vx855.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/wl1273-core.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/altera-stapl 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/apds9802als.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/apds990x.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/bh1770glc.ko 2024-06-13T06:33:58.348 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/cb710 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/cb710/cb710.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/eeprom 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/eeprom/at24.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/eeprom/max6875.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/enclosure.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/hpilo.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/isl29003.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/isl29020.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/lis3lv02d 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/mei 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/mei/mei-me.ko 2024-06-13T06:33:58.349 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/mei/mei-txe.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/mei/mei.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-gru 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-gru/gru.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-xp 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-xp/xp.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-xp/xpc.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/tifm_7xx1.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/tifm_core.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/tsl2550.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/vmw_balloon.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/vmw_vmci 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net 2024-06-13T06:33:58.350 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/bonding 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/bonding/bonding.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/b53 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/b53/b53_common.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/bcm-sf2.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/mv88e6060.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/mv88e6xxx 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dummy.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/eql.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-06-13T06:33:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390/8390.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/adaptec 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/agere 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/agere/et131x.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/alteon 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/altera 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-06-13T06:33:58.352 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/amd 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/alx 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atl1c 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atl1e 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atlx 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom 2024-06-13T06:33:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/genet 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/brocade 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/brocade/bna 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-06-13T06:33:58.354 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/cisco 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/cisco/enic 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-06-13T06:33:58.355 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dlink 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dnet.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/emulex 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/emulex/benet 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/ethoc.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/fealnx.ko 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel 2024-06-13T06:33:58.356 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e100.ko 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e1000 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e1000e 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/fm10k 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/i40e 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/iavf 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/igb 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/igbvf 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-06-13T06:33:58.357 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/ixgbe 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/ixgbevf 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/jme.ko 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/marvell 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/marvell/skge.ko 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx4 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-06-13T06:33:58.358 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx5 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/micrel 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/myricom 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/myricom/myri10ge 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/natsemi 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/neterion 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/nvidia 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-06-13T06:33:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/packetengines 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/netxen 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/rdc 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek/atp.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-06-13T06:33:58.360 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/rocker 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sfc 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/silan 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sis 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sis/sis190.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sis/sis900.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-06-13T06:33:58.361 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/stmicro 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/stmicro/stmmac 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun/cassini.ko 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun/niu.ko 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun/sungem.ko 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/tehuti 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/ti 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/ti/tlan.ko 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/via 2024-06-13T06:33:58.362 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/wiznet 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/xircom 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/geneve.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/hyperv 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ieee802154 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ieee802154/fakelb.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ifb.ko 2024-06-13T06:33:58.363 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ipvlan 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ipvlan/ipvlan.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/macvlan.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/macvtap.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio/mdio-regmap.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mii.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/net_failover.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/netconsole.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/nlmon.ko 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/pcs 2024-06-13T06:33:58.364 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/pcs/pcs-lynx.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/amd.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/bcm7xxx.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/bcm87xx.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/broadcom.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/cicada.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/davicom.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/dp83640.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/et1011c.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/icplus.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/lxt.ko 2024-06-13T06:33:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/marvell.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/micrel.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/national.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/phylink.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/qsemi.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/realtek.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/smsc.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/ste10Xp.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/vitesse.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/bsd_comp.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_async.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_deflate.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_generic.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_mppe.ko 2024-06-13T06:33:58.366 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_synctty.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/pppoe.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/pppox.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/pptp.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/slip 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/slip/slhc.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/slip/slip.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/sungem_phy.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/tap.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_activebackup.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_broadcast.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-06-13T06:33:58.367 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_random.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/tun.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/veth.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/virtio_net.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/vmxnet3 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/vxlan 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/vxlan/vxlan.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/xen-netback 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/xen-netback/xen-netback.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/xen-netfront.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ntb 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ntb/ntb.ko 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/nvme 2024-06-13T06:33:58.368 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/nvme/host 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/nvme/host/nvme-core.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/nvme/host/nvme.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport/parport.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport/parport_cs.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport/parport_pc.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport/parport_serial.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/hotplug 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/pcie 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/pcie/aer_inject.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/xen-pcifront.ko 2024-06-13T06:33:58.369 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pcmcia 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pcmcia/i82092.ko 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pcmcia/pd6729.ko 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pcmcia/yenta_socket.ko 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pinctrl 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pinctrl/intel 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/acer-wmi.ko 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/acerhdf.ko 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/apple-gmux.ko 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/asus-laptop.ko 2024-06-13T06:33:58.370 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/asus-wmi.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/classmate-laptop.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/hdaps.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/intel 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/intel_ips.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/msi-wmi.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/mxm-wmi.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-06-13T06:33:58.371 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/samsung-laptop.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/samsung-q10.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/topstar-laptop.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/toshiba_haps.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/wmi-bmof.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/wmi.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/powercap 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/powercap/intel_rapl_common.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/powercap/intel_rapl_msr.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/clients 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/clients/pps-gpio.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/clients/pps-ldisc.ko 2024-06-13T06:33:58.372 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/clients/pps_parport.ko 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/pps_core.ko 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ptp 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ptp/ptp.ko 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ptp/ptp_kvm.ko 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras/amd 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras/amd/atl 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras/amd/fmpm.ko 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-bq32k.ko 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1286.ko 2024-06-13T06:33:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1307.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1374.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1511.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1553.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1672.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1742.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds2404.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds3232.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-em3027.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-fm3130.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-isl12022.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-isl1208.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-m41t80.ko 2024-06-13T06:33:58.374 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-m48t35.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-m48t59.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-max6900.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-msm6242.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf2127.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf85063.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf8523.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf8563.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf8583.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rp5c01.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rs5c372.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rx8025.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rx8581.ko 2024-06-13T06:33:58.375 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-x1205.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/3w-9xxx.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/3w-sas.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/3w-xxxx.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/BusLogic.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/a100u2w.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aacraid 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aacraid/aacraid.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/advansys.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aic7xxx 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-06-13T06:33:58.376 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/am53c974.ko 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/arcmsr 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/atp870u.ko 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/be2iscsi 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bfa 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bfa/bfa.ko 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bnx2fc 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bnx2i 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/ch.ko 2024-06-13T06:33:58.377 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/csiostor 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/csiostor/csiostor.ko 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/cxgb3i 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/cxgb4i 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/dc395x.ko 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-06-13T06:33:58.378 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/dmx3191d.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/esas2r 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/esas2r/esas2r.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/esp_scsi.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fcoe 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fcoe/fcoe.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fnic 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fnic/fnic.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/hpsa.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/hptiop.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/hv_storvsc.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/initio.ko 2024-06-13T06:33:58.379 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/ipr.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/ips.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/isci 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/isci/isci.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/iscsi_tcp.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libfc 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libfc/libfc.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libiscsi.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libiscsi_tcp.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libsas 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libsas/libsas.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/lpfc 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/lpfc/lpfc.ko 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid 2024-06-13T06:33:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mpt3sas 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mvsas 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mvsas/mvsas.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mvumi.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/pm8001 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/pmcraid.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla1280.ko 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla2xxx 2024-06-13T06:33:58.381 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla4xxx 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/raid_class.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_debug.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_fc.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_sas.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_spi.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_srp.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/ses.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/st.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/stex.ko 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/sym53c8xx_2 2024-06-13T06:33:58.382 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/virtio_scsi.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/vmw_pvscsi.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/wd719x.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ssb 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ssb/ssb.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/iscsi 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/loopback 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/loopback/tcm_loop.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_file.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_iblock.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_mod.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_pscsi.ko 2024-06-13T06:33:58.383 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_user.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/tcm_fc 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-06-13T06:33:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/ipwireless 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/n_gsm.ko 2024-06-13T06:33:58.385 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/n_hdlc.ko 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/nozomi.ko 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/8250 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/arc_uart.ko 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/jsm 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/jsm/jsm.ko 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs/core 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs/core/ufshcd-core.ko 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs/host 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio.ko 2024-06-13T06:33:58.386 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio_aec.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio_cif.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio_pci_generic.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio_sercos3.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class/cdc-acm.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class/cdc-wdm.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class/usblp.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class/usbtmc.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/host 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/host/sl811-hcd.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/image 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/image/mdc800.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/image/microtek.ko 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc 2024-06-13T06:33:58.387 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/adutux.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/appledisplay.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/emi26.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/emi62.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/ezusb.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/idmouse.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/iowarrior.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/isight_firmware.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/ldusb.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/legousbtower.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/sisusbvga 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/trancevibrator.ko 2024-06-13T06:33:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/usb3503.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/usblcd.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/usbsevseg.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/uss720.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/yurex.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/phy 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/phy/phy-generic.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/aircable.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ark3116.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/belkin_sa.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ch341.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/cp210x.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/cyberjack.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/cypress_m8.ko 2024-06-13T06:33:58.389 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/digi_acceleport.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/empeg.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ftdi_sio.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/garmin_gps.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/io_edgeport.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/io_ti.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ipaq.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ipw.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ir-usb.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/keyspan.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/keyspan_pda.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/kl5kusb105.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/kobil_sct.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/mct_u232.ko 2024-06-13T06:33:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/mos7720.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/mos7840.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/navman.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/omninet.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/opticon.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/option.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/oti6858.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/pl2303.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/qcaux.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/qcserial.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/quatech2.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/safe_serial.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/sierra.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/spcp8x5.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ssu100.ko 2024-06-13T06:33:58.391 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/symbolserial.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/usb_debug.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/usb_wwan.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/visor.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/whiteheat.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/xsens_mt.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/uas.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-alauda.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-cypress.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-datafab.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-freecom.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-isd200.ko 2024-06-13T06:33:58.392 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-karma.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-onetouch.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-realtek.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-sddr09.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-sddr55.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-usbat.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/usb-storage.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/usbip 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/usbip/usbip-core.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/usbip/usbip-host.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/pci 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-06-13T06:33:58.393 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/pci/vfio-pci.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/vfio.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost/vhost.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost/vhost_iotlb.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost/vhost_net.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost/vhost_scsi.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video/backlight 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video/backlight/apple_bl.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video/backlight/lcd.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video/backlight/platform_lcd.ko 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio 2024-06-13T06:33:58.394 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_balloon.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_mmio.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_pci.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_ring.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/masters 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/masters/ds2482.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/masters/ds2490.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-06-13T06:33:58.395 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_smem.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_therm.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/wire.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/alim1535_wdt.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/alim7101_wdt.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/f71808e_wdt.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/hpwdt.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/i6300esb.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-06-13T06:33:58.396 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/iTCO_wdt.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/ib700wdt.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/ibmasr.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/it8712f_wdt.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/it87_wdt.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/machzwd.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/nv_tco.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/pcwd_pci.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/pcwd_usb.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/sch311x_wdt.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/softdog.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/sp5100_tco.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/via_wdt.ko 2024-06-13T06:33:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/w83877f_wdt.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/w83977f_wdt.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/wdt_pci.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/xen_wdt.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-acpi-processor.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-evtchn.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-gntalloc.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-gntdev.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-pciback 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-06-13T06:33:58.398 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-privcmd.ko 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xenfs 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xenfs/xenfs.ko 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/9p 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/9p/9p.ko 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/affs 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/affs/affs.ko 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/befs 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/befs/befs.ko 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/binfmt_misc.ko 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/btrfs 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/btrfs/btrfs.ko 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/cachefiles 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/cachefiles/cachefiles.ko 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ceph 2024-06-13T06:33:58.399 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ceph/ceph.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/coda 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/coda/coda.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/cramfs 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/cramfs/cramfs.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/dlm 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/dlm/dlm.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ecryptfs 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ecryptfs/ecryptfs.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/f2fs 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/f2fs/f2fs.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fat 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fat/fat.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fat/msdos.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fat/vfat.ko 2024-06-13T06:33:58.400 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fuse 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fuse/cuse.ko 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fuse/fuse.ko 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/gfs2 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/gfs2/gfs2.ko 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/hfs 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/hfs/hfs.ko 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/hfsplus 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/hfsplus/hfsplus.ko 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/isofs 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/isofs/isofs.ko 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/jfs 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/jfs/jfs.ko 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/lockd 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/lockd/lockd.ko 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/minix 2024-06-13T06:33:58.401 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/minix/minix.ko 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/netfs 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/netfs/netfs.ko 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/blocklayout 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/filelayout 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/flexfilelayout 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/nfs.ko 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/nfsv3.ko 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/nfsv4.ko 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs_common 2024-06-13T06:33:58.402 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs_common/grace.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs_common/nfs_acl.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfsd 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfsd/nfsd.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nilfs2 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nilfs2/nilfs2.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-celtic.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-centeuro.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-croatian.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-cyrillic.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-gaelic.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-greek.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-iceland.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-inuit.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-roman.ko 2024-06-13T06:33:58.403 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-romanian.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-turkish.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp1250.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp1251.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp1255.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp737.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp775.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp850.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp852.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp855.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp857.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp860.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp861.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp862.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp863.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp864.ko 2024-06-13T06:33:58.404 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp865.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp866.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp869.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp874.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp932.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp936.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp949.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp950.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_euc-jp.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-1.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-13.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-14.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-15.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-2.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-3.ko 2024-06-13T06:33:58.405 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-4.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-5.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-6.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-7.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-9.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_koi8-r.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_koi8-ru.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_koi8-u.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_ucs2_utils.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_utf8.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/cluster 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/dlm 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/dlmfs 2024-06-13T06:33:58.406 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/ocfs2.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/overlayfs 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/overlayfs/overlay.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/pstore 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/pstore/ramoops.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/reiserfs 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/reiserfs/reiserfs.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/romfs 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/romfs/romfs.ko 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/client 2024-06-13T06:33:58.407 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/client/cifs.ko 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/common 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/common/cifs_arc4.ko 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/common/cifs_md4.ko 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/squashfs 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/squashfs/squashfs.ko 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/sysv 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/sysv/sysv.ko 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/udf 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/udf/udf.ko 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ufs 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ufs/ufs.ko 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/xfs 2024-06-13T06:33:58.408 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/xfs/xfs.ko 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/rcu 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/rcu/rcutorture.ko 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/torture.ko 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/trace 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/asn1_encoder.ko 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crc-itu-t.ko 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crc8.ko 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crypto 2024-06-13T06:33:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crypto/libarc4.ko 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crypto/libdes.ko 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/libcrc32c.ko 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/lru_cache.ko 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/lz4 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/lz4/lz4_compress.ko 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/lz4/lz4hc_compress.ko 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/math 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/math/cordic.ko 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/raid6 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/raid6/raid6_pq.ko 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/reed_solomon 2024-06-13T06:33:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/reed_solomon/reed_solomon.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/ts_bm.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/ts_fsm.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/ts_kmp.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/zstd 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/zstd/zstd_compress.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/mm 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/mm/hwpoison-inject.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/6lowpan.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_dest.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_fragment.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_hop.ko 2024-06-13T06:33:58.411 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_ipv6.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_mobility.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_routing.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_udp.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/garp.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/mrp.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/p8022.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/psnap.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/stp.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/8021q 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/8021q/8021q.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p/9pnet.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p/9pnet_fd.ko 2024-06-13T06:33:58.412 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p/9pnet_rdma.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p/9pnet_virtio.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/batman-adv 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/batman-adv/batman-adv.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/br_netfilter.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/bridge.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_among.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_arp.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-06-13T06:33:58.413 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_ip.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_limit.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_log.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_mark.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_snat.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_stp.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-06-13T06:33:58.414 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebtables.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ceph 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ceph/libceph.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/core 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/core/failover.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/core/pktgen.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp/dccp.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp/dccp_diag.ko 2024-06-13T06:33:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp/dccp_ipv4.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp/dccp_ipv6.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dns_resolver 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dns_resolver/dns_resolver.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/dsa_core.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/tag_brcm.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/tag_dsa.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/tag_none.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/tag_trailer.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154/6lowpan 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154/ieee802154.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154/ieee802154_socket.ko 2024-06-13T06:33:58.416 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ah4.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/esp4.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/fou.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/gre.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/inet_diag.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ip_gre.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ip_tunnel.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ip_vti.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ipcomp.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ipip.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/arp_tables.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-06-13T06:33:58.417 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/iptable_security.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-06-13T06:33:58.418 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_bic.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_dctcp.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_diag.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_highspeed.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_htcp.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_hybla.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_illinois.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_lp.ko 2024-06-13T06:33:58.419 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_scalable.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_vegas.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_veno.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_westwood.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_yeah.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tunnel4.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/udp_diag.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/udp_tunnel.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/xfrm4_tunnel.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ah6.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/esp6.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/fou6.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ip6_tunnel.ko 2024-06-13T06:33:58.420 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ip6_vti.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ipcomp6.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-06-13T06:33:58.421 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-06-13T06:33:58.422 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/sit.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/tunnel6.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/xfrm6_tunnel.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/key 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/key/af_key.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_core.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_debugfs.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_eth.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_ip.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_ip6.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_netlink.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_ppp.ko 2024-06-13T06:33:58.423 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/llc 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/llc/llc.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/mac802154 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/mac802154/mac802154.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/mpls 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/mpls/mpls_gso.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-06-13T06:33:58.424 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-06-13T06:33:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conncount.ko 2024-06-13T06:33:58.426 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_h323.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_irc.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_sane.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_sip.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-06-13T06:33:58.427 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_dup_netdev.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_flow_table.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_flow_table_inet.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_log_syslog.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_amanda.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_ftp.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_irc.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_sip.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_tftp.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_synproxy_core.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_tables.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink_acct.ko 2024-06-13T06:33:58.428 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink_log.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink_osf.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink_queue.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_chain_nat.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_compat.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_ct.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_dup_netdev.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_fib.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_fwd_netdev.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_hash.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_limit.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_log.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_masq.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_nat.ko 2024-06-13T06:33:58.429 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_numgen.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_queue.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_quota.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_redir.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_reject.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_reject_inet.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_socket.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_tunnel.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_AUDIT.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_CHECKSUM.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_CLASSIFY.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_CT.ko 2024-06-13T06:33:58.430 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_DSCP.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_HL.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_HMARK.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_IDLETIMER.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_LED.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_LOG.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_MASQUERADE.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_NETMAP.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_NFLOG.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_NFQUEUE.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_RATEEST.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_REDIRECT.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_SECMARK.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TCPMSS.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-06-13T06:33:58.431 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TEE.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TPROXY.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TRACE.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_addrtype.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_bpf.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_cgroup.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_cluster.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_comment.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_connbytes.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_connlabel.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_connlimit.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_connmark.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_conntrack.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_cpu.ko 2024-06-13T06:33:58.432 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_dccp.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_devgroup.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_dscp.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_ecn.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_esp.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_hashlimit.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_helper.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_hl.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_ipcomp.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_iprange.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_ipvs.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_l2tp.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_length.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_limit.ko 2024-06-13T06:33:58.433 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_mac.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_mark.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_multiport.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_nat.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_nfacct.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_osf.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_owner.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_physdev.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_pkttype.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_policy.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_quota.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_rateest.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_realm.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_recent.ko 2024-06-13T06:33:58.434 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_sctp.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_set.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_socket.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_state.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_statistic.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_string.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_tcpmss.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_time.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_u32.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netlink 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netlink/netlink_diag.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/nsh 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/nsh/nsh.ko 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch 2024-06-13T06:33:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch/openvswitch.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch/vport-geneve.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch/vport-gre.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch/vport-vxlan.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/packet 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/packet/af_packet_diag.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/rds 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/rds/rds.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/rds/rds_rdma.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/rds/rds_tcp.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_csum.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_gact.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_mirred.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_nat.ko 2024-06-13T06:33:58.436 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_pedit.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_police.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_simple.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_skbedit.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_vlan.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_basic.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_bpf.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_flow.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_fw.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_route.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_u32.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_cmp.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_ipset.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_meta.ko 2024-06-13T06:33:58.437 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_nbyte.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_text.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_u32.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_choke.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_codel.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_drr.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_fq.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_gred.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_hfsc.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_hhf.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_htb.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_ingress.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_mqprio.ko 2024-06-13T06:33:58.438 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_mqprio_lib.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_multiq.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_netem.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_pie.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_plug.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_prio.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_qfq.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_red.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_sfb.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_sfq.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_tbf.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_teql.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sctp 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sctp/sctp.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sctp/sctp_diag.ko 2024-06-13T06:33:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/auth_gss 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/sunrpc.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/xprtrdma 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/tipc 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/tipc/tipc.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/tipc/tipc_diag.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/unix 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/unix/unix_diag.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-06-13T06:33:58.440 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vsock.ko 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vsock_diag.ko 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vsock_loopback.ko 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/xfrm 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/xfrm/xfrm_ipcomp.ko 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys/encrypted-keys 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys/trusted-keys 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys/trusted-keys/trusted.ko 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.alias 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.alias.bin 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.builtin 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.builtin.alias.bin 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.builtin.bin 2024-06-13T06:33:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.builtin.modinfo 2024-06-13T06:33:58.442 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.dep 2024-06-13T06:33:58.442 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.dep.bin 2024-06-13T06:33:58.442 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.devname 2024-06-13T06:33:58.442 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.order 2024-06-13T06:33:58.442 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.softdep 2024-06-13T06:33:58.442 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.symbols 2024-06-13T06:33:58.442 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.symbols.bin 2024-06-13T06:33:58.442 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-gff88c41504f3/vmlinuz 2024-06-13T06:33:58.445 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gff88c41504f3 2024-06-13T06:33:58.446 DEBUG:teuthology.orchestra.run.smithi026:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-06-13T06:33:58.469 INFO:teuthology.orchestra.run.smithi026.stdout:postinstall scriptlet (using /bin/sh): 2024-06-13T06:33:58.469 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout: /usr/bin/kernel-install add 6.9.0-gff88c41504f3 /lib/modules/6.9.0-gff88c41504f3/vmlinuz 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout:for file in vmlinuz System.map config; do 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout: if ! cmp --silent "/lib/modules/6.9.0-gff88c41504f3/${file}" "/boot/${file}-6.9.0-gff88c41504f3"; then 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout: cp "/lib/modules/6.9.0-gff88c41504f3/${file}" "/boot/${file}-6.9.0-gff88c41504f3" 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout: fi 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout:done 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -d "/lib/modules/6.9.0-gff88c41504f3/dtb" ] && \ 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout: ! diff -rq "/lib/modules/6.9.0-gff88c41504f3/dtb" "/boot/dtb-6.9.0-gff88c41504f3" >/dev/null 2>&1; then 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout: rm -rf "/boot/dtb-6.9.0-gff88c41504f3" 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout: cp -r "/lib/modules/6.9.0-gff88c41504f3/dtb" "/boot/dtb-6.9.0-gff88c41504f3" 2024-06-13T06:33:58.470 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-06-13T06:33:58.471 INFO:teuthology.orchestra.run.smithi026.stdout:if [ ! -e "/lib/modules/6.9.0-gff88c41504f3/modules.dep" ]; then 2024-06-13T06:33:58.471 INFO:teuthology.orchestra.run.smithi026.stdout: /usr/sbin/depmod 6.9.0-gff88c41504f3 2024-06-13T06:33:58.471 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-06-13T06:33:58.471 INFO:teuthology.orchestra.run.smithi026.stdout:preuninstall scriptlet (using /bin/sh): 2024-06-13T06:33:58.471 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-13T06:33:58.471 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-install remove 6.9.0-gff88c41504f3 2024-06-13T06:33:58.471 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-06-13T06:33:58.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.06 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 68.el9 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:Group : Unspecified 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 8264254 2024-06-13T06:33:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : builder@centos.org 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : CentOS 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-13T06:33:58.546 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2024-06-13T06:33:58.547 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:33:58.547 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2024-06-13T06:33:58.547 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-13T06:33:58.548 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 6.9.0-gff88c41504f3 2024-06-13T06:33:58.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-13T06:33:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-06-13T06:34:00.554 INFO:teuthology.orchestra.run.smithi026.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-13T06:34:00.581 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-06-13T06:34:00.583 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-06-13T06:34:00.598 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.qymlmJXS5l 2024-06-13T06:34:00.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qymlmJXS5l 2024-06-13T06:34:00.675 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.qymlmJXS5l 2024-06-13T06:34:00.813 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.qymlmJXS5l is 6KB 2024-06-13T06:34:00.860 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.qymlmJXS5l 2024-06-13T06:34:00.876 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2024-06-13T06:34:00.938 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-13T06:34:00.938 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-13T06:34:00.938 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-13T06:34:00.938 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gff88c41504f3.conf 2024-06-13T06:34:00.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gff88c41504f3 2024-06-13T06:34:01.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-06-13T06:34:31.110 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-13T06:34:31.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-06-13T06:34:31.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:34:49.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-06-13T06:34:58.503 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-06-13T06:34:58.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:35:16.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-06-13T06:35:28.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-06-13T06:35:28.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:35:32.279 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-06-13T06:35:32.975 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-06-13T06:35:32.975 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gff88c41504f3"... 2024-06-13T06:35:32.976 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-06-13T06:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout:6.9.0-gff88c41504f3 2024-06-13T06:35:33.033 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gff88c41504f3 vs 6.9.0-gff88c41504f3 2024-06-13T06:35:33.034 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-13T06:35:33.034 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-13T06:35:34.035 DEBUG:teuthology.parallel:result is None 2024-06-13T06:35:34.239 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-13T06:35:34.251 INFO:teuthology.task.internal:Creating test directory... 2024-06-13T06:35:34.252 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-13T06:35:34.261 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-13T06:35:34.264 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-13T06:35:34.299 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-13T06:35:34.307 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-13T06:35:34.314 INFO:teuthology.task.internal:Creating archive directory... 2024-06-13T06:35:34.314 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-13T06:35:34.346 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-13T06:35:34.349 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-13T06:35:34.474 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-13T06:35:34.481 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-13T06:35:34.481 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-13T06:35:34.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:35:34.498 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-13T06:35:34.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:35:34.515 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-13T06:35:34.520 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-13T06:35:34.541 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-13T06:35:34.579 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:35:34.592 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:35:34.597 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:35:34.629 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:35:34.657 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:35:34.705 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:35:34.711 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-13T06:35:34.719 INFO:teuthology.task.internal:Configuring sudo... 2024-06-13T06:35:34.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-13T06:35:34.723 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-13T06:35:34.726 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-13T06:35:34.783 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-13T06:35:34.792 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-13T06:35:34.792 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-13T06:35:34.797 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-13T06:35:34.800 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-13T06:35:34.839 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:35:34.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:35:35.081 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:35:35.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:35:35.217 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:35:35.218 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-13T06:35:35.305 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:35:35.342 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:35:35.420 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:35:35.461 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:35:35.504 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:35:35.504 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-13T06:35:35.570 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:35:35.614 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:35:35.678 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:35:35.718 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:35:35.762 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:35:35.762 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-13T06:35:35.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-06-13T06:35:35.835 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-06-13T06:35:35.837 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-06-13T06:35:35.869 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T06:35:35.903 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T06:35:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T06:35:36.320 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-13T06:35:36.327 INFO:teuthology.task.internal:Starting timer... 2024-06-13T06:35:36.327 INFO:teuthology.run_tasks:Running task pcp... 2024-06-13T06:35:36.362 INFO:teuthology.run_tasks:Running task selinux... 2024-06-13T06:35:36.427 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-13T06:35:36.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-06-13T06:35:36.559 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-06-13T06:35:36.563 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-06-13T06:35:36.613 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-06-13T06:35:36.615 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-06-13T06:35:36.684 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-06-13T06:35:36.685 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-13T06:35:36.686 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-06-13T06:35:36.736 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-06-13T06:35:36.737 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-06-13T06:35:36.774 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-06-13T06:35:36.774 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-06-13T06:35:36.808 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-06-13T06:35:36.808 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2024-06-13T06:35:36.808 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T06:35:36.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:35:36.881 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T06:35:36.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:35:36.908 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T06:35:36.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:35:36.932 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-13T06:35:36.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-06-13T06:35:36.998 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-06-13T06:35:37.023 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-06-13T06:35:37.052 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-13T06:35:37.061 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-13T06:35:37.064 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-13T06:35:37.168 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-13T06:35:37.179 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-13T06:35:37.179 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi135.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-13T06:41:54.421 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-06-13T06:41:54.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-06-13T06:41:54.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:41:54.533 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-06-13T06:41:54.694 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-06-13T06:41:54.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-06-13T06:41:54.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:41:54.769 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-06-13T06:41:54.846 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-06-13T06:41:54.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-06-13T06:41:54.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:41:54.926 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-06-13T06:41:55.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-06-13T06:41:55.013 INFO:teuthology.run_tasks:Running task clock... 2024-06-13T06:41:55.024 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-13T06:41:55.024 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-13T06:41:55.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T06:41:55.032 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-13T06:41:55.032 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T06:41:55.035 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-13T06:41:55.035 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T06:41:55.063 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-13T06:41:55.077 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-13T06:41:55.084 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-13T06:41:55.099 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-13T06:41:55.100 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-06-13T06:41:55.111 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-06-13T06:41:55.123 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-06-13T06:41:55.125 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-13T06:41:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-13T06:41:55.134 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-06-13T06:41:55.138 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-13T06:41:55.148 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-13T06:41:55.162 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-13T06:41:55.175 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-13T06:41:55.195 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-06-13T06:41:55.199 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T06:41:55.199 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-06-13T06:41:55.199 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.199 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.199 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.200 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.219 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-06-13T06:41:55.222 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T06:41:55.223 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-06-13T06:41:55.223 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.223 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.223 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.223 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.250 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-06-13T06:41:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-06-13T06:41:55.326 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-13T06:41:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-13T06:41:55.518 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-06-13T06:41:55.529 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T06:41:55.529 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-06-13T06:41:55.529 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.529 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.529 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.529 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:41:55.530 INFO:teuthology.run_tasks:Running task install... 2024-06-13T06:41:55.539 DEBUG:teuthology.task.install:project ceph 2024-06-13T06:41:55.539 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-13T06:41:55.539 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-13T06:41:55.539 INFO:teuthology.task.install:Using flavor: default 2024-06-13T06:41:55.545 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-13T06:41:55.545 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-13T06:41:55.545 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-13T06:41:55.546 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:41:55.547 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-13T06:41:55.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:41:55.549 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-13T06:41:55.549 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:41:55.696 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-13T06:41:55.696 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-13T06:41:55.700 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-13T06:41:55.700 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-13T06:41:55.774 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-13T06:41:55.774 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-13T06:41:55.995 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-13T06:41:55.995 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:41:55.996 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-13T06:41:56.022 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-13T06:41:56.022 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-13T06:41:56.051 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-13T06:41:56.052 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:41:56.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-13T06:41:56.066 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-13T06:41:56.066 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:41:56.066 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-13T06:41:56.088 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-13T06:41:56.094 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-13T06:41:56.094 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-13T06:41:56.146 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-13T06:41:56.147 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-13T06:41:56.161 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-13T06:41:56.162 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-13T06:41:56.226 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-06-13T06:41:56.228 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-06-13T06:41:56.237 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-13T06:41:56.282 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-13T06:41:56.301 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-06-13T06:41:56.302 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-06-13T06:41:56.401 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-13T06:41:56.452 INFO:teuthology.orchestra.run.smithi135.stdout:45 files removed 2024-06-13T06:41:56.479 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-13T06:41:56.485 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-06-13T06:41:56.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-06-13T06:41:56.528 INFO:teuthology.orchestra.run.smithi136.stdout:45 files removed 2024-06-13T06:41:56.555 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-13T06:41:56.819 INFO:teuthology.orchestra.run.smithi026.stdout:45 files removed 2024-06-13T06:41:56.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-13T06:41:57.592 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 98 kB/s | 83 kB 00:00 2024-06-13T06:41:57.690 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 95 kB/s | 83 kB 00:00 2024-06-13T06:41:58.242 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-06-13T06:41:58.251 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 89 kB/s | 83 kB 00:00 2024-06-13T06:41:58.365 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-06-13T06:41:59.010 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-06-13T06:41:59.074 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-06-13T06:41:59.210 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 13 kB/s | 11 kB 00:00 2024-06-13T06:41:59.684 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-06-13T06:42:00.127 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-06-13T06:42:00.853 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.1 MB 00:01 2024-06-13T06:42:02.705 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.1 MB 00:02 2024-06-13T06:42:02.730 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-06-13T06:42:07.896 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.3 MB 00:01 2024-06-13T06:42:09.469 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 17 kB 00:00 2024-06-13T06:42:09.606 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 20 MB 00:05 2024-06-13T06:42:10.556 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-06-13T06:42:15.761 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-06-13T06:42:16.288 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 20 MB 00:14 2024-06-13T06:42:16.896 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.3 MB 00:03 2024-06-13T06:42:18.065 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:42:18.065 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:42:18.075 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-13T06:42:18.076 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-13T06:42:18.077 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-13T06:42:18.078 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-13T06:42:18.079 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-13T06:42:18.081 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-13T06:42:18.082 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-13T06:42:18.144 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-06-13T06:42:18.164 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-06-13T06:42:18.164 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-06-13T06:42:18.164 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-06-13T06:42:18.164 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-06-13T06:42:18.164 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-13T06:42:18.164 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-13T06:42:18.164 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-13T06:42:18.164 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-13T06:42:18.165 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.2.2-1.el9 appstream 4.4 M 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-13T06:42:18.166 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.4.0-1.el9 appstream 95 k 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-13T06:42:18.167 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.2.2-1.el9 baseos 1.4 M 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.2.2-1.el9 baseos 2.4 M 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs x86_64 6.4.0-1.el9 baseos 1.1 M 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-13T06:42:18.168 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-13T06:42:18.169 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-13T06:42:18.170 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-13T06:42:18.171 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-13T06:42:18.172 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-13T06:42:18.173 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-13T06:42:18.174 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-13T06:42:18.174 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-13T06:42:18.174 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-13T06:42:18.174 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-13T06:42:18.174 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-13T06:42:18.174 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-13T06:42:18.174 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-13T06:42:18.174 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-13T06:42:18.175 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-13T06:42:18.175 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-13T06:42:18.175 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-13T06:42:18.175 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-13T06:42:18.175 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-13T06:42:18.175 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-13T06:42:18.176 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-13T06:42:18.176 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-13T06:42:18.176 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-13T06:42:18.176 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-13T06:42:18.176 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-13T06:42:18.177 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-13T06:42:18.177 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-13T06:42:18.177 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-13T06:42:18.177 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-13T06:42:18.177 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-13T06:42:18.177 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-13T06:42:18.178 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-13T06:42:18.178 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-13T06:42:18.178 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-13T06:42:18.178 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-13T06:42:18.178 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-13T06:42:18.178 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-13T06:42:18.178 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-13T06:42:18.179 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout:Install 148 Packages 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 12 Packages 2024-06-13T06:42:18.180 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:42:18.181 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 226 M 2024-06-13T06:42:18.181 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-06-13T06:42:18.589 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 17 kB 00:00 2024-06-13T06:42:19.079 INFO:teuthology.orchestra.run.smithi136.stdout:(1/160): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-06-13T06:42:19.579 INFO:teuthology.orchestra.run.smithi136.stdout:(2/160): ceph-base-19.0.0-2946.g3d89accf.el9.x8 7.2 MB/s | 5.4 MB 00:00 2024-06-13T06:42:19.688 INFO:teuthology.orchestra.run.smithi136.stdout:(3/160): ceph-immutable-object-cache-19.0.0-294 1.3 MB/s | 145 kB 00:00 2024-06-13T06:42:19.905 INFO:teuthology.orchestra.run.smithi136.stdout:(4/160): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 890 kB 00:00 2024-06-13T06:42:20.076 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-06-13T06:42:20.313 INFO:teuthology.orchestra.run.smithi136.stdout:(5/160): ceph-common-19.0.0-2946.g3d89accf.el9. 14 MB/s | 21 MB 00:01 2024-06-13T06:42:20.363 INFO:teuthology.orchestra.run.smithi136.stdout:(6/160): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 3.5 MB/s | 2.4 MB 00:00 2024-06-13T06:42:20.655 INFO:teuthology.orchestra.run.smithi136.stdout:(7/160): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 14 MB/s | 4.7 MB 00:00 2024-06-13T06:42:20.689 INFO:teuthology.orchestra.run.smithi136.stdout:(8/160): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-06-13T06:42:20.865 INFO:teuthology.orchestra.run.smithi136.stdout:(9/160): ceph-selinux-19.0.0-2946.g3d89accf.el9 144 kB/s | 25 kB 00:00 2024-06-13T06:42:21.390 INFO:teuthology.orchestra.run.smithi136.stdout:(10/160): ceph-radosgw-19.0.0-2946.g3d89accf.el 14 MB/s | 10 MB 00:00 2024-06-13T06:42:21.457 INFO:teuthology.orchestra.run.smithi136.stdout:(11/160): cephfs-mirror-19.0.0-2946.g3d89accf.e 3.3 MB/s | 221 kB 00:00 2024-06-13T06:42:21.515 INFO:teuthology.orchestra.run.smithi136.stdout:(12/160): cephfs-shell-19.0.0-2946.g3d89accf.el 465 kB/s | 27 kB 00:00 2024-06-13T06:42:21.574 INFO:teuthology.orchestra.run.smithi136.stdout:(13/160): libcephfs-devel-19.0.0-2946.g3d89accf 544 kB/s | 32 kB 00:00 2024-06-13T06:42:21.666 INFO:teuthology.orchestra.run.smithi136.stdout:(14/160): libcephfs2-19.0.0-2946.g3d89accf.el9. 8.0 MB/s | 747 kB 00:00 2024-06-13T06:42:21.696 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.3 MB 00:01 2024-06-13T06:42:21.833 INFO:teuthology.orchestra.run.smithi136.stdout:(15/160): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 11 MB/s | 17 MB 00:01 2024-06-13T06:42:21.858 INFO:teuthology.orchestra.run.smithi136.stdout:(16/160): libcephsqlite-19.0.0-2946.g3d89accf.e 851 kB/s | 163 kB 00:00 2024-06-13T06:42:21.883 INFO:teuthology.orchestra.run.smithi136.stdout:(17/160): librados-devel-19.0.0-2946.g3d89accf. 2.5 MB/s | 127 kB 00:00 2024-06-13T06:42:21.917 INFO:teuthology.orchestra.run.smithi136.stdout:(18/160): libradosstriper1-19.0.0-2946.g3d89acc 8.4 MB/s | 501 kB 00:00 2024-06-13T06:42:21.967 INFO:teuthology.orchestra.run.smithi136.stdout:(19/160): python3-ceph-argparse-19.0.0-2946.g3d 903 kB/s | 45 kB 00:00 2024-06-13T06:42:22.068 INFO:teuthology.orchestra.run.smithi136.stdout:(20/160): python3-ceph-common-19.0.0-2946.g3d89 1.3 MB/s | 135 kB 00:00 2024-06-13T06:42:22.143 INFO:teuthology.orchestra.run.smithi136.stdout:(21/160): python3-cephfs-19.0.0-2946.g3d89accf. 2.1 MB/s | 162 kB 00:00 2024-06-13T06:42:22.218 INFO:teuthology.orchestra.run.smithi136.stdout:(22/160): librgw2-19.0.0-2946.g3d89accf.el9.x86 16 MB/s | 5.3 MB 00:00 2024-06-13T06:42:22.252 INFO:teuthology.orchestra.run.smithi136.stdout:(23/160): python3-rados-19.0.0-2946.g3d89accf.e 2.9 MB/s | 321 kB 00:00 2024-06-13T06:42:22.277 INFO:teuthology.orchestra.run.smithi136.stdout:(24/160): python3-rbd-19.0.0-2946.g3d89accf.el9 5.0 MB/s | 300 kB 00:00 2024-06-13T06:42:22.303 INFO:teuthology.orchestra.run.smithi136.stdout:(25/160): python3-rgw-19.0.0-2946.g3d89accf.el9 1.9 MB/s | 99 kB 00:00 2024-06-13T06:42:22.328 INFO:teuthology.orchestra.run.smithi136.stdout:(26/160): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-13T06:42:22.453 INFO:teuthology.orchestra.run.smithi136.stdout:(27/160): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.3 MB/s | 171 kB 00:00 2024-06-13T06:42:22.520 INFO:teuthology.orchestra.run.smithi136.stdout:(28/160): rbd-mirror-19.0.0-2946.g3d89accf.el9. 14 MB/s | 3.1 MB 00:00 2024-06-13T06:42:22.546 INFO:teuthology.orchestra.run.smithi136.stdout:(29/160): ceph-grafana-dashboards-19.0.0-2946.g 344 kB/s | 32 kB 00:00 2024-06-13T06:42:22.571 INFO:teuthology.orchestra.run.smithi136.stdout:(30/160): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.8 MB/s | 142 kB 00:00 2024-06-13T06:42:22.930 INFO:teuthology.orchestra.run.smithi136.stdout:(31/160): ceph-mgr-dashboard-19.0.0-2946.g3d89a 9.2 MB/s | 3.5 MB 00:00 2024-06-13T06:42:23.013 INFO:teuthology.orchestra.run.smithi136.stdout:(32/160): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-13T06:42:23.147 INFO:teuthology.orchestra.run.smithi136.stdout:(33/160): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-06-13T06:42:23.180 INFO:teuthology.orchestra.run.smithi136.stdout:(34/160): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 295 kB/s | 49 kB 00:00 2024-06-13T06:42:23.206 INFO:teuthology.orchestra.run.smithi136.stdout:(35/160): ceph-prometheus-alerts-19.0.0-2946.g3 282 kB/s | 16 kB 00:00 2024-06-13T06:42:23.231 INFO:teuthology.orchestra.run.smithi136.stdout:(36/160): ceph-volume-19.0.0-2946.g3d89accf.el9 5.1 MB/s | 264 kB 00:00 2024-06-13T06:42:23.298 INFO:teuthology.orchestra.run.smithi136.stdout:(37/160): cephadm-19.0.0-2946.g3d89accf.el9.noa 8.1 MB/s | 763 kB 00:00 2024-06-13T06:42:23.315 INFO:teuthology.orchestra.run.smithi136.stdout:(38/160): cephfs-top-19.0.0-2946.g3d89accf.el9. 218 kB/s | 18 kB 00:00 2024-06-13T06:42:23.372 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 17 kB 00:00 2024-06-13T06:42:23.372 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:23.379 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:23.482 INFO:teuthology.orchestra.run.smithi136.stdout:(39/160): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-06-13T06:42:23.485 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:23.549 INFO:teuthology.orchestra.run.smithi136.stdout:(40/160): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-06-13T06:42:23.574 INFO:teuthology.orchestra.run.smithi136.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-06-13T06:42:23.600 INFO:teuthology.orchestra.run.smithi136.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-06-13T06:42:23.641 INFO:teuthology.orchestra.run.smithi136.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-13T06:42:23.684 INFO:teuthology.orchestra.run.smithi136.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-06-13T06:42:23.709 INFO:teuthology.orchestra.run.smithi136.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-06-13T06:42:23.743 INFO:teuthology.orchestra.run.smithi136.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-13T06:42:23.743 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-13T06:42:23.785 INFO:teuthology.orchestra.run.smithi136.stdout:(47/160): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-06-13T06:42:23.788 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:23.826 INFO:teuthology.orchestra.run.smithi136.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 8.0 MB/s | 689 kB 00:00 2024-06-13T06:42:23.828 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:23.860 INFO:teuthology.orchestra.run.smithi136.stdout:(49/160): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-06-13T06:42:23.885 INFO:teuthology.orchestra.run.smithi136.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-06-13T06:42:23.911 INFO:teuthology.orchestra.run.smithi136.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 429 kB/s | 22 kB 00:00 2024-06-13T06:42:23.936 INFO:teuthology.orchestra.run.smithi136.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-13T06:42:23.961 INFO:teuthology.orchestra.run.smithi136.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-06-13T06:42:24.012 INFO:teuthology.orchestra.run.smithi136.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-06-13T06:42:24.063 INFO:teuthology.orchestra.run.smithi136.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-06-13T06:42:24.088 INFO:teuthology.orchestra.run.smithi136.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-06-13T06:42:24.113 INFO:teuthology.orchestra.run.smithi136.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-13T06:42:24.138 INFO:teuthology.orchestra.run.smithi136.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-06-13T06:42:24.180 INFO:teuthology.orchestra.run.smithi136.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-06-13T06:42:24.222 INFO:teuthology.orchestra.run.smithi136.stdout:(60/160): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-06-13T06:42:24.248 INFO:teuthology.orchestra.run.smithi136.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 379 kB/s | 25 kB 00:00 2024-06-13T06:42:24.273 INFO:teuthology.orchestra.run.smithi136.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-06-13T06:42:24.318 INFO:teuthology.orchestra.run.smithi136.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 292 kB 00:00 2024-06-13T06:42:24.348 INFO:teuthology.orchestra.run.smithi136.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-06-13T06:42:24.374 INFO:teuthology.orchestra.run.smithi136.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 630 kB/s | 35 kB 00:00 2024-06-13T06:42:24.375 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-06-13T06:42:24.458 INFO:teuthology.orchestra.run.smithi136.stdout:(66/160): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-06-13T06:42:24.524 INFO:teuthology.orchestra.run.smithi136.stdout:(67/160): openssl-devel-3.2.2-1.el9.x86_64.rpm 30 MB/s | 4.4 MB 00:00 2024-06-13T06:42:24.566 INFO:teuthology.orchestra.run.smithi136.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-13T06:42:24.600 INFO:teuthology.orchestra.run.smithi136.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-06-13T06:42:24.625 INFO:teuthology.orchestra.run.smithi136.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-13T06:42:24.684 INFO:teuthology.orchestra.run.smithi136.stdout:(71/160): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-06-13T06:42:24.759 INFO:teuthology.orchestra.run.smithi136.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-06-13T06:42:24.784 INFO:teuthology.orchestra.run.smithi136.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-06-13T06:42:24.818 INFO:teuthology.orchestra.run.smithi136.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-06-13T06:42:24.902 INFO:teuthology.orchestra.run.smithi136.stdout:(75/160): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-06-13T06:42:24.927 INFO:teuthology.orchestra.run.smithi136.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-13T06:42:24.952 INFO:teuthology.orchestra.run.smithi136.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-13T06:42:24.978 INFO:teuthology.orchestra.run.smithi136.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-06-13T06:42:25.003 INFO:teuthology.orchestra.run.smithi136.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-06-13T06:42:25.030 INFO:teuthology.orchestra.run.smithi136.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-13T06:42:25.054 INFO:teuthology.orchestra.run.smithi136.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 685 kB/s | 35 kB 00:00 2024-06-13T06:42:25.096 INFO:teuthology.orchestra.run.smithi136.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-13T06:42:25.605 INFO:teuthology.orchestra.run.smithi136.stdout:(83/160): ceph-test-19.0.0-2946.g3d89accf.el9.x 10 MB/s | 48 MB 00:04 2024-06-13T06:42:25.630 INFO:teuthology.orchestra.run.smithi136.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 416 kB/s | 222 kB 00:00 2024-06-13T06:42:25.697 INFO:teuthology.orchestra.run.smithi136.stdout:(85/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.6 MB/s | 5.1 MB 00:00 2024-06-13T06:42:25.722 INFO:teuthology.orchestra.run.smithi136.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 459 kB/s | 42 kB 00:00 2024-06-13T06:42:25.747 INFO:teuthology.orchestra.run.smithi136.stdout:(87/160): python3-packaging-20.9-5.el9.noarch.r 543 kB/s | 77 kB 00:00 2024-06-13T06:42:25.773 INFO:teuthology.orchestra.run.smithi136.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-13T06:42:25.798 INFO:teuthology.orchestra.run.smithi136.stdout:(89/160): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-06-13T06:42:25.823 INFO:teuthology.orchestra.run.smithi136.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-13T06:42:25.849 INFO:teuthology.orchestra.run.smithi136.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-13T06:42:25.874 INFO:teuthology.orchestra.run.smithi136.stdout:(92/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-06-13T06:42:25.899 INFO:teuthology.orchestra.run.smithi136.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-06-13T06:42:25.924 INFO:teuthology.orchestra.run.smithi136.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 876 kB/s | 44 kB 00:00 2024-06-13T06:42:25.949 INFO:teuthology.orchestra.run.smithi136.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-13T06:42:25.975 INFO:teuthology.orchestra.run.smithi136.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-06-13T06:42:26.000 INFO:teuthology.orchestra.run.smithi136.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-06-13T06:42:26.025 INFO:teuthology.orchestra.run.smithi136.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-06-13T06:42:26.051 INFO:teuthology.orchestra.run.smithi136.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-13T06:42:26.076 INFO:teuthology.orchestra.run.smithi136.stdout:(100/160): xfsprogs-devel-6.4.0-1.el9.x86_64.rp 1.8 MB/s | 95 kB 00:00 2024-06-13T06:42:26.102 INFO:teuthology.orchestra.run.smithi136.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-13T06:42:26.127 INFO:teuthology.orchestra.run.smithi136.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-06-13T06:42:26.127 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:26.128 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:26.361 INFO:teuthology.orchestra.run.smithi136.stdout:(103/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-06-13T06:42:26.394 INFO:teuthology.orchestra.run.smithi136.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 16 MB/s | 548 kB 00:00 2024-06-13T06:42:26.428 INFO:teuthology.orchestra.run.smithi136.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-06-13T06:42:26.461 INFO:teuthology.orchestra.run.smithi136.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 582 kB/s | 19 kB 00:00 2024-06-13T06:42:26.587 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-06-13T06:42:26.620 INFO:teuthology.orchestra.run.smithi136.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-06-13T06:42:26.646 INFO:teuthology.orchestra.run.smithi136.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-13T06:42:26.671 INFO:teuthology.orchestra.run.smithi136.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-13T06:42:26.696 INFO:teuthology.orchestra.run.smithi136.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-06-13T06:42:26.730 INFO:teuthology.orchestra.run.smithi136.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-06-13T06:42:26.780 INFO:teuthology.orchestra.run.smithi136.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-06-13T06:42:26.814 INFO:teuthology.orchestra.run.smithi136.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-06-13T06:42:26.830 INFO:teuthology.orchestra.run.smithi136.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-06-13T06:42:26.847 INFO:teuthology.orchestra.run.smithi136.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-06-13T06:42:26.864 INFO:teuthology.orchestra.run.smithi136.stdout:(116/160): python3-certifi-2021.10.8-2.el9.noar 896 kB/s | 15 kB 00:00 2024-06-13T06:42:26.890 INFO:teuthology.orchestra.run.smithi136.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-06-13T06:42:26.923 INFO:teuthology.orchestra.run.smithi136.stdout:(118/160): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-06-13T06:42:26.957 INFO:teuthology.orchestra.run.smithi136.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-06-13T06:42:26.982 INFO:teuthology.orchestra.run.smithi136.stdout:(120/160): python3-google-auth-2.29.0-2.el9.noa 8.1 MB/s | 205 kB 00:00 2024-06-13T06:42:27.058 INFO:teuthology.orchestra.run.smithi136.stdout:(121/160): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-06-13T06:42:27.083 INFO:teuthology.orchestra.run.smithi136.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-06-13T06:42:27.108 INFO:teuthology.orchestra.run.smithi136.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-06-13T06:42:27.133 INFO:teuthology.orchestra.run.smithi136.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-06-13T06:42:27.159 INFO:teuthology.orchestra.run.smithi136.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 921 kB/s | 23 kB 00:00 2024-06-13T06:42:27.184 INFO:teuthology.orchestra.run.smithi136.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-06-13T06:42:27.209 INFO:teuthology.orchestra.run.smithi136.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 785 kB/s | 20 kB 00:00 2024-06-13T06:42:27.285 INFO:teuthology.orchestra.run.smithi136.stdout:(128/160): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-06-13T06:42:27.310 INFO:teuthology.orchestra.run.smithi136.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-13T06:42:27.335 INFO:teuthology.orchestra.run.smithi136.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-13T06:42:27.360 INFO:teuthology.orchestra.run.smithi136.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-13T06:42:27.394 INFO:teuthology.orchestra.run.smithi136.stdout:(132/160): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-06-13T06:42:27.419 INFO:teuthology.orchestra.run.smithi136.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-06-13T06:42:27.445 INFO:teuthology.orchestra.run.smithi136.stdout:(134/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-06-13T06:42:27.478 INFO:teuthology.orchestra.run.smithi136.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 841 kB/s | 28 kB 00:00 2024-06-13T06:42:27.504 INFO:teuthology.orchestra.run.smithi136.stdout:(136/160): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-06-13T06:42:27.529 INFO:teuthology.orchestra.run.smithi136.stdout:(137/160): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-06-13T06:42:27.554 INFO:teuthology.orchestra.run.smithi136.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-13T06:42:27.580 INFO:teuthology.orchestra.run.smithi136.stdout:(139/160): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-06-13T06:42:27.605 INFO:teuthology.orchestra.run.smithi136.stdout:(140/160): python3-typing-extensions-4.4.0-2.el 2.0 MB/s | 51 kB 00:00 2024-06-13T06:42:27.639 INFO:teuthology.orchestra.run.smithi136.stdout:(141/160): python3-webob-1.8.7-6.el9.noarch.rpm 6.7 MB/s | 230 kB 00:00 2024-06-13T06:42:27.664 INFO:teuthology.orchestra.run.smithi136.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-13T06:42:27.689 INFO:teuthology.orchestra.run.smithi136.stdout:(143/160): lua-devel-5.4.4-4.el9.x86_64.rpm 14 kB/s | 22 kB 00:01 2024-06-13T06:42:27.723 INFO:teuthology.orchestra.run.smithi136.stdout:(144/160): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-06-13T06:42:27.740 INFO:teuthology.orchestra.run.smithi136.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-06-13T06:42:27.765 INFO:teuthology.orchestra.run.smithi136.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-06-13T06:42:27.824 INFO:teuthology.orchestra.run.smithi136.stdout:(147/160): protobuf-compiler-3.14.0-13.el9.x86_ 509 kB/s | 863 kB 00:01 2024-06-13T06:42:27.874 INFO:teuthology.orchestra.run.smithi136.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-06-13T06:42:28.033 INFO:teuthology.orchestra.run.smithi136.stdout:(149/160): NetworkManager-1.47.91-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-06-13T06:42:28.100 INFO:teuthology.orchestra.run.smithi136.stdout:(150/160): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-13T06:42:28.142 INFO:teuthology.orchestra.run.smithi136.stdout:(151/160): NetworkManager-team-1.47.91-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-06-13T06:42:28.200 INFO:teuthology.orchestra.run.smithi136.stdout:(152/160): librados2-19.0.0-2946.g3d89accf.el9. 7.8 MB/s | 3.4 MB 00:00 2024-06-13T06:42:28.225 INFO:teuthology.orchestra.run.smithi136.stdout:(153/160): NetworkManager-tui-1.47.91-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-06-13T06:42:28.284 INFO:teuthology.orchestra.run.smithi136.stdout:(154/160): librbd1-19.0.0-2946.g3d89accf.el9.x8 6.8 MB/s | 3.1 MB 00:00 2024-06-13T06:42:28.309 INFO:teuthology.orchestra.run.smithi136.stdout:(155/160): elfutils-debuginfod-client-0.191-4.e 337 kB/s | 37 kB 00:00 2024-06-13T06:42:28.335 INFO:teuthology.orchestra.run.smithi136.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-06-13T06:42:28.410 INFO:teuthology.orchestra.run.smithi136.stdout:(157/160): openssl-libs-3.2.2-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-13T06:42:28.435 INFO:teuthology.orchestra.run.smithi136.stdout:(158/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.7 MB/s | 260 kB 00:00 2024-06-13T06:42:28.477 INFO:teuthology.orchestra.run.smithi136.stdout:(159/160): xfsprogs-6.4.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-06-13T06:42:28.511 INFO:teuthology.orchestra.run.smithi136.stdout:(160/160): openssl-3.2.2-1.el9.x86_64.rpm 6.7 MB/s | 1.4 MB 00:00 2024-06-13T06:42:28.556 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:42:28.556 INFO:teuthology.orchestra.run.smithi136.stdout:Total 22 MB/s | 226 MB 00:10 2024-06-13T06:42:29.265 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:42:29.266 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:42:29.281 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-13T06:42:29.282 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-13T06:42:29.283 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-13T06:42:29.284 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-13T06:42:29.286 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-13T06:42:29.288 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-13T06:42:29.289 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-13T06:42:29.360 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-13T06:42:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-13T06:42:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.2.2-1.el9 appstream 4.4 M 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-13T06:42:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.4.0-1.el9 appstream 95 k 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-13T06:42:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.2.2-1.el9 baseos 1.4 M 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.2.2-1.el9 baseos 2.4 M 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs x86_64 6.4.0-1.el9 baseos 1.1 M 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-13T06:42:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-13T06:42:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-13T06:42:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-13T06:42:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-13T06:42:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-13T06:42:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-13T06:42:29.408 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-13T06:42:29.409 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-13T06:42:29.410 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-06-13T06:42:29.411 INFO:teuthology.orchestra.run.smithi026.stdout:Install 148 Packages 2024-06-13T06:42:29.412 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 12 Packages 2024-06-13T06:42:29.412 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:42:29.412 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 226 M 2024-06-13T06:42:29.412 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-06-13T06:42:29.632 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-06-13T06:42:29.759 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-06-13T06:42:29.952 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-06-13T06:42:29.952 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-06-13T06:42:30.547 INFO:teuthology.orchestra.run.smithi026.stdout:(1/160): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-06-13T06:42:30.848 INFO:teuthology.orchestra.run.smithi026.stdout:(2/160): ceph-base-19.0.0-2946.g3d89accf.el9.x8 8.4 MB/s | 5.4 MB 00:00 2024-06-13T06:42:30.923 INFO:teuthology.orchestra.run.smithi026.stdout:(3/160): ceph-immutable-object-cache-19.0.0-294 1.9 MB/s | 145 kB 00:00 2024-06-13T06:42:31.106 INFO:teuthology.orchestra.run.smithi026.stdout:(4/160): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 13 MB/s | 2.4 MB 00:00 2024-06-13T06:42:31.248 INFO:teuthology.orchestra.run.smithi026.stdout:(5/160): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 11 MB/s | 1.5 MB 00:00 2024-06-13T06:42:31.357 INFO:teuthology.orchestra.run.smithi026.stdout:(6/160): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 890 kB 00:00 2024-06-13T06:42:31.632 INFO:teuthology.orchestra.run.smithi026.stdout:(7/160): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 12 MB/s | 4.7 MB 00:00 2024-06-13T06:42:31.755 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-06-13T06:42:31.755 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-06-13T06:42:31.921 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:42:31.921 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:42:31.932 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-13T06:42:31.932 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-13T06:42:31.933 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-13T06:42:31.934 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-13T06:42:31.936 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-13T06:42:31.938 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-13T06:42:31.939 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-13T06:42:32.001 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-06-13T06:42:32.021 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-13T06:42:32.021 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-06-13T06:42:32.021 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-13T06:42:32.021 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-06-13T06:42:32.021 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-13T06:42:32.021 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-13T06:42:32.022 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-13T06:42:32.023 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:3.2.2-1.el9 appstream 4.4 M 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 6.4.0-1.el9 appstream 95 k 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-13T06:42:32.024 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: openssl x86_64 1:3.2.2-1.el9 baseos 1.4 M 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs x86_64 1:3.2.2-1.el9 baseos 2.4 M 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs x86_64 6.4.0-1.el9 baseos 1.1 M 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-13T06:42:32.025 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-13T06:42:32.026 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-13T06:42:32.027 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-13T06:42:32.028 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-13T06:42:32.029 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-13T06:42:32.030 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-13T06:42:32.031 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-13T06:42:32.032 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-13T06:42:32.033 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-13T06:42:32.034 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-06-13T06:42:32.034 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-13T06:42:32.034 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:42:32.034 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-06-13T06:42:32.034 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-13T06:42:32.034 INFO:teuthology.orchestra.run.smithi135.stdout:Install 148 Packages 2024-06-13T06:42:32.034 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 12 Packages 2024-06-13T06:42:32.034 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:42:32.035 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 226 M 2024-06-13T06:42:32.035 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-06-13T06:42:32.291 INFO:teuthology.orchestra.run.smithi026.stdout:(8/160): ceph-radosgw-19.0.0-2946.g3d89accf.el9 16 MB/s | 10 MB 00:00 2024-06-13T06:42:32.366 INFO:teuthology.orchestra.run.smithi026.stdout:(9/160): ceph-selinux-19.0.0-2946.g3d89accf.el9 339 kB/s | 25 kB 00:00 2024-06-13T06:42:33.108 INFO:teuthology.orchestra.run.smithi135.stdout:(1/160): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-06-13T06:42:33.934 INFO:teuthology.orchestra.run.smithi135.stdout:(2/160): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 890 kB 00:00 2024-06-13T06:42:34.084 INFO:teuthology.orchestra.run.smithi135.stdout:(3/160): ceph-immutable-object-cache-19.0.0-294 965 kB/s | 145 kB 00:00 2024-06-13T06:42:34.558 INFO:teuthology.orchestra.run.smithi026.stdout:(10/160): ceph-common-19.0.0-2946.g3d89accf.el9 4.8 MB/s | 21 MB 00:04 2024-06-13T06:42:34.691 INFO:teuthology.orchestra.run.smithi026.stdout:(11/160): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.6 MB/s | 221 kB 00:00 2024-06-13T06:42:34.808 INFO:teuthology.orchestra.run.smithi026.stdout:(12/160): cephfs-shell-19.0.0-2946.g3d89accf.el 234 kB/s | 27 kB 00:00 2024-06-13T06:42:34.901 INFO:teuthology.orchestra.run.smithi135.stdout:(4/160): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.9 MB/s | 2.4 MB 00:00 2024-06-13T06:42:34.908 INFO:teuthology.orchestra.run.smithi026.stdout:(13/160): libcephfs-devel-19.0.0-2946.g3d89accf 320 kB/s | 32 kB 00:00 2024-06-13T06:42:35.225 INFO:teuthology.orchestra.run.smithi026.stdout:(14/160): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.3 MB/s | 747 kB 00:00 2024-06-13T06:42:35.384 INFO:teuthology.orchestra.run.smithi135.stdout:(5/160): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-06-13T06:42:35.584 INFO:teuthology.orchestra.run.smithi026.stdout:(15/160): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 4.0 MB/s | 17 MB 00:04 2024-06-13T06:42:35.601 INFO:teuthology.orchestra.run.smithi135.stdout:(6/160): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-06-13T06:42:35.618 INFO:teuthology.orchestra.run.smithi026.stdout:(16/160): libcephsqlite-19.0.0-2946.g3d89accf.e 418 kB/s | 163 kB 00:00 2024-06-13T06:42:35.676 INFO:teuthology.orchestra.run.smithi026.stdout:(17/160): librados-devel-19.0.0-2946.g3d89accf. 1.4 MB/s | 127 kB 00:00 2024-06-13T06:42:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:(18/160): libradosstriper1-19.0.0-2946.g3d89acc 2.8 MB/s | 501 kB 00:00 2024-06-13T06:42:35.885 INFO:teuthology.orchestra.run.smithi026.stdout:(19/160): python3-ceph-argparse-19.0.0-2946.g3d 496 kB/s | 45 kB 00:00 2024-06-13T06:42:36.020 INFO:teuthology.orchestra.run.smithi026.stdout:(20/160): python3-ceph-common-19.0.0-2946.g3d89 1.0 MB/s | 135 kB 00:00 2024-06-13T06:42:36.552 INFO:teuthology.orchestra.run.smithi135.stdout:(7/160): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 4.0 MB/s | 4.7 MB 00:01 2024-06-13T06:42:36.565 INFO:teuthology.orchestra.run.smithi026.stdout:(21/160): ceph-test-19.0.0-2946.g3d89accf.el9.x 12 MB/s | 48 MB 00:04 2024-06-13T06:42:36.603 INFO:teuthology.orchestra.run.smithi026.stdout:(22/160): python3-cephfs-19.0.0-2946.g3d89accf. 278 kB/s | 162 kB 00:00 2024-06-13T06:42:36.762 INFO:teuthology.orchestra.run.smithi026.stdout:(23/160): librgw2-19.0.0-2946.g3d89accf.el9.x86 4.9 MB/s | 5.3 MB 00:01 2024-06-13T06:42:36.795 INFO:teuthology.orchestra.run.smithi026.stdout:(24/160): python3-rados-19.0.0-2946.g3d89accf.e 1.4 MB/s | 321 kB 00:00 2024-06-13T06:42:36.854 INFO:teuthology.orchestra.run.smithi026.stdout:(25/160): python3-rbd-19.0.0-2946.g3d89accf.el9 1.2 MB/s | 300 kB 00:00 2024-06-13T06:42:36.879 INFO:teuthology.orchestra.run.smithi026.stdout:(26/160): python3-rgw-19.0.0-2946.g3d89accf.el9 854 kB/s | 99 kB 00:00 2024-06-13T06:42:36.904 INFO:teuthology.orchestra.run.smithi026.stdout:(27/160): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 789 kB/s | 85 kB 00:00 2024-06-13T06:42:36.980 INFO:teuthology.orchestra.run.smithi026.stdout:(28/160): ceph-grafana-dashboards-19.0.0-2946.g 430 kB/s | 32 kB 00:00 2024-06-13T06:42:37.013 INFO:teuthology.orchestra.run.smithi026.stdout:(29/160): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.5 MB/s | 171 kB 00:00 2024-06-13T06:42:37.047 INFO:teuthology.orchestra.run.smithi026.stdout:(30/160): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.1 MB/s | 142 kB 00:00 2024-06-13T06:42:37.598 INFO:teuthology.orchestra.run.smithi026.stdout:(31/160): rbd-mirror-19.0.0-2946.g3d89accf.el9. 4.2 MB/s | 3.1 MB 00:00 2024-06-13T06:42:37.698 INFO:teuthology.orchestra.run.smithi026.stdout:(32/160): ceph-mgr-dashboard-19.0.0-2946.g3d89a 5.1 MB/s | 3.5 MB 00:00 2024-06-13T06:42:37.731 INFO:teuthology.orchestra.run.smithi026.stdout:(33/160): ceph-mgr-modules-core-19.0.0-2946.g3d 1.8 MB/s | 249 kB 00:00 2024-06-13T06:42:37.807 INFO:teuthology.orchestra.run.smithi026.stdout:(34/160): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 458 kB/s | 49 kB 00:00 2024-06-13T06:42:37.832 INFO:teuthology.orchestra.run.smithi026.stdout:(35/160): ceph-prometheus-alerts-19.0.0-2946.g3 165 kB/s | 16 kB 00:00 2024-06-13T06:42:37.974 INFO:teuthology.orchestra.run.smithi026.stdout:(36/160): ceph-mgr-diskprediction-local-19.0.0- 8.0 MB/s | 7.4 MB 00:00 2024-06-13T06:42:38.007 INFO:teuthology.orchestra.run.smithi026.stdout:(37/160): ceph-volume-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 264 kB 00:00 2024-06-13T06:42:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-06-13T06:42:38.125 INFO:teuthology.orchestra.run.smithi026.stdout:(38/160): cephfs-top-19.0.0-2946.g3d89accf.el9. 122 kB/s | 18 kB 00:00 2024-06-13T06:42:38.134 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.2.2-1.el9.x86_64 1/172 2024-06-13T06:42:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-06-13T06:42:38.225 INFO:teuthology.orchestra.run.smithi026.stdout:(39/160): cephadm-19.0.0-2946.g3d89accf.el9.noa 1.9 MB/s | 763 kB 00:00 2024-06-13T06:42:38.233 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:38.234 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-06-13T06:42:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-06-13T06:42:38.300 INFO:teuthology.orchestra.run.smithi026.stdout:(40/160): fuse-2.9.9-15.el9.x86_64.rpm 276 kB/s | 80 kB 00:00 2024-06-13T06:42:38.317 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-06-13T06:42:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-06-13T06:42:38.367 INFO:teuthology.orchestra.run.smithi026.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-06-13T06:42:38.392 INFO:teuthology.orchestra.run.smithi026.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-06-13T06:42:38.451 INFO:teuthology.orchestra.run.smithi026.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-06-13T06:42:38.476 INFO:teuthology.orchestra.run.smithi026.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-06-13T06:42:38.510 INFO:teuthology.orchestra.run.smithi026.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-13T06:42:38.543 INFO:teuthology.orchestra.run.smithi026.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-06-13T06:42:38.546 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-13T06:42:38.577 INFO:teuthology.orchestra.run.smithi026.stdout:(47/160): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-06-13T06:42:38.580 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:38.636 INFO:teuthology.orchestra.run.smithi026.stdout:(48/160): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-06-13T06:42:38.638 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-06-13T06:42:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:42:38.810 INFO:teuthology.orchestra.run.smithi135.stdout:(8/160): ceph-radosgw-19.0.0-2946.g3d89accf.el9 4.5 MB/s | 10 MB 00:02 2024-06-13T06:42:38.828 INFO:teuthology.orchestra.run.smithi026.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 560 kB/s | 107 kB 00:00 2024-06-13T06:42:38.886 INFO:teuthology.orchestra.run.smithi135.stdout:(9/160): ceph-selinux-19.0.0-2946.g3d89accf.el9 336 kB/s | 25 kB 00:00 2024-06-13T06:42:38.903 INFO:teuthology.orchestra.run.smithi026.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-06-13T06:42:38.914 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:42:38.920 INFO:teuthology.orchestra.run.smithi026.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 238 kB/s | 22 kB 00:00 2024-06-13T06:42:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:42:38.987 INFO:teuthology.orchestra.run.smithi026.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 943 kB 00:00 2024-06-13T06:42:39.004 INFO:teuthology.orchestra.run.smithi026.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 394 kB/s | 33 kB 00:00 2024-06-13T06:42:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:42:39.038 INFO:teuthology.orchestra.run.smithi026.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-06-13T06:42:39.054 INFO:teuthology.orchestra.run.smithi026.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 368 kB/s | 18 kB 00:00 2024-06-13T06:42:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:42:39.088 INFO:teuthology.orchestra.run.smithi026.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 226 kB/s | 11 kB 00:00 2024-06-13T06:42:39.098 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-06-13T06:42:39.147 INFO:teuthology.orchestra.run.smithi026.stdout:(57/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 787 kB/s | 45 kB 00:00 2024-06-13T06:42:39.180 INFO:teuthology.orchestra.run.smithi026.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-06-13T06:42:39.281 INFO:teuthology.orchestra.run.smithi026.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 4.9 MB/s | 662 kB 00:00 2024-06-13T06:42:39.296 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-06-13T06:42:39.323 INFO:teuthology.orchestra.run.smithi026.stdout:(60/160): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-06-13T06:42:39.346 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-06-13T06:42:39.348 INFO:teuthology.orchestra.run.smithi026.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 382 kB/s | 25 kB 00:00 2024-06-13T06:42:39.366 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:42:39.382 INFO:teuthology.orchestra.run.smithi026.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 824 kB/s | 47 kB 00:00 2024-06-13T06:42:39.449 INFO:teuthology.orchestra.run.smithi026.stdout:(63/160): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-06-13T06:42:39.452 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:42:39.476 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:42:39.507 INFO:teuthology.orchestra.run.smithi026.stdout:(64/160): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 35 kB 00:00 2024-06-13T06:42:39.529 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:42:39.541 INFO:teuthology.orchestra.run.smithi026.stdout:(65/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-06-13T06:42:39.551 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:42:39.573 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:42:39.761 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:42:39.855 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:42:40.056 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : xfsprogs-6.4.0-1.el9.x86_64 18/172 2024-06-13T06:42:40.069 INFO:teuthology.orchestra.run.smithi135.stdout:(10/160): ceph-common-19.0.0-2946.g3d89accf.el9 2.9 MB/s | 21 MB 00:07 2024-06-13T06:42:40.083 INFO:teuthology.orchestra.run.smithi026.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-06-13T06:42:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:42:40.228 INFO:teuthology.orchestra.run.smithi135.stdout:(11/160): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.4 MB/s | 221 kB 00:00 2024-06-13T06:42:40.328 INFO:teuthology.orchestra.run.smithi135.stdout:(12/160): cephfs-shell-19.0.0-2946.g3d89accf.el 270 kB/s | 27 kB 00:00 2024-06-13T06:42:40.420 INFO:teuthology.orchestra.run.smithi135.stdout:(13/160): libcephfs-devel-19.0.0-2946.g3d89accf 346 kB/s | 32 kB 00:00 2024-06-13T06:42:40.522 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:42:40.643 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:42:40.662 INFO:teuthology.orchestra.run.smithi135.stdout:(14/160): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.0 MB/s | 747 kB 00:00 2024-06-13T06:42:40.729 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:42:40.762 INFO:teuthology.orchestra.run.smithi135.stdout:(15/160): libcephsqlite-19.0.0-2946.g3d89accf.e 1.6 MB/s | 163 kB 00:00 2024-06-13T06:42:40.855 INFO:teuthology.orchestra.run.smithi135.stdout:(16/160): librados-devel-19.0.0-2946.g3d89accf. 1.3 MB/s | 127 kB 00:00 2024-06-13T06:42:40.941 INFO:teuthology.orchestra.run.smithi026.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-06-13T06:42:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:42:41.030 INFO:teuthology.orchestra.run.smithi135.stdout:(17/160): libradosstriper1-19.0.0-2946.g3d89acc 2.8 MB/s | 501 kB 00:00 2024-06-13T06:42:41.059 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-06-13T06:42:41.154 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-06-13T06:42:41.184 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-06-13T06:42:41.219 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-06-13T06:42:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-06-13T06:42:41.300 INFO:teuthology.orchestra.run.smithi026.stdout:(68/160): openblas-openmp-0.3.21-2.el9.x86_64.r 2.6 MB/s | 4.7 MB 00:01 2024-06-13T06:42:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-06-13T06:42:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-06-13T06:42:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-06-13T06:42:41.422 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-06-13T06:42:41.425 INFO:teuthology.orchestra.run.smithi026.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-06-13T06:42:41.455 INFO:teuthology.orchestra.run.smithi135.stdout:(18/160): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.9 MB/s | 17 MB 00:05 2024-06-13T06:42:41.491 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-06-13T06:42:41.530 INFO:teuthology.orchestra.run.smithi135.stdout:(19/160): python3-ceph-argparse-19.0.0-2946.g3d 604 kB/s | 45 kB 00:00 2024-06-13T06:42:41.544 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/172 2024-06-13T06:42:41.550 INFO:teuthology.orchestra.run.smithi026.stdout:(70/160): openssl-devel-3.2.2-1.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:02 2024-06-13T06:42:41.583 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-06-13T06:42:41.592 INFO:teuthology.orchestra.run.smithi026.stdout:(71/160): postgresql-contrib-13.14-1.el9.x86_64 1.4 MB/s | 911 kB 00:00 2024-06-13T06:42:41.618 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-06-13T06:42:41.623 INFO:teuthology.orchestra.run.smithi135.stdout:(20/160): python3-ceph-common-19.0.0-2946.g3d89 1.4 MB/s | 135 kB 00:00 2024-06-13T06:42:41.684 INFO:teuthology.orchestra.run.smithi026.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 963 kB/s | 87 kB 00:00 2024-06-13T06:42:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-06-13T06:42:41.715 INFO:teuthology.orchestra.run.smithi135.stdout:(21/160): python3-cephfs-19.0.0-2946.g3d89accf. 1.7 MB/s | 162 kB 00:00 2024-06-13T06:42:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 35/172 2024-06-13T06:42:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 36/172 2024-06-13T06:42:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 37/172 2024-06-13T06:42:41.872 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-06-13T06:42:41.873 INFO:teuthology.orchestra.run.smithi135.stdout:(22/160): python3-rados-19.0.0-2946.g3d89accf.e 2.0 MB/s | 321 kB 00:00 2024-06-13T06:42:41.916 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-06-13T06:42:41.990 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-06-13T06:42:42.024 INFO:teuthology.orchestra.run.smithi135.stdout:(23/160): python3-rbd-19.0.0-2946.g3d89accf.el9 1.9 MB/s | 300 kB 00:00 2024-06-13T06:42:42.122 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-06-13T06:42:42.124 INFO:teuthology.orchestra.run.smithi135.stdout:(24/160): python3-rgw-19.0.0-2946.g3d89accf.el9 994 kB/s | 99 kB 00:00 2024-06-13T06:42:42.154 INFO:teuthology.orchestra.run.smithi026.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 1.7 MB/s | 1.0 MB 00:00 2024-06-13T06:42:42.175 INFO:teuthology.orchestra.run.smithi135.stdout:(25/160): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-13T06:42:42.176 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-06-13T06:42:42.244 INFO:teuthology.orchestra.run.smithi026.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 692 kB/s | 61 kB 00:00 2024-06-13T06:42:42.402 INFO:teuthology.orchestra.run.smithi026.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-06-13T06:42:42.475 INFO:teuthology.orchestra.run.smithi135.stdout:(26/160): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.7 MB/s | 5.3 MB 00:01 2024-06-13T06:42:42.550 INFO:teuthology.orchestra.run.smithi135.stdout:(27/160): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-06-13T06:42:42.552 INFO:teuthology.orchestra.run.smithi026.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-06-13T06:42:42.611 INFO:teuthology.orchestra.run.smithi026.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-06-13T06:42:42.659 INFO:teuthology.orchestra.run.smithi135.stdout:(28/160): rbd-mirror-19.0.0-2946.g3d89accf.el9. 6.4 MB/s | 3.1 MB 00:00 2024-06-13T06:42:42.684 INFO:teuthology.orchestra.run.smithi135.stdout:(29/160): ceph-grafana-dashboards-19.0.0-2946.g 237 kB/s | 32 kB 00:00 2024-06-13T06:42:42.686 INFO:teuthology.orchestra.run.smithi026.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 258 kB/s | 19 kB 00:00 2024-06-13T06:42:42.696 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-06-13T06:42:42.710 INFO:teuthology.orchestra.run.smithi135.stdout:(30/160): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.8 MB/s | 142 kB 00:00 2024-06-13T06:42:42.772 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-06-13T06:42:42.801 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-06-13T06:42:42.812 INFO:teuthology.orchestra.run.smithi026.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-06-13T06:42:42.837 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-06-13T06:42:42.870 INFO:teuthology.orchestra.run.smithi026.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 605 kB/s | 35 kB 00:00 2024-06-13T06:42:42.878 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-06-13T06:42:42.916 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 48/172 2024-06-13T06:42:42.957 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-06-13T06:42:43.109 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-06-13T06:42:43.160 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.2.2-1.el9.x86_64 51/172 2024-06-13T06:42:43.244 INFO:teuthology.orchestra.run.smithi135.stdout:(31/160): ceph-mgr-dashboard-19.0.0-2946.g3d89a 6.3 MB/s | 3.5 MB 00:00 2024-06-13T06:42:43.311 INFO:teuthology.orchestra.run.smithi135.stdout:(32/160): ceph-mgr-modules-core-19.0.0-2946.g3d 3.7 MB/s | 249 kB 00:00 2024-06-13T06:42:43.324 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-06-13T06:42:43.378 INFO:teuthology.orchestra.run.smithi135.stdout:(33/160): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 739 kB/s | 49 kB 00:00 2024-06-13T06:42:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 53/172 2024-06-13T06:42:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 54/172 2024-06-13T06:42:43.445 INFO:teuthology.orchestra.run.smithi135.stdout:(34/160): ceph-prometheus-alerts-19.0.0-2946.g3 247 kB/s | 16 kB 00:00 2024-06-13T06:42:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 55/172 2024-06-13T06:42:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 56/172 2024-06-13T06:42:43.520 INFO:teuthology.orchestra.run.smithi135.stdout:(35/160): ceph-volume-19.0.0-2946.g3d89accf.el9 3.4 MB/s | 264 kB 00:00 2024-06-13T06:42:43.555 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 57/172 2024-06-13T06:42:43.598 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 58/172 2024-06-13T06:42:43.640 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/172 2024-06-13T06:42:43.679 INFO:teuthology.orchestra.run.smithi135.stdout:(36/160): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-06-13T06:42:43.721 INFO:teuthology.orchestra.run.smithi135.stdout:(37/160): cephadm-19.0.0-2946.g3d89accf.el9.noa 3.7 MB/s | 763 kB 00:00 2024-06-13T06:42:43.755 INFO:teuthology.orchestra.run.smithi135.stdout:(38/160): cephfs-top-19.0.0-2946.g3d89accf.el9. 243 kB/s | 18 kB 00:00 2024-06-13T06:42:43.776 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/172 2024-06-13T06:42:43.813 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:43.819 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:43.825 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/172 2024-06-13T06:42:43.866 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/172 2024-06-13T06:42:43.960 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/172 2024-06-13T06:42:43.991 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 64/172 2024-06-13T06:42:44.031 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/172 2024-06-13T06:42:44.057 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/172 2024-06-13T06:42:44.081 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 67/172 2024-06-13T06:42:44.157 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 68/172 2024-06-13T06:42:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 69/172 2024-06-13T06:42:44.322 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 70/172 2024-06-13T06:42:44.350 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 71/172 2024-06-13T06:42:44.384 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 72/172 2024-06-13T06:42:44.415 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 73/172 2024-06-13T06:42:44.421 INFO:teuthology.orchestra.run.smithi026.stdout:(81/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.3 MB/s | 5.1 MB 00:01 2024-06-13T06:42:44.446 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 74/172 2024-06-13T06:42:44.476 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 75/172 2024-06-13T06:42:44.487 INFO:teuthology.orchestra.run.smithi026.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.9 MB/s | 250 kB 00:00 2024-06-13T06:42:44.511 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 76/172 2024-06-13T06:42:44.544 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 77/172 2024-06-13T06:42:44.554 INFO:teuthology.orchestra.run.smithi026.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-06-13T06:42:44.585 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/172 2024-06-13T06:42:44.613 INFO:teuthology.orchestra.run.smithi026.stdout:(84/160): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-06-13T06:42:44.620 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 79/172 2024-06-13T06:42:44.701 INFO:teuthology.orchestra.run.smithi026.stdout:(85/160): python3-prettytable-0.7.2-27.el9.noar 484 kB/s | 42 kB 00:00 2024-06-13T06:42:44.706 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 80/172 2024-06-13T06:42:44.750 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 81/172 2024-06-13T06:42:44.781 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 82/172 2024-06-13T06:42:44.820 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 83/172 2024-06-13T06:42:44.822 INFO:teuthology.orchestra.run.smithi026.stdout:(86/160): postgresql-server-13.14-1.el9.x86_64. 1.7 MB/s | 5.8 MB 00:03 2024-06-13T06:42:44.856 INFO:teuthology.orchestra.run.smithi026.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-06-13T06:42:44.918 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 84/172 2024-06-13T06:42:44.923 INFO:teuthology.orchestra.run.smithi026.stdout:(88/160): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-06-13T06:42:44.948 INFO:teuthology.orchestra.run.smithi026.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-06-13T06:42:44.973 INFO:teuthology.orchestra.run.smithi026.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-13T06:42:44.974 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:42:44.998 INFO:teuthology.orchestra.run.smithi026.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-06-13T06:42:45.065 INFO:teuthology.orchestra.run.smithi026.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 632 kB/s | 42 kB 00:00 2024-06-13T06:42:45.213 INFO:teuthology.orchestra.run.smithi026.stdout:(93/160): python3-babel-2.9.1-2.el9.noarch.rpm 1.7 MB/s | 6.0 MB 00:03 2024-06-13T06:42:45.233 INFO:teuthology.orchestra.run.smithi026.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 266 kB/s | 44 kB 00:00 2024-06-13T06:42:45.283 INFO:teuthology.orchestra.run.smithi026.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 729 kB/s | 36 kB 00:00 2024-06-13T06:42:45.342 INFO:teuthology.orchestra.run.smithi026.stdout:(96/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 825 kB/s | 47 kB 00:00 2024-06-13T06:42:45.392 INFO:teuthology.orchestra.run.smithi026.stdout:(97/160): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-06-13T06:42:45.438 INFO:teuthology.orchestra.run.smithi135.stdout:(39/160): ceph-test-19.0.0-2946.g3d89accf.el9.x 7.4 MB/s | 48 MB 00:06 2024-06-13T06:42:45.450 INFO:teuthology.orchestra.run.smithi026.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-06-13T06:42:45.455 INFO:teuthology.orchestra.run.smithi135.stdout:(40/160): fuse-2.9.9-15.el9.x86_64.rpm 46 kB/s | 80 kB 00:01 2024-06-13T06:42:45.456 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:45.456 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] python3-cffi-1.14.5-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.14.5-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:45.542 INFO:teuthology.orchestra.run.smithi026.stdout:(99/160): xfsprogs-devel-6.4.0-1.el9.x86_64.rpm 1.0 MB/s | 95 kB 00:00 2024-06-13T06:42:45.568 INFO:teuthology.orchestra.run.smithi026.stdout:(100/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-06-13T06:42:45.695 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-06-13T06:42:45.757 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-06-13T06:42:45.789 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-06-13T06:42:45.801 INFO:teuthology.orchestra.run.smithi026.stdout:(101/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-06-13T06:42:45.827 INFO:teuthology.orchestra.run.smithi026.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 173 kB/s | 45 kB 00:00 2024-06-13T06:42:45.856 INFO:teuthology.orchestra.run.smithi135.stdout:(41/160): libgfortran-11.4.1-3.el9.x86_64.rpm 382 kB/s | 803 kB 00:02 2024-06-13T06:42:45.887 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-06-13T06:42:45.927 INFO:teuthology.orchestra.run.smithi026.stdout:(103/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 166 kB/s | 64 kB 00:00 2024-06-13T06:42:45.935 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-06-13T06:42:45.979 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-06-13T06:42:45.995 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:46.048 INFO:teuthology.orchestra.run.smithi026.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:46.059 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-06-13T06:42:46.094 INFO:teuthology.orchestra.run.smithi026.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-06-13T06:42:46.107 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-06-13T06:42:46.136 INFO:teuthology.orchestra.run.smithi026.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-06-13T06:42:46.161 INFO:teuthology.orchestra.run.smithi026.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 803 kB/s | 19 kB 00:00 2024-06-13T06:42:46.187 INFO:teuthology.orchestra.run.smithi026.stdout:(107/160): lua-devel-5.4.4-4.el9.x86_64.rpm 58 kB/s | 22 kB 00:00 2024-06-13T06:42:46.212 INFO:teuthology.orchestra.run.smithi026.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-06-13T06:42:46.246 INFO:teuthology.orchestra.run.smithi026.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-06-13T06:42:46.277 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-06-13T06:42:46.297 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-06-13T06:42:46.304 INFO:teuthology.orchestra.run.smithi026.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-06-13T06:42:46.338 INFO:teuthology.orchestra.run.smithi026.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.6 MB/s | 151 kB 00:00 2024-06-13T06:42:46.366 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-06-13T06:42:46.413 INFO:teuthology.orchestra.run.smithi026.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-06-13T06:42:46.480 INFO:teuthology.orchestra.run.smithi026.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-06-13T06:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout:(114/160): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-06-13T06:42:46.547 INFO:teuthology.orchestra.run.smithi026.stdout:(115/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 657 kB/s | 43 kB 00:00 2024-06-13T06:42:46.564 INFO:teuthology.orchestra.run.smithi135.stdout:(42/160): libquadmath-11.4.1-3.el9.x86_64.rpm 174 kB/s | 195 kB 00:01 2024-06-13T06:42:46.573 INFO:teuthology.orchestra.run.smithi026.stdout:(116/160): python3-cachetools-4.2.4-1.el9.noarc 652 kB/s | 32 kB 00:00 2024-06-13T06:42:46.589 INFO:teuthology.orchestra.run.smithi135.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 223 kB/s | 253 kB 00:01 2024-06-13T06:42:46.598 INFO:teuthology.orchestra.run.smithi026.stdout:(117/160): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2024-06-13T06:42:46.640 INFO:teuthology.orchestra.run.smithi026.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-06-13T06:42:46.690 INFO:teuthology.orchestra.run.smithi026.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-06-13T06:42:46.724 INFO:teuthology.orchestra.run.smithi026.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-06-13T06:42:46.758 INFO:teuthology.orchestra.run.smithi026.stdout:(121/160): python3-google-auth-2.29.0-2.el9.noa 3.0 MB/s | 205 kB 00:00 2024-06-13T06:42:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 2.5 MB/s | 144 kB 00:00 2024-06-13T06:42:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 664 kB/s | 11 kB 00:00 2024-06-13T06:42:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 734 kB/s | 18 kB 00:00 2024-06-13T06:42:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-06-13T06:42:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-06-13T06:42:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 814 kB/s | 20 kB 00:00 2024-06-13T06:42:47.003 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-06-13T06:42:47.018 INFO:teuthology.orchestra.run.smithi026.stdout:(128/160): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-06-13T06:42:47.035 INFO:teuthology.orchestra.run.smithi026.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 2.9 MB/s | 46 kB 00:00 2024-06-13T06:42:47.069 INFO:teuthology.orchestra.run.smithi026.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.4 MB/s | 79 kB 00:00 2024-06-13T06:42:47.094 INFO:teuthology.orchestra.run.smithi026.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-13T06:42:47.119 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-06-13T06:42:47.144 INFO:teuthology.orchestra.run.smithi026.stdout:(132/160): python3-pecan-1.4.2-3.el9.noarch.rpm 5.4 MB/s | 272 kB 00:00 2024-06-13T06:42:47.162 INFO:teuthology.orchestra.run.smithi026.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 1.0 MB/s | 16 kB 00:00 2024-06-13T06:42:47.198 INFO:teuthology.orchestra.run.smithi135.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 935 kB/s | 1.2 MB 00:01 2024-06-13T06:42:47.262 INFO:teuthology.orchestra.run.smithi026.stdout:(134/160): libarrow-9.0.0-11.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-06-13T06:42:47.474 INFO:teuthology.orchestra.run.smithi135.stdout:(45/160): python3-ply-3.11-14.el9.noarch.rpm 117 kB/s | 106 kB 00:00 2024-06-13T06:42:47.479 INFO:teuthology.orchestra.run.smithi026.stdout:(135/160): python3-grpcio-1.46.7-10.el9.x86_64. 2.7 MB/s | 2.0 MB 00:00 2024-06-13T06:42:47.513 INFO:teuthology.orchestra.run.smithi026.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 113 kB/s | 28 kB 00:00 2024-06-13T06:42:47.524 INFO:teuthology.orchestra.run.smithi135.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 145 kB/s | 135 kB 00:00 2024-06-13T06:42:47.525 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-13T06:42:47.538 INFO:teuthology.orchestra.run.smithi026.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 537 kB/s | 31 kB 00:00 2024-06-13T06:42:47.563 INFO:teuthology.orchestra.run.smithi026.stdout:(138/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 224 kB/s | 90 kB 00:00 2024-06-13T06:42:47.588 INFO:teuthology.orchestra.run.smithi026.stdout:(139/160): python3-tempora-5.0.0-2.el9.noarch.r 1.5 MB/s | 36 kB 00:00 2024-06-13T06:42:47.614 INFO:teuthology.orchestra.run.smithi026.stdout:(140/160): python3-rsa-4.9-2.el9.noarch.rpm 792 kB/s | 59 kB 00:00 2024-06-13T06:42:47.647 INFO:teuthology.orchestra.run.smithi026.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 884 kB/s | 51 kB 00:00 2024-06-13T06:42:47.681 INFO:teuthology.orchestra.run.smithi026.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-06-13T06:42:47.691 INFO:teuthology.orchestra.run.smithi135.stdout:(47/160): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-06-13T06:42:47.692 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:47.715 INFO:teuthology.orchestra.run.smithi026.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-06-13T06:42:47.740 INFO:teuthology.orchestra.run.smithi026.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 830 kB/s | 20 kB 00:00 2024-06-13T06:42:47.794 INFO:teuthology.orchestra.run.smithi026.stdout:(145/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-06-13T06:42:47.832 INFO:teuthology.orchestra.run.smithi026.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-06-13T06:42:47.866 INFO:teuthology.orchestra.run.smithi026.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-06-13T06:42:47.933 INFO:teuthology.orchestra.run.smithi135.stdout:(48/160): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-06-13T06:42:47.938 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:47.983 INFO:teuthology.orchestra.run.smithi135.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-06-13T06:42:48.017 INFO:teuthology.orchestra.run.smithi135.stdout:(50/160): elfutils-libelf-devel-0.191-4.el9.x86 649 kB/s | 22 kB 00:00 2024-06-13T06:42:48.024 INFO:teuthology.orchestra.run.smithi026.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 6.9 MB/s | 1.6 MB 00:00 2024-06-13T06:42:48.118 INFO:teuthology.orchestra.run.smithi135.stdout:(51/160): python3-requests-2.25.1-8.el9.noarch. 136 kB/s | 125 kB 00:00 2024-06-13T06:42:48.151 INFO:teuthology.orchestra.run.smithi135.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-06-13T06:42:48.208 INFO:teuthology.orchestra.run.smithi026.stdout:(149/160): NetworkManager-1.47.91-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-06-13T06:42:48.210 INFO:teuthology.orchestra.run.smithi135.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-06-13T06:42:48.252 INFO:teuthology.orchestra.run.smithi135.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-06-13T06:42:48.294 INFO:teuthology.orchestra.run.smithi135.stdout:(55/160): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-06-13T06:42:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-06-13T06:42:48.333 INFO:teuthology.orchestra.run.smithi026.stdout:(150/160): NetworkManager-libnm-1.47.91-1.el9.x 15 MB/s | 1.8 MB 00:00 2024-06-13T06:42:48.367 INFO:teuthology.orchestra.run.smithi026.stdout:(151/160): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-13T06:42:48.377 INFO:teuthology.orchestra.run.smithi135.stdout:(56/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-06-13T06:42:48.417 INFO:teuthology.orchestra.run.smithi026.stdout:(152/160): NetworkManager-tui-1.47.91-1.el9.x86 5.0 MB/s | 248 kB 00:00 2024-06-13T06:42:48.420 INFO:teuthology.orchestra.run.smithi135.stdout:(57/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-13T06:42:48.451 INFO:teuthology.orchestra.run.smithi026.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-06-13T06:42:48.495 INFO:teuthology.orchestra.run.smithi135.stdout:(58/160): python3-urllib3-1.26.5-5.el9.noarch.r 216 kB/s | 215 kB 00:00 2024-06-13T06:42:48.520 INFO:teuthology.orchestra.run.smithi135.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-06-13T06:42:48.543 INFO:teuthology.orchestra.run.smithi026.stdout:(154/160): librados2-19.0.0-2946.g3d89accf.el9. 4.8 MB/s | 3.4 MB 00:00 2024-06-13T06:42:48.562 INFO:teuthology.orchestra.run.smithi135.stdout:(60/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 612 kB/s | 25 kB 00:00 2024-06-13T06:42:48.567 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-06-13T06:42:48.576 INFO:teuthology.orchestra.run.smithi026.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.6 MB/s | 207 kB 00:00 2024-06-13T06:42:48.595 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-06-13T06:42:48.604 INFO:teuthology.orchestra.run.smithi135.stdout:(61/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-06-13T06:42:48.628 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-06-13T06:42:48.668 INFO:teuthology.orchestra.run.smithi026.stdout:(156/160): librbd1-19.0.0-2946.g3d89accf.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-06-13T06:42:48.702 INFO:teuthology.orchestra.run.smithi026.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.6 MB/s | 260 kB 00:00 2024-06-13T06:42:48.722 INFO:teuthology.orchestra.run.smithi135.stdout:(62/160): libtool-2.4.6-45.el9.x86_64.rpm 2.5 MB/s | 582 kB 00:00 2024-06-13T06:42:48.747 INFO:teuthology.orchestra.run.smithi135.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-06-13T06:42:48.769 INFO:teuthology.orchestra.run.smithi026.stdout:(158/160): openssl-3.2.2-1.el9.x86_64.rpm 7.1 MB/s | 1.4 MB 00:00 2024-06-13T06:42:48.772 INFO:teuthology.orchestra.run.smithi135.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-13T06:42:48.798 INFO:teuthology.orchestra.run.smithi135.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-06-13T06:42:48.836 INFO:teuthology.orchestra.run.smithi026.stdout:(159/160): xfsprogs-6.4.0-1.el9.x86_64.rpm 8.1 MB/s | 1.1 MB 00:00 2024-06-13T06:42:48.940 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-06-13T06:42:48.953 INFO:teuthology.orchestra.run.smithi026.stdout:(160/160): openssl-libs-3.2.2-1.el9.x86_64.rpm 8.4 MB/s | 2.4 MB 00:00 2024-06-13T06:42:48.966 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:42:48.966 INFO:teuthology.orchestra.run.smithi026.stdout:Total 12 MB/s | 226 MB 00:19 2024-06-13T06:42:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:42:49.081 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:42:49.112 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 105/172 2024-06-13T06:42:49.139 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-06-13T06:42:49.148 INFO:teuthology.orchestra.run.smithi135.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:00 2024-06-13T06:42:49.615 INFO:teuthology.orchestra.run.smithi135.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-06-13T06:42:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-06-13T06:42:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:42:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:42:49.745 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 109/172 2024-06-13T06:42:49.891 INFO:teuthology.orchestra.run.smithi135.stdout:(68/160): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 911 kB 00:00 2024-06-13T06:42:49.974 INFO:teuthology.orchestra.run.smithi135.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-06-13T06:42:50.066 INFO:teuthology.orchestra.run.smithi135.stdout:(70/160): openblas-openmp-0.3.21-2.el9.x86_64.r 3.7 MB/s | 4.7 MB 00:01 2024-06-13T06:42:50.217 INFO:teuthology.orchestra.run.smithi135.stdout:(71/160): openssl-devel-3.2.2-1.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-06-13T06:42:50.275 INFO:teuthology.orchestra.run.smithi135.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-06-13T06:42:50.442 INFO:teuthology.orchestra.run.smithi135.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-06-13T06:42:50.484 INFO:teuthology.orchestra.run.smithi135.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-13T06:42:50.576 INFO:teuthology.orchestra.run.smithi135.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-06-13T06:42:50.668 INFO:teuthology.orchestra.run.smithi135.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-06-13T06:42:50.702 INFO:teuthology.orchestra.run.smithi135.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 768 kB/s | 26 kB 00:00 2024-06-13T06:42:50.744 INFO:teuthology.orchestra.run.smithi135.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-06-13T06:42:50.820 INFO:teuthology.orchestra.run.smithi135.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-06-13T06:42:50.862 INFO:teuthology.orchestra.run.smithi135.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 834 kB/s | 35 kB 00:00 2024-06-13T06:42:51.604 INFO:teuthology.orchestra.run.smithi135.stdout:(81/160): postgresql-server-13.14-1.el9.x86_64. 3.5 MB/s | 5.8 MB 00:01 2024-06-13T06:42:51.704 INFO:teuthology.orchestra.run.smithi135.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-06-13T06:42:51.796 INFO:teuthology.orchestra.run.smithi135.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-06-13T06:42:51.846 INFO:teuthology.orchestra.run.smithi135.stdout:(84/160): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-13T06:42:51.888 INFO:teuthology.orchestra.run.smithi135.stdout:(85/160): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-06-13T06:42:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:42:51.997 INFO:teuthology.orchestra.run.smithi135.stdout:(86/160): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-06-13T06:42:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:42:52.073 INFO:teuthology.orchestra.run.smithi135.stdout:(87/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-06-13T06:42:52.198 INFO:teuthology.orchestra.run.smithi135.stdout:(88/160): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-06-13T06:42:52.207 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-06-13T06:42:52.240 INFO:teuthology.orchestra.run.smithi135.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-06-13T06:42:52.266 INFO:teuthology.orchestra.run.smithi135.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 759 kB/s | 51 kB 00:00 2024-06-13T06:42:52.291 INFO:teuthology.orchestra.run.smithi135.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-13T06:42:52.333 INFO:teuthology.orchestra.run.smithi135.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 994 kB/s | 42 kB 00:00 2024-06-13T06:42:52.375 INFO:teuthology.orchestra.run.smithi135.stdout:(93/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-13T06:42:52.467 INFO:teuthology.orchestra.run.smithi135.stdout:(94/160): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-06-13T06:42:52.518 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-06-13T06:42:52.518 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-06-13T06:42:52.543 INFO:teuthology.orchestra.run.smithi135.stdout:(95/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.0 MB/s | 5.1 MB 00:01 2024-06-13T06:42:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:42:52.568 INFO:teuthology.orchestra.run.smithi135.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 356 kB/s | 36 kB 00:00 2024-06-13T06:42:52.577 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:42:52.593 INFO:teuthology.orchestra.run.smithi135.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 939 kB/s | 47 kB 00:00 2024-06-13T06:42:52.635 INFO:teuthology.orchestra.run.smithi135.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-06-13T06:42:52.671 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:42:52.671 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-13T06:42:52.671 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-13T06:42:52.671 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:42:52.694 INFO:teuthology.orchestra.run.smithi135.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-06-13T06:42:52.703 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:42:52.719 INFO:teuthology.orchestra.run.smithi135.stdout:(100/160): xfsprogs-devel-6.4.0-1.el9.x86_64.rp 1.1 MB/s | 95 kB 00:00 2024-06-13T06:42:52.920 INFO:teuthology.orchestra.run.smithi135.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 282 kB/s | 64 kB 00:00 2024-06-13T06:42:52.945 INFO:teuthology.orchestra.run.smithi135.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 198 kB/s | 45 kB 00:00 2024-06-13T06:42:52.946 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:52.948 INFO:teuthology.orchestra.run.smithi135.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:42:53.054 INFO:teuthology.orchestra.run.smithi135.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-06-13T06:42:53.088 INFO:teuthology.orchestra.run.smithi135.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 16 MB/s | 548 kB 00:00 2024-06-13T06:42:53.121 INFO:teuthology.orchestra.run.smithi135.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-06-13T06:42:53.147 INFO:teuthology.orchestra.run.smithi135.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-06-13T06:42:53.256 INFO:teuthology.orchestra.run.smithi135.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-06-13T06:42:53.281 INFO:teuthology.orchestra.run.smithi135.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-13T06:42:53.306 INFO:teuthology.orchestra.run.smithi135.stdout:(109/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-06-13T06:42:53.332 INFO:teuthology.orchestra.run.smithi135.stdout:(110/160): liboath-2.6.7-2.el9.x86_64.rpm 957 kB/s | 49 kB 00:00 2024-06-13T06:42:53.382 INFO:teuthology.orchestra.run.smithi135.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-06-13T06:42:53.416 INFO:teuthology.orchestra.run.smithi135.stdout:(112/160): libunwind-1.6.2-1.el9.x86_64.rpm 617 kB/s | 67 kB 00:00 2024-06-13T06:42:53.450 INFO:teuthology.orchestra.run.smithi135.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-06-13T06:42:53.475 INFO:teuthology.orchestra.run.smithi135.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-13T06:42:53.500 INFO:teuthology.orchestra.run.smithi135.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-13T06:42:53.534 INFO:teuthology.orchestra.run.smithi135.stdout:(116/160): python3-asyncssh-2.13.2-5.el9.noarch 4.5 MB/s | 548 kB 00:00 2024-06-13T06:42:53.559 INFO:teuthology.orchestra.run.smithi135.stdout:(117/160): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-06-13T06:42:53.585 INFO:teuthology.orchestra.run.smithi135.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-13T06:42:53.610 INFO:teuthology.orchestra.run.smithi135.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-06-13T06:42:53.635 INFO:teuthology.orchestra.run.smithi135.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-06-13T06:42:53.661 INFO:teuthology.orchestra.run.smithi135.stdout:(121/160): python3-google-auth-2.29.0-2.el9.noa 4.0 MB/s | 205 kB 00:00 2024-06-13T06:42:53.678 INFO:teuthology.orchestra.run.smithi135.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 8.4 MB/s | 144 kB 00:00 2024-06-13T06:42:53.703 INFO:teuthology.orchestra.run.smithi135.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 423 kB/s | 11 kB 00:00 2024-06-13T06:42:53.729 INFO:teuthology.orchestra.run.smithi135.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-06-13T06:42:53.754 INFO:teuthology.orchestra.run.smithi135.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 920 kB/s | 23 kB 00:00 2024-06-13T06:42:53.779 INFO:teuthology.orchestra.run.smithi135.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 772 kB/s | 19 kB 00:00 2024-06-13T06:42:53.805 INFO:teuthology.orchestra.run.smithi135.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-06-13T06:42:53.864 INFO:teuthology.orchestra.run.smithi135.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 8.9 MB/s | 2.0 MB 00:00 2024-06-13T06:42:53.897 INFO:teuthology.orchestra.run.smithi135.stdout:(129/160): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-06-13T06:42:53.914 INFO:teuthology.orchestra.run.smithi135.stdout:(130/160): python3-logutils-0.3.5-21.el9.noarch 921 kB/s | 46 kB 00:00 2024-06-13T06:42:53.940 INFO:teuthology.orchestra.run.smithi135.stdout:(131/160): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-06-13T06:42:53.965 INFO:teuthology.orchestra.run.smithi135.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-06-13T06:42:53.999 INFO:teuthology.orchestra.run.smithi135.stdout:(133/160): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-06-13T06:42:54.024 INFO:teuthology.orchestra.run.smithi135.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-06-13T06:42:54.050 INFO:teuthology.orchestra.run.smithi135.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-06-13T06:42:54.075 INFO:teuthology.orchestra.run.smithi135.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 558 kB/s | 28 kB 00:00 2024-06-13T06:42:54.101 INFO:teuthology.orchestra.run.smithi135.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-06-13T06:42:54.126 INFO:teuthology.orchestra.run.smithi135.stdout:(138/160): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-06-13T06:42:54.151 INFO:teuthology.orchestra.run.smithi135.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-06-13T06:42:54.177 INFO:teuthology.orchestra.run.smithi135.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 707 kB/s | 36 kB 00:00 2024-06-13T06:42:54.202 INFO:teuthology.orchestra.run.smithi135.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-13T06:42:54.227 INFO:teuthology.orchestra.run.smithi135.stdout:(142/160): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-06-13T06:42:54.267 INFO:teuthology.orchestra.run.smithi135.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.4 MB/s | 90 kB 00:00 2024-06-13T06:42:54.337 INFO:teuthology.orchestra.run.smithi135.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 286 kB/s | 20 kB 00:00 2024-06-13T06:42:54.362 INFO:teuthology.orchestra.run.smithi135.stdout:(145/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-06-13T06:42:54.388 INFO:teuthology.orchestra.run.smithi135.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-13T06:42:54.446 INFO:teuthology.orchestra.run.smithi135.stdout:(147/160): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-06-13T06:42:54.747 INFO:teuthology.orchestra.run.smithi135.stdout:(148/160): librados2-19.0.0-2946.g3d89accf.el9. 9.4 MB/s | 3.4 MB 00:00 2024-06-13T06:42:54.889 INFO:teuthology.orchestra.run.smithi135.stdout:(149/160): librbd1-19.0.0-2946.g3d89accf.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-06-13T06:42:55.187 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-06-13T06:42:55.187 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-06-13T06:42:55.998 INFO:teuthology.orchestra.run.smithi135.stdout:(150/160): NetworkManager-1.47.91-1.el9.x86_64. 1.8 MB/s | 2.3 MB 00:01 2024-06-13T06:42:56.115 INFO:teuthology.orchestra.run.smithi135.stdout:(151/160): NetworkManager-libnm-1.47.91-1.el9.x 1.5 MB/s | 1.8 MB 00:01 2024-06-13T06:42:56.732 INFO:teuthology.orchestra.run.smithi135.stdout:(152/160): NetworkManager-team-1.47.91-1.el9.x8 56 kB/s | 41 kB 00:00 2024-06-13T06:42:57.057 INFO:teuthology.orchestra.run.smithi135.stdout:(153/160): NetworkManager-tui-1.47.91-1.el9.x86 263 kB/s | 248 kB 00:00 2024-06-13T06:42:57.458 INFO:teuthology.orchestra.run.smithi135.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 51 kB/s | 37 kB 00:00 2024-06-13T06:42:57.983 INFO:teuthology.orchestra.run.smithi135.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 223 kB/s | 207 kB 00:00 2024-06-13T06:42:58.183 INFO:teuthology.orchestra.run.smithi135.stdout:(156/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.7 MB/s | 16 MB 00:05 2024-06-13T06:42:58.559 INFO:teuthology.orchestra.run.smithi135.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 236 kB/s | 260 kB 00:01 2024-06-13T06:42:59.267 INFO:teuthology.orchestra.run.smithi135.stdout:(158/160): openssl-3.2.2-1.el9.x86_64.rpm 1.1 MB/s | 1.4 MB 00:01 2024-06-13T06:42:59.542 INFO:teuthology.orchestra.run.smithi135.stdout:(159/160): openssl-libs-3.2.2-1.el9.x86_64.rpm 1.8 MB/s | 2.4 MB 00:01 2024-06-13T06:42:59.659 INFO:teuthology.orchestra.run.smithi135.stdout:(160/160): xfsprogs-6.4.0-1.el9.x86_64.rpm 1.0 MB/s | 1.1 MB 00:01 2024-06-13T06:42:59.665 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:42:59.665 INFO:teuthology.orchestra.run.smithi135.stdout:Total 8.2 MB/s | 226 MB 00:27 2024-06-13T06:43:00.838 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-06-13T06:43:01.030 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-06-13T06:43:01.030 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-06-13T06:43:02.421 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-06-13T06:43:02.552 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.2.2-1.el9.x86_64 1/172 2024-06-13T06:43:02.671 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-06-13T06:43:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-06-13T06:43:02.885 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-06-13T06:43:02.886 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-06-13T06:43:02.902 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-06-13T06:43:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-06-13T06:43:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-06-13T06:43:03.591 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-06-13T06:43:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:43:03.869 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:43:03.902 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:43:04.068 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:43:04.110 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:43:04.182 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-06-13T06:43:04.300 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:43:04.300 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-06-13T06:43:04.300 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-06-13T06:43:04.301 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-06-13T06:43:04.301 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-06-13T06:43:04.301 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-06-13T06:43:04.301 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-06-13T06:43:04.301 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-06-13T06:43:04.301 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:04.468 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:43:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:43:04.484 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:04.484 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-13T06:43:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:04.495 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-06-13T06:43:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-06-13T06:43:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:43:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:43:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:43:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:43:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:43:04.986 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:43:05.295 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:43:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:43:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : xfsprogs-6.4.0-1.el9.x86_64 18/172 2024-06-13T06:43:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:43:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:43:05.973 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:05.973 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-13T06:43:05.973 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:43:05.973 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:43:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:06.090 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:43:06.180 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:43:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:43:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:43:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-13T06:43:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:43:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:43:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:43:06.453 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:43:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:43:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:43:06.687 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:06.688 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-13T06:43:06.688 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:43:06.688 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:43:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-06-13T06:43:06.820 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:43:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-06-13T06:43:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-06-13T06:43:07.031 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-06-13T06:43:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-06-13T06:43:07.141 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-06-13T06:43:07.205 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-06-13T06:43:07.219 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-06-13T06:43:07.260 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-06-13T06:43:07.308 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-06-13T06:43:07.313 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-06-13T06:43:07.422 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-06-13T06:43:07.455 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-06-13T06:43:07.473 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-06-13T06:43:07.490 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-06-13T06:43:07.531 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-06-13T06:43:07.537 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-06-13T06:43:07.559 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-06-13T06:43:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-06-13T06:43:07.626 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-06-13T06:43:07.748 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-06-13T06:43:07.826 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/172 2024-06-13T06:43:07.861 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-06-13T06:43:07.941 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-06-13T06:43:07.986 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:43:08.016 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:43:08.047 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 129/172 2024-06-13T06:43:08.060 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-06-13T06:43:08.109 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 130/172 2024-06-13T06:43:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 35/172 2024-06-13T06:43:08.139 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:43:08.184 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 36/172 2024-06-13T06:43:08.222 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 37/172 2024-06-13T06:43:08.337 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-06-13T06:43:08.417 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-06-13T06:43:08.529 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-06-13T06:43:08.708 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-06-13T06:43:08.787 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-06-13T06:43:09.165 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:43:09.207 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:43:09.418 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-06-13T06:43:09.489 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-libs-1:3.2.2-1.el9.x86_64 1/172 2024-06-13T06:43:09.532 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-06-13T06:43:09.536 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-06-13T06:43:09.589 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-06-13T06:43:09.626 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-06-13T06:43:09.637 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-06-13T06:43:09.651 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-06-13T06:43:09.677 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-06-13T06:43:09.694 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-06-13T06:43:09.715 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-06-13T06:43:09.771 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-06-13T06:43:09.818 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 48/172 2024-06-13T06:43:09.894 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-06-13T06:43:10.017 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-06-13T06:43:10.127 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:43:10.131 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-06-13T06:43:10.165 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:43:10.191 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:43:10.223 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.2.2-1.el9.x86_64 51/172 2024-06-13T06:43:10.248 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:43:10.278 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:43:10.292 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:43:10.307 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-06-13T06:43:10.368 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:43:10.399 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:43:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 53/172 2024-06-13T06:43:10.432 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-06-13T06:43:10.462 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 134/172 2024-06-13T06:43:10.472 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 54/172 2024-06-13T06:43:10.502 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:43:10.510 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 55/172 2024-06-13T06:43:10.525 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:43:10.525 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:10.525 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-13T06:43:10.525 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:43:10.525 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:43:10.526 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:10.569 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:43:10.571 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 56/172 2024-06-13T06:43:10.581 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:43:10.607 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-06-13T06:43:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 57/172 2024-06-13T06:43:10.655 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-06-13T06:43:10.675 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:43:10.678 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 137/172 2024-06-13T06:43:10.697 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 58/172 2024-06-13T06:43:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 138/172 2024-06-13T06:43:10.758 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/172 2024-06-13T06:43:10.778 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:43:10.801 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:43:10.863 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:43:10.885 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:43:10.907 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:43:10.969 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/172 2024-06-13T06:43:11.021 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/172 2024-06-13T06:43:11.076 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/172 2024-06-13T06:43:11.094 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:43:11.197 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:43:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/172 2024-06-13T06:43:11.253 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 64/172 2024-06-13T06:43:11.322 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/172 2024-06-13T06:43:11.361 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/172 2024-06-13T06:43:11.363 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : xfsprogs-6.4.0-1.el9.x86_64 18/172 2024-06-13T06:43:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 67/172 2024-06-13T06:43:11.510 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:43:11.570 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:43:11.585 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 68/172 2024-06-13T06:43:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-45.el9.x86_64 139/172 2024-06-13T06:43:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:43:11.667 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:43:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:43:11.670 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:11.671 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-13T06:43:11.671 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:43:11.671 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:43:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:43:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:43:11.736 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:11.736 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-13T06:43:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:11.746 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:43:11.781 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:43:11.797 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 69/172 2024-06-13T06:43:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:43:11.804 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:11.804 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-13T06:43:11.804 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:43:11.805 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:43:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:11.849 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 70/172 2024-06-13T06:43:11.889 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 71/172 2024-06-13T06:43:11.936 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 72/172 2024-06-13T06:43:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:43:11.996 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 73/172 2024-06-13T06:43:12.051 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-06-13T06:43:12.054 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 74/172 2024-06-13T06:43:12.096 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 75/172 2024-06-13T06:43:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:43:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:43:12.130 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:12.130 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-13T06:43:12.130 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:43:12.130 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:43:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:12.146 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-06-13T06:43:12.150 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 76/172 2024-06-13T06:43:12.184 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-06-13T06:43:12.216 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 77/172 2024-06-13T06:43:12.228 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-06-13T06:43:12.292 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-06-13T06:43:12.295 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/172 2024-06-13T06:43:12.327 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-06-13T06:43:12.339 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 79/172 2024-06-13T06:43:12.370 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-06-13T06:43:12.403 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-06-13T06:43:12.431 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-06-13T06:43:12.488 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 80/172 2024-06-13T06:43:12.500 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-06-13T06:43:12.552 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/172 2024-06-13T06:43:12.555 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 81/172 2024-06-13T06:43:12.583 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-06-13T06:43:12.597 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 82/172 2024-06-13T06:43:12.619 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-06-13T06:43:12.669 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 83/172 2024-06-13T06:43:12.694 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-06-13T06:43:12.730 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 35/172 2024-06-13T06:43:12.773 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 36/172 2024-06-13T06:43:12.801 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 37/172 2024-06-13T06:43:12.829 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 84/172 2024-06-13T06:43:12.870 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-06-13T06:43:12.909 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:43:12.925 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-06-13T06:43:12.999 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-06-13T06:43:13.131 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-06-13T06:43:13.185 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-06-13T06:43:13.708 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-06-13T06:43:13.789 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-06-13T06:43:13.818 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-06-13T06:43:13.862 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-06-13T06:43:13.911 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-06-13T06:43:13.918 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-06-13T06:43:13.949 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 48/172 2024-06-13T06:43:13.975 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-06-13T06:43:13.991 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-06-13T06:43:14.009 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-06-13T06:43:14.137 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-06-13T06:43:14.140 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-06-13T06:43:14.203 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-1:3.2.2-1.el9.x86_64 51/172 2024-06-13T06:43:14.228 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-06-13T06:43:14.257 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-06-13T06:43:14.284 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-06-13T06:43:14.329 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 53/172 2024-06-13T06:43:14.361 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 54/172 2024-06-13T06:43:14.398 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 55/172 2024-06-13T06:43:14.398 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-06-13T06:43:14.444 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 56/172 2024-06-13T06:43:14.455 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-06-13T06:43:14.480 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 57/172 2024-06-13T06:43:14.523 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 58/172 2024-06-13T06:43:14.565 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/172 2024-06-13T06:43:14.694 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-06-13T06:43:14.700 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/172 2024-06-13T06:43:14.734 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-06-13T06:43:14.759 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/172 2024-06-13T06:43:14.800 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/172 2024-06-13T06:43:14.844 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-06-13T06:43:14.895 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/172 2024-06-13T06:43:14.933 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 64/172 2024-06-13T06:43:14.973 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/172 2024-06-13T06:43:14.999 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/172 2024-06-13T06:43:15.023 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 67/172 2024-06-13T06:43:15.097 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 68/172 2024-06-13T06:43:15.225 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 69/172 2024-06-13T06:43:15.256 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 70/172 2024-06-13T06:43:15.284 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 71/172 2024-06-13T06:43:15.318 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 72/172 2024-06-13T06:43:15.357 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-55.el9.x86_64 73/172 2024-06-13T06:43:15.388 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 74/172 2024-06-13T06:43:15.418 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 75/172 2024-06-13T06:43:15.453 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 76/172 2024-06-13T06:43:15.486 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 77/172 2024-06-13T06:43:15.527 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/172 2024-06-13T06:43:15.562 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 79/172 2024-06-13T06:43:15.648 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 80/172 2024-06-13T06:43:15.692 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 81/172 2024-06-13T06:43:15.716 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-06-13T06:43:15.724 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 82/172 2024-06-13T06:43:15.771 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 83/172 2024-06-13T06:43:15.869 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 84/172 2024-06-13T06:43:15.872 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-06-13T06:43:15.917 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:43:16.636 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-06-13T06:43:16.652 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 144/172 2024-06-13T06:43:16.680 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-06-13T06:43:16.693 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 145/172 2024-06-13T06:43:16.714 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-06-13T06:43:16.734 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 146/172 2024-06-13T06:43:16.766 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.4.0-1.el9.x86_64 147/172 2024-06-13T06:43:16.794 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-06-13T06:43:16.812 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-06-13T06:43:16.861 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-06-13T06:43:16.906 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-06-13T06:43:16.977 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:43:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-06-13T06:43:17.024 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-06-13T06:43:17.025 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-06-13T06:43:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 151/172 2024-06-13T06:43:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-06-13T06:43:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:43:17.164 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 154/172 2024-06-13T06:43:17.203 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-06-13T06:43:17.223 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-06-13T06:43:17.300 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-06-13T06:43:17.562 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-06-13T06:43:17.673 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-06-13T06:43:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:17.845 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-06-13T06:43:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:17.888 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-06-13T06:43:17.945 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-06-13T06:43:18.062 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-06-13T06:43:18.288 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-06-13T06:43:18.327 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:43:18.450 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:43:18.484 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 105/172 2024-06-13T06:43:18.540 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-06-13T06:43:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 156/172 2024-06-13T06:43:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.2.2-1.el9.x86_64 157/172 2024-06-13T06:43:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-06-13T06:43:18.916 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-06-13T06:43:18.946 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-06-13T06:43:18.971 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-06-13T06:43:18.997 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:43:19.023 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-06-13T06:43:19.023 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:43:19.100 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:43:19.100 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-06-13T06:43:19.132 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-06-13T06:43:19.132 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:43:19.147 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:43:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:43:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-06-13T06:43:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:19.276 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-06-13T06:43:19.283 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:43:19.316 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 109/172 2024-06-13T06:43:19.368 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-06-13T06:43:19.396 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-06-13T06:43:19.419 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:19.419 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:43:19.429 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-06-13T06:43:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:43:19.536 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:43:19.562 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-06-13T06:43:19.626 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:43:19.749 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-06-13T06:43:19.800 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:43:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:43:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 105/172 2024-06-13T06:43:19.958 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-06-13T06:43:20.437 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-06-13T06:43:20.470 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:43:20.541 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:43:20.563 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 109/172 2024-06-13T06:43:22.113 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:43:22.162 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:43:22.835 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:43:22.862 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:43:22.868 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:43:22.915 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:43:23.064 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:43:23.064 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-13T06:43:23.064 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-13T06:43:23.064 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:23.099 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:43:23.403 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:43:23.438 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:43:23.527 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:43:23.527 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-13T06:43:23.527 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-13T06:43:23.527 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:23.555 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:43:25.097 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/172 2024-06-13T06:43:25.098 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/172 2024-06-13T06:43:25.099 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-06-13T06:43:25.100 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-06-13T06:43:25.101 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.2.2-1.el9.x86_64 68/172 2024-06-13T06:43:25.102 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-06-13T06:43:25.103 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-06-13T06:43:25.104 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.4.0-1.el9.x86_64 101/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-06-13T06:43:25.105 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/172 2024-06-13T06:43:25.106 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-06-13T06:43:25.107 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/172 2024-06-13T06:43:25.108 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-06-13T06:43:25.109 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-06-13T06:43:25.110 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.2-1.el9.x86_64 167/172 2024-06-13T06:43:25.110 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-06-13T06:43:25.110 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.2-1.el9.x86_64 169/172 2024-06-13T06:43:25.110 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:43:25.110 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-6.4.0-1.el9.x86_64 171/172 2024-06-13T06:43:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:43:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:27.946 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-06-13T06:43:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-13T06:43:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-13T06:43:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-13T06:43:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.2.2-1.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.2.2-1.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-6.4.0-1.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-06-13T06:43:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-13T06:43:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-13T06:43:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.2.2-1.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.14-1.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-13T06:43:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-13T06:43:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-13T06:43:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-13T06:43:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-13T06:43:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-3.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-13T06:43:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-13T06:43:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.4.0-1.el9.x86_64 2024-06-13T06:43:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-13T06:43:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-13T06:43:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:43:27.959 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-06-13T06:43:28.413 DEBUG:teuthology.parallel:result is None 2024-06-13T06:43:35.719 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:43:35.719 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-06-13T06:43:35.720 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-06-13T06:43:35.720 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-06-13T06:43:35.720 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-06-13T06:43:35.720 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-06-13T06:43:35.720 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-06-13T06:43:35.720 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-06-13T06:43:35.720 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:35.881 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:43:35.900 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:43:35.900 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:35.900 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-13T06:43:35.900 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-06-13T06:43:36.119 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:36.356 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:43:36.405 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:43:36.406 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:36.406 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-13T06:43:36.406 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:37.385 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:43:37.410 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:43:37.410 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:37.410 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-13T06:43:37.410 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:43:37.411 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:43:37.411 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:37.635 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:43:37.658 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:43:37.658 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:37.658 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-13T06:43:37.658 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:43:37.658 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:43:37.658 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:38.083 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:43:38.102 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:43:38.125 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:43:38.125 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:38.125 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-13T06:43:38.125 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:43:38.125 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:43:38.125 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:38.166 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:43:38.166 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:38.166 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-13T06:43:38.166 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:43:38.166 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:43:38.166 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:38.224 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-06-13T06:43:38.272 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-06-13T06:43:38.328 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-06-13T06:43:38.425 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:43:38.495 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:43:38.496 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:38.496 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-13T06:43:38.496 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:43:38.496 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:43:38.496 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:38.512 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-06-13T06:43:38.562 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-06-13T06:43:38.639 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-06-13T06:43:38.728 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-06-13T06:43:38.875 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-06-13T06:43:38.912 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-06-13T06:43:38.943 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-06-13T06:43:38.979 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-06-13T06:43:39.034 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:43:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:43:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-13T06:43:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:43:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:43:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:39.279 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-06-13T06:43:39.352 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-06-13T06:43:39.427 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-06-13T06:43:39.465 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:43:39.503 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:43:39.534 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 129/172 2024-06-13T06:43:39.596 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 130/172 2024-06-13T06:43:39.634 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:43:39.692 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-06-13T06:43:39.766 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-06-13T06:43:39.859 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-06-13T06:43:39.987 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-06-13T06:43:40.185 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-06-13T06:43:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-06-13T06:43:40.294 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-06-13T06:43:40.326 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-06-13T06:43:40.676 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:43:40.711 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:43:40.921 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:43:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:43:40.999 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 129/172 2024-06-13T06:43:41.123 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 130/172 2024-06-13T06:43:41.156 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:43:41.681 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:43:41.703 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:43:41.805 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:43:41.966 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 134/172 2024-06-13T06:43:41.996 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:43:42.020 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:43:42.020 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:42.020 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-13T06:43:42.020 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:43:42.020 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:43:42.020 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:42.064 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:43:42.075 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:43:42.172 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 137/172 2024-06-13T06:43:42.220 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 138/172 2024-06-13T06:43:42.529 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:43:42.575 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:43:43.128 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-45.el9.x86_64 139/172 2024-06-13T06:43:43.158 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:43:43.181 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:43:43.181 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:43.181 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-13T06:43:43.181 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:43:43.181 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:43:43.181 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:43.225 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:43:43.248 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:43:43.248 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:43.248 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-13T06:43:43.248 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:43.293 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:43:43.315 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:43:43.315 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:43.315 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-13T06:43:43.315 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:43:43.316 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:43:43.316 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:43.618 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:43:43.641 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:43:43.641 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:43.641 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-13T06:43:43.641 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:43:43.641 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:43:43.641 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:43.744 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:43:43.775 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:43:43.929 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:43:44.116 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 134/172 2024-06-13T06:43:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:43:44.229 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:43:44.229 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:44.229 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-13T06:43:44.230 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:43:44.230 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:43:44.230 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:44.286 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:43:44.325 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:43:44.434 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 137/172 2024-06-13T06:43:44.505 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 138/172 2024-06-13T06:43:45.567 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-45.el9.x86_64 139/172 2024-06-13T06:43:45.605 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:43:45.699 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:43:45.699 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:45.699 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-13T06:43:45.699 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:43:45.699 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:43:45.699 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:45.764 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:43:45.867 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:43:45.867 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:45.867 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-13T06:43:45.867 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:45.956 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:43:46.069 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:43:46.069 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:46.069 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-13T06:43:46.069 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:43:46.069 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:43:46.069 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:46.490 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:43:46.569 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:43:46.570 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:43:46.570 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-13T06:43:46.570 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:43:46.570 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:43:46.570 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 144/172 2024-06-13T06:43:48.229 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 145/172 2024-06-13T06:43:48.270 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 146/172 2024-06-13T06:43:48.302 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-6.4.0-1.el9.x86_64 147/172 2024-06-13T06:43:48.330 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-06-13T06:43:48.512 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:43:48.559 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-06-13T06:43:48.591 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 151/172 2024-06-13T06:43:48.632 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-06-13T06:43:48.670 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:43:48.708 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 154/172 2024-06-13T06:43:49.279 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:50.161 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 156/172 2024-06-13T06:43:50.244 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:3.2.2-1.el9.x86_64 157/172 2024-06-13T06:43:50.284 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-06-13T06:43:50.373 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-06-13T06:43:50.413 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-06-13T06:43:50.438 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-06-13T06:43:50.558 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:43:50.558 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-06-13T06:43:50.558 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:43:50.574 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:43:50.633 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-06-13T06:43:50.633 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:43:50.717 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:43:50.742 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-06-13T06:43:50.746 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:50.746 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:50.927 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:50.927 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:43:51.011 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:43:51.036 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:43:51.062 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-06-13T06:43:51.125 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:43:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 144/172 2024-06-13T06:43:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 145/172 2024-06-13T06:43:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 146/172 2024-06-13T06:43:51.971 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.4.0-1.el9.x86_64 147/172 2024-06-13T06:43:52.009 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-06-13T06:43:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:43:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-06-13T06:43:52.394 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 151/172 2024-06-13T06:43:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-06-13T06:43:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:43:52.541 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 154/172 2024-06-13T06:43:53.545 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:53.653 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:43:55.155 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 156/172 2024-06-13T06:43:55.262 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.2.2-1.el9.x86_64 157/172 2024-06-13T06:43:55.319 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-06-13T06:43:55.453 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-06-13T06:43:55.488 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-06-13T06:43:55.521 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-06-13T06:43:55.556 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:43:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-06-13T06:43:55.597 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:43:55.756 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:43:55.801 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-06-13T06:43:55.802 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:43:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:43:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-06-13T06:43:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:43:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:43:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:43:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:43:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-06-13T06:43:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:43:56.867 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:43:56.867 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/172 2024-06-13T06:43:56.868 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/172 2024-06-13T06:43:56.869 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/172 2024-06-13T06:43:56.882 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/172 2024-06-13T06:43:56.882 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/172 2024-06-13T06:43:56.882 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/172 2024-06-13T06:43:56.882 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/172 2024-06-13T06:43:56.882 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/172 2024-06-13T06:43:56.882 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-06-13T06:43:56.883 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-06-13T06:43:56.884 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:3.2.2-1.el9.x86_64 68/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-06-13T06:43:56.885 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-06-13T06:43:56.886 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-06-13T06:43:56.887 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-6.4.0-1.el9.x86_64 101/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-06-13T06:43:56.888 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-06-13T06:43:56.889 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-06-13T06:43:56.890 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-06-13T06:43:56.891 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-06-13T06:43:56.892 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.2.2-1.el9.x86_64 167/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.2.2-1.el9.x86_64 169/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:43:56.893 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-6.4.0-1.el9.x86_64 171/172 2024-06-13T06:43:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:43:57.000 INFO:teuthology.orchestra.run.smithi026.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-06-13T06:43:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:43:59.222 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:43:59.222 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:59.222 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-06-13T06:43:59.222 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-13T06:43:59.222 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-13T06:43:59.222 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-13T06:43:59.222 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-1:3.2.2-1.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs-1:3.2.2-1.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-6.4.0-1.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-13T06:43:59.223 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.2-8.el9.noarch 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.7.4-5.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.224 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.225 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.4-9.el9.x86_64 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-13T06:43:59.226 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-13T06:43:59.227 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-13T06:43:59.228 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: lua-5.4.4-4.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:3.2.2-1.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-13.14-1.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-13T06:43:59.229 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-13T06:43:59.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-13T06:43:59.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-13T06:43:59.233 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-13T06:43:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:43:59.235 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-3.el9.x86_64 2024-06-13T06:43:59.236 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-13T06:43:59.236 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-13T06:43:59.236 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-13T06:43:59.236 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-6.4.0-1.el9.x86_64 2024-06-13T06:43:59.236 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-13T06:43:59.236 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-13T06:43:59.236 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-13T06:43:59.236 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-06-13T06:43:59.690 DEBUG:teuthology.parallel:result is None 2024-06-13T06:44:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/172 2024-06-13T06:44:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/172 2024-06-13T06:44:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/172 2024-06-13T06:44:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/172 2024-06-13T06:44:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/172 2024-06-13T06:44:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/172 2024-06-13T06:44:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/172 2024-06-13T06:44:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/172 2024-06-13T06:44:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/172 2024-06-13T06:44:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/172 2024-06-13T06:44:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/172 2024-06-13T06:44:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/172 2024-06-13T06:44:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/172 2024-06-13T06:44:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/172 2024-06-13T06:44:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/172 2024-06-13T06:44:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/172 2024-06-13T06:44:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/172 2024-06-13T06:44:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/172 2024-06-13T06:44:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-06-13T06:44:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-06-13T06:44:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.2.2-1.el9.x86_64 68/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-06-13T06:44:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-06-13T06:44:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-06-13T06:44:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-06-13T06:44:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-06-13T06:44:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-06-13T06:44:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.4.0-1.el9.x86_64 101/172 2024-06-13T06:44:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-06-13T06:44:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-06-13T06:44:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-06-13T06:44:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-06-13T06:44:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-06-13T06:44:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-06-13T06:44:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/172 2024-06-13T06:44:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.2-1.el9.x86_64 167/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.2-1.el9.x86_64 169/172 2024-06-13T06:44:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:44:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-6.4.0-1.el9.x86_64 171/172 2024-06-13T06:44:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:44:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:44:05.457 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-06-13T06:44:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-13T06:44:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.2.2-1.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.2.2-1.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-6.4.0-1.el9.x86_64 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-06-13T06:44:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-13T06:44:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.2.2-1.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.14-1.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-13T06:44:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-13T06:44:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-13T06:44:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-13T06:44:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-13T06:44:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.4.0-1.el9.x86_64 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:44:05.470 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-06-13T06:44:06.136 DEBUG:teuthology.parallel:result is None 2024-06-13T06:44:06.136 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:44:06.368 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-13T06:44:06.425 INFO:teuthology.orchestra.run.smithi026.stdout:19.0.0-2946.g3d89accf.el9 2024-06-13T06:44:06.426 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-13T06:44:06.426 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-13T06:44:06.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:44:06.645 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-13T06:44:06.671 INFO:teuthology.orchestra.run.smithi135.stdout:19.0.0-2946.g3d89accf.el9 2024-06-13T06:44:06.672 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-13T06:44:06.672 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-13T06:44:06.674 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:44:06.819 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-13T06:44:06.845 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-2946.g3d89accf.el9 2024-06-13T06:44:06.845 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-13T06:44:06.845 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-13T06:44:06.847 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-13T06:44:06.847 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:44:06.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-13T06:44:06.911 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:44:06.911 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-13T06:44:06.943 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:44:06.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-13T06:44:06.974 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-13T06:44:06.974 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:44:06.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-06-13T06:44:07.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-13T06:44:07.150 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:44:07.151 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-06-13T06:44:07.179 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-13T06:44:07.265 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:44:07.265 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-06-13T06:44:07.293 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-13T06:44:07.359 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-13T06:44:07.359 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:44:07.359 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-13T06:44:07.435 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-13T06:44:07.547 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:44:07.547 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-13T06:44:07.570 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-13T06:44:07.635 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:44:07.635 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-13T06:44:07.660 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-13T06:44:07.728 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-13T06:44:07.728 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:44:07.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-06-13T06:44:07.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-13T06:44:07.891 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:44:07.892 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-06-13T06:44:07.915 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-13T06:44:07.980 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:44:07.980 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-06-13T06:44:08.003 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-13T06:44:08.067 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-13T06:44:08.168 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-06-13T06:44:08.168 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:44:08.169 INFO:tasks.cephadm:Cluster fsid is 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:44:08.169 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-13T06:44:08.169 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.26', 'mon.b': '172.21.15.135', 'mon.c': '172.21.15.136'} 2024-06-13T06:44:08.169 INFO:tasks.cephadm:First mon is mon.a on smithi026 2024-06-13T06:44:08.169 INFO:tasks.cephadm:First mgr is x 2024-06-13T06:44:08.169 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-13T06:44:08.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2024-06-13T06:44:08.242 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname $(hostname -s) 2024-06-13T06:44:08.266 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-06-13T06:44:08.291 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-13T06:44:08.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:44:08.441 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-13T06:44:08.664 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:44:08.666 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-13T06:44:08.666 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:09.084 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 13 06:44 /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:09.084 DEBUG:teuthology.orchestra.run.smithi135:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:09.459 INFO:teuthology.orchestra.run.smithi135.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 13 06:44 /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:09.460 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:10.003 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 13 06:44 /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:10.003 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:10.052 DEBUG:teuthology.orchestra.run.smithi135:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:10.071 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-13T06:44:10.106 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-13T06:44:10.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-13T06:44:10.110 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-13T06:44:10.114 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-13T06:44:11.017 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:44:11.105 INFO:teuthology.orchestra.run.smithi135.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:44:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:44:46.555 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-13T06:44:46.556 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-13T06:44:46.556 INFO:teuthology.orchestra.run.smithi135.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-13T06:44:46.556 INFO:teuthology.orchestra.run.smithi135.stdout: "repo_digests": [ 2024-06-13T06:44:46.556 INFO:teuthology.orchestra.run.smithi135.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-13T06:44:46.556 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-06-13T06:44:46.556 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-13T06:44:52.567 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-13T06:44:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-13T06:44:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-13T06:44:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-06-13T06:44:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-13T06:44:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-06-13T06:44:52.567 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-13T06:44:56.185 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T06:44:56.185 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-13T06:44:56.185 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-13T06:44:56.185 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2024-06-13T06:44:56.185 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-13T06:44:56.185 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T06:44:56.186 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T06:44:56.252 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2024-06-13T06:44:56.346 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph 2024-06-13T06:44:56.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-06-13T06:44:56.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2024-06-13T06:44:56.505 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /etc/ceph 2024-06-13T06:44:56.533 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-06-13T06:44:56.560 INFO:tasks.cephadm:Writing seed config... 2024-06-13T06:44:56.561 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-13T06:44:56.561 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-13T06:44:56.562 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-13T06:44:56.563 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-13T06:44:56.564 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-13T06:44:56.565 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-06-13T06:44:56.565 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:44:56.565 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-13T06:44:56.623 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 559474f0-2950-11ef-bc9c-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-13T06:44:56.624 DEBUG:teuthology.orchestra.run.smithi026:mon.a> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a.service 2024-06-13T06:44:56.671 DEBUG:teuthology.orchestra.run.smithi026:mgr.x> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.x.service 2024-06-13T06:44:56.720 INFO:tasks.cephadm:Bootstrapping... 2024-06-13T06:44:56.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid 559474f0-2950-11ef-bc9c-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.26 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-13T06:44:57.000 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:44:57.000 INFO:teuthology.orchestra.run.smithi026.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', '559474f0-2950-11ef-bc9c-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.26', '--skip-admin-label'] 2024-06-13T06:44:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-13T06:44:57.001 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying podman|docker is present... 2024-06-13T06:44:57.056 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 5.1.0 2024-06-13T06:44:57.057 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying lvm2 is present... 2024-06-13T06:44:57.057 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying time synchronization is in place... 2024-06-13T06:44:57.073 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-13T06:44:57.073 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-13T06:44:57.091 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-13T06:44:57.092 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-06-13T06:44:57.113 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2024-06-13T06:44:57.132 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2024-06-13T06:44:57.132 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2024-06-13T06:44:57.132 INFO:teuthology.orchestra.run.smithi026.stdout:Repeating the final host check... 2024-06-13T06:44:57.192 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 5.1.0 2024-06-13T06:44:57.192 INFO:teuthology.orchestra.run.smithi026.stdout:podman (/bin/podman) version 5.1.0 is present 2024-06-13T06:44:57.193 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl is present 2024-06-13T06:44:57.193 INFO:teuthology.orchestra.run.smithi026.stdout:lvcreate is present 2024-06-13T06:44:57.207 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-13T06:44:57.207 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-13T06:44:57.221 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-13T06:44:57.221 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-06-13T06:44:57.244 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2024-06-13T06:44:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2024-06-13T06:44:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2024-06-13T06:44:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:Host looks OK 2024-06-13T06:44:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:Cluster fsid: 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:44:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:Acquiring lock 139912397579456 on /run/cephadm/559474f0-2950-11ef-bc9c-c7b262605968.lock 2024-06-13T06:44:57.263 INFO:teuthology.orchestra.run.smithi026.stdout:Lock 139912397579456 acquired on /run/cephadm/559474f0-2950-11ef-bc9c-c7b262605968.lock 2024-06-13T06:44:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 3300 ... 2024-06-13T06:44:57.264 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 6789 ... 2024-06-13T06:44:57.265 INFO:teuthology.orchestra.run.smithi026.stdout:Base mon IP(s) is [172.21.15.26:3300, 172.21.15.26:6789], mon addrv is [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-06-13T06:44:57.274 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.26 metric 100 2024-06-13T06:44:57.274 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.26 metric 100 2024-06-13T06:44:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-13T06:44:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-13T06:44:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-13T06:44:57.289 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-13T06:44:57.289 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-13T06:44:57.289 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-13T06:44:57.290 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-13T06:44:57.290 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-06-13T06:44:57.290 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-13T06:44:57.290 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2024-06-13T06:44:57.291 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2024-06-13T06:44:57.291 INFO:teuthology.orchestra.run.smithi026.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-13T06:44:57.292 INFO:teuthology.orchestra.run.smithi026.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-13T06:44:57.293 INFO:teuthology.orchestra.run.smithi026.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:44:58.074 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-13T06:44:58.074 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:44:58.074 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Getting image source signatures 2024-06-13T06:44:58.074 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-13T06:44:58.075 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-13T06:44:58.075 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-13T06:44:58.075 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-13T06:44:59.282 INFO:teuthology.orchestra.run.smithi026.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-13T06:44:59.283 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-13T06:44:59.283 INFO:teuthology.orchestra.run.smithi026.stdout:Extracting ceph user uid/gid from container image... 2024-06-13T06:45:00.380 INFO:teuthology.orchestra.run.smithi026.stdout:stat: stdout 167 167 2024-06-13T06:45:00.381 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial keys... 2024-06-13T06:45:01.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQBslWpmtfadNxAA0nLRfFiE0xWo7WGsVSPtkw== 2024-06-13T06:45:02.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQBulWpm8etaCxAAl5G2CC8DpdcQkcfIjaGSiA== 2024-06-13T06:45:03.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQBvlWpmVG+ZEBAAxzpI/G/Q3PMnhMqcsybjbw== 2024-06-13T06:45:03.869 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial monmap... 2024-06-13T06:45:05.021 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool for a [v2:172.21.15.26:3300,v1:172.21.15.26:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = quincy 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: set fsid to 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:45:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mon... 2024-06-13T06:45:06.248 INFO:teuthology.orchestra.run.smithi026.stdout:create mon.a on 2024-06-13T06:45:06.650 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-13T06:45:06.892 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-13T06:45:07.196 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-559474f0-2950-11ef-bc9c-c7b262605968.target → /etc/systemd/system/ceph-559474f0-2950-11ef-bc9c-c7b262605968.target. 2024-06-13T06:45:07.196 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-559474f0-2950-11ef-bc9c-c7b262605968.target → /etc/systemd/system/ceph-559474f0-2950-11ef-bc9c-c7b262605968.target. 2024-06-13T06:45:07.604 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a 2024-06-13T06:45:07.604 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a.service: Unit ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a.service not loaded. 2024-06-13T06:45:07.914 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-559474f0-2950-11ef-bc9c-c7b262605968.target.wants/ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a.service → /etc/systemd/system/ceph-559474f0-2950-11ef-bc9c-c7b262605968@.service. 2024-06-13T06:45:08.580 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:08 smithi026 podman[31652]: 2024-06-13 06:45:08.358104411 +0000 UTC m=+0.170261661 container create 8bdab0c08104793b07bdf9c54ea3988928032ba5c4730126d2fd0e2364d7c41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-13T06:45:08.581 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:08 smithi026 podman[31652]: 2024-06-13 06:45:08.578663645 +0000 UTC m=+0.390820902 container init 8bdab0c08104793b07bdf9c54ea3988928032ba5c4730126d2fd0e2364d7c41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-13T06:45:08.738 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-13T06:45:08.738 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-06-13T06:45:08.759 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-13T06:45:08.759 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-06-13T06:45:08.760 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-06-13T06:45:08.760 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2024-06-13T06:45:08.762 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon to start... 2024-06-13T06:45:08.762 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon... 2024-06-13T06:45:08.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:08 smithi026 podman[31652]: 2024-06-13 06:45:08.591495255 +0000 UTC m=+0.403652500 container start 8bdab0c08104793b07bdf9c54ea3988928032ba5c4730126d2fd0e2364d7c41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-13T06:45:08.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:08 smithi026 ceph-mon[31671]: mkfs 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:08.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:08 smithi026 ceph-mon[31671]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-13T06:45:08.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:08 smithi026 bash[31652]: 8bdab0c08104793b07bdf9c54ea3988928032ba5c4730126d2fd0e2364d7c41b 2024-06-13T06:45:08.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:08 smithi026 systemd[1]: Started Ceph mon.a for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:45:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-13T06:45:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: monmap epoch 1 2024-06-13T06:45:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: last_changed 2024-06-13T06:45:04.384219+0000 2024-06-13T06:45:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:45:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: min_mon_release 19 (squid) 2024-06-13T06:45:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: election_strategy: 1 2024-06-13T06:45:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:45:09.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: fsmap 2024-06-13T06:45:09.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: osdmap e1: 0 total, 0 up, 0 in 2024-06-13T06:45:09.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: mgrmap e1: no daemons active 2024-06-13T06:45:09.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:09 smithi026 ceph-mon[31671]: from='client.? 172.21.15.26:0/2893887033' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-13T06:45:10.087 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout cluster: 2024-06-13T06:45:10.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout id: 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:10.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-13T06:45:10.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:10.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout services: 2024-06-13T06:45:10.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.770342s) 2024-06-13T06:45:10.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-13T06:45:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-13T06:45:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout data: 2024-06-13T06:45:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-13T06:45:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-13T06:45:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-13T06:45:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pgs: 2024-06-13T06:45:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.346+0000 7f2593200700 1 Processor -- start 2024-06-13T06:45:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.347+0000 7f2593200700 1 -- start start 2024-06-13T06:45:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.347+0000 7f2593200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f259407ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.347+0000 7f2593200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f259407f230 con 0x7f259407e840 2024-06-13T06:45:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.348+0000 7f2591e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f259407ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.348+0000 7f2591e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f259407ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37282/0 (socket says 172.21.15.26:37282) 2024-06-13T06:45:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.348+0000 7f2591e00700 1 -- 172.21.15.26:0/1752099044 learned_addr learned my addr 172.21.15.26:0/1752099044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:10.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.348+0000 7f2591e00700 1 -- 172.21.15.26:0/1752099044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259407fa90 con 0x7f259407e840 2024-06-13T06:45:10.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.349+0000 7f2591e00700 1 --2- 172.21.15.26:0/1752099044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f259407ec60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25840098a0 tx=0x7f2584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68a69b978203abc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:10.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.349+0000 7f2590a00700 1 -- 172.21.15.26:0/1752099044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2584004160 con 0x7f259407e840 2024-06-13T06:45:10.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.349+0000 7f2590a00700 1 -- 172.21.15.26:0/1752099044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2584004300 con 0x7f259407e840 2024-06-13T06:45:10.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.349+0000 7f2590a00700 1 -- 172.21.15.26:0/1752099044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25840036a0 con 0x7f259407e840 2024-06-13T06:45:10.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.350+0000 7f2593200700 1 -- 172.21.15.26:0/1752099044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 msgr2=0x7f259407ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:10.096 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.350+0000 7f2593200700 1 --2- 172.21.15.26:0/1752099044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f259407ec60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25840098a0 tx=0x7f2584009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:10.096 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.350+0000 7f2593200700 1 -- 172.21.15.26:0/1752099044 shutdown_connections 2024-06-13T06:45:10.096 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.350+0000 7f2593200700 1 --2- 172.21.15.26:0/1752099044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f259407ec60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:10.096 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.350+0000 7f2593200700 1 -- 172.21.15.26:0/1752099044 >> 172.21.15.26:0/1752099044 conn(0x7f259407d310 msgr2=0x7f259407d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:10.096 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.350+0000 7f2593200700 1 -- 172.21.15.26:0/1752099044 shutdown_connections 2024-06-13T06:45:10.096 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.350+0000 7f2593200700 1 -- 172.21.15.26:0/1752099044 wait complete. 2024-06-13T06:45:10.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.351+0000 7f2593200700 1 Processor -- start 2024-06-13T06:45:10.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.351+0000 7f2593200700 1 -- start start 2024-06-13T06:45:10.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.352+0000 7f2593200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f25941a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.352+0000 7f2593200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f259407ffc0 con 0x7f259407e840 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.352+0000 7f2591e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f25941a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.352+0000 7f2591e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f25941a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37294/0 (socket says 172.21.15.26:37294) 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.352+0000 7f2591e00700 1 -- 172.21.15.26:0/2893887033 learned_addr learned my addr 172.21.15.26:0/2893887033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.353+0000 7f2591e00700 1 -- 172.21.15.26:0/2893887033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25941a1f20 con 0x7f259407e840 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.353+0000 7f2591e00700 1 --2- 172.21.15.26:0/2893887033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f25941a19e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2584000c00 tx=0x7f2584004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.353+0000 7f258aa00700 1 -- 172.21.15.26:0/2893887033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2584003db0 con 0x7f259407e840 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.353+0000 7f258aa00700 1 -- 172.21.15.26:0/2893887033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2584011630 con 0x7f259407e840 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.353+0000 7f2593200700 1 -- 172.21.15.26:0/2893887033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25941a21b0 con 0x7f259407e840 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.353+0000 7f258aa00700 1 -- 172.21.15.26:0/2893887033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2584011950 con 0x7f259407e840 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.354+0000 7f2593200700 1 -- 172.21.15.26:0/2893887033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25941a4ea0 con 0x7f259407e840 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.354+0000 7f258aa00700 1 -- 172.21.15.26:0/2893887033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2584011af0 con 0x7f259407e840 2024-06-13T06:45:10.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.354+0000 7f258aa00700 1 -- 172.21.15.26:0/2893887033 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f258401c790 con 0x7f259407e840 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.355+0000 7f257fe00700 1 -- 172.21.15.26:0/2893887033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2594063af0 con 0x7f259407e840 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.358+0000 7f258aa00700 1 -- 172.21.15.26:0/2893887033 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2584032020 con 0x7f259407e840 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.438+0000 7f257fe00700 1 -- 172.21.15.26:0/2893887033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f25940630a0 con 0x7f259407e840 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.439+0000 7f258aa00700 1 -- 172.21.15.26:0/2893887033 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2584004260 con 0x7f259407e840 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.441+0000 7f257fe00700 1 -- 172.21.15.26:0/2893887033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 msgr2=0x7f25941a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.441+0000 7f257fe00700 1 --2- 172.21.15.26:0/2893887033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f25941a19e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2584000c00 tx=0x7f2584004750 comp rx=0 tx=0).stop 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.441+0000 7f257fe00700 1 -- 172.21.15.26:0/2893887033 shutdown_connections 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.441+0000 7f257fe00700 1 --2- 172.21.15.26:0/2893887033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259407e840 0x7f25941a19e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.441+0000 7f257fe00700 1 -- 172.21.15.26:0/2893887033 >> 172.21.15.26:0/2893887033 conn(0x7f259407d310 msgr2=0x7f25941079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.441+0000 7f257fe00700 1 -- 172.21.15.26:0/2893887033 shutdown_connections 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:09.442+0000 7f257fe00700 1 -- 172.21.15.26:0/2893887033 wait complete. 2024-06-13T06:45:10.101 INFO:teuthology.orchestra.run.smithi026.stdout:mon is available 2024-06-13T06:45:10.102 INFO:teuthology.orchestra.run.smithi026.stdout:Assimilating anything we can from ceph.conf... 2024-06-13T06:45:11.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:10 smithi026 ceph-mon[31671]: from='client.? 172.21.15.26:0/3250272198' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-13T06:45:11.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:10 smithi026 ceph-mon[31671]: from='client.? 172.21.15.26:0/3250272198' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-13T06:45:11.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.714+0000 7ff142400700 1 Processor -- start 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.715+0000 7ff142400700 1 -- start start 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.715+0000 7ff142400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c10a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:11.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.715+0000 7ff142400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c10aa90 con 0x7ff13c10a0a0 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.715+0000 7ff13b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c10a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.716+0000 7ff13b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c10a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37296/0 (socket says 172.21.15.26:37296) 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.716+0000 7ff13b400700 1 -- 172.21.15.26:0/2190902793 learned_addr learned my addr 172.21.15.26:0/2190902793 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.716+0000 7ff13b400700 1 -- 172.21.15.26:0/2190902793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c10b300 con 0x7ff13c10a0a0 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.716+0000 7ff13b400700 1 --2- 172.21.15.26:0/2190902793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c10a4c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0098a0 tx=0x7ff12c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9342d306c34ff79f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.717+0000 7ff13a000700 1 -- 172.21.15.26:0/2190902793 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff12c004160 con 0x7ff13c10a0a0 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.717+0000 7ff13a000700 1 -- 172.21.15.26:0/2190902793 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c004300 con 0x7ff13c10a0a0 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.717+0000 7ff13a000700 1 -- 172.21.15.26:0/2190902793 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff12c0036a0 con 0x7ff13c10a0a0 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.717+0000 7ff142400700 1 -- 172.21.15.26:0/2190902793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 msgr2=0x7ff13c10a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.717+0000 7ff142400700 1 --2- 172.21.15.26:0/2190902793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c10a4c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0098a0 tx=0x7ff12c009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.718+0000 7ff142400700 1 -- 172.21.15.26:0/2190902793 shutdown_connections 2024-06-13T06:45:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.718+0000 7ff142400700 1 --2- 172.21.15.26:0/2190902793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c10a4c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.718+0000 7ff142400700 1 -- 172.21.15.26:0/2190902793 >> 172.21.15.26:0/2190902793 conn(0x7ff13c07d6f0 msgr2=0x7ff13c07db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.718+0000 7ff142400700 1 -- 172.21.15.26:0/2190902793 shutdown_connections 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.718+0000 7ff142400700 1 -- 172.21.15.26:0/2190902793 wait complete. 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.719+0000 7ff142400700 1 Processor -- start 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.719+0000 7ff142400700 1 -- start start 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.719+0000 7ff142400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.719+0000 7ff142400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c10b830 con 0x7ff13c10a0a0 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.720+0000 7ff13b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.720+0000 7ff13b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c19d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37308/0 (socket says 172.21.15.26:37308) 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.720+0000 7ff13b400700 1 -- 172.21.15.26:0/3250272198 learned_addr learned my addr 172.21.15.26:0/3250272198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.720+0000 7ff13b400700 1 -- 172.21.15.26:0/3250272198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c19ddc0 con 0x7ff13c10a0a0 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.720+0000 7ff13b400700 1 --2- 172.21.15.26:0/3250272198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c19d880 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff12c000c00 tx=0x7ff12c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.721+0000 7ff133e00700 1 -- 172.21.15.26:0/3250272198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff12c003db0 con 0x7ff13c10a0a0 2024-06-13T06:45:11.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.721+0000 7ff133e00700 1 -- 172.21.15.26:0/3250272198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c011630 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.721+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c19e050 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.721+0000 7ff133e00700 1 -- 172.21.15.26:0/3250272198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff12c011950 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.721+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c1a0d40 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.721+0000 7ff133e00700 1 -- 172.21.15.26:0/3250272198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff12c011af0 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.721+0000 7ff133e00700 1 -- 172.21.15.26:0/3250272198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff12c01c450 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.722+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13c063af0 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.725+0000 7ff133e00700 1 -- 172.21.15.26:0/3250272198 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff12c032020 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.802+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff13c19e530 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.814+0000 7ff133e00700 1 -- 172.21.15.26:0/3250272198 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7ff13c19e530 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.815+0000 7ff133e00700 1 -- 172.21.15.26:0/3250272198 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff12c004260 con 0x7ff13c10a0a0 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.816+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 msgr2=0x7ff13c19d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.816+0000 7ff142400700 1 --2- 172.21.15.26:0/3250272198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c19d880 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff12c000c00 tx=0x7ff12c004750 comp rx=0 tx=0).stop 2024-06-13T06:45:11.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.816+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 shutdown_connections 2024-06-13T06:45:11.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.816+0000 7ff142400700 1 --2- 172.21.15.26:0/3250272198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13c10a0a0 0x7ff13c19d880 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:11.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.816+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 >> 172.21.15.26:0/3250272198 conn(0x7ff13c07d6f0 msgr2=0x7ff13c194450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:11.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.817+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 shutdown_connections 2024-06-13T06:45:11.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:10.817+0000 7ff142400700 1 -- 172.21.15.26:0/3250272198 wait complete. 2024-06-13T06:45:11.469 INFO:teuthology.orchestra.run.smithi026.stdout:Generating new minimal ceph.conf... 2024-06-13T06:45:12.484 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:12 smithi026 ceph-mon[31671]: from='client.? 172.21.15.26:0/106711929' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:12.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.074+0000 7f466d000700 1 Processor -- start 2024-06-13T06:45:12.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.074+0000 7f466d000700 1 -- start start 2024-06-13T06:45:12.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.075+0000 7f466d000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466810a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:12.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.075+0000 7f466d000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f466810aa90 con 0x7f466810a0a0 2024-06-13T06:45:12.752 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.075+0000 7f4665e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466810a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:12.752 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.075+0000 7f4665e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466810a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37310/0 (socket says 172.21.15.26:37310) 2024-06-13T06:45:12.752 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.075+0000 7f4665e00700 1 -- 172.21.15.26:0/3453072431 learned_addr learned my addr 172.21.15.26:0/3453072431 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:12.752 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.076+0000 7f4665e00700 1 -- 172.21.15.26:0/3453072431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466810b300 con 0x7f466810a0a0 2024-06-13T06:45:12.752 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.076+0000 7f4665e00700 1 --2- 172.21.15.26:0/3453072431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466810a4c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f46580098a0 tx=0x7f4658009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d8b8a00f22a5fd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:12.752 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.076+0000 7f4664a00700 1 -- 172.21.15.26:0/3453072431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4658004160 con 0x7f466810a0a0 2024-06-13T06:45:12.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.076+0000 7f4664a00700 1 -- 172.21.15.26:0/3453072431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4658004300 con 0x7f466810a0a0 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.076+0000 7f4664a00700 1 -- 172.21.15.26:0/3453072431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4658011b50 con 0x7f466810a0a0 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.077+0000 7f466d000700 1 -- 172.21.15.26:0/3453072431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 msgr2=0x7f466810a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.077+0000 7f466d000700 1 --2- 172.21.15.26:0/3453072431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466810a4c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f46580098a0 tx=0x7f4658009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.077+0000 7f466d000700 1 -- 172.21.15.26:0/3453072431 shutdown_connections 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.077+0000 7f466d000700 1 --2- 172.21.15.26:0/3453072431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466810a4c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.077+0000 7f466d000700 1 -- 172.21.15.26:0/3453072431 >> 172.21.15.26:0/3453072431 conn(0x7f466807d6f0 msgr2=0x7f466807db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.077+0000 7f466d000700 1 -- 172.21.15.26:0/3453072431 shutdown_connections 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.077+0000 7f466d000700 1 -- 172.21.15.26:0/3453072431 wait complete. 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.078+0000 7f466d000700 1 Processor -- start 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.078+0000 7f466d000700 1 -- start start 2024-06-13T06:45:12.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.079+0000 7f466d000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.079+0000 7f466d000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f466810e6b0 con 0x7f466810a0a0 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.079+0000 7f4665e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.079+0000 7f4665e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466819e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37320/0 (socket says 172.21.15.26:37320) 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.079+0000 7f4665e00700 1 -- 172.21.15.26:0/106711929 learned_addr learned my addr 172.21.15.26:0/106711929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.079+0000 7f4665e00700 1 -- 172.21.15.26:0/106711929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466819e730 con 0x7f466810a0a0 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.080+0000 7f4665e00700 1 --2- 172.21.15.26:0/106711929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466819e1f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4658011850 tx=0x7f4658011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.080+0000 7f465ea00700 1 -- 172.21.15.26:0/106711929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4658003af0 con 0x7f466810a0a0 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.080+0000 7f465ea00700 1 -- 172.21.15.26:0/106711929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4658020070 con 0x7f466810a0a0 2024-06-13T06:45:12.756 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.080+0000 7f465ea00700 1 -- 172.21.15.26:0/106711929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46580195a0 con 0x7f466810a0a0 2024-06-13T06:45:12.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.080+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466819e9c0 con 0x7f466810a0a0 2024-06-13T06:45:12.758 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.080+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46681a16b0 con 0x7f466810a0a0 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.081+0000 7f465ea00700 1 -- 172.21.15.26:0/106711929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4658019be0 con 0x7f466810a0a0 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.081+0000 7f465ea00700 1 -- 172.21.15.26:0/106711929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4658011e00 con 0x7f466810a0a0 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.082+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4668063af0 con 0x7f466810a0a0 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.084+0000 7f465ea00700 1 -- 172.21.15.26:0/106711929 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4658003c90 con 0x7f466810a0a0 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.163+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f46681a1bb0 con 0x7f466810a0a0 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.164+0000 7f465ea00700 1 -- 172.21.15.26:0/106711929 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f4658025460 con 0x7f466810a0a0 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.165+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 msgr2=0x7f466819e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.165+0000 7f466d000700 1 --2- 172.21.15.26:0/106711929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466819e1f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4658011850 tx=0x7f4658011c10 comp rx=0 tx=0).stop 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.165+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 shutdown_connections 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.165+0000 7f466d000700 1 --2- 172.21.15.26:0/106711929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466810a0a0 0x7f466819e1f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:12.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.165+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 >> 172.21.15.26:0/106711929 conn(0x7f466807d6f0 msgr2=0x7f4668107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:12.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.166+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 shutdown_connections 2024-06-13T06:45:12.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:12.166+0000 7f466d000700 1 -- 172.21.15.26:0/106711929 wait complete. 2024-06-13T06:45:12.760 INFO:teuthology.orchestra.run.smithi026.stdout:Restarting the monitor... 2024-06-13T06:45:13.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:12 smithi026 systemd[1]: Stopping Ceph mon.a for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:45:13.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:13 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[31666]: 2024-06-13T06:45:13.214+0000 7f3be9000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T06:45:13.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:13 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[31666]: 2024-06-13T06:45:13.214+0000 7f3be9000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-13T06:45:13.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:13 smithi026 podman[31915]: 2024-06-13 06:45:13.400645891 +0000 UTC m=+0.302921725 container died 8bdab0c08104793b07bdf9c54ea3988928032ba5c4730126d2fd0e2364d7c41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-13T06:45:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:13 smithi026 podman[31915]: 2024-06-13 06:45:13.898397063 +0000 UTC m=+0.800672891 container cleanup 8bdab0c08104793b07bdf9c54ea3988928032ba5c4730126d2fd0e2364d7c41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:45:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:13 smithi026 bash[31915]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a 2024-06-13T06:45:14.441 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:14 smithi026 podman[31930]: 2024-06-13 06:45:14.190708403 +0000 UTC m=+0.767368780 container remove 8bdab0c08104793b07bdf9c54ea3988928032ba5c4730126d2fd0e2364d7c41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-13T06:45:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:14 smithi026 systemd[1]: ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-13T06:45:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:14 smithi026 systemd[1]: Stopped Ceph mon.a for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:45:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:14 smithi026 systemd[1]: ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a.service: Consumed 1.916s CPU time. 2024-06-13T06:45:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:14 smithi026 systemd[1]: Starting Ceph mon.a for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:45:15.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:14 smithi026 podman[32023]: 2024-06-13 06:45:14.728747425 +0000 UTC m=+0.038173490 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:45:17.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:17 smithi026 podman[32023]: 2024-06-13 06:45:17.209596166 +0000 UTC m=+2.519022241 container create 9dcd7ed340094569bf40d4ad44fec9babd9ea284548e291cd24d575e35cd80ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-13T06:45:23.515 INFO:teuthology.orchestra.run.smithi026.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-13T06:45:23.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 podman[32023]: 2024-06-13 06:45:23.406917479 +0000 UTC m=+8.716343598 container init 9dcd7ed340094569bf40d4ad44fec9babd9ea284548e291cd24d575e35cd80ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:45:23.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 podman[32023]: 2024-06-13 06:45:23.418898021 +0000 UTC m=+8.728324100 container start 9dcd7ed340094569bf40d4ad44fec9babd9ea284548e291cd24d575e35cd80ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-13T06:45:23.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: set uid:gid to 167:167 (ceph:ceph) 2024-06-13T06:45:23.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-13T06:45:23.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: pidfile_write: ignore empty --pid-file 2024-06-13T06:45:23.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: load: jerasure load: lrc 2024-06-13T06:45:23.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: RocksDB version: 7.9.2 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Git sha 0 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: DB SUMMARY 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: DB Session ID: BEY3G64YQ7IH21Q184VS 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: CURRENT file: CURRENT 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: IDENTITY file: IDENTITY 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104264 ; 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.error_if_exists: 0 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.create_if_missing: 0 2024-06-13T06:45:23.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.paranoid_checks: 1 2024-06-13T06:45:23.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-13T06:45:23.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-13T06:45:23.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-13T06:45:23.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.env: 0x5589657b7c60 2024-06-13T06:45:23.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.fs: PosixFileSystem 2024-06-13T06:45:23.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.info_log: 0x55899d607b60 2024-06-13T06:45:23.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_file_opening_threads: 16 2024-06-13T06:45:23.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.statistics: (nil) 2024-06-13T06:45:23.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.use_fsync: 0 2024-06-13T06:45:23.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_log_file_size: 0 2024-06-13T06:45:23.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-13T06:45:23.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-13T06:45:23.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.keep_log_file_num: 1000 2024-06-13T06:45:23.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.recycle_log_file_num: 0 2024-06-13T06:45:23.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.allow_fallocate: 1 2024-06-13T06:45:23.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.allow_mmap_reads: 0 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.allow_mmap_writes: 0 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.use_direct_reads: 0 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.create_missing_column_families: 0 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.db_log_dir: 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.wal_dir: 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-13T06:45:23.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.advise_random_on_open: 1 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.db_write_buffer_size: 0 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.write_buffer_manager: 0x55899d5d81e0 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.rate_limiter: (nil) 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.wal_recovery_mode: 2 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.enable_thread_tracking: 0 2024-06-13T06:45:23.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.enable_pipelined_write: 0 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.unordered_write: 0 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.row_cache: None 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.wal_filter: None 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.allow_ingest_behind: 0 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.two_write_queues: 0 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.manual_wal_flush: 0 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.wal_compression: 0 2024-06-13T06:45:23.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.atomic_flush: 0 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.log_readahead_size: 0 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.best_efforts_recovery: 0 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.allow_data_in_errors: 0 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.db_host_id: __hostname__ 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_background_jobs: 2 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_background_compactions: -1 2024-06-13T06:45:23.674 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_subcompactions: 1 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_total_wal_size: 0 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_open_files: -1 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bytes_per_sync: 0 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_readahead_size: 0 2024-06-13T06:45:23.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_background_flushes: -1 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Compression algorithms supported: 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: kZSTD supported: 0 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: kXpressCompression supported: 0 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: kLZ4HCCompression supported: 1 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: kZlibCompression supported: 1 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: kSnappyCompression supported: 1 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: kLZ4Compression supported: 1 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: kBZip2Compression supported: 0 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-13T06:45:23.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.merge_operator: 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_filter: None 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_filter_factory: None 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.sst_partitioner_factory: None 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55899d607c80) 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks: 1 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_top_level_index_and_filter: 1 2024-06-13T06:45:23.677 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_type: 0 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_index_type: 0 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_shortening: 1 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: checksum: 4 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: no_block_cache: 0 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache: 0x55899d609090 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_name: BinnedLRUCache 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_options: 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: capacity : 536870912 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_shard_bits : 4 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: strict_capacity_limit : 0 2024-06-13T06:45:23.678 INFO:journalctl@ceph.mon.a.smithi026.stdout: high_pri_pool_ratio: 0.000 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_compressed: (nil) 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: persistent_cache: (nil) 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size: 4096 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size_deviation: 10 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_restart_interval: 16 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_block_restart_interval: 1 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: metadata_block_size: 4096 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: partition_filters: 0 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: use_delta_encoding: 1 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: filter_policy: bloomfilter 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: whole_key_filtering: 1 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: verify_compression: 0 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: read_amp_bytes_per_bit: 0 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: format_version: 5 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_index_compression: 1 2024-06-13T06:45:23.679 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_align: 0 2024-06-13T06:45:23.680 INFO:journalctl@ceph.mon.a.smithi026.stdout: max_auto_readahead_size: 262144 2024-06-13T06:45:23.680 INFO:journalctl@ceph.mon.a.smithi026.stdout: prepopulate_block_cache: 0 2024-06-13T06:45:23.680 INFO:journalctl@ceph.mon.a.smithi026.stdout: initial_auto_readahead_size: 8192 2024-06-13T06:45:23.680 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.write_buffer_size: 33554432 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_write_buffer_number: 2 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression: NoCompression 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression: Disabled 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.prefix_extractor: nullptr 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.num_levels: 7 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-13T06:45:23.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.level: 32767 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.strategy: 0 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.enabled: false 2024-06-13T06:45:23.682 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.target_file_size_base: 67108864 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-13T06:45:23.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.arena_block_size: 1048576 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.disable_auto_compactions: 0 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-13T06:45:23.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.table_properties_collectors: 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.inplace_update_support: 0 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.bloom_locality: 0 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.max_successive_merges: 0 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.paranoid_file_checks: 0 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.force_consistency_checks: 1 2024-06-13T06:45:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.report_bg_io_stats: 0 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.ttl: 2592000 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.enable_blob_files: false 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.min_blob_size: 0 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.blob_file_size: 268435456 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.blob_file_starting_level: 0 2024-06-13T06:45:23.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3af7bd3b-9b82-405c-8f03-c05796ed0591 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718261123478984, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718261123481272, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99425, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97333, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14056, "raw_average_key_size": 49, "raw_value_size": 89703, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718261123, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3af7bd3b-9b82-405c-8f03-c05796ed0591", "db_session_id": "BEY3G64YQ7IH21Q184VS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718261123481433, "job": 1, "event": "recovery_finished"} 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55899d6f8000 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: DB pointer 0x55899d6e2000 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** DB Stats ** 2024-06-13T06:45:23.687 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: L0 2/0 98.93 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Sum 2/0 98.93 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-13T06:45:23.688 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 59.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative compaction: 0.00 GB write, 13.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-13T06:45:23.689 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval compaction: 0.00 GB write, 13.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache BinnedLRUCache@0x55899d609090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: starting mon.a rank 0 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a@-1(???) e1 preinit fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a@-1(???).mds e1 new map 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a@-1(???).mds e1 print_map 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: e1 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: legacy client fscid: -1 2024-06-13T06:45:23.690 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout: No filesystems configured 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 bash[32023]: 9dcd7ed340094569bf40d4ad44fec9babd9ea284548e291cd24d575e35cd80ff 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 systemd[1]: Started Ceph mon.a for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: monmap epoch 1 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: last_changed 2024-06-13T06:45:04.384219+0000 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:45:23.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: min_mon_release 19 (squid) 2024-06-13T06:45:23.692 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: election_strategy: 1 2024-06-13T06:45:23.692 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:45:23.692 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: fsmap 2024-06-13T06:45:23.692 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: osdmap e1: 0 total, 0 up, 0 in 2024-06-13T06:45:23.692 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:23 smithi026 ceph-mon[32041]: mgrmap e1: no daemons active 2024-06-13T06:45:24.981 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.205+0000 7f0830c00700 1 Processor -- start 2024-06-13T06:45:24.983 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.206+0000 7f0830c00700 1 -- start start 2024-06-13T06:45:24.983 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.206+0000 7f0830c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c10ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:24.983 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.206+0000 7f0830c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f082c07be50 con 0x7f082c108950 2024-06-13T06:45:24.983 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.207+0000 7f0829e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c10ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:24.983 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.207+0000 7f0829e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c10ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40320/0 (socket says 172.21.15.26:40320) 2024-06-13T06:45:24.983 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.207+0000 7f0829e00700 1 -- 172.21.15.26:0/1581022850 learned_addr learned my addr 172.21.15.26:0/1581022850 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:24.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.207+0000 7f0829e00700 1 -- 172.21.15.26:0/1581022850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c10b280 con 0x7f082c108950 2024-06-13T06:45:24.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.208+0000 7f0829e00700 1 --2- 172.21.15.26:0/1581022850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c10ad40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f081c0098a0 tx=0x7f081c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed54ce2b001dae7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:24.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.208+0000 7f0828a00700 1 -- 172.21.15.26:0/1581022850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f081c004160 con 0x7f082c108950 2024-06-13T06:45:24.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.208+0000 7f0828a00700 1 -- 172.21.15.26:0/1581022850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f081c004300 con 0x7f082c108950 2024-06-13T06:45:24.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.208+0000 7f0828a00700 1 -- 172.21.15.26:0/1581022850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f081c011b50 con 0x7f082c108950 2024-06-13T06:45:24.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.209+0000 7f0830c00700 1 -- 172.21.15.26:0/1581022850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 msgr2=0x7f082c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.209+0000 7f0830c00700 1 --2- 172.21.15.26:0/1581022850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c10ad40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f081c0098a0 tx=0x7f081c009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.209+0000 7f0830c00700 1 -- 172.21.15.26:0/1581022850 shutdown_connections 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.209+0000 7f0830c00700 1 --2- 172.21.15.26:0/1581022850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c10ad40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.209+0000 7f0830c00700 1 -- 172.21.15.26:0/1581022850 >> 172.21.15.26:0/1581022850 conn(0x7f082c1023c0 msgr2=0x7f082c1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.209+0000 7f0830c00700 1 -- 172.21.15.26:0/1581022850 shutdown_connections 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.209+0000 7f0830c00700 1 -- 172.21.15.26:0/1581022850 wait complete. 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.210+0000 7f0830c00700 1 Processor -- start 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.210+0000 7f0830c00700 1 -- start start 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.211+0000 7f0830c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c19e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.211+0000 7f0830c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f082c10c2a0 con 0x7f082c108950 2024-06-13T06:45:24.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.211+0000 7f0829e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c19e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.211+0000 7f0829e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c19e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40332/0 (socket says 172.21.15.26:40332) 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.211+0000 7f0829e00700 1 -- 172.21.15.26:0/2515802420 learned_addr learned my addr 172.21.15.26:0/2515802420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.211+0000 7f0829e00700 1 -- 172.21.15.26:0/2515802420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c19e5e0 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.212+0000 7f0829e00700 1 --2- 172.21.15.26:0/2515802420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c19e0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f081c011670 tx=0x7f081c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.212+0000 7f0822a00700 1 -- 172.21.15.26:0/2515802420 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f081c0039c0 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.212+0000 7f0822a00700 1 -- 172.21.15.26:0/2515802420 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f081c011e50 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.212+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c19e870 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.212+0000 7f0822a00700 1 -- 172.21.15.26:0/2515802420 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f081c01a5c0 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.212+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c1a1560 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.213+0000 7f0822a00700 1 -- 172.21.15.26:0/2515802420 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f081c01ac50 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.213+0000 7f0822a00700 1 -- 172.21.15.26:0/2515802420 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f081c019700 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.214+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082c063af0 con 0x7f082c108950 2024-06-13T06:45:24.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.217+0000 7f0822a00700 1 -- 172.21.15.26:0/2515802420 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f081c003b60 con 0x7f082c108950 2024-06-13T06:45:24.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.296+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f082c1a1730 con 0x7f082c108950 2024-06-13T06:45:24.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.301+0000 7f0822a00700 1 -- 172.21.15.26:0/2515802420 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f081c019900 con 0x7f082c108950 2024-06-13T06:45:24.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.302+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 msgr2=0x7f082c19e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:24.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.302+0000 7f0830c00700 1 --2- 172.21.15.26:0/2515802420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c19e0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f081c011670 tx=0x7f081c011b60 comp rx=0 tx=0).stop 2024-06-13T06:45:24.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.303+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 shutdown_connections 2024-06-13T06:45:24.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.303+0000 7f0830c00700 1 --2- 172.21.15.26:0/2515802420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c108950 0x7f082c19e0a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:24.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.303+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 >> 172.21.15.26:0/2515802420 conn(0x7f082c1023c0 msgr2=0x7f082c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:24.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.303+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 shutdown_connections 2024-06-13T06:45:24.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:24.303+0000 7f0830c00700 1 -- 172.21.15.26:0/2515802420 wait complete. 2024-06-13T06:45:24.990 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-13T06:45:25.010 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-13T06:45:25.011 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mgr... 2024-06-13T06:45:25.012 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-13T06:45:25.014 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-13T06:45:25.442 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.x 2024-06-13T06:45:25.442 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.x.service: Unit ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.x.service not loaded. 2024-06-13T06:45:25.705 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-559474f0-2950-11ef-bc9c-c7b262605968.target.wants/ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-559474f0-2950-11ef-bc9c-c7b262605968@.service. 2024-06-13T06:45:25.722 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:25 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2515802420' entity='client.admin' 2024-06-13T06:45:26.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:25 smithi026 systemd[1]: Starting Ceph mgr.x for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:45:26.361 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:26 smithi026 podman[32266]: 2024-06-13 06:45:26.001743164 +0000 UTC m=+0.032955926 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:45:26.361 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:26 smithi026 podman[32266]: 2024-06-13 06:45:26.148297946 +0000 UTC m=+0.179510745 container create 0d1fbd0ba318f8bd48a0378be6dd95467d6e1c779fdd2291d0f959e7a4eab050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-13T06:45:26.361 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:26 smithi026 podman[32266]: 2024-06-13 06:45:26.360539205 +0000 UTC m=+0.391752057 container init 0d1fbd0ba318f8bd48a0378be6dd95467d6e1c779fdd2291d0f959e7a4eab050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, ceph=True) 2024-06-13T06:45:26.527 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-13T06:45:26.527 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-06-13T06:45:26.545 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-13T06:45:26.545 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-06-13T06:45:26.545 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-06-13T06:45:26.545 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2024-06-13T06:45:26.573 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-13T06:45:26.573 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-06-13T06:45:26.594 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-13T06:45:26.594 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-06-13T06:45:26.594 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-06-13T06:45:26.594 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-13T06:45:26.594 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr to start... 2024-06-13T06:45:26.594 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr... 2024-06-13T06:45:26.627 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:26 smithi026 podman[32266]: 2024-06-13 06:45:26.371422858 +0000 UTC m=+0.402635633 container start 0d1fbd0ba318f8bd48a0378be6dd95467d6e1c779fdd2291d0f959e7a4eab050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-13T06:45:26.627 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:26 smithi026 bash[32266]: 0d1fbd0ba318f8bd48a0378be6dd95467d6e1c779fdd2291d0f959e7a4eab050 2024-06-13T06:45:26.627 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:26 smithi026 systemd[1]: Started Ceph mgr.x for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:45:26.892 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:26.627+0000 7f5973dc0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:45:26.893 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:26.892+0000 7f5973dc0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:45:27.162 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:27.030+0000 7f5973dc0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:45:27.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:27 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1999900218' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-13T06:45:27.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:27.457+0000 7f5973dc0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:45:27.915 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:27.574+0000 7f5973dc0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:45:28.684 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:28.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:45:28.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:28.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-13T06:45:28.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-13T06:45:28.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-13T06:45:28.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-13T06:45:28.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-13T06:45:28.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:28.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:45:28.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-06-13T06:45:08.671890+0000", 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.231+0000 7f5a9e600700 1 Processor -- start 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.231+0000 7f5a9e600700 1 -- start start 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.232+0000 7f5a9e600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a9810ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.232+0000 7f5a9e600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9807be50 con 0x7f5a98108950 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.233+0000 7f5a97400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a9810ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:28.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.233+0000 7f5a97400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a9810ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40356/0 (socket says 172.21.15.26:40356) 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.233+0000 7f5a97400700 1 -- 172.21.15.26:0/2243665947 learned_addr learned my addr 172.21.15.26:0/2243665947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.233+0000 7f5a97400700 1 -- 172.21.15.26:0/2243665947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9810b280 con 0x7f5a98108950 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.233+0000 7f5a97400700 1 --2- 172.21.15.26:0/2243665947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a9810ad40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5a880098a0 tx=0x7f5a88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1411e480f55626a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.234+0000 7f5a96000700 1 -- 172.21.15.26:0/2243665947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88004160 con 0x7f5a98108950 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.234+0000 7f5a96000700 1 -- 172.21.15.26:0/2243665947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a88004300 con 0x7f5a98108950 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.234+0000 7f5a96000700 1 -- 172.21.15.26:0/2243665947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88011b50 con 0x7f5a98108950 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.234+0000 7f5a9e600700 1 -- 172.21.15.26:0/2243665947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 msgr2=0x7f5a9810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.234+0000 7f5a9e600700 1 --2- 172.21.15.26:0/2243665947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a9810ad40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5a880098a0 tx=0x7f5a88009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.235+0000 7f5a9e600700 1 -- 172.21.15.26:0/2243665947 shutdown_connections 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.235+0000 7f5a9e600700 1 --2- 172.21.15.26:0/2243665947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a9810ad40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.235+0000 7f5a9e600700 1 -- 172.21.15.26:0/2243665947 >> 172.21.15.26:0/2243665947 conn(0x7f5a981023c0 msgr2=0x7f5a981047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.235+0000 7f5a9e600700 1 -- 172.21.15.26:0/2243665947 shutdown_connections 2024-06-13T06:45:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.235+0000 7f5a9e600700 1 -- 172.21.15.26:0/2243665947 wait complete. 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.236+0000 7f5a9e600700 1 Processor -- start 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.236+0000 7f5a9e600700 1 -- start start 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.237+0000 7f5a9e600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a981081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.237+0000 7f5a9e600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9810c2a0 con 0x7f5a98108950 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.237+0000 7f5a97400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a981081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.237+0000 7f5a97400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a981081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40372/0 (socket says 172.21.15.26:40372) 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.237+0000 7f5a97400700 1 -- 172.21.15.26:0/1999900218 learned_addr learned my addr 172.21.15.26:0/1999900218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.238+0000 7f5a97400700 1 -- 172.21.15.26:0/1999900218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a98108710 con 0x7f5a98108950 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.238+0000 7f5a97400700 1 --2- 172.21.15.26:0/1999900218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a981081d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a88011670 tx=0x7f5a88011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.238+0000 7f5a8fe00700 1 -- 172.21.15.26:0/1999900218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a880039c0 con 0x7f5a98108950 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.238+0000 7f5a8fe00700 1 -- 172.21.15.26:0/1999900218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a88011e50 con 0x7f5a98108950 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.238+0000 7f5a8fe00700 1 -- 172.21.15.26:0/1999900218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a8801a5c0 con 0x7f5a98108950 2024-06-13T06:45:28.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.238+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a98106900 con 0x7f5a98108950 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.238+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a98106e40 con 0x7f5a98108950 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.239+0000 7f5a8fe00700 1 -- 172.21.15.26:0/1999900218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5a8801ac00 con 0x7f5a98108950 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.239+0000 7f5a8fe00700 1 -- 172.21.15.26:0/1999900218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5a880193c0 con 0x7f5a98108950 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.240+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a98063af0 con 0x7f5a98108950 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.242+0000 7f5a8fe00700 1 -- 172.21.15.26:0/1999900218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5a88003b60 con 0x7f5a98108950 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.323+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5a98107370 con 0x7f5a98108950 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.324+0000 7f5a8fe00700 1 -- 172.21.15.26:0/1999900218 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5a88024770 con 0x7f5a98108950 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.325+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 msgr2=0x7f5a981081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.325+0000 7f5a9e600700 1 --2- 172.21.15.26:0/1999900218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a981081d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a88011670 tx=0x7f5a88011b60 comp rx=0 tx=0).stop 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.325+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 shutdown_connections 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.325+0000 7f5a9e600700 1 --2- 172.21.15.26:0/1999900218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a98108950 0x7f5a981081d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.325+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 >> 172.21.15.26:0/1999900218 conn(0x7f5a981023c0 msgr2=0x7f5a98102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:28.702 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.326+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 shutdown_connections 2024-06-13T06:45:28.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:27.326+0000 7f5a9e600700 1 -- 172.21.15.26:0/1999900218 wait complete. 2024-06-13T06:45:28.703 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (1/15)... 2024-06-13T06:45:29.724 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:29.275+0000 7f5973dc0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:45:29.724 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:29.506+0000 7f5973dc0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:45:29.724 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:29.723+0000 7f5973dc0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:45:30.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:30.111+0000 7f5973dc0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:45:30.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:30.236+0000 7f5973dc0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:45:30.768 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:30.509+0000 7f5973dc0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:45:30.768 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:30.768+0000 7f5973dc0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:45:31.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:31 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2515425170' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-13T06:45:31.752 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:31.603+0000 7f5973dc0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:45:32.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:32.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:45:32.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T06:45:32.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-06-13T06:45:32.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-13T06:45:32.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-13T06:45:32.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-13T06:45:32.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:32.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-13T06:45:32.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-13T06:45:32.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-06-13T06:45:32.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:32.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-13T06:45:32.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-06-13T06:45:32.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:32.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-06-13T06:45:32.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-13T06:45:32.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:32.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-13T06:45:32.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-13T06:45:32.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:32.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-13T06:45:32.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:32.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-13T06:45:32.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-13T06:45:32.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-13T06:45:32.307 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-13T06:45:32.308 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-13T06:45:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-06-13T06:45:08.671890+0000", 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-13T06:45:32.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.359+0000 7fefe8c00700 1 Processor -- start 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.359+0000 7fefe8c00700 1 -- start start 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.360+0000 7fefe8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe4108180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.360+0000 7fefe8c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefe4108750 con 0x7fefe4107d60 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.360+0000 7fefe3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe4108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.360+0000 7fefe3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe4108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40386/0 (socket says 172.21.15.26:40386) 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.360+0000 7fefe3400700 1 -- 172.21.15.26:0/1875805325 learned_addr learned my addr 172.21.15.26:0/1875805325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.361+0000 7fefe3400700 1 -- 172.21.15.26:0/1875805325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe4108f10 con 0x7fefe4107d60 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.361+0000 7fefe3400700 1 --2- 172.21.15.26:0/1875805325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe4108180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefd40098a0 tx=0x7fefd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f98dda37b9c767a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.361+0000 7fefe2000700 1 -- 172.21.15.26:0/1875805325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefd4004160 con 0x7fefe4107d60 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.361+0000 7fefe2000700 1 -- 172.21.15.26:0/1875805325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fefd4004300 con 0x7fefe4107d60 2024-06-13T06:45:32.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.361+0000 7fefe2000700 1 -- 172.21.15.26:0/1875805325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefd4011b50 con 0x7fefe4107d60 2024-06-13T06:45:32.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.362+0000 7fefe8c00700 1 -- 172.21.15.26:0/1875805325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 msgr2=0x7fefe4108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:32.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.362+0000 7fefe8c00700 1 --2- 172.21.15.26:0/1875805325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe4108180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefd40098a0 tx=0x7fefd4009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:32.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.362+0000 7fefe8c00700 1 -- 172.21.15.26:0/1875805325 shutdown_connections 2024-06-13T06:45:32.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.362+0000 7fefe8c00700 1 --2- 172.21.15.26:0/1875805325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe4108180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:32.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.362+0000 7fefe8c00700 1 -- 172.21.15.26:0/1875805325 >> 172.21.15.26:0/1875805325 conn(0x7fefe4103360 msgr2=0x7fefe4105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.362+0000 7fefe8c00700 1 -- 172.21.15.26:0/1875805325 shutdown_connections 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.362+0000 7fefe8c00700 1 -- 172.21.15.26:0/1875805325 wait complete. 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.363+0000 7fefe8c00700 1 Processor -- start 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.364+0000 7fefe8c00700 1 -- start start 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.364+0000 7fefe8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.364+0000 7fefe8c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefe410c2c0 con 0x7fefe4107d60 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.364+0000 7fefe3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.364+0000 7fefe3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40400/0 (socket says 172.21.15.26:40400) 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.364+0000 7fefe3400700 1 -- 172.21.15.26:0/2515425170 learned_addr learned my addr 172.21.15.26:0/2515425170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.365+0000 7fefe3400700 1 -- 172.21.15.26:0/2515425170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe419e4d0 con 0x7fefe4107d60 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.365+0000 7fefe3400700 1 --2- 172.21.15.26:0/2515425170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe419df90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009fb0 tx=0x7fefd4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.365+0000 7fefdbe00700 1 -- 172.21.15.26:0/2515425170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefd40038b0 con 0x7fefe4107d60 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.365+0000 7fefdbe00700 1 -- 172.21.15.26:0/2515425170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fefd4011e50 con 0x7fefe4107d60 2024-06-13T06:45:32.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.365+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe419e760 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.365+0000 7fefdbe00700 1 -- 172.21.15.26:0/2515425170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefd4019550 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.365+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe41a1450 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.366+0000 7fefdbe00700 1 -- 172.21.15.26:0/2515425170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fefd4019b90 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.366+0000 7fefdbe00700 1 -- 172.21.15.26:0/2515425170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefd4024730 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.367+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc4005330 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.369+0000 7fefdbe00700 1 -- 172.21.15.26:0/2515425170 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fefd4003a50 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.449+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fefc40058f0 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.450+0000 7fefdbe00700 1 -- 172.21.15.26:0/2515425170 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fefd4024930 con 0x7fefe4107d60 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.451+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 msgr2=0x7fefe419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.451+0000 7fefe8c00700 1 --2- 172.21.15.26:0/2515425170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe419df90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009fb0 tx=0x7fefd4011bf0 comp rx=0 tx=0).stop 2024-06-13T06:45:32.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.452+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 shutdown_connections 2024-06-13T06:45:32.315 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.452+0000 7fefe8c00700 1 --2- 172.21.15.26:0/2515425170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe4107d60 0x7fefe419df90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:32.315 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.452+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 >> 172.21.15.26:0/2515425170 conn(0x7fefe4103360 msgr2=0x7fefe4103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:32.315 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.452+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 shutdown_connections 2024-06-13T06:45:32.315 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:31.452+0000 7fefe8c00700 1 -- 172.21.15.26:0/2515425170 wait complete. 2024-06-13T06:45:32.315 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (2/15)... 2024-06-13T06:45:32.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:32.174+0000 7f5973dc0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:45:32.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:32.307+0000 7f5973dc0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:45:33.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:32.578+0000 7f5973dc0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:45:33.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:33.243+0000 7f5973dc0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:45:33.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:33.368+0000 7f5973dc0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:45:34.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:33.695+0000 7f5973dc0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:45:34.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:33.879+0000 7f5973dc0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:45:34.460 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:34.007+0000 7f5973dc0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:45:34.460 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:34.162+0000 7f5973dc0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:45:34.752 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:34.540+0000 7f5973dc0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:45:35.353 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:35 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3201788838' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-13T06:45:35.663 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:35.352+0000 7f5973dc0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:45:35.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:35.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:35.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-13T06:45:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-13T06:45:35.746 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-13T06:45:35.747 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-06-13T06:45:08.671890+0000", 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.924+0000 7f88db600700 1 Processor -- start 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.925+0000 7f88db600700 1 -- start start 2024-06-13T06:45:35.748 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.925+0000 7f88db600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d41082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.925+0000 7f88db600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4108820 con 0x7f88d4107ec0 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.926+0000 7f88d8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d41082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.926+0000 7f88d8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d41082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46908/0 (socket says 172.21.15.26:46908) 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.926+0000 7f88d8c00700 1 -- 172.21.15.26:0/2846910071 learned_addr learned my addr 172.21.15.26:0/2846910071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.926+0000 7f88d8c00700 1 -- 172.21.15.26:0/2846910071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d4109060 con 0x7f88d4107ec0 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.926+0000 7f88d8c00700 1 --2- 172.21.15.26:0/2846910071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d41082e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009e80 tx=0x7f88c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d130c9d870ea81 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.927+0000 7f88d3e00700 1 -- 172.21.15.26:0/2846910071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88c4017070 con 0x7f88d4107ec0 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.927+0000 7f88d3e00700 1 -- 172.21.15.26:0/2846910071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f88c400b8c0 con 0x7f88d4107ec0 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.927+0000 7f88d3e00700 1 -- 172.21.15.26:0/2846910071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88c40045c0 con 0x7f88d4107ec0 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.928+0000 7f88db600700 1 -- 172.21.15.26:0/2846910071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 msgr2=0x7f88d41082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.928+0000 7f88db600700 1 --2- 172.21.15.26:0/2846910071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d41082e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009e80 tx=0x7f88c400b330 comp rx=0 tx=0).stop 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.928+0000 7f88db600700 1 -- 172.21.15.26:0/2846910071 shutdown_connections 2024-06-13T06:45:35.749 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.928+0000 7f88db600700 1 --2- 172.21.15.26:0/2846910071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d41082e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.928+0000 7f88db600700 1 -- 172.21.15.26:0/2846910071 >> 172.21.15.26:0/2846910071 conn(0x7f88d41034d0 msgr2=0x7f88d41058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.928+0000 7f88db600700 1 -- 172.21.15.26:0/2846910071 shutdown_connections 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.928+0000 7f88db600700 1 -- 172.21.15.26:0/2846910071 wait complete. 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.930+0000 7f88db600700 1 Processor -- start 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.930+0000 7f88db600700 1 -- start start 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.930+0000 7f88db600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d4199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.930+0000 7f88db600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88d407c0c0 con 0x7f88d4107ec0 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.930+0000 7f88d8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d4199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.931+0000 7f88d8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d4199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46924/0 (socket says 172.21.15.26:46924) 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.931+0000 7f88d8c00700 1 -- 172.21.15.26:0/3201788838 learned_addr learned my addr 172.21.15.26:0/3201788838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.931+0000 7f88d8c00700 1 -- 172.21.15.26:0/3201788838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d419a1a0 con 0x7f88d4107ec0 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.931+0000 7f88d8c00700 1 --2- 172.21.15.26:0/3201788838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d4199c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f88c4004210 tx=0x7f88c400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.932+0000 7f88d2000700 1 -- 172.21.15.26:0/3201788838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88c4020020 con 0x7f88d4107ec0 2024-06-13T06:45:35.750 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.932+0000 7f88d2000700 1 -- 172.21.15.26:0/3201788838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f88c4003b50 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.932+0000 7f88d2000700 1 -- 172.21.15.26:0/3201788838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88c4017040 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.932+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d419a430 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.932+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d419d120 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.932+0000 7f88d2000700 1 -- 172.21.15.26:0/3201788838 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f88c4023020 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.932+0000 7f88d2000700 1 -- 172.21.15.26:0/3201788838 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f88c400f050 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.933+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d4063af0 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:34.936+0000 7f88d2000700 1 -- 172.21.15.26:0/3201788838 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f88c402a020 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.015+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f88d402de60 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.015+0000 7f88d2000700 1 -- 172.21.15.26:0/3201788838 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f88c400b860 con 0x7f88d4107ec0 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.017+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 msgr2=0x7f88d4199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.017+0000 7f88db600700 1 --2- 172.21.15.26:0/3201788838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d4199c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f88c4004210 tx=0x7f88c400bd50 comp rx=0 tx=0).stop 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.017+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 shutdown_connections 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.017+0000 7f88db600700 1 --2- 172.21.15.26:0/3201788838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4107ec0 0x7f88d4199c60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.017+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 >> 172.21.15.26:0/3201788838 conn(0x7f88d41034d0 msgr2=0x7f88d41057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:35.751 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.017+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 shutdown_connections 2024-06-13T06:45:35.752 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:35.017+0000 7f88db600700 1 -- 172.21.15.26:0/3201788838 wait complete. 2024-06-13T06:45:35.752 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (3/15)... 2024-06-13T06:45:36.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:35.661+0000 7f5973dc0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:45:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: Activating manager daemon x 2024-06-13T06:45:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: mgrmap e2: x(active, starting, since 0.00702713s) 2024-06-13T06:45:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: Manager daemon x is now available 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-13T06:45:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:36 smithi026 ceph-mon[32041]: from='mgr.14100 172.21.15.26:0/587344644' entity='mgr.x' 2024-06-13T06:45:38.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:37 smithi026 ceph-mon[32041]: mgrmap e3: x(active, since 1.02139s) 2024-06-13T06:45:38.998 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:38 smithi026 ceph-mon[32041]: mgrmap e4: x(active, since 2s) 2024-06-13T06:45:38.998 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:38 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1941001369' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-13T06:45:39.383 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:39.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:45:39.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T06:45:39.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-06-13T06:45:39.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-13T06:45:39.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-13T06:45:39.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-13T06:45:39.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:39.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-13T06:45:39.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-13T06:45:39.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-06-13T06:45:39.386 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:39.386 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-13T06:45:39.386 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-06-13T06:45:39.386 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:39.386 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-06-13T06:45:39.387 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-13T06:45:39.387 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:39.387 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-13T06:45:39.387 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-13T06:45:39.388 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:39.388 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-13T06:45:39.388 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:39.388 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-13T06:45:39.388 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-13T06:45:39.388 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-13T06:45:39.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-13T06:45:39.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-13T06:45:39.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-13T06:45:39.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:39.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-13T06:45:39.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-13T06:45:39.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:39.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-13T06:45:39.392 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-13T06:45:39.392 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:39.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-13T06:45:39.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-13T06:45:39.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-13T06:45:39.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-13T06:45:39.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-06-13T06:45:08.671890+0000", 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.351+0000 7f24efe00700 1 Processor -- start 2024-06-13T06:45:39.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.351+0000 7f24efe00700 1 -- start start 2024-06-13T06:45:39.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.351+0000 7f24efe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f010a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:39.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.351+0000 7f24efe00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010a9f0 con 0x7f24f010a090 2024-06-13T06:45:39.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.352+0000 7f24eea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f010a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:39.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.352+0000 7f24eea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f010a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46998/0 (socket says 172.21.15.26:46998) 2024-06-13T06:45:39.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.352+0000 7f24eea00700 1 -- 172.21.15.26:0/236830536 learned_addr learned my addr 172.21.15.26:0/236830536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:39.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.352+0000 7f24eea00700 1 -- 172.21.15.26:0/236830536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f010ab70 con 0x7f24f010a090 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.353+0000 7f24eea00700 1 --2- 172.21.15.26:0/236830536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f010a4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24e00098a0 tx=0x7f24e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbae6739bf50e37 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.353+0000 7f24ed600700 1 -- 172.21.15.26:0/236830536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24e0004160 con 0x7f24f010a090 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.353+0000 7f24ed600700 1 -- 172.21.15.26:0/236830536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f24e0004300 con 0x7f24f010a090 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.353+0000 7f24ed600700 1 -- 172.21.15.26:0/236830536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24e0011b50 con 0x7f24f010a090 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.354+0000 7f24efe00700 1 -- 172.21.15.26:0/236830536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 msgr2=0x7f24f010a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.354+0000 7f24efe00700 1 --2- 172.21.15.26:0/236830536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f010a4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24e00098a0 tx=0x7f24e0009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.354+0000 7f24efe00700 1 -- 172.21.15.26:0/236830536 shutdown_connections 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.354+0000 7f24efe00700 1 --2- 172.21.15.26:0/236830536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f010a4b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:39.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.354+0000 7f24efe00700 1 -- 172.21.15.26:0/236830536 >> 172.21.15.26:0/236830536 conn(0x7f24f007d2e0 msgr2=0x7f24f007d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.354+0000 7f24efe00700 1 -- 172.21.15.26:0/236830536 shutdown_connections 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.354+0000 7f24efe00700 1 -- 172.21.15.26:0/236830536 wait complete. 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.355+0000 7f24efe00700 1 Processor -- start 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.355+0000 7f24efe00700 1 -- start start 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.355+0000 7f24efe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.355+0000 7f24efe00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010bf20 con 0x7f24f010a090 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.356+0000 7f24eea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.356+0000 7f24eea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f019ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47012/0 (socket says 172.21.15.26:47012) 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.356+0000 7f24eea00700 1 -- 172.21.15.26:0/1941001369 learned_addr learned my addr 172.21.15.26:0/1941001369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.356+0000 7f24eea00700 1 -- 172.21.15.26:0/1941001369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f019e410 con 0x7f24f010a090 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.356+0000 7f24eea00700 1 --2- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f019ded0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24e0011850 tx=0x7f24e0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.357+0000 7f24e7400700 1 -- 172.21.15.26:0/1941001369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24e0003af0 con 0x7f24f010a090 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.357+0000 7f24e7400700 1 -- 172.21.15.26:0/1941001369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f24e0020070 con 0x7f24f010a090 2024-06-13T06:45:39.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.357+0000 7f24e7400700 1 -- 172.21.15.26:0/1941001369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24e0019550 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.357+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f019e700 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.357+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01a1390 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.358+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f0063af0 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.358+0000 7f24e7400700 1 -- 172.21.15.26:0/1941001369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f24e0019800 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.359+0000 7f24e7400700 1 --2- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f24d4039050 0x7f24d403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.359+0000 7f24e7400700 1 -- 172.21.15.26:0/1941001369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f24e0011e00 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.359+0000 7f24ee000700 1 --2- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f24d4039050 0x7f24d403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.360+0000 7f24ee000700 1 --2- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f24d4039050 0x7f24d403b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24dc006fd0 tx=0x7f24dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.366+0000 7f24e7400700 1 -- 172.21.15.26:0/1941001369 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f24e000e260 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.678+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f24f01a1a80 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.679+0000 7f24e7400700 1 -- 172.21.15.26:0/1941001369 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f24e004cdc0 con 0x7f24f010a090 2024-06-13T06:45:39.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f24d4039050 msgr2=0x7f24d403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 --2- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f24d4039050 0x7f24d403b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24dc006fd0 tx=0x7f24dc006e40 comp rx=0 tx=0).stop 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 msgr2=0x7f24f019ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 --2- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f019ded0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24e0011850 tx=0x7f24e0011c10 comp rx=0 tx=0).stop 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 shutdown_connections 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 --2- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f24d4039050 0x7f24d403b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 --2- 172.21.15.26:0/1941001369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f010a090 0x7f24f019ded0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 >> 172.21.15.26:0/1941001369 conn(0x7f24f007d2e0 msgr2=0x7f24f0107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.683+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 shutdown_connections 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:38.684+0000 7f24efe00700 1 -- 172.21.15.26:0/1941001369 wait complete. 2024-06-13T06:45:39.404 INFO:teuthology.orchestra.run.smithi026.stdout:mgr is available 2024-06-13T06:45:40.665 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:40 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3105825203' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-13T06:45:41.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:41.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2024-06-13T06:45:41.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:41.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-13T06:45:41.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-13T06:45:41.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-13T06:45:41.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.038+0000 7fbc75800700 1 Processor -- start 2024-06-13T06:45:41.033 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.039+0000 7fbc75800700 1 -- start start 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.039+0000 7fbc75800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc70106b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.039+0000 7fbc75800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7007c700 con 0x7fbc70106770 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.040+0000 7fbc6ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc70106b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.040+0000 7fbc6ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc70106b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47014/0 (socket says 172.21.15.26:47014) 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.040+0000 7fbc6ea00700 1 -- 172.21.15.26:0/2155124682 learned_addr learned my addr 172.21.15.26:0/2155124682 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.040+0000 7fbc6ea00700 1 -- 172.21.15.26:0/2155124682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc701070d0 con 0x7fbc70106770 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.040+0000 7fbc6ea00700 1 --2- 172.21.15.26:0/2155124682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc70106b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbc600098a0 tx=0x7fbc60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6ac5da981f245 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.041+0000 7fbc6d600700 1 -- 172.21.15.26:0/2155124682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc60004160 con 0x7fbc70106770 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.041+0000 7fbc6d600700 1 -- 172.21.15.26:0/2155124682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc60004300 con 0x7fbc70106770 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.041+0000 7fbc6d600700 1 -- 172.21.15.26:0/2155124682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc60011b50 con 0x7fbc70106770 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.041+0000 7fbc75800700 1 -- 172.21.15.26:0/2155124682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 msgr2=0x7fbc70106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:41.034 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.041+0000 7fbc75800700 1 --2- 172.21.15.26:0/2155124682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc70106b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbc600098a0 tx=0x7fbc60009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.042+0000 7fbc75800700 1 -- 172.21.15.26:0/2155124682 shutdown_connections 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.042+0000 7fbc75800700 1 --2- 172.21.15.26:0/2155124682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc70106b90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.042+0000 7fbc75800700 1 -- 172.21.15.26:0/2155124682 >> 172.21.15.26:0/2155124682 conn(0x7fbc701023f0 msgr2=0x7fbc70104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.042+0000 7fbc75800700 1 -- 172.21.15.26:0/2155124682 shutdown_connections 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.042+0000 7fbc75800700 1 -- 172.21.15.26:0/2155124682 wait complete. 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.043+0000 7fbc75800700 1 Processor -- start 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.043+0000 7fbc75800700 1 -- start start 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.043+0000 7fbc75800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc7019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.043+0000 7fbc75800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70108160 con 0x7fbc70106770 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.044+0000 7fbc6ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc7019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.044+0000 7fbc6ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc7019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47024/0 (socket says 172.21.15.26:47024) 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.044+0000 7fbc6ea00700 1 -- 172.21.15.26:0/3105825203 learned_addr learned my addr 172.21.15.26:0/3105825203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:41.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.044+0000 7fbc6ea00700 1 -- 172.21.15.26:0/3105825203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7019e610 con 0x7fbc70106770 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.044+0000 7fbc6ea00700 1 --2- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc7019e0d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbc60011b60 tx=0x7fbc60011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.045+0000 7fbc67400700 1 -- 172.21.15.26:0/3105825203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc60003970 con 0x7fbc70106770 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.045+0000 7fbc67400700 1 -- 172.21.15.26:0/3105825203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc60003b10 con 0x7fbc70106770 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.045+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7019e8a0 con 0x7fbc70106770 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.045+0000 7fbc67400700 1 -- 172.21.15.26:0/3105825203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc60024610 con 0x7fbc70106770 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.045+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc701a1590 con 0x7fbc70106770 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.046+0000 7fbc67400700 1 -- 172.21.15.26:0/3105825203 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fbc600248c0 con 0x7fbc70106770 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.046+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc70063af0 con 0x7fbc70106770 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.047+0000 7fbc67400700 1 --2- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7fbc54039050 0x7fbc5403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:41.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.047+0000 7fbc67400700 1 -- 172.21.15.26:0/3105825203 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc60011e00 con 0x7fbc70106770 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.047+0000 7fbc6e000700 1 --2- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7fbc54039050 0x7fbc5403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.048+0000 7fbc6e000700 1 --2- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7fbc54039050 0x7fbc5403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c006fd0 tx=0x7fbc5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.052+0000 7fbc67400700 1 -- 172.21.15.26:0/3105825203 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc6000bcc0 con 0x7fbc70106770 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.283+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbc7002df10 con 0x7fbc70106770 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.284+0000 7fbc67400700 1 -- 172.21.15.26:0/3105825203 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fbc6004cd30 con 0x7fbc70106770 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.287+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7fbc54039050 msgr2=0x7fbc5403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.287+0000 7fbc75800700 1 --2- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7fbc54039050 0x7fbc5403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c006fd0 tx=0x7fbc5c006e40 comp rx=0 tx=0).stop 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.287+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 msgr2=0x7fbc7019e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.287+0000 7fbc75800700 1 --2- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc7019e0d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbc60011b60 tx=0x7fbc60011c40 comp rx=0 tx=0).stop 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.288+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 shutdown_connections 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.288+0000 7fbc75800700 1 --2- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7fbc54039050 0x7fbc5403b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.288+0000 7fbc75800700 1 --2- 172.21.15.26:0/3105825203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc70106770 0x7fbc7019e0d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:41.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.288+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 >> 172.21.15.26:0/3105825203 conn(0x7fbc701023f0 msgr2=0x7fbc70194d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:41.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.288+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 shutdown_connections 2024-06-13T06:45:41.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:40.288+0000 7fbc75800700 1 -- 172.21.15.26:0/3105825203 wait complete. 2024-06-13T06:45:41.038 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling cephadm module... 2024-06-13T06:45:42.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:42 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2246877358' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-13T06:45:42.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: ignoring --setuser ceph since I am not root 2024-06-13T06:45:42.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: ignoring --setgroup ceph since I am not root 2024-06-13T06:45:42.575 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:42.323+0000 7f2672a15200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:45:42.575 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:42.575+0000 7f2672a15200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:45:42.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.661+0000 7f4866200700 1 Processor -- start 2024-06-13T06:45:42.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.662+0000 7f4866200700 1 -- start start 2024-06-13T06:45:42.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.662+0000 7f4866200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f4860108300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:42.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.662+0000 7f4866200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4860108840 con 0x7f4860107ee0 2024-06-13T06:45:42.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.662+0000 7f485f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f4860108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:42.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.662+0000 7f485f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f4860108300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47034/0 (socket says 172.21.15.26:47034) 2024-06-13T06:45:42.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.662+0000 7f485f400700 1 -- 172.21.15.26:0/3157762433 learned_addr learned my addr 172.21.15.26:0/3157762433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.663+0000 7f485f400700 1 -- 172.21.15.26:0/3157762433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4860108ff0 con 0x7f4860107ee0 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.663+0000 7f485f400700 1 --2- 172.21.15.26:0/3157762433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f4860108300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4850009e80 tx=0x7f485000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d55adc7d2b1f38e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.663+0000 7f485e000700 1 -- 172.21.15.26:0/3157762433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4850017070 con 0x7f4860107ee0 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.664+0000 7f485e000700 1 -- 172.21.15.26:0/3157762433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f485000b8c0 con 0x7f4860107ee0 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.664+0000 7f485e000700 1 -- 172.21.15.26:0/3157762433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48500045c0 con 0x7f4860107ee0 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.664+0000 7f4866200700 1 -- 172.21.15.26:0/3157762433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 msgr2=0x7f4860108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.664+0000 7f4866200700 1 --2- 172.21.15.26:0/3157762433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f4860108300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4850009e80 tx=0x7f485000b330 comp rx=0 tx=0).stop 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.664+0000 7f4866200700 1 -- 172.21.15.26:0/3157762433 shutdown_connections 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.664+0000 7f4866200700 1 --2- 172.21.15.26:0/3157762433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f4860108300 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.664+0000 7f4866200700 1 -- 172.21.15.26:0/3157762433 >> 172.21.15.26:0/3157762433 conn(0x7f4860103460 msgr2=0x7f48601058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.665+0000 7f4866200700 1 -- 172.21.15.26:0/3157762433 shutdown_connections 2024-06-13T06:45:42.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.665+0000 7f4866200700 1 -- 172.21.15.26:0/3157762433 wait complete. 2024-06-13T06:45:42.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.666+0000 7f4866200700 1 Processor -- start 2024-06-13T06:45:42.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.666+0000 7f4866200700 1 -- start start 2024-06-13T06:45:42.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.666+0000 7f4866200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f486007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:42.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.666+0000 7f4866200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f486010c430 con 0x7f4860107ee0 2024-06-13T06:45:42.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.667+0000 7f485f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f486007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:42.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.667+0000 7f485f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f486007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47046/0 (socket says 172.21.15.26:47046) 2024-06-13T06:45:42.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.667+0000 7f485f400700 1 -- 172.21.15.26:0/2246877358 learned_addr learned my addr 172.21.15.26:0/2246877358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:42.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.667+0000 7f485f400700 1 -- 172.21.15.26:0/2246877358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486007e090 con 0x7f4860107ee0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.667+0000 7f485f400700 1 --2- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f486007db50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4850009530 tx=0x7f485000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.668+0000 7f4857e00700 1 -- 172.21.15.26:0/2246877358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4850020020 con 0x7f4860107ee0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.668+0000 7f4857e00700 1 -- 172.21.15.26:0/2246877358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4850003b50 con 0x7f4860107ee0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.668+0000 7f4857e00700 1 -- 172.21.15.26:0/2246877358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4850017040 con 0x7f4860107ee0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.668+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486007c280 con 0x7f4860107ee0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.668+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486007c7c0 con 0x7f4860107ee0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.669+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4860063af0 con 0x7f4860107ee0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.670+0000 7f4857e00700 1 -- 172.21.15.26:0/2246877358 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f4850023020 con 0x7f4860107ee0 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.670+0000 7f4857e00700 1 --2- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f4844039050 0x7f484403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:42.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.670+0000 7f4857e00700 1 -- 172.21.15.26:0/2246877358 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f485000f050 con 0x7f4860107ee0 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.670+0000 7f485ea00700 1 --2- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f4844039050 0x7f484403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.671+0000 7f485ea00700 1 --2- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f4844039050 0x7f484403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f484c006fd0 tx=0x7f484c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.676+0000 7f4857e00700 1 -- 172.21.15.26:0/2246877358 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f485002a070 con 0x7f4860107ee0 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:41.964+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f486007cf10 con 0x7f4860107ee0 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.018+0000 7f4857e00700 1 -- 172.21.15.26:0/2246877358 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f4850003db0 con 0x7f4860107ee0 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.021+0000 7f4857e00700 1 -- 172.21.15.26:0/2246877358 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f4850012480 con 0x7f4860107ee0 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.021+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f4844039050 msgr2=0x7f484403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.021+0000 7f4866200700 1 --2- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f4844039050 0x7f484403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f484c006fd0 tx=0x7f484c006e40 comp rx=0 tx=0).stop 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.021+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 msgr2=0x7f486007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.021+0000 7f4866200700 1 --2- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f486007db50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4850009530 tx=0x7f485000bd50 comp rx=0 tx=0).stop 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.022+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 shutdown_connections 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.022+0000 7f4866200700 1 --2- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f4844039050 0x7f484403b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.022+0000 7f4866200700 1 --2- 172.21.15.26:0/2246877358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4860107ee0 0x7f486007db50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:42.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.022+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 >> 172.21.15.26:0/2246877358 conn(0x7f4860103460 msgr2=0x7f4860104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:42.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.023+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 shutdown_connections 2024-06-13T06:45:42.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:42.023+0000 7f4866200700 1 -- 172.21.15.26:0/2246877358 wait complete. 2024-06-13T06:45:42.886 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:42.712+0000 7f2672a15200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:45:43.180 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:43 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2246877358' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-13T06:45:43.180 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:43 smithi026 ceph-mon[32041]: mgrmap e5: x(active, since 6s) 2024-06-13T06:45:43.180 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:43 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:43.133+0000 7f2672a15200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:45:43.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:43 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:43.253+0000 7f2672a15200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:45:44.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:44 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/125105950' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-13T06:45:44.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:45:44.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-13T06:45:44.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-13T06:45:44.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-13T06:45:44.389 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-13T06:45:44.389 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:45:44.389 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.335+0000 7f0dc1e00700 1 Processor -- start 2024-06-13T06:45:44.389 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.335+0000 7f0dc1e00700 1 -- start start 2024-06-13T06:45:44.389 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.336+0000 7f0dc1e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:44.389 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.336+0000 7f0dc1e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc07c6d0 con 0x7f0dbc1065b0 2024-06-13T06:45:44.389 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.336+0000 7f0dc0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.336+0000 7f0dc0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35606/0 (socket says 172.21.15.26:35606) 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.336+0000 7f0dc0a00700 1 -- 172.21.15.26:0/115053259 learned_addr learned my addr 172.21.15.26:0/115053259 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.337+0000 7f0dc0a00700 1 -- 172.21.15.26:0/115053259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc108f20 con 0x7f0dbc1065b0 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.337+0000 7f0dc0a00700 1 --2- 172.21.15.26:0/115053259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1089e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0db00098a0 tx=0x7f0db0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c5f7e60515b703 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.337+0000 7f0dbb400700 1 -- 172.21.15.26:0/115053259 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db0004160 con 0x7f0dbc1065b0 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.337+0000 7f0dbb400700 1 -- 172.21.15.26:0/115053259 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0db0004300 con 0x7f0dbc1065b0 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.337+0000 7f0dbb400700 1 -- 172.21.15.26:0/115053259 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db0011b50 con 0x7f0dbc1065b0 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.338+0000 7f0dc1e00700 1 -- 172.21.15.26:0/115053259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 msgr2=0x7f0dbc1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.338+0000 7f0dc1e00700 1 --2- 172.21.15.26:0/115053259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1089e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0db00098a0 tx=0x7f0db0009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.338+0000 7f0dc1e00700 1 -- 172.21.15.26:0/115053259 shutdown_connections 2024-06-13T06:45:44.390 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.338+0000 7f0dc1e00700 1 --2- 172.21.15.26:0/115053259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1089e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.338+0000 7f0dc1e00700 1 -- 172.21.15.26:0/115053259 >> 172.21.15.26:0/115053259 conn(0x7f0dbc1021f0 msgr2=0x7f0dbc104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.338+0000 7f0dc1e00700 1 -- 172.21.15.26:0/115053259 shutdown_connections 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.338+0000 7f0dc1e00700 1 -- 172.21.15.26:0/115053259 wait complete. 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.339+0000 7f0dc1e00700 1 Processor -- start 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.340+0000 7f0dc1e00700 1 -- start start 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.340+0000 7f0dc1e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.340+0000 7f0dc1e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc109fb0 con 0x7f0dbc1065b0 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.340+0000 7f0dc0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.340+0000 7f0dc0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35614/0 (socket says 172.21.15.26:35614) 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.340+0000 7f0dc0a00700 1 -- 172.21.15.26:0/125105950 learned_addr learned my addr 172.21.15.26:0/125105950 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.341+0000 7f0dc0a00700 1 -- 172.21.15.26:0/125105950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc1a2870 con 0x7f0dbc1065b0 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.341+0000 7f0dc0a00700 1 --2- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1a2330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0db0011b60 tx=0x7f0db0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:44.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.341+0000 7f0db9600700 1 -- 172.21.15.26:0/125105950 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db0003970 con 0x7f0dbc1065b0 2024-06-13T06:45:44.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.341+0000 7f0db9600700 1 -- 172.21.15.26:0/125105950 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0db0003b10 con 0x7f0dbc1065b0 2024-06-13T06:45:44.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.341+0000 7f0db9600700 1 -- 172.21.15.26:0/125105950 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db0024610 con 0x7f0dbc1065b0 2024-06-13T06:45:44.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.341+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc1a2b00 con 0x7f0dbc1065b0 2024-06-13T06:45:44.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.342+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc1a57f0 con 0x7f0dbc1065b0 2024-06-13T06:45:44.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.343+0000 7f0db9600700 1 -- 172.21.15.26:0/125105950 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0db001a9f0 con 0x7f0dbc1065b0 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.343+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dbc19b890 con 0x7f0dbc1065b0 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.343+0000 7f0db9600700 1 --2- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f0da40390a0 0x7f0da403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.343+0000 7f0db9600700 1 -- 172.21.15.26:0/125105950 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0db0011e00 con 0x7f0dbc1065b0 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.343+0000 7f0dbbe00700 1 -- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f0da40390a0 msgr2=0x7f0da403b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2776242437 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.343+0000 7f0dbbe00700 1 --2- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f0da40390a0 0x7f0da403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.353+0000 7f0db9600700 1 -- 172.21.15.26:0/125105950 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0db0016260 con 0x7f0dbc1065b0 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.544+0000 7f0dbbe00700 1 -- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f0da40390a0 msgr2=0x7f0da403b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2776242437 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.544+0000 7f0dbbe00700 1 --2- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f0da40390a0 0x7f0da403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.641+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0dbc02df50 con 0x7f0dbc1065b0 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.642+0000 7f0db9600700 1 -- 172.21.15.26:0/125105950 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f0db0019e10 con 0x7f0dbc1065b0 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.645+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f0da40390a0 msgr2=0x7f0da403b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.645+0000 7f0dc1e00700 1 --2- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f0da40390a0 0x7f0da403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:44.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.645+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 msgr2=0x7f0dbc1a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.645+0000 7f0dc1e00700 1 --2- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1a2330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0db0011b60 tx=0x7f0db0011c40 comp rx=0 tx=0).stop 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.645+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 shutdown_connections 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.645+0000 7f0dc1e00700 1 --2- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7f0da40390a0 0x7f0da403b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.645+0000 7f0dc1e00700 1 --2- 172.21.15.26:0/125105950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dbc1065b0 0x7f0dbc1a2330 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.645+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 >> 172.21.15.26:0/125105950 conn(0x7f0dbc1021f0 msgr2=0x7f0dbc104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.646+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 shutdown_connections 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:43.646+0000 7f0dc1e00700 1 -- 172.21.15.26:0/125105950 wait complete. 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2024-06-13T06:45:44.396 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 5... 2024-06-13T06:45:44.924 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:44 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:44.898+0000 7f2672a15200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:45:45.176 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:45.133+0000 7f2672a15200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:45:45.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:45.356+0000 7f2672a15200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:45:45.877 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:45.750+0000 7f2672a15200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:45:45.878 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:45.877+0000 7f2672a15200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:45:46.402 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:46 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:46.138+0000 7f2672a15200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:45:46.752 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:46 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:46.402+0000 7f2672a15200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:45:47.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:47 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:47.224+0000 7f2672a15200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:45:48.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:47 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:47.795+0000 7f2672a15200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:45:48.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:47 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:47.928+0000 7f2672a15200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:45:48.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:48 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:48.203+0000 7f2672a15200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:45:49.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:48 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:48.884+0000 7f2672a15200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:45:49.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:49 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:49.011+0000 7f2672a15200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:45:49.667 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:49 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:49.349+0000 7f2672a15200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:45:49.667 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:49 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:49.536+0000 7f2672a15200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:45:49.667 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:49 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:49.667+0000 7f2672a15200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:45:50.001 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:49 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:49.826+0000 7f2672a15200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:45:50.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:50 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:50.204+0000 7f2672a15200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:45:51.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:50 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:50.994+0000 7f2672a15200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:45:51.752 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:45:51.295+0000 7f2672a15200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:45:51.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: Active manager daemon x restarted 2024-06-13T06:45:51.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: Activating manager daemon x 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: osdmap e2: 0 total, 0 up, 0 in 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: mgrmap e6: x(active, starting, since 0.00914138s) 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: Manager daemon x is now available 2024-06-13T06:45:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:51 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:45:52.680 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: Found migration_current of "None". Setting to last migration. 2024-06-13T06:45:52.680 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:45:52.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:52.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:52.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-13T06:45:52.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-13T06:45:52.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:45:52.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:45:52.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:52 smithi026 ceph-mon[32041]: mgrmap e7: x(active, since 1.01131s) 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.967+0000 7ff191600700 1 Processor -- start 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.967+0000 7ff191600700 1 -- start start 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.968+0000 7ff191600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c1061f0 0x7ff18c106610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.968+0000 7ff191600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c106be0 con 0x7ff18c1061f0 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.968+0000 7ff18aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c1061f0 0x7ff18c106610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.968+0000 7ff18aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c1061f0 0x7ff18c106610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35616/0 (socket says 172.21.15.26:35616) 2024-06-13T06:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.968+0000 7ff18aa00700 1 -- 172.21.15.26:0/4063440666 learned_addr learned my addr 172.21.15.26:0/4063440666 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.969+0000 7ff18aa00700 1 -- 172.21.15.26:0/4063440666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18c109180 con 0x7ff18c1061f0 2024-06-13T06:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.969+0000 7ff18aa00700 1 --2- 172.21.15.26:0/4063440666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c1061f0 0x7ff18c106610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009e80 tx=0x7ff17c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dd3f456958360a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.970+0000 7ff189600700 1 -- 172.21.15.26:0/4063440666 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff17c017070 con 0x7ff18c1061f0 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.970+0000 7ff189600700 1 -- 172.21.15.26:0/4063440666 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff17c00b8c0 con 0x7ff18c1061f0 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.970+0000 7ff189600700 1 -- 172.21.15.26:0/4063440666 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff17c0045c0 con 0x7ff18c1061f0 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.970+0000 7ff191600700 1 -- 172.21.15.26:0/4063440666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c1061f0 msgr2=0x7ff18c106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.970+0000 7ff191600700 1 --2- 172.21.15.26:0/4063440666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c1061f0 0x7ff18c106610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009e80 tx=0x7ff17c00b330 comp rx=0 tx=0).stop 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.970+0000 7ff191600700 1 -- 172.21.15.26:0/4063440666 shutdown_connections 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.970+0000 7ff191600700 1 --2- 172.21.15.26:0/4063440666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c1061f0 0x7ff18c106610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.970+0000 7ff191600700 1 -- 172.21.15.26:0/4063440666 >> 172.21.15.26:0/4063440666 conn(0x7ff18c101b40 msgr2=0x7ff18c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.971+0000 7ff191600700 1 -- 172.21.15.26:0/4063440666 shutdown_connections 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.971+0000 7ff191600700 1 -- 172.21.15.26:0/4063440666 wait complete. 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.972+0000 7ff191600700 1 Processor -- start 2024-06-13T06:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.972+0000 7ff191600700 1 -- start start 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.972+0000 7ff191600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c19e240 0x7ff18c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.972+0000 7ff191600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c10c530 con 0x7ff18c19e240 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.972+0000 7ff18aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c19e240 0x7ff18c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.972+0000 7ff18aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c19e240 0x7ff18c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35632/0 (socket says 172.21.15.26:35632) 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.972+0000 7ff18aa00700 1 -- 172.21.15.26:0/2977517084 learned_addr learned my addr 172.21.15.26:0/2977517084 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.973+0000 7ff18aa00700 1 -- 172.21.15.26:0/2977517084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18c19eba0 con 0x7ff18c19e240 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.973+0000 7ff18aa00700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c19e240 0x7ff18c19e660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff17c004350 tx=0x7ff17c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.973+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff17c021020 con 0x7ff18c19e240 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.973+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff17c01d470 con 0x7ff18c19e240 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.973+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18c19ee30 con 0x7ff18c19e240 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.973+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff17c017040 con 0x7ff18c19e240 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.974+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18c1a1990 con 0x7ff18c19e240 2024-06-13T06:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.975+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff17c024020 con 0x7ff18c19e240 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.975+0000 7ff183400700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.975+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 --> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff17003bc70 con 0x7ff1700390a0 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.975+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff17c00f050 con 0x7ff18c19e240 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.975+0000 7ff18a000700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 msgr2=0x7ff17003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2776242437 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:44.975+0000 7ff18a000700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:45.176+0000 7ff18a000700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 msgr2=0x7ff17003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2776242437 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:45.176+0000 7ff18a000700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:45.577+0000 7ff18a000700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 msgr2=0x7ff17003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2776242437 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:45.577+0000 7ff18a000700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:46.378+0000 7ff18a000700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 msgr2=0x7ff17003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2776242437 2024-06-13T06:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:46.378+0000 7ff18a000700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-13T06:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:47.980+0000 7ff18a000700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 msgr2=0x7ff17003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2776242437 2024-06-13T06:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:47.980+0000 7ff18a000700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-13T06:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:51.183+0000 7ff18a000700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 msgr2=0x7ff17003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2776242437 2024-06-13T06:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:51.184+0000 7ff18a000700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-06-13T06:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:51.305+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7ff17c003a20 con 0x7ff18c19e240 2024-06-13T06:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:51.305+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 msgr2=0x7ff17003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:51.305+0000 7ff183400700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.308+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7ff17c04e680 con 0x7ff18c19e240 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.309+0000 7ff183400700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7ff17003c9d0 0x7ff17003edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.309+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff17003bc70 con 0x7ff17003c9d0 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.311+0000 7ff18a000700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7ff17003c9d0 0x7ff17003edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.316+0000 7ff18a000700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7ff17003c9d0 0x7ff17003edc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff178003a10 tx=0x7ff1780092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.318+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mgr.14120 v2:172.21.15.26:6800/3558618334 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff17003bc70 con 0x7ff17003c9d0 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.327+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff18c063af0 con 0x7ff17003c9d0 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.328+0000 7ff183400700 1 -- 172.21.15.26:0/2977517084 <== mgr.14120 v2:172.21.15.26:6800/3558618334 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff18c063af0 con 0x7ff17003c9d0 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.328+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7ff17003c9d0 msgr2=0x7ff17003edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.328+0000 7ff191600700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7ff17003c9d0 0x7ff17003edc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff178003a10 tx=0x7ff1780092b0 comp rx=0 tx=0).stop 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.328+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c19e240 msgr2=0x7ff18c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.328+0000 7ff191600700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c19e240 0x7ff18c19e660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff17c004350 tx=0x7ff17c00bd30 comp rx=0 tx=0).stop 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.329+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 shutdown_connections 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.329+0000 7ff191600700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7ff17003c9d0 0x7ff17003edc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.329+0000 7ff191600700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18c19e240 0x7ff18c19e660 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.329+0000 7ff191600700 1 --2- 172.21.15.26:0/2977517084 >> [v2:172.21.15.26:6800/2776242437,v1:172.21.15.26:6801/2776242437] conn(0x7ff1700390a0 0x7ff17003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.329+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 >> 172.21.15.26:0/2977517084 conn(0x7ff18c101b40 msgr2=0x7ff18c1026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.329+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 shutdown_connections 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:52.329+0000 7ff191600700 1 -- 172.21.15.26:0/2977517084 wait complete. 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 5 is available 2024-06-13T06:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:Setting orchestrator backend to cephadm... 2024-06-13T06:45:53.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:53 smithi026 ceph-mon[32041]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-13T06:45:53.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:53 smithi026 ceph-mon[32041]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-13T06:45:53.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:53 smithi026 ceph-mon[32041]: [13/Jun/2024:06:45:52] ENGINE Bus STARTING 2024-06-13T06:45:53.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:53 smithi026 ceph-mon[32041]: [13/Jun/2024:06:45:52] ENGINE Serving on https://172.21.15.26:7150 2024-06-13T06:45:53.623 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:53 smithi026 ceph-mon[32041]: [13/Jun/2024:06:45:52] ENGINE Serving on http://172.21.15.26:8765 2024-06-13T06:45:53.623 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:53 smithi026 ceph-mon[32041]: [13/Jun/2024:06:45:52] ENGINE Bus STARTED 2024-06-13T06:45:53.623 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:53 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:54.627 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:54 smithi026 ceph-mon[32041]: mgrmap e8: x(active, since 2s) 2024-06-13T06:45:54.627 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:54 smithi026 ceph-mon[32041]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:54.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:54 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:45:54.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:54 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.690+0000 7f9e7d800700 1 Processor -- start 2024-06-13T06:45:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.690+0000 7f9e7d800700 1 -- start start 2024-06-13T06:45:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.690+0000 7f9e7d800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7810a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.690+0000 7f9e7d800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7810a9f0 con 0x7f9e7810a090 2024-06-13T06:45:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.691+0000 7f9e77e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7810a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.691+0000 7f9e77e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7810a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49122/0 (socket says 172.21.15.26:49122) 2024-06-13T06:45:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.691+0000 7f9e77e00700 1 -- 172.21.15.26:0/4253825383 learned_addr learned my addr 172.21.15.26:0/4253825383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.691+0000 7f9e77e00700 1 -- 172.21.15.26:0/4253825383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7810ab70 con 0x7f9e7810a090 2024-06-13T06:45:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.691+0000 7f9e77e00700 1 --2- 172.21.15.26:0/4253825383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7810a4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9e680098a0 tx=0x7f9e68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b97d7b12d4698f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.692+0000 7f9e76a00700 1 -- 172.21.15.26:0/4253825383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68004160 con 0x7f9e7810a090 2024-06-13T06:45:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.692+0000 7f9e76a00700 1 -- 172.21.15.26:0/4253825383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e68004300 con 0x7f9e7810a090 2024-06-13T06:45:54.649 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.692+0000 7f9e76a00700 1 -- 172.21.15.26:0/4253825383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68011b50 con 0x7f9e7810a090 2024-06-13T06:45:54.649 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.692+0000 7f9e7d800700 1 -- 172.21.15.26:0/4253825383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 msgr2=0x7f9e7810a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:54.649 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.692+0000 7f9e7d800700 1 --2- 172.21.15.26:0/4253825383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7810a4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9e680098a0 tx=0x7f9e68009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:54.649 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.693+0000 7f9e7d800700 1 -- 172.21.15.26:0/4253825383 shutdown_connections 2024-06-13T06:45:54.649 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.693+0000 7f9e7d800700 1 --2- 172.21.15.26:0/4253825383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7810a4b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:54.649 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.693+0000 7f9e7d800700 1 -- 172.21.15.26:0/4253825383 >> 172.21.15.26:0/4253825383 conn(0x7f9e7807d2e0 msgr2=0x7f9e7807d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.693+0000 7f9e7d800700 1 -- 172.21.15.26:0/4253825383 shutdown_connections 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.693+0000 7f9e7d800700 1 -- 172.21.15.26:0/4253825383 wait complete. 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.694+0000 7f9e7d800700 1 Processor -- start 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.694+0000 7f9e7d800700 1 -- start start 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.694+0000 7f9e7d800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.694+0000 7f9e7d800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7810bf20 con 0x7f9e7810a090 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.694+0000 7f9e77e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.694+0000 7f9e77e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49136/0 (socket says 172.21.15.26:49136) 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.695+0000 7f9e77e00700 1 -- 172.21.15.26:0/130361560 learned_addr learned my addr 172.21.15.26:0/130361560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.695+0000 7f9e77e00700 1 -- 172.21.15.26:0/130361560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7819e410 con 0x7f9e7810a090 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.695+0000 7f9e77e00700 1 --2- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7819ded0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9e68011850 tx=0x7f9e68011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.695+0000 7f9e74c00700 1 -- 172.21.15.26:0/130361560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68003af0 con 0x7f9e7810a090 2024-06-13T06:45:54.651 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.695+0000 7f9e74c00700 1 -- 172.21.15.26:0/130361560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e68020070 con 0x7f9e7810a090 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.695+0000 7f9e74c00700 1 -- 172.21.15.26:0/130361560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68019550 con 0x7f9e7810a090 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.695+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7819e700 con 0x7f9e7810a090 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.696+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e781a1390 con 0x7f9e7810a090 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.697+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e78063af0 con 0x7f9e7810a090 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.697+0000 7f9e74c00700 1 -- 172.21.15.26:0/130361560 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9e68019800 con 0x7f9e7810a090 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.697+0000 7f9e74c00700 1 --2- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9e5c0390a0 0x7f9e5c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.698+0000 7f9e74c00700 1 -- 172.21.15.26:0/130361560 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9e68011e00 con 0x7f9e7810a090 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.698+0000 7f9e77400700 1 --2- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9e5c0390a0 0x7f9e5c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:54.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.698+0000 7f9e77400700 1 --2- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9e5c0390a0 0x7f9e5c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006fd0 tx=0x7f9e64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.703+0000 7f9e74c00700 1 -- 172.21.15.26:0/130361560 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9e6800e260 con 0x7f9e7810a090 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.938+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f9e781084f0 con 0x7f9e5c0390a0 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.947+0000 7f9e74c00700 1 -- 172.21.15.26:0/130361560 <== mgr.14120 v2:172.21.15.26:6800/3558618334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9e781084f0 con 0x7f9e5c0390a0 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.950+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9e5c0390a0 msgr2=0x7f9e5c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.950+0000 7f9e7d800700 1 --2- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9e5c0390a0 0x7f9e5c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006fd0 tx=0x7f9e64006e40 comp rx=0 tx=0).stop 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.950+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 msgr2=0x7f9e7819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.950+0000 7f9e7d800700 1 --2- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7819ded0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9e68011850 tx=0x7f9e68011c10 comp rx=0 tx=0).stop 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.951+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 shutdown_connections 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.951+0000 7f9e7d800700 1 --2- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9e5c0390a0 0x7f9e5c03b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.951+0000 7f9e7d800700 1 --2- 172.21.15.26:0/130361560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e7810a090 0x7f9e7819ded0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.951+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 >> 172.21.15.26:0/130361560 conn(0x7f9e7807d2e0 msgr2=0x7f9e78107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.951+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 shutdown_connections 2024-06-13T06:45:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:53.951+0000 7f9e7d800700 1 -- 172.21.15.26:0/130361560 wait complete. 2024-06-13T06:45:56.231 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-13T06:45:56.231 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.266+0000 7f9c8ee00700 1 Processor -- start 2024-06-13T06:45:56.231 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.266+0000 7f9c8ee00700 1 -- start start 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.266+0000 7f9c8ee00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c88082350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.266+0000 7f9c8ee00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88082920 con 0x7f9c88081f30 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.267+0000 7f9c87e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c88082350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.267+0000 7f9c87e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c88082350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49146/0 (socket says 172.21.15.26:49146) 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.267+0000 7f9c87e00700 1 -- 172.21.15.26:0/400272675 learned_addr learned my addr 172.21.15.26:0/400272675 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.267+0000 7f9c87e00700 1 -- 172.21.15.26:0/400272675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8810f700 con 0x7f9c88081f30 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.268+0000 7f9c87e00700 1 --2- 172.21.15.26:0/400272675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c88082350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c780098a0 tx=0x7f9c78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a4ea1ab378947a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.268+0000 7f9c86a00700 1 -- 172.21.15.26:0/400272675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c78004160 con 0x7f9c88081f30 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.268+0000 7f9c86a00700 1 -- 172.21.15.26:0/400272675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c78004300 con 0x7f9c88081f30 2024-06-13T06:45:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.268+0000 7f9c86a00700 1 -- 172.21.15.26:0/400272675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c78011b50 con 0x7f9c88081f30 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.268+0000 7f9c8ee00700 1 -- 172.21.15.26:0/400272675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 msgr2=0x7f9c88082350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.268+0000 7f9c8ee00700 1 --2- 172.21.15.26:0/400272675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c88082350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c780098a0 tx=0x7f9c78009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.269+0000 7f9c8ee00700 1 -- 172.21.15.26:0/400272675 shutdown_connections 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.269+0000 7f9c8ee00700 1 --2- 172.21.15.26:0/400272675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c88082350 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.269+0000 7f9c8ee00700 1 -- 172.21.15.26:0/400272675 >> 172.21.15.26:0/400272675 conn(0x7f9c8807d6a0 msgr2=0x7f9c8807fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.269+0000 7f9c8ee00700 1 -- 172.21.15.26:0/400272675 shutdown_connections 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.269+0000 7f9c8ee00700 1 -- 172.21.15.26:0/400272675 wait complete. 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.270+0000 7f9c8ee00700 1 Processor -- start 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.270+0000 7f9c8ee00700 1 -- start start 2024-06-13T06:45:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.270+0000 7f9c8ee00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c881a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.271+0000 7f9c8ee00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88112ab0 con 0x7f9c88081f30 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.271+0000 7f9c87e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c881a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.271+0000 7f9c87e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c881a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49158/0 (socket says 172.21.15.26:49158) 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.271+0000 7f9c87e00700 1 -- 172.21.15.26:0/3728833993 learned_addr learned my addr 172.21.15.26:0/3728833993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.271+0000 7f9c87e00700 1 -- 172.21.15.26:0/3728833993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c881a2a30 con 0x7f9c88081f30 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.272+0000 7f9c87e00700 1 --2- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c881a24f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c78011850 tx=0x7f9c78011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.272+0000 7f9c84c00700 1 -- 172.21.15.26:0/3728833993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c78003af0 con 0x7f9c88081f30 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.272+0000 7f9c84c00700 1 -- 172.21.15.26:0/3728833993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c78020070 con 0x7f9c88081f30 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.272+0000 7f9c84c00700 1 -- 172.21.15.26:0/3728833993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c780195a0 con 0x7f9c88081f30 2024-06-13T06:45:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.272+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c881a2cc0 con 0x7f9c88081f30 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.272+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c881a59b0 con 0x7f9c88081f30 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.273+0000 7f9c84c00700 1 -- 172.21.15.26:0/3728833993 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9c7801a430 con 0x7f9c88081f30 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.273+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c88063af0 con 0x7f9c88081f30 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.274+0000 7f9c84c00700 1 --2- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9c6c0390a0 0x7f9c6c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.274+0000 7f9c84c00700 1 -- 172.21.15.26:0/3728833993 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9c78011e00 con 0x7f9c88081f30 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.274+0000 7f9c87400700 1 --2- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9c6c0390a0 0x7f9c6c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.275+0000 7f9c87400700 1 --2- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9c6c0390a0 0x7f9c6c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c74006fd0 tx=0x7f9c74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.279+0000 7f9c84c00700 1 -- 172.21.15.26:0/3728833993 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c7800e260 con 0x7f9c88081f30 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.509+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9c8807f930 con 0x7f9c6c0390a0 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.511+0000 7f9c84c00700 1 -- 172.21.15.26:0/3728833993 <== mgr.14120 v2:172.21.15.26:6800/3558618334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9c8807f930 con 0x7f9c6c0390a0 2024-06-13T06:45:56.235 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9c6c0390a0 msgr2=0x7f9c6c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 --2- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9c6c0390a0 0x7f9c6c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c74006fd0 tx=0x7f9c74006e40 comp rx=0 tx=0).stop 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 msgr2=0x7f9c881a24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 --2- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c881a24f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c78011850 tx=0x7f9c78011c10 comp rx=0 tx=0).stop 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 shutdown_connections 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 --2- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f9c6c0390a0 0x7f9c6c03b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 --2- 172.21.15.26:0/3728833993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c88081f30 0x7f9c881a24f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 >> 172.21.15.26:0/3728833993 conn(0x7f9c8807d6a0 msgr2=0x7f9c8807f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 shutdown_connections 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:55.514+0000 7f9c8ee00700 1 -- 172.21.15.26:0/3728833993 wait complete. 2024-06-13T06:45:56.236 INFO:teuthology.orchestra.run.smithi026.stdout:Generating ssh key... 2024-06-13T06:45:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:56 smithi026 ceph-mon[32041]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:57 smithi026 ceph-mon[32041]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:57 smithi026 ceph-mon[32041]: Generating ssh key... 2024-06-13T06:45:58.753 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: Generating public/private rsa key pair. 2024-06-13T06:45:58.753 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: Your identification has been saved in /tmp/tmpi1khu62g/key. 2024-06-13T06:45:58.753 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: Your public key has been saved in /tmp/tmpi1khu62g/key.pub. 2024-06-13T06:45:58.753 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: The key fingerprint is: 2024-06-13T06:45:58.753 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: SHA256:ABKQGwotACepbsEKjh4teH3p4Bm3Em7bZ0UfA8wNAsw ceph-559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:45:58.753 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: The key's randomart image is: 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: +---[RSA 3072]----+ 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: |B=+..o...o.o | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: |*o.. .E .+ . | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: |=+ . . | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: |=o . . o | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: |B o. .S. . o | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: |+*..= + . . | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: |o.oo O . . | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: | . *.o o | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: | ..o.o | 2024-06-13T06:45:58.754 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:45:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: +----[SHA256]-----+ 2024-06-13T06:45:59.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.847+0000 7f10b0a00700 1 Processor -- start 2024-06-13T06:45:59.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.847+0000 7f10b0a00700 1 -- start start 2024-06-13T06:45:59.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.848+0000 7f10b0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:59.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.848+0000 7f10b0a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac07c800 con 0x7f10ac07de10 2024-06-13T06:45:59.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.848+0000 7f10ab400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:59.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.848+0000 7f10ab400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49164/0 (socket says 172.21.15.26:49164) 2024-06-13T06:45:59.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.848+0000 7f10ab400700 1 -- 172.21.15.26:0/1295468710 learned_addr learned my addr 172.21.15.26:0/1295468710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:59.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.849+0000 7f10ab400700 1 -- 172.21.15.26:0/1295468710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac07c980 con 0x7f10ac07de10 2024-06-13T06:45:59.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.849+0000 7f10ab400700 1 --2- 172.21.15.26:0/1295468710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac07c2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f109c0098a0 tx=0x7f109c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d6452dc35a5279 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:59.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.850+0000 7f10aa000700 1 -- 172.21.15.26:0/1295468710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f109c004160 con 0x7f10ac07de10 2024-06-13T06:45:59.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.850+0000 7f10aa000700 1 -- 172.21.15.26:0/1295468710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f109c004300 con 0x7f10ac07de10 2024-06-13T06:45:59.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.850+0000 7f10aa000700 1 -- 172.21.15.26:0/1295468710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f109c011b50 con 0x7f10ac07de10 2024-06-13T06:45:59.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.850+0000 7f10b0a00700 1 -- 172.21.15.26:0/1295468710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 msgr2=0x7f10ac07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:59.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.850+0000 7f10b0a00700 1 --2- 172.21.15.26:0/1295468710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac07c2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f109c0098a0 tx=0x7f109c009bb0 comp rx=0 tx=0).stop 2024-06-13T06:45:59.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.851+0000 7f10b0a00700 1 -- 172.21.15.26:0/1295468710 shutdown_connections 2024-06-13T06:45:59.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.851+0000 7f10b0a00700 1 --2- 172.21.15.26:0/1295468710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac07c2c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:59.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.851+0000 7f10b0a00700 1 -- 172.21.15.26:0/1295468710 >> 172.21.15.26:0/1295468710 conn(0x7f10ac103300 msgr2=0x7f10ac105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:59.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.851+0000 7f10b0a00700 1 -- 172.21.15.26:0/1295468710 shutdown_connections 2024-06-13T06:45:59.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.851+0000 7f10b0a00700 1 -- 172.21.15.26:0/1295468710 wait complete. 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.852+0000 7f10b0a00700 1 Processor -- start 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.852+0000 7f10b0a00700 1 -- start start 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.852+0000 7f10b0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.852+0000 7f10b0a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac10e310 con 0x7f10ac07de10 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.853+0000 7f10ab400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.853+0000 7f10ab400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac19dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49166/0 (socket says 172.21.15.26:49166) 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.853+0000 7f10ab400700 1 -- 172.21.15.26:0/3129057282 learned_addr learned my addr 172.21.15.26:0/3129057282 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.853+0000 7f10ab400700 1 -- 172.21.15.26:0/3129057282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac19e3e0 con 0x7f10ac07de10 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.853+0000 7f10ab400700 1 --2- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac19dea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f109c011b60 tx=0x7f109c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.854+0000 7f10a3e00700 1 -- 172.21.15.26:0/3129057282 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f109c003b20 con 0x7f10ac07de10 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.854+0000 7f10a3e00700 1 -- 172.21.15.26:0/3129057282 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f109c003cc0 con 0x7f10ac07de10 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.854+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac19e670 con 0x7f10ac07de10 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.854+0000 7f10a3e00700 1 -- 172.21.15.26:0/3129057282 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f109c024840 con 0x7f10ac07de10 2024-06-13T06:45:59.165 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.854+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac1a1360 con 0x7f10ac07de10 2024-06-13T06:45:59.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.855+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ac063af0 con 0x7f10ac07de10 2024-06-13T06:45:59.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.855+0000 7f10a3e00700 1 -- 172.21.15.26:0/3129057282 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f109c022020 con 0x7f10ac07de10 2024-06-13T06:45:59.166 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.856+0000 7f10a3e00700 1 --2- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f10900390a0 0x7f109003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:59.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.856+0000 7f10a3e00700 1 -- 172.21.15.26:0/3129057282 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f109c011e00 con 0x7f10ac07de10 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.857+0000 7f10aaa00700 1 --2- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f10900390a0 0x7f109003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.857+0000 7f10aaa00700 1 --2- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f10900390a0 0x7f109003b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1098006fd0 tx=0x7f1098006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:56.861+0000 7f10a3e00700 1 -- 172.21.15.26:0/3129057282 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f109c04cb60 con 0x7f10ac07de10 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:57.091+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f10ac104570 con 0x7f10900390a0 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.429+0000 7f10a3e00700 1 -- 172.21.15.26:0/3129057282 <== mgr.14120 v2:172.21.15.26:6800/3558618334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10ac104570 con 0x7f10900390a0 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.432+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f10900390a0 msgr2=0x7f109003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.432+0000 7f10b0a00700 1 --2- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f10900390a0 0x7f109003b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1098006fd0 tx=0x7f1098006e40 comp rx=0 tx=0).stop 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.432+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 msgr2=0x7f10ac19dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.432+0000 7f10b0a00700 1 --2- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac19dea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f109c011b60 tx=0x7f109c011c40 comp rx=0 tx=0).stop 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.433+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 shutdown_connections 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.433+0000 7f10b0a00700 1 --2- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f10900390a0 0x7f109003b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.433+0000 7f10b0a00700 1 --2- 172.21.15.26:0/3129057282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10ac07de10 0x7f10ac19dea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:59.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.433+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 >> 172.21.15.26:0/3129057282 conn(0x7f10ac103300 msgr2=0x7f10ac103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:59.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.433+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 shutdown_connections 2024-06-13T06:45:59.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:58.433+0000 7f10b0a00700 1 -- 172.21.15.26:0/3129057282 wait complete. 2024-06-13T06:45:59.741 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:59 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:45:59.741 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:45:59 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:46:00.736 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:00 smithi026 ceph-mon[32041]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:00.753 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7dU9LwRuR25l1pjEPBiXI9gsw2GkjrBMDenBurX/j/fMw6QnJYgxs4WWrtA1zNDy3Ge1lubbHtgr8yRE7YJDMMzU3yw0OGpCZMUvUosZxvz97OMl4C0q6rZJEV9g6n21nzYIBCN5oiTpBoJ6ZZ05iOH2LHZ4yLNw0D6Ut4m8WVF/zUkMsWOOcn6IU1rasrckhlgMl14fZFMyuObaIheccnYnAsdFpC6+6cWImsPmNmaWyLJ+tXNtuvKpFnqDdKowRbXtWvHQMvXsyw1BWKi+gqFhC1q4H5mkAz0yCyWsoZQalt2cQAd+2f/TeLEty9m35HH7qZmSYk2Zhgfd2W6+ok4xfNp5z8I5R7mWwo+11L2m01HRVukZyO5wXVUFWcbkNTEG14+C7THAGEclbl2w1liZ2jWaDNIvzJ16Z+HhsRiZo8EPT5Tg4K+x0TiI+K1lrJwR94u8QwWdf7uiRdpZe7/yPiRFHM+gxGrgxGSeGXhuGThsc6T/V1nHbU3Rz9/E= ceph-559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:46:00.753 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.802+0000 7f1d6f400700 1 Processor -- start 2024-06-13T06:46:00.753 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.802+0000 7f1d6f400700 1 -- start start 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.803+0000 7f1d6f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d68108280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.803+0000 7f1d6f400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681087c0 con 0x7f1d68107e60 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.803+0000 7f1d6ca00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d68108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.803+0000 7f1d6ca00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d68108280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49180/0 (socket says 172.21.15.26:49180) 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.803+0000 7f1d6ca00700 1 -- 172.21.15.26:0/1061229444 learned_addr learned my addr 172.21.15.26:0/1061229444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.804+0000 7f1d6ca00700 1 -- 172.21.15.26:0/1061229444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d68109050 con 0x7f1d68107e60 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.804+0000 7f1d6ca00700 1 --2- 172.21.15.26:0/1061229444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d68108280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0098a0 tx=0x7f1d5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413142560d307854 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.804+0000 7f1d67400700 1 -- 172.21.15.26:0/1061229444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d5c004160 con 0x7f1d68107e60 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.804+0000 7f1d67400700 1 -- 172.21.15.26:0/1061229444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d5c004300 con 0x7f1d68107e60 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.804+0000 7f1d67400700 1 -- 172.21.15.26:0/1061229444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d5c011b50 con 0x7f1d68107e60 2024-06-13T06:46:00.754 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.805+0000 7f1d6f400700 1 -- 172.21.15.26:0/1061229444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 msgr2=0x7f1d68108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.805+0000 7f1d6f400700 1 --2- 172.21.15.26:0/1061229444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d68108280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0098a0 tx=0x7f1d5c009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.805+0000 7f1d6f400700 1 -- 172.21.15.26:0/1061229444 shutdown_connections 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.805+0000 7f1d6f400700 1 --2- 172.21.15.26:0/1061229444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d68108280 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.805+0000 7f1d6f400700 1 -- 172.21.15.26:0/1061229444 >> 172.21.15.26:0/1061229444 conn(0x7f1d68103470 msgr2=0x7f1d68105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.805+0000 7f1d6f400700 1 -- 172.21.15.26:0/1061229444 shutdown_connections 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.805+0000 7f1d6f400700 1 -- 172.21.15.26:0/1061229444 wait complete. 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.806+0000 7f1d6f400700 1 Processor -- start 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.806+0000 7f1d6f400700 1 -- start start 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.807+0000 7f1d6f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d6819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.807+0000 7f1d6f400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810c310 con 0x7f1d68107e60 2024-06-13T06:46:00.755 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.807+0000 7f1d6ca00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d6819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.807+0000 7f1d6ca00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d6819e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49186/0 (socket says 172.21.15.26:49186) 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.807+0000 7f1d6ca00700 1 -- 172.21.15.26:0/811751961 learned_addr learned my addr 172.21.15.26:0/811751961 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.808+0000 7f1d6ca00700 1 -- 172.21.15.26:0/811751961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6819e660 con 0x7f1d68107e60 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.808+0000 7f1d6ca00700 1 --2- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d6819e120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009fb0 tx=0x7f1d5c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.808+0000 7f1d65600700 1 -- 172.21.15.26:0/811751961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d5c0038b0 con 0x7f1d68107e60 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.808+0000 7f1d65600700 1 -- 172.21.15.26:0/811751961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d5c011e50 con 0x7f1d68107e60 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.808+0000 7f1d65600700 1 -- 172.21.15.26:0/811751961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d5c019550 con 0x7f1d68107e60 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.808+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6819e8f0 con 0x7f1d68107e60 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.808+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d681a15e0 con 0x7f1d68107e60 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.810+0000 7f1d65600700 1 -- 172.21.15.26:0/811751961 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1d5c01a430 con 0x7f1d68107e60 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.810+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6810c490 con 0x7f1d68107e60 2024-06-13T06:46:00.759 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.810+0000 7f1d65600700 1 --2- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f1d500390b0 0x7f1d5003b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.810+0000 7f1d65600700 1 -- 172.21.15.26:0/811751961 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1d5c04d350 con 0x7f1d68107e60 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.811+0000 7f1d67e00700 1 --2- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f1d500390b0 0x7f1d5003b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.811+0000 7f1d67e00700 1 --2- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f1d500390b0 0x7f1d5003b570 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006fd0 tx=0x7f1d54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:45:59.816+0000 7f1d65600700 1 -- 172.21.15.26:0/811751961 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d5c029d50 con 0x7f1d68107e60 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.047+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f1d68104680 con 0x7f1d500390b0 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.048+0000 7f1d65600700 1 -- 172.21.15.26:0/811751961 <== mgr.14120 v2:172.21.15.26:6800/3558618334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f1d68104680 con 0x7f1d500390b0 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f1d500390b0 msgr2=0x7f1d5003b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 --2- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f1d500390b0 0x7f1d5003b570 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006fd0 tx=0x7f1d54006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 msgr2=0x7f1d6819e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 --2- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d6819e120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009fb0 tx=0x7f1d5c011bf0 comp rx=0 tx=0).stop 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 shutdown_connections 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 --2- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f1d500390b0 0x7f1d5003b570 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 --2- 172.21.15.26:0/811751961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d68107e60 0x7f1d6819e120 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 >> 172.21.15.26:0/811751961 conn(0x7f1d68103470 msgr2=0x7f1d68103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 shutdown_connections 2024-06-13T06:46:00.761 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:00.052+0000 7f1d6f400700 1 -- 172.21.15.26:0/811751961 wait complete. 2024-06-13T06:46:00.761 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-13T06:46:00.761 INFO:teuthology.orchestra.run.smithi026.stdout:Adding key to root@localhost authorized_keys... 2024-06-13T06:46:00.761 INFO:teuthology.orchestra.run.smithi026.stdout:Adding host smithi026... 2024-06-13T06:46:02.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:02 smithi026 ceph-mon[32041]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:03.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:03 smithi026 ceph-mon[32041]: Deploying cephadm binary to smithi026 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Added host 'smithi026' with addr '172.21.15.26' 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.392+0000 7f142a800700 1 Processor -- start 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.392+0000 7f142a800700 1 -- start start 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.393+0000 7f142a800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142410ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.393+0000 7f142a800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f142407be90 con 0x7f1424108910 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.393+0000 7f1423400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142410ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.393+0000 7f1423400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142410ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49200/0 (socket says 172.21.15.26:49200) 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.393+0000 7f1423400700 1 -- 172.21.15.26:0/2554238167 learned_addr learned my addr 172.21.15.26:0/2554238167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:08.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.394+0000 7f1423400700 1 -- 172.21.15.26:0/2554238167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142410b240 con 0x7f1424108910 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.394+0000 7f1423400700 1 --2- 172.21.15.26:0/2554238167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142410ad00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14140098a0 tx=0x7f1414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd12b261b8d4a0c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.395+0000 7f1422000700 1 -- 172.21.15.26:0/2554238167 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1414004160 con 0x7f1424108910 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.395+0000 7f1422000700 1 -- 172.21.15.26:0/2554238167 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1414004300 con 0x7f1424108910 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.395+0000 7f1422000700 1 -- 172.21.15.26:0/2554238167 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1414011b50 con 0x7f1424108910 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.396+0000 7f142a800700 1 -- 172.21.15.26:0/2554238167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 msgr2=0x7f142410ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.396+0000 7f142a800700 1 --2- 172.21.15.26:0/2554238167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142410ad00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14140098a0 tx=0x7f1414009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.396+0000 7f142a800700 1 -- 172.21.15.26:0/2554238167 shutdown_connections 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.396+0000 7f142a800700 1 --2- 172.21.15.26:0/2554238167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142410ad00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.396+0000 7f142a800700 1 -- 172.21.15.26:0/2554238167 >> 172.21.15.26:0/2554238167 conn(0x7f14241023c0 msgr2=0x7f14241047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.396+0000 7f142a800700 1 -- 172.21.15.26:0/2554238167 shutdown_connections 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.396+0000 7f142a800700 1 -- 172.21.15.26:0/2554238167 wait complete. 2024-06-13T06:46:08.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.397+0000 7f142a800700 1 Processor -- start 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.397+0000 7f142a800700 1 -- start start 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.398+0000 7f142a800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.398+0000 7f142a800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f142410c270 con 0x7f1424108910 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.398+0000 7f1423400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.398+0000 7f1423400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142419be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49202/0 (socket says 172.21.15.26:49202) 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.398+0000 7f1423400700 1 -- 172.21.15.26:0/1776457630 learned_addr learned my addr 172.21.15.26:0/1776457630 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.398+0000 7f1423400700 1 -- 172.21.15.26:0/1776457630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142419c390 con 0x7f1424108910 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.399+0000 7f1423400700 1 --2- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142419be50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1414011b60 tx=0x7f1414011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.399+0000 7f141be00700 1 -- 172.21.15.26:0/1776457630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1414003b20 con 0x7f1424108910 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.399+0000 7f141be00700 1 -- 172.21.15.26:0/1776457630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1414003cc0 con 0x7f1424108910 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.399+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142419c620 con 0x7f1424108910 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.399+0000 7f141be00700 1 -- 172.21.15.26:0/1776457630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1414024840 con 0x7f1424108910 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.399+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142419f310 con 0x7f1424108910 2024-06-13T06:46:08.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.400+0000 7f141be00700 1 -- 172.21.15.26:0/1776457630 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1414022020 con 0x7f1424108910 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.400+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1424063af0 con 0x7f1424108910 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.401+0000 7f141be00700 1 --2- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f14080390a0 0x7f140803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.401+0000 7f141be00700 1 -- 172.21.15.26:0/1776457630 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1414011e00 con 0x7f1424108910 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.401+0000 7f1422a00700 1 --2- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f14080390a0 0x7f140803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.402+0000 7f1422a00700 1 --2- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f14080390a0 0x7f140803b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1410006fd0 tx=0x7f1410006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.406+0000 7f141be00700 1 -- 172.21.15.26:0/1776457630 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f141400b3e0 con 0x7f1424108910 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:01.638+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}) -- 0x7f14241035c0 con 0x7f14080390a0 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.111+0000 7f141be00700 1 -- 172.21.15.26:0/1776457630 <== mgr.14120 v2:172.21.15.26:6800/3558618334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f14241035c0 con 0x7f14080390a0 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.115+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f14080390a0 msgr2=0x7f140803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.115+0000 7f142a800700 1 --2- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f14080390a0 0x7f140803b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1410006fd0 tx=0x7f1410006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.115+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 msgr2=0x7f142419be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.115+0000 7f142a800700 1 --2- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142419be50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1414011b60 tx=0x7f1414011c40 comp rx=0 tx=0).stop 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.115+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 shutdown_connections 2024-06-13T06:46:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.115+0000 7f142a800700 1 --2- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f14080390a0 0x7f140803b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:08.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.115+0000 7f142a800700 1 --2- 172.21.15.26:0/1776457630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424108910 0x7f142419be50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:08.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.115+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 >> 172.21.15.26:0/1776457630 conn(0x7f14241023c0 msgr2=0x7f1424102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:08.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.116+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 shutdown_connections 2024-06-13T06:46:08.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:08.116+0000 7f142a800700 1 -- 172.21.15.26:0/1776457630 wait complete. 2024-06-13T06:46:08.870 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mon service... 2024-06-13T06:46:09.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:09 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:46:09.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:09 smithi026 ceph-mon[32041]: Added host smithi026 2024-06-13T06:46:09.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:09 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:11.117 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:10 smithi026 ceph-mon[32041]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:11.117 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:10 smithi026 ceph-mon[32041]: Saving service mon spec with placement count:5 2024-06-13T06:46:11.117 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:10 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.629+0000 7f5165200700 1 Processor -- start 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.629+0000 7f5165200700 1 -- start start 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.630+0000 7f5165200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.630+0000 7f5165200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f516007c7e0 con 0x7f516007ddf0 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.630+0000 7f515de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.631+0000 7f515de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40604/0 (socket says 172.21.15.26:40604) 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.631+0000 7f515de00700 1 -- 172.21.15.26:0/2110663740 learned_addr learned my addr 172.21.15.26:0/2110663740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.631+0000 7f515de00700 1 -- 172.21.15.26:0/2110663740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516007c960 con 0x7f516007ddf0 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.631+0000 7f515de00700 1 --2- 172.21.15.26:0/2110663740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516007c2a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f51500098a0 tx=0x7f5150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66456d971751f3d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.632+0000 7f515ca00700 1 -- 172.21.15.26:0/2110663740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5150004160 con 0x7f516007ddf0 2024-06-13T06:46:11.133 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.632+0000 7f515ca00700 1 -- 172.21.15.26:0/2110663740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5150004300 con 0x7f516007ddf0 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.632+0000 7f515ca00700 1 -- 172.21.15.26:0/2110663740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5150011b50 con 0x7f516007ddf0 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.633+0000 7f5165200700 1 -- 172.21.15.26:0/2110663740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 msgr2=0x7f516007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.633+0000 7f5165200700 1 --2- 172.21.15.26:0/2110663740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516007c2a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f51500098a0 tx=0x7f5150009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.633+0000 7f5165200700 1 -- 172.21.15.26:0/2110663740 shutdown_connections 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.633+0000 7f5165200700 1 --2- 172.21.15.26:0/2110663740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516007c2a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.633+0000 7f5165200700 1 -- 172.21.15.26:0/2110663740 >> 172.21.15.26:0/2110663740 conn(0x7f5160103490 msgr2=0x7f51601058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.633+0000 7f5165200700 1 -- 172.21.15.26:0/2110663740 shutdown_connections 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.633+0000 7f5165200700 1 -- 172.21.15.26:0/2110663740 wait complete. 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.634+0000 7f5165200700 1 Processor -- start 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.634+0000 7f5165200700 1 -- start start 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.634+0000 7f5165200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:11.134 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.635+0000 7f5165200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f516010e4e0 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.635+0000 7f515de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.635+0000 7f515de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516019e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40610/0 (socket says 172.21.15.26:40610) 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.635+0000 7f515de00700 1 -- 172.21.15.26:0/2139785558 learned_addr learned my addr 172.21.15.26:0/2139785558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.635+0000 7f515de00700 1 -- 172.21.15.26:0/2139785558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516019e5f0 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.636+0000 7f515de00700 1 --2- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516019e0b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5150011b60 tx=0x7f5150011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.636+0000 7f5156a00700 1 -- 172.21.15.26:0/2139785558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5150003b20 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.636+0000 7f5156a00700 1 -- 172.21.15.26:0/2139785558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5150003cc0 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.636+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516019e880 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.636+0000 7f5156a00700 1 -- 172.21.15.26:0/2139785558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5150024840 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.636+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51601a1570 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.637+0000 7f5156a00700 1 -- 172.21.15.26:0/2139785558 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5150022020 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.637+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5160063af0 con 0x7f516007ddf0 2024-06-13T06:46:11.135 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.638+0000 7f5156a00700 1 --2- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f51440390a0 0x7f514403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.638+0000 7f5156a00700 1 -- 172.21.15.26:0/2139785558 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5150011e00 con 0x7f516007ddf0 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.638+0000 7f515d400700 1 --2- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f51440390a0 0x7f514403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.639+0000 7f515d400700 1 --2- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f51440390a0 0x7f514403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f514c006fd0 tx=0x7f514c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.644+0000 7f5156a00700 1 -- 172.21.15.26:0/2139785558 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f515004cb10 con 0x7f516007ddf0 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.893+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5160104730 con 0x7f51440390a0 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.903+0000 7f5156a00700 1 -- 172.21.15.26:0/2139785558 <== mgr.14120 v2:172.21.15.26:6800/3558618334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5160104730 con 0x7f51440390a0 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.906+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f51440390a0 msgr2=0x7f514403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.906+0000 7f5165200700 1 --2- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f51440390a0 0x7f514403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f514c006fd0 tx=0x7f514c006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.907+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 msgr2=0x7f516019e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.907+0000 7f5165200700 1 --2- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516019e0b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5150011b60 tx=0x7f5150011c40 comp rx=0 tx=0).stop 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.907+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 shutdown_connections 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.907+0000 7f5165200700 1 --2- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f51440390a0 0x7f514403b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.907+0000 7f5165200700 1 --2- 172.21.15.26:0/2139785558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516007ddf0 0x7f516019e0b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.136 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.907+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 >> 172.21.15.26:0/2139785558 conn(0x7f5160103490 msgr2=0x7f5160104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:11.137 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.907+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 shutdown_connections 2024-06-13T06:46:11.137 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:09.907+0000 7f5165200700 1 -- 172.21.15.26:0/2139785558 wait complete. 2024-06-13T06:46:11.137 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mgr service... 2024-06-13T06:46:12.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:12 smithi026 ceph-mon[32041]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:12.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:12 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:46:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:12 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:46:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:12 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:46:13.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-13T06:46:13.197 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.225+0000 7f3a65600700 1 Processor -- start 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.225+0000 7f3a65600700 1 -- start start 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.225+0000 7f3a65600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.226+0000 7f3a65600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6007c800 con 0x7f3a6007de10 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.226+0000 7f3a5fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.226+0000 7f3a5fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40612/0 (socket says 172.21.15.26:40612) 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.226+0000 7f3a5fe00700 1 -- 172.21.15.26:0/1153265502 learned_addr learned my addr 172.21.15.26:0/1153265502 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.227+0000 7f3a5fe00700 1 -- 172.21.15.26:0/1153265502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6007c980 con 0x7f3a6007de10 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.227+0000 7f3a5fe00700 1 --2- 172.21.15.26:0/1153265502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6007c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a500098a0 tx=0x7f3a50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511fa7c0a5ff58f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.227+0000 7f3a5ea00700 1 -- 172.21.15.26:0/1153265502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a50004160 con 0x7f3a6007de10 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.227+0000 7f3a5ea00700 1 -- 172.21.15.26:0/1153265502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a50004300 con 0x7f3a6007de10 2024-06-13T06:46:13.198 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.227+0000 7f3a5ea00700 1 -- 172.21.15.26:0/1153265502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a50011b20 con 0x7f3a6007de10 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.228+0000 7f3a65600700 1 -- 172.21.15.26:0/1153265502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 msgr2=0x7f3a6007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.228+0000 7f3a65600700 1 --2- 172.21.15.26:0/1153265502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6007c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a500098a0 tx=0x7f3a50009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.228+0000 7f3a65600700 1 -- 172.21.15.26:0/1153265502 shutdown_connections 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.228+0000 7f3a65600700 1 --2- 172.21.15.26:0/1153265502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6007c2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.228+0000 7f3a65600700 1 -- 172.21.15.26:0/1153265502 >> 172.21.15.26:0/1153265502 conn(0x7f3a60103300 msgr2=0x7f3a60105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.228+0000 7f3a65600700 1 -- 172.21.15.26:0/1153265502 shutdown_connections 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.229+0000 7f3a65600700 1 -- 172.21.15.26:0/1153265502 wait complete. 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.230+0000 7f3a65600700 1 Processor -- start 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.230+0000 7f3a65600700 1 -- start start 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.230+0000 7f3a65600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.230+0000 7f3a65600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6010e300 con 0x7f3a6007de10 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.230+0000 7f3a5fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:13.199 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.230+0000 7f3a5fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6019de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40620/0 (socket says 172.21.15.26:40620) 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.230+0000 7f3a5fe00700 1 -- 172.21.15.26:0/986814732 learned_addr learned my addr 172.21.15.26:0/986814732 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.231+0000 7f3a5fe00700 1 -- 172.21.15.26:0/986814732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6019e370 con 0x7f3a6007de10 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.231+0000 7f3a5fe00700 1 --2- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6019de30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a500117e0 tx=0x7f3a50011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.231+0000 7f3a5cc00700 1 -- 172.21.15.26:0/986814732 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5001a430 con 0x7f3a6007de10 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.231+0000 7f3a5cc00700 1 -- 172.21.15.26:0/986814732 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5001aab0 con 0x7f3a6007de10 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.231+0000 7f3a5cc00700 1 -- 172.21.15.26:0/986814732 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a50024a60 con 0x7f3a6007de10 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.231+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6019e600 con 0x7f3a6007de10 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.232+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6019ea60 con 0x7f3a6007de10 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.233+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a60063af0 con 0x7f3a6007de10 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.233+0000 7f3a5cc00700 1 -- 172.21.15.26:0/986814732 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3a50022020 con 0x7f3a6007de10 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.233+0000 7f3a5cc00700 1 --2- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f3a440390f0 0x7f3a4403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:13.200 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.233+0000 7f3a5cc00700 1 -- 172.21.15.26:0/986814732 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3a5004d400 con 0x7f3a6007de10 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.234+0000 7f3a5f400700 1 --2- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f3a440390f0 0x7f3a4403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.235+0000 7f3a5f400700 1 --2- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f3a440390f0 0x7f3a4403b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c006fd0 tx=0x7f3a4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.239+0000 7f3a5cc00700 1 -- 172.21.15.26:0/986814732 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3a500212c0 con 0x7f3a6007de10 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.485+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3a601a1470 con 0x7f3a440390f0 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.492+0000 7f3a5cc00700 1 -- 172.21.15.26:0/986814732 <== mgr.14120 v2:172.21.15.26:6800/3558618334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3a601a1470 con 0x7f3a440390f0 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.494+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f3a440390f0 msgr2=0x7f3a4403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.494+0000 7f3a65600700 1 --2- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f3a440390f0 0x7f3a4403b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c006fd0 tx=0x7f3a4c006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.494+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 msgr2=0x7f3a6019de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.494+0000 7f3a65600700 1 --2- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6019de30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a500117e0 tx=0x7f3a50011ce0 comp rx=0 tx=0).stop 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.494+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 shutdown_connections 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.494+0000 7f3a65600700 1 --2- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f3a440390f0 0x7f3a4403b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:13.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.494+0000 7f3a65600700 1 --2- 172.21.15.26:0/986814732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6007de10 0x7f3a6019de30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:13.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.494+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 >> 172.21.15.26:0/986814732 conn(0x7f3a60103300 msgr2=0x7f3a60103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:13.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.495+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 shutdown_connections 2024-06-13T06:46:13.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:12.495+0000 7f3a65600700 1 -- 172.21.15.26:0/986814732 wait complete. 2024-06-13T06:46:13.820 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:13 smithi026 ceph-mon[32041]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:13.820 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:13 smithi026 ceph-mon[32041]: Saving service mgr spec with placement count:2 2024-06-13T06:46:14.846 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:14 smithi026 ceph-mon[32041]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:14.846 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:14 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3382498619' entity='client.admin' 2024-06-13T06:46:15.493 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.898+0000 7f23a3400700 1 Processor -- start 2024-06-13T06:46:15.493 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.898+0000 7f23a3400700 1 -- start start 2024-06-13T06:46:15.493 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.898+0000 7f23a3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c10ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:15.493 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.898+0000 7f23a3400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f239c07be90 con 0x7f239c108910 2024-06-13T06:46:15.493 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.898+0000 7f23a0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c10ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:15.493 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.899+0000 7f23a0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c10ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53436/0 (socket says 172.21.15.26:53436) 2024-06-13T06:46:15.493 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.899+0000 7f23a0a00700 1 -- 172.21.15.26:0/3875763175 learned_addr learned my addr 172.21.15.26:0/3875763175 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.899+0000 7f23a0a00700 1 -- 172.21.15.26:0/3875763175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c10b240 con 0x7f239c108910 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.899+0000 7f23a0a00700 1 --2- 172.21.15.26:0/3875763175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c10ad00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23900098a0 tx=0x7f2390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f103dd582565cccb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.900+0000 7f239b400700 1 -- 172.21.15.26:0/3875763175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2390004160 con 0x7f239c108910 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.900+0000 7f239b400700 1 -- 172.21.15.26:0/3875763175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2390004300 con 0x7f239c108910 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.900+0000 7f239b400700 1 -- 172.21.15.26:0/3875763175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2390011b20 con 0x7f239c108910 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.901+0000 7f23a3400700 1 -- 172.21.15.26:0/3875763175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 msgr2=0x7f239c10ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.901+0000 7f23a3400700 1 --2- 172.21.15.26:0/3875763175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c10ad00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23900098a0 tx=0x7f2390009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.901+0000 7f23a3400700 1 -- 172.21.15.26:0/3875763175 shutdown_connections 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.901+0000 7f23a3400700 1 --2- 172.21.15.26:0/3875763175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c10ad00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.901+0000 7f23a3400700 1 -- 172.21.15.26:0/3875763175 >> 172.21.15.26:0/3875763175 conn(0x7f239c1023c0 msgr2=0x7f239c1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.901+0000 7f23a3400700 1 -- 172.21.15.26:0/3875763175 shutdown_connections 2024-06-13T06:46:15.494 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.901+0000 7f23a3400700 1 -- 172.21.15.26:0/3875763175 wait complete. 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.902+0000 7f23a3400700 1 Processor -- start 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.902+0000 7f23a3400700 1 -- start start 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.903+0000 7f23a3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.903+0000 7f23a3400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f239c10c2b0 con 0x7f239c108910 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.903+0000 7f23a0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.903+0000 7f23a0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53446/0 (socket says 172.21.15.26:53446) 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.903+0000 7f23a0a00700 1 -- 172.21.15.26:0/3382498619 learned_addr learned my addr 172.21.15.26:0/3382498619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.903+0000 7f23a0a00700 1 -- 172.21.15.26:0/3382498619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c19e510 con 0x7f239c108910 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.904+0000 7f23a0a00700 1 --2- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c19dfd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23900117e0 tx=0x7f2390011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.904+0000 7f2399600700 1 -- 172.21.15.26:0/3382498619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2390003ba0 con 0x7f239c108910 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.904+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c19e7a0 con 0x7f239c108910 2024-06-13T06:46:15.495 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.904+0000 7f2399600700 1 -- 172.21.15.26:0/3382498619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239001a910 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.904+0000 7f2399600700 1 -- 172.21.15.26:0/3382498619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23900248f0 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.904+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c19ec00 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.905+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239c063af0 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.905+0000 7f2399600700 1 -- 172.21.15.26:0/3382498619 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2390022020 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.906+0000 7f2399600700 1 --2- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f2384039050 0x7f238403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.906+0000 7f2399600700 1 -- 172.21.15.26:0/3382498619 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f239004da30 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.906+0000 7f239be00700 1 --2- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f2384039050 0x7f238403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.907+0000 7f239be00700 1 --2- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f2384039050 0x7f238403b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2388006fd0 tx=0x7f2388006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:13.913+0000 7f2399600700 1 -- 172.21.15.26:0/3382498619 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f239002cdf0 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.164+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f239c1a1490 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.167+0000 7f2399600700 1 -- 172.21.15.26:0/3382498619 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f239000b5a0 con 0x7f239c108910 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.170+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f2384039050 msgr2=0x7f238403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:15.496 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.170+0000 7f23a3400700 1 --2- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f2384039050 0x7f238403b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2388006fd0 tx=0x7f2388006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:15.497 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.170+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 msgr2=0x7f239c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:15.497 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.170+0000 7f23a3400700 1 --2- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c19dfd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23900117e0 tx=0x7f2390011ba0 comp rx=0 tx=0).stop 2024-06-13T06:46:15.497 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.171+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 shutdown_connections 2024-06-13T06:46:15.497 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.171+0000 7f23a3400700 1 --2- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f2384039050 0x7f238403b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:15.497 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.171+0000 7f23a3400700 1 --2- 172.21.15.26:0/3382498619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239c108910 0x7f239c19dfd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:15.497 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.171+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 >> 172.21.15.26:0/3382498619 conn(0x7f239c1023c0 msgr2=0x7f239c102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:15.497 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.171+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 shutdown_connections 2024-06-13T06:46:15.497 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:14.171+0000 7f23a3400700 1 -- 172.21.15.26:0/3382498619 wait complete. 2024-06-13T06:46:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:16 smithi026 ceph-mon[32041]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:16 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:46:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:16 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/265674237' entity='client.admin' 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.188+0000 7fd985600700 1 Processor -- start 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.189+0000 7fd985600700 1 -- start start 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.189+0000 7fd985600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98007cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.189+0000 7fd985600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd98007d180 con 0x7fd98007e750 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.190+0000 7fd97ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98007cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.190+0000 7fd97ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98007cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53452/0 (socket says 172.21.15.26:53452) 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.190+0000 7fd97ea00700 1 -- 172.21.15.26:0/2891402598 learned_addr learned my addr 172.21.15.26:0/2891402598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.190+0000 7fd97ea00700 1 -- 172.21.15.26:0/2891402598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98007d300 con 0x7fd98007e750 2024-06-13T06:46:17.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.191+0000 7fd97ea00700 1 --2- 172.21.15.26:0/2891402598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98007cbb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9700098a0 tx=0x7fd970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4e4f4ff73b47f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.191+0000 7fd97e000700 1 -- 172.21.15.26:0/2891402598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd970004160 con 0x7fd98007e750 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.191+0000 7fd97e000700 1 -- 172.21.15.26:0/2891402598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd970004300 con 0x7fd98007e750 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.191+0000 7fd97e000700 1 -- 172.21.15.26:0/2891402598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd970011b20 con 0x7fd98007e750 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.192+0000 7fd985600700 1 -- 172.21.15.26:0/2891402598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 msgr2=0x7fd98007cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.192+0000 7fd985600700 1 --2- 172.21.15.26:0/2891402598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98007cbb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9700098a0 tx=0x7fd970009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.192+0000 7fd985600700 1 -- 172.21.15.26:0/2891402598 shutdown_connections 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.192+0000 7fd985600700 1 --2- 172.21.15.26:0/2891402598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98007cbb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.192+0000 7fd985600700 1 -- 172.21.15.26:0/2891402598 >> 172.21.15.26:0/2891402598 conn(0x7fd980103480 msgr2=0x7fd9801058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.193+0000 7fd985600700 1 -- 172.21.15.26:0/2891402598 shutdown_connections 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.193+0000 7fd985600700 1 -- 172.21.15.26:0/2891402598 wait complete. 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.194+0000 7fd985600700 1 Processor -- start 2024-06-13T06:46:17.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.194+0000 7fd985600700 1 -- start start 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.195+0000 7fd985600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.195+0000 7fd985600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010e530 con 0x7fd98007e750 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.195+0000 7fd97ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.195+0000 7fd97ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98019e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53458/0 (socket says 172.21.15.26:53458) 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.195+0000 7fd97ea00700 1 -- 172.21.15.26:0/265674237 learned_addr learned my addr 172.21.15.26:0/265674237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.196+0000 7fd97ea00700 1 -- 172.21.15.26:0/265674237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98019e650 con 0x7fd98007e750 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.196+0000 7fd97ea00700 1 --2- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98019e110 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd970000c00 tx=0x7fd970011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.196+0000 7fd977e00700 1 -- 172.21.15.26:0/265674237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd970003b60 con 0x7fd98007e750 2024-06-13T06:46:17.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.196+0000 7fd977e00700 1 -- 172.21.15.26:0/265674237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd970003d00 con 0x7fd98007e750 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.196+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98019e8e0 con 0x7fd98007e750 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.197+0000 7fd977e00700 1 -- 172.21.15.26:0/265674237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9700248a0 con 0x7fd98007e750 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.197+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98019ed40 con 0x7fd98007e750 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.198+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd980063af0 con 0x7fd98007e750 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.198+0000 7fd977e00700 1 -- 172.21.15.26:0/265674237 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd970022020 con 0x7fd98007e750 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.199+0000 7fd977e00700 1 --2- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fd9640390a0 0x7fd96403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.199+0000 7fd977e00700 1 -- 172.21.15.26:0/265674237 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd97004d020 con 0x7fd98007e750 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.199+0000 7fd976000700 1 --2- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fd9640390a0 0x7fd96403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.200+0000 7fd976000700 1 --2- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fd9640390a0 0x7fd96403b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd968006fd0 tx=0x7fd968006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.204+0000 7fd977e00700 1 -- 172.21.15.26:0/265674237 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd970019bf0 con 0x7fd98007e750 2024-06-13T06:46:17.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.434+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fd9801a17d0 con 0x7fd98007e750 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.438+0000 7fd977e00700 1 -- 172.21.15.26:0/265674237 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fd97001acc0 con 0x7fd98007e750 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.442+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fd9640390a0 msgr2=0x7fd96403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.442+0000 7fd985600700 1 --2- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fd9640390a0 0x7fd96403b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd968006fd0 tx=0x7fd968006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.442+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 msgr2=0x7fd98019e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.442+0000 7fd985600700 1 --2- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98019e110 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd970000c00 tx=0x7fd970011c10 comp rx=0 tx=0).stop 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.443+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 shutdown_connections 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.443+0000 7fd985600700 1 --2- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fd9640390a0 0x7fd96403b560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.443+0000 7fd985600700 1 --2- 172.21.15.26:0/265674237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98007e750 0x7fd98019e110 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.443+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 >> 172.21.15.26:0/265674237 conn(0x7fd980103480 msgr2=0x7fd980103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.443+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 shutdown_connections 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:16.443+0000 7fd985600700 1 -- 172.21.15.26:0/265674237 wait complete. 2024-06-13T06:46:17.160 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling the dashboard module... 2024-06-13T06:46:19.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:18 smithi026 ceph-mon[32041]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:19.055 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:18 smithi026 ceph-mon[32041]: from='mgr.14120 172.21.15.26:0/3709379197' entity='mgr.x' 2024-06-13T06:46:19.055 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:18 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1177417395' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-13T06:46:19.055 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:18 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: ignoring --setuser ceph since I am not root 2024-06-13T06:46:19.055 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:18 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: ignoring --setgroup ceph since I am not root 2024-06-13T06:46:19.369 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:19 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:19.115+0000 7f6881efa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:46:19.369 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:19 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:19.368+0000 7f6881efa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:46:19.706 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:19 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:19.504+0000 7f6881efa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:46:20.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:19 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1177417395' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-13T06:46:20.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:19 smithi026 ceph-mon[32041]: mgrmap e9: x(active, since 27s) 2024-06-13T06:46:20.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:19 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:19.917+0000 7f6881efa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:46:20.498 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:20 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:20.034+0000 7f6881efa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:46:20.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.760+0000 7fb4b9c00700 1 Processor -- start 2024-06-13T06:46:20.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.761+0000 7fb4b9c00700 1 -- start start 2024-06-13T06:46:20.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.761+0000 7fb4b9c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b4106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:20.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.761+0000 7fb4b9c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4106b50 con 0x7fb4b4106160 2024-06-13T06:46:20.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.761+0000 7fb4b2a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b4106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:20.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.762+0000 7fb4b2a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b4106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53468/0 (socket says 172.21.15.26:53468) 2024-06-13T06:46:20.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.762+0000 7fb4b2a00700 1 -- 172.21.15.26:0/3885989490 learned_addr learned my addr 172.21.15.26:0/3885989490 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:20.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.762+0000 7fb4b2a00700 1 -- 172.21.15.26:0/3885989490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b41090f0 con 0x7fb4b4106160 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.762+0000 7fb4b2a00700 1 --2- 172.21.15.26:0/3885989490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b4106580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40098a0 tx=0x7fb4a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d375627e316113e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.763+0000 7fb4b1600700 1 -- 172.21.15.26:0/3885989490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4a4004160 con 0x7fb4b4106160 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.763+0000 7fb4b1600700 1 -- 172.21.15.26:0/3885989490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a4004300 con 0x7fb4b4106160 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.763+0000 7fb4b1600700 1 -- 172.21.15.26:0/3885989490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4a4011b20 con 0x7fb4b4106160 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.763+0000 7fb4b9c00700 1 -- 172.21.15.26:0/3885989490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 msgr2=0x7fb4b4106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.763+0000 7fb4b9c00700 1 --2- 172.21.15.26:0/3885989490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b4106580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40098a0 tx=0x7fb4a4009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.763+0000 7fb4b9c00700 1 -- 172.21.15.26:0/3885989490 shutdown_connections 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.763+0000 7fb4b9c00700 1 --2- 172.21.15.26:0/3885989490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b4106580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.763+0000 7fb4b9c00700 1 -- 172.21.15.26:0/3885989490 >> 172.21.15.26:0/3885989490 conn(0x7fb4b4101af0 msgr2=0x7fb4b4103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.764+0000 7fb4b9c00700 1 -- 172.21.15.26:0/3885989490 shutdown_connections 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.764+0000 7fb4b9c00700 1 -- 172.21.15.26:0/3885989490 wait complete. 2024-06-13T06:46:20.513 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.765+0000 7fb4b9c00700 1 Processor -- start 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.765+0000 7fb4b9c00700 1 -- start start 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.765+0000 7fb4b9c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.765+0000 7fb4b9c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b410c4a0 con 0x7fb4b4106160 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.765+0000 7fb4b2a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.766+0000 7fb4b2a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53480/0 (socket says 172.21.15.26:53480) 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.766+0000 7fb4b2a00700 1 -- 172.21.15.26:0/1177417395 learned_addr learned my addr 172.21.15.26:0/1177417395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.766+0000 7fb4b2a00700 1 -- 172.21.15.26:0/1177417395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b419e520 con 0x7fb4b4106160 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.766+0000 7fb4b2a00700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b419dfe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40117e0 tx=0x7fb4a4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.766+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4a4003bf0 con 0x7fb4b4106160 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.767+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b419e7b0 con 0x7fb4b4106160 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.767+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a401a910 con 0x7fb4b4106160 2024-06-13T06:46:20.514 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.767+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4a40249a0 con 0x7fb4b4106160 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.767+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b419ec10 con 0x7fb4b4106160 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.768+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb4a4022020 con 0x7fb4b4106160 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.768+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b4063af0 con 0x7fb4b4106160 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.768+0000 7fb4ab400700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 0x7fb49803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.768+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4a404d410 con 0x7fb4b4106160 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.769+0000 7fb4b2000700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 0x7fb49803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.769+0000 7fb4b2000700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 0x7fb49803b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0006fd0 tx=0x7fb4a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:17.774+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb4a4011e20 con 0x7fb4b4106160 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.073+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fb4b41a15e0 con 0x7fb4b4106160 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.802+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fb4a4004300 con 0x7fb4b4106160 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.806+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 msgr2=0x7fb49803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.806+0000 7fb4b9c00700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 0x7fb49803b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0006fd0 tx=0x7fb4a0006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.806+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 msgr2=0x7fb4b419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:20.515 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.806+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb4a402a2a0 con 0x7fb4b4106160 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.807+0000 7fb4b9c00700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b419dfe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40117e0 tx=0x7fb4a4011c10 comp rx=0 tx=0).stop 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.807+0000 7fb4ab400700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 0x7fb49803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.807+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 shutdown_connections 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.807+0000 7fb4b9c00700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 0x7fb49803b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.807+0000 7fb4b9c00700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4b4106160 0x7fb4b419dfe0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40117e0 tx=0x7fb4a4011c10 comp rx=0 tx=0).stop 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.807+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 >> 172.21.15.26:0/1177417395 conn(0x7fb4b4101af0 msgr2=0x7fb4b4102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.808+0000 7fb4ab400700 1 -- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 msgr2=0x7fb49803b4c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.808+0000 7fb4ab400700 1 --2- 172.21.15.26:0/1177417395 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7fb498039000 0x7fb49803b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.808+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 shutdown_connections 2024-06-13T06:46:20.516 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:18.808+0000 7fb4b9c00700 1 -- 172.21.15.26:0/1177417395 wait complete. 2024-06-13T06:46:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:21 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2686467911' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-13T06:46:21.752 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:21 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:21.648+0000 7f6881efa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:46:22.105 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:21 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:21.883+0000 7f6881efa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:46:22.106 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:22 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:22.105+0000 7f6881efa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:46:22.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:46:22.167 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.143+0000 7f6387e00700 1 Processor -- start 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.143+0000 7f6387e00700 1 -- start start 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.143+0000 7f6387e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f6388107060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.143+0000 7f6387e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6388107630 con 0x7f6388106c40 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.144+0000 7f6386a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f6388107060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.144+0000 7f6386a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f6388107060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53508/0 (socket says 172.21.15.26:53508) 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.144+0000 7f6386a00700 1 -- 172.21.15.26:0/3989850612 learned_addr learned my addr 172.21.15.26:0/3989850612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.145+0000 7f6386a00700 1 -- 172.21.15.26:0/3989850612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6388107e40 con 0x7f6388106c40 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.145+0000 7f6386a00700 1 --2- 172.21.15.26:0/3989850612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f6388107060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f63780098a0 tx=0x7f6378009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf436b18c94d919b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.145+0000 7f6385600700 1 -- 172.21.15.26:0/3989850612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6378004160 con 0x7f6388106c40 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.145+0000 7f6385600700 1 -- 172.21.15.26:0/3989850612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6378004300 con 0x7f6388106c40 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.145+0000 7f6385600700 1 -- 172.21.15.26:0/3989850612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6378011b20 con 0x7f6388106c40 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.146+0000 7f6387e00700 1 -- 172.21.15.26:0/3989850612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 msgr2=0x7f6388107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.146+0000 7f6387e00700 1 --2- 172.21.15.26:0/3989850612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f6388107060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f63780098a0 tx=0x7f6378009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.146+0000 7f6387e00700 1 -- 172.21.15.26:0/3989850612 shutdown_connections 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.146+0000 7f6387e00700 1 --2- 172.21.15.26:0/3989850612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f6388107060 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.146+0000 7f6387e00700 1 -- 172.21.15.26:0/3989850612 >> 172.21.15.26:0/3989850612 conn(0x7f63881021a0 msgr2=0x7f6388104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.146+0000 7f6387e00700 1 -- 172.21.15.26:0/3989850612 shutdown_connections 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.146+0000 7f6387e00700 1 -- 172.21.15.26:0/3989850612 wait complete. 2024-06-13T06:46:22.169 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.147+0000 7f6387e00700 1 Processor -- start 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.147+0000 7f6387e00700 1 -- start start 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.148+0000 7f6387e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f638810cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.148+0000 7f6387e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f638807b130 con 0x7f6388106c40 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.148+0000 7f6386a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f638810cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.148+0000 7f6386a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f638810cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53510/0 (socket says 172.21.15.26:53510) 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.148+0000 7f6386a00700 1 -- 172.21.15.26:0/2686467911 learned_addr learned my addr 172.21.15.26:0/2686467911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.148+0000 7f6386a00700 1 -- 172.21.15.26:0/2686467911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638810d040 con 0x7f6388106c40 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.149+0000 7f6386a00700 1 --2- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f638810cb00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6378000c00 tx=0x7f6378011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.149+0000 7f638ca00700 1 -- 172.21.15.26:0/2686467911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6378003b60 con 0x7f6388106c40 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.149+0000 7f638ca00700 1 -- 172.21.15.26:0/2686467911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f637801a910 con 0x7f6388106c40 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.149+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638810b230 con 0x7f6388106c40 2024-06-13T06:46:22.170 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.149+0000 7f638ca00700 1 -- 172.21.15.26:0/2686467911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6378024930 con 0x7f6388106c40 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.149+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638810b6f0 con 0x7f6388106c40 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.150+0000 7f638ca00700 1 -- 172.21.15.26:0/2686467911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f6378022020 con 0x7f6388106c40 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.150+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6388063af0 con 0x7f6388106c40 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.151+0000 7f638ca00700 1 --2- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f636c0390a0 0x7f636c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.151+0000 7f638ca00700 1 -- 172.21.15.26:0/2686467911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f637804d2b0 con 0x7f6388106c40 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.151+0000 7f6386000700 1 -- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f636c0390a0 msgr2=0x7f636c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558618334 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.151+0000 7f6386000700 1 --2- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f636c0390a0 0x7f636c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.156+0000 7f638ca00700 1 -- 172.21.15.26:0/2686467911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f637801ab70 con 0x7f6388106c40 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.352+0000 7f6386000700 1 -- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f636c0390a0 msgr2=0x7f636c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558618334 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.352+0000 7f6386000700 1 --2- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f636c0390a0 0x7f636c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.440+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6388195560 con 0x7f6388106c40 2024-06-13T06:46:22.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.441+0000 7f638ca00700 1 -- 172.21.15.26:0/2686467911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f637802d3f0 con 0x7f6388106c40 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.445+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f636c0390a0 msgr2=0x7f636c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.445+0000 7f6387e00700 1 --2- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f636c0390a0 0x7f636c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.445+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 msgr2=0x7f638810cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.445+0000 7f6387e00700 1 --2- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f638810cb00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6378000c00 tx=0x7f6378011c10 comp rx=0 tx=0).stop 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.445+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 shutdown_connections 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.446+0000 7f6387e00700 1 --2- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f636c0390a0 0x7f636c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.446+0000 7f6387e00700 1 --2- 172.21.15.26:0/2686467911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6388106c40 0x7f638810cb00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.446+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 >> 172.21.15.26:0/2686467911 conn(0x7f63881021a0 msgr2=0x7f63881029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.446+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 shutdown_connections 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:21.446+0000 7f6387e00700 1 -- 172.21.15.26:0/2686467911 wait complete. 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2024-06-13T06:46:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 9... 2024-06-13T06:46:22.739 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:22 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:22.512+0000 7f6881efa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:46:22.739 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:22 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:22.641+0000 7f6881efa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:46:23.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:22 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:22.905+0000 7f6881efa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:46:23.405 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:23 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:23.167+0000 7f6881efa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:46:24.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:24 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:24.017+0000 7f6881efa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:46:25.001 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:24 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:24.593+0000 7f6881efa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:46:25.001 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:24 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:24.726+0000 7f6881efa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:46:25.001 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:25.001+0000 7f6881efa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:46:25.809 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:25.676+0000 7f6881efa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:46:26.153 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:25.809+0000 7f6881efa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:46:26.479 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:26.153+0000 7f6881efa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:46:26.479 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:26.344+0000 7f6881efa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:46:26.479 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:26.478+0000 7f6881efa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:46:26.751 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:26.639+0000 7f6881efa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:46:27.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:27.020+0000 7f6881efa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:46:28.105 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:27.809+0000 7f6881efa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:46:28.105+0000 7f6881efa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: Active manager daemon x restarted 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: Activating manager daemon x 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: osdmap e3: 0 total, 0 up, 0 in 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: mgrmap e10: x(active, starting, since 0.00882237s) 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-13T06:46:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-13T06:46:28.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-13T06:46:28.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:28 smithi026 ceph-mon[32041]: Manager daemon x is now available 2024-06-13T06:46:29.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:29.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-13T06:46:29.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-13T06:46:29.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:29 smithi026 ceph-mon[32041]: mgrmap e11: x(active, since 1.00994s) 2024-06-13T06:46:30.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:46:30.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-13T06:46:30.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.794+0000 7f71fb000700 1 Processor -- start 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.794+0000 7f71fb000700 1 -- start start 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.795+0000 7f71fb000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.795+0000 7f71fb000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71f407c890 con 0x7f71f407de60 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.795+0000 7f71f3e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.795+0000 7f71f3e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57054/0 (socket says 172.21.15.26:57054) 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.795+0000 7f71f3e00700 1 -- 172.21.15.26:0/3037430080 learned_addr learned my addr 172.21.15.26:0/3037430080 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:30.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.796+0000 7f71f3e00700 1 -- 172.21.15.26:0/3037430080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f407ca10 con 0x7f71f407de60 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.796+0000 7f71f3e00700 1 --2- 172.21.15.26:0/3037430080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f407c2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71e40098a0 tx=0x7f71e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2c0441b6ea81c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.796+0000 7f71f2a00700 1 -- 172.21.15.26:0/3037430080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71e4004160 con 0x7f71f407de60 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.796+0000 7f71f2a00700 1 -- 172.21.15.26:0/3037430080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e4004300 con 0x7f71f407de60 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.796+0000 7f71f2a00700 1 -- 172.21.15.26:0/3037430080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71e4011b20 con 0x7f71f407de60 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.797+0000 7f71fb000700 1 -- 172.21.15.26:0/3037430080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 msgr2=0x7f71f407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.797+0000 7f71fb000700 1 --2- 172.21.15.26:0/3037430080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f407c2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71e40098a0 tx=0x7f71e4009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.797+0000 7f71fb000700 1 -- 172.21.15.26:0/3037430080 shutdown_connections 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.797+0000 7f71fb000700 1 --2- 172.21.15.26:0/3037430080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f407c2c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.797+0000 7f71fb000700 1 -- 172.21.15.26:0/3037430080 >> 172.21.15.26:0/3037430080 conn(0x7f71f4103400 msgr2=0x7f71f4105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.797+0000 7f71fb000700 1 -- 172.21.15.26:0/3037430080 shutdown_connections 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.798+0000 7f71fb000700 1 -- 172.21.15.26:0/3037430080 wait complete. 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.799+0000 7f71fb000700 1 Processor -- start 2024-06-13T06:46:30.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.799+0000 7f71fb000700 1 -- start start 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.800+0000 7f71fb000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.800+0000 7f71fb000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71f410e4e0 con 0x7f71f407de60 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.800+0000 7f71f3e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.800+0000 7f71f3e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57058/0 (socket says 172.21.15.26:57058) 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.800+0000 7f71f3e00700 1 -- 172.21.15.26:0/1475820198 learned_addr learned my addr 172.21.15.26:0/1475820198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.800+0000 7f71f3e00700 1 -- 172.21.15.26:0/1475820198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f419e520 con 0x7f71f407de60 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.801+0000 7f71f3e00700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f419dfe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71e4011b60 tx=0x7f71e4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.801+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71e4003b60 con 0x7f71f407de60 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.801+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e401a910 con 0x7f71f407de60 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.801+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71e4024930 con 0x7f71f407de60 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.801+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f419e7b0 con 0x7f71f407de60 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.801+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f419ec10 con 0x7f71f407de60 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.803+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f71e4022020 con 0x7f71f407de60 2024-06-13T06:46:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.803+0000 7f71f0c00700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 0x7f71d803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.803+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 --> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f71d803bcd0 con 0x7f71d8039100 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.803+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f71e404d300 con 0x7f71f407de60 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.803+0000 7f71f3400700 1 -- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 msgr2=0x7f71d803b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558618334 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:22.803+0000 7f71f3400700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 0x7f71d803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:23.004+0000 7f71f3400700 1 -- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 msgr2=0x7f71d803b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558618334 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:23.004+0000 7f71f3400700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 0x7f71d803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:23.405+0000 7f71f3400700 1 -- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 msgr2=0x7f71d803b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558618334 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:23.405+0000 7f71f3400700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 0x7f71d803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:24.206+0000 7f71f3400700 1 -- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 msgr2=0x7f71d803b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558618334 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:24.206+0000 7f71f3400700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 0x7f71d803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-13T06:46:30.105 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:25.808+0000 7f71f3400700 1 -- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 msgr2=0x7f71d803b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558618334 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:25.808+0000 7f71f3400700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 0x7f71d803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:28.114+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f71e4024ad0 con 0x7f71f407de60 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:28.115+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 msgr2=0x7f71d803b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:28.115+0000 7f71f0c00700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 0x7f71d803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.116+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f71e404e7e0 con 0x7f71f407de60 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.116+0000 7f71f0c00700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f71d803cbc0 0x7f71d803efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.116+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f71e404b7c0 con 0x7f71d803cbc0 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.117+0000 7f71f3400700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f71d803cbc0 0x7f71d803efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.119+0000 7f71f3400700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f71d803cbc0 0x7f71d803efb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f71e0003a10 tx=0x7f71e0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.120+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f71e404b7c0 con 0x7f71d803cbc0 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.128+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f71f4063af0 con 0x7f71d803cbc0 2024-06-13T06:46:30.106 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.128+0000 7f71f0c00700 1 -- 172.21.15.26:0/1475820198 <== mgr.14152 v2:172.21.15.26:6800/662324914 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f71f4063af0 con 0x7f71d803cbc0 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f71d803cbc0 msgr2=0x7f71d803efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f71d803cbc0 0x7f71d803efb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f71e0003a10 tx=0x7f71e0009270 comp rx=0 tx=0).stop 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 msgr2=0x7f71f419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f419dfe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71e4011b60 tx=0x7f71e4011c40 comp rx=0 tx=0).stop 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 shutdown_connections 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f71d803cbc0 0x7f71d803efb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f407de60 0x7f71f419dfe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 --2- 172.21.15.26:0/1475820198 >> [v2:172.21.15.26:6800/3558618334,v1:172.21.15.26:6801/3558618334] conn(0x7f71d8039100 0x7f71d803b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.129+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 >> 172.21.15.26:0/1475820198 conn(0x7f71f4103400 msgr2=0x7f71f4103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.130+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 shutdown_connections 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:29.130+0000 7f71fb000700 1 -- 172.21.15.26:0/1475820198 wait complete. 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 9 is available 2024-06-13T06:46:30.107 INFO:teuthology.orchestra.run.smithi026.stdout:Generating a dashboard self-signed certificate... 2024-06-13T06:46:30.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:30 smithi026 ceph-mon[32041]: [13/Jun/2024:06:46:28] ENGINE Bus STARTING 2024-06-13T06:46:30.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:30 smithi026 ceph-mon[32041]: [13/Jun/2024:06:46:29] ENGINE Serving on https://172.21.15.26:7150 2024-06-13T06:46:30.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:30 smithi026 ceph-mon[32041]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-13T06:46:30.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:30 smithi026 ceph-mon[32041]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-13T06:46:30.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:30 smithi026 ceph-mon[32041]: [13/Jun/2024:06:46:29] ENGINE Serving on http://172.21.15.26:8765 2024-06-13T06:46:30.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:30 smithi026 ceph-mon[32041]: [13/Jun/2024:06:46:29] ENGINE Bus STARTED 2024-06-13T06:46:31.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:31 smithi026 ceph-mon[32041]: mgrmap e12: x(active, since 2s) 2024-06-13T06:46:31.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.128+0000 7f105f200700 1 Processor -- start 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.128+0000 7f105f200700 1 -- start start 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.129+0000 7f105f200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.129+0000 7f105f200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f106007be90 con 0x7f10601065a0 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.129+0000 7f105de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.130+0000 7f105de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57142/0 (socket says 172.21.15.26:57142) 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.130+0000 7f105de00700 1 -- 172.21.15.26:0/1976022800 learned_addr learned my addr 172.21.15.26:0/1976022800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.130+0000 7f105de00700 1 -- 172.21.15.26:0/1976022800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1060108f10 con 0x7f10601065a0 2024-06-13T06:46:32.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.130+0000 7f105de00700 1 --2- 172.21.15.26:0/1976022800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601089d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10500098a0 tx=0x7f1050009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bcb0eff7169f62 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.131+0000 7f105ca00700 1 -- 172.21.15.26:0/1976022800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1050004160 con 0x7f10601065a0 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.131+0000 7f105ca00700 1 -- 172.21.15.26:0/1976022800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1050004300 con 0x7f10601065a0 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.131+0000 7f105ca00700 1 -- 172.21.15.26:0/1976022800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1050011b20 con 0x7f10601065a0 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.131+0000 7f105f200700 1 -- 172.21.15.26:0/1976022800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 msgr2=0x7f10601089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.131+0000 7f105f200700 1 --2- 172.21.15.26:0/1976022800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601089d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10500098a0 tx=0x7f1050009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.132+0000 7f105f200700 1 -- 172.21.15.26:0/1976022800 shutdown_connections 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.132+0000 7f105f200700 1 --2- 172.21.15.26:0/1976022800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601089d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.132+0000 7f105f200700 1 -- 172.21.15.26:0/1976022800 >> 172.21.15.26:0/1976022800 conn(0x7f10601021a0 msgr2=0x7f1060104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.132+0000 7f105f200700 1 -- 172.21.15.26:0/1976022800 shutdown_connections 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.132+0000 7f105f200700 1 -- 172.21.15.26:0/1976022800 wait complete. 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.133+0000 7f105f200700 1 Processor -- start 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.133+0000 7f105f200700 1 -- start start 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.133+0000 7f105f200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.133+0000 7f105f200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1060109fe0 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.134+0000 7f105de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.134+0000 7f105de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601a22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57152/0 (socket says 172.21.15.26:57152) 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.134+0000 7f105de00700 1 -- 172.21.15.26:0/330551212 learned_addr learned my addr 172.21.15.26:0/330551212 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.134+0000 7f105de00700 1 -- 172.21.15.26:0/330551212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10601a27f0 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.134+0000 7f105de00700 1 --2- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601a22b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10500116a0 tx=0x7f1050011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.135+0000 7f1056a00700 1 -- 172.21.15.26:0/330551212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1050003a40 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.135+0000 7f1056a00700 1 -- 172.21.15.26:0/330551212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1050003be0 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.135+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10601a2a80 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.135+0000 7f1056a00700 1 -- 172.21.15.26:0/330551212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f105001a750 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.135+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a2ee0 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.136+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1060063af0 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.136+0000 7f1056a00700 1 -- 172.21.15.26:0/330551212 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f1050024430 con 0x7f10601065a0 2024-06-13T06:46:32.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.137+0000 7f1056a00700 1 --2- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10440390b0 0x7f104403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.137+0000 7f1056a00700 1 -- 172.21.15.26:0/330551212 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f105004d810 con 0x7f10601065a0 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.137+0000 7f105d400700 1 --2- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10440390b0 0x7f104403b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.138+0000 7f105d400700 1 --2- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10440390b0 0x7f104403b570 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.142+0000 7f1056a00700 1 -- 172.21.15.26:0/330551212 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1050011e20 con 0x7f10601065a0 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.379+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f10600628d0 con 0x7f10440390b0 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.536+0000 7f1056a00700 1 -- 172.21.15.26:0/330551212 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f10600628d0 con 0x7f10440390b0 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10440390b0 msgr2=0x7f104403b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 --2- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10440390b0 0x7f104403b570 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 msgr2=0x7f10601a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 --2- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601a22b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10500116a0 tx=0x7f1050011ba0 comp rx=0 tx=0).stop 2024-06-13T06:46:32.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 shutdown_connections 2024-06-13T06:46:32.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 --2- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10440390b0 0x7f104403b570 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:32.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 --2- 172.21.15.26:0/330551212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10601065a0 0x7f10601a22b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:32.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 >> 172.21.15.26:0/330551212 conn(0x7f10601021a0 msgr2=0x7f1060104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:32.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 shutdown_connections 2024-06-13T06:46:32.288 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:31.540+0000 7f105f200700 1 -- 172.21.15.26:0/330551212 wait complete. 2024-06-13T06:46:32.288 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial admin user... 2024-06-13T06:46:32.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:32 smithi026 ceph-mon[32041]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:32.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:32.832 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:34.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:33 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:34.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:33 smithi026 ceph-mon[32041]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:34.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:33 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:34.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$rZ51ft9EHLLFm8lAvBZzE.8TZHl03HruuFlD9gFDnsKu0zy.Lgab.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1718261193, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-13T06:46:34.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.011+0000 7f8cc8400700 1 Processor -- start 2024-06-13T06:46:34.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.011+0000 7f8cc8400700 1 -- start start 2024-06-13T06:46:34.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.012+0000 7f8cc8400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:34.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.012+0000 7f8cc8400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc007c8f0 con 0x7f8cc007de70 2024-06-13T06:46:34.287 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.012+0000 7f8cc5a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:34.288 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.012+0000 7f8cc5a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32938/0 (socket says 172.21.15.26:32938) 2024-06-13T06:46:34.288 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.012+0000 7f8cc5a00700 1 -- 172.21.15.26:0/4026389737 learned_addr learned my addr 172.21.15.26:0/4026389737 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:34.288 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.013+0000 7f8cc5a00700 1 -- 172.21.15.26:0/4026389737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc007ca70 con 0x7f8cc007de70 2024-06-13T06:46:34.288 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.013+0000 7f8cc5a00700 1 --2- 172.21.15.26:0/4026389737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc007c320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40098a0 tx=0x7f8cb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c353879f348ee9ce server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:34.288 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.014+0000 7f8cbfe00700 1 -- 172.21.15.26:0/4026389737 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb4004160 con 0x7f8cc007de70 2024-06-13T06:46:34.288 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.014+0000 7f8cbfe00700 1 -- 172.21.15.26:0/4026389737 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb4004300 con 0x7f8cc007de70 2024-06-13T06:46:34.288 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.014+0000 7f8cbfe00700 1 -- 172.21.15.26:0/4026389737 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb4011b20 con 0x7f8cc007de70 2024-06-13T06:46:34.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.015+0000 7f8cc8400700 1 -- 172.21.15.26:0/4026389737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 msgr2=0x7f8cc007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:34.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.015+0000 7f8cc8400700 1 --2- 172.21.15.26:0/4026389737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc007c320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40098a0 tx=0x7f8cb4009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:34.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.015+0000 7f8cc8400700 1 -- 172.21.15.26:0/4026389737 shutdown_connections 2024-06-13T06:46:34.293 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.015+0000 7f8cc8400700 1 --2- 172.21.15.26:0/4026389737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc007c320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.015+0000 7f8cc8400700 1 -- 172.21.15.26:0/4026389737 >> 172.21.15.26:0/4026389737 conn(0x7f8cc01035b0 msgr2=0x7f8cc01059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.015+0000 7f8cc8400700 1 -- 172.21.15.26:0/4026389737 shutdown_connections 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.015+0000 7f8cc8400700 1 -- 172.21.15.26:0/4026389737 wait complete. 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.016+0000 7f8cc8400700 1 Processor -- start 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.016+0000 7f8cc8400700 1 -- start start 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.017+0000 7f8cc8400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.017+0000 7f8cc8400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc010e5b0 con 0x7f8cc007de70 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.017+0000 7f8cc5a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.017+0000 7f8cc5a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32942/0 (socket says 172.21.15.26:32942) 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.017+0000 7f8cc5a00700 1 -- 172.21.15.26:0/2522391927 learned_addr learned my addr 172.21.15.26:0/2522391927 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.017+0000 7f8cc5a00700 1 -- 172.21.15.26:0/2522391927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc019e590 con 0x7f8cc007de70 2024-06-13T06:46:34.294 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.018+0000 7f8cc5a00700 1 --2- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc019e050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4011b60 tx=0x7f8cb4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.018+0000 7f8cbe000700 1 -- 172.21.15.26:0/2522391927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb4003b60 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.018+0000 7f8cbe000700 1 -- 172.21.15.26:0/2522391927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb401a910 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.018+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc019e820 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.018+0000 7f8cbe000700 1 -- 172.21.15.26:0/2522391927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb4024930 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.018+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc019ec80 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.019+0000 7f8cbe000700 1 -- 172.21.15.26:0/2522391927 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f8cb4022020 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.019+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc0063af0 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.020+0000 7f8cbe000700 1 --2- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ca80390a0 0x7f8ca803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.020+0000 7f8cbe000700 1 -- 172.21.15.26:0/2522391927 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8cb404d380 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.020+0000 7f8cc5000700 1 --2- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ca80390a0 0x7f8ca803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.021+0000 7f8cc5000700 1 --2- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ca80390a0 0x7f8ca803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006fd0 tx=0x7f8cac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.027+0000 7f8cbe000700 1 -- 172.21.15.26:0/2522391927 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8cb400b3e0 con 0x7f8cc007de70 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.306+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f8cc01a15d0 con 0x7f8ca80390a0 2024-06-13T06:46:34.295 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.571+0000 7f8cbe000700 1 -- 172.21.15.26:0/2522391927 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f8cc01a15d0 con 0x7f8ca80390a0 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ca80390a0 msgr2=0x7f8ca803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 --2- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ca80390a0 0x7f8ca803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006fd0 tx=0x7f8cac006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 msgr2=0x7f8cc019e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 --2- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc019e050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4011b60 tx=0x7f8cb4011c40 comp rx=0 tx=0).stop 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 shutdown_connections 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 --2- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ca80390a0 0x7f8ca803b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 --2- 172.21.15.26:0/2522391927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc007de70 0x7f8cc019e050 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 >> 172.21.15.26:0/2522391927 conn(0x7f8cc01035b0 msgr2=0x7f8cc01040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 shutdown_connections 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:33.574+0000 7f8cc8400700 1 -- 172.21.15.26:0/2522391927 wait complete. 2024-06-13T06:46:34.296 INFO:teuthology.orchestra.run.smithi026.stdout:Fetching dashboard port number... 2024-06-13T06:46:36.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:35 smithi026 ceph-mon[32041]: mgrmap e13: x(active, since 6s) 2024-06-13T06:46:36.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:35 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1217367601' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 8443 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.095+0000 7fc00f200700 1 Processor -- start 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.095+0000 7fc00f200700 1 -- start start 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.095+0000 7fc00f200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.095+0000 7fc00f200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc00810ab70 con 0x7fc00810a180 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.096+0000 7fc007e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.096+0000 7fc007e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00810a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32956/0 (socket says 172.21.15.26:32956) 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.096+0000 7fc007e00700 1 -- 172.21.15.26:0/1913085753 learned_addr learned my addr 172.21.15.26:0/1913085753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:36.663 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.096+0000 7fc007e00700 1 -- 172.21.15.26:0/1913085753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00810b2e0 con 0x7fc00810a180 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.097+0000 7fc007e00700 1 --2- 172.21.15.26:0/1913085753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00810a5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbff80098a0 tx=0x7fbff8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82af8b199b1717f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.097+0000 7fc006a00700 1 -- 172.21.15.26:0/1913085753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbff8004160 con 0x7fc00810a180 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.097+0000 7fc006a00700 1 -- 172.21.15.26:0/1913085753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff8004300 con 0x7fc00810a180 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.097+0000 7fc006a00700 1 -- 172.21.15.26:0/1913085753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbff8011b20 con 0x7fc00810a180 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.097+0000 7fc00f200700 1 -- 172.21.15.26:0/1913085753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 msgr2=0x7fc00810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.097+0000 7fc00f200700 1 --2- 172.21.15.26:0/1913085753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00810a5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbff80098a0 tx=0x7fbff8009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.098+0000 7fc00f200700 1 -- 172.21.15.26:0/1913085753 shutdown_connections 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.098+0000 7fc00f200700 1 --2- 172.21.15.26:0/1913085753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00810a5a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.098+0000 7fc00f200700 1 -- 172.21.15.26:0/1913085753 >> 172.21.15.26:0/1913085753 conn(0x7fc00807d6a0 msgr2=0x7fc00807dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.098+0000 7fc00f200700 1 -- 172.21.15.26:0/1913085753 shutdown_connections 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.098+0000 7fc00f200700 1 -- 172.21.15.26:0/1913085753 wait complete. 2024-06-13T06:46:36.664 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.099+0000 7fc00f200700 1 Processor -- start 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.099+0000 7fc00f200700 1 -- start start 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.099+0000 7fc00f200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.099+0000 7fc00f200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc00810e690 con 0x7fc00810a180 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.099+0000 7fc007e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.099+0000 7fc007e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32970/0 (socket says 172.21.15.26:32970) 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.099+0000 7fc007e00700 1 -- 172.21.15.26:0/1217367601 learned_addr learned my addr 172.21.15.26:0/1217367601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.100+0000 7fc007e00700 1 -- 172.21.15.26:0/1217367601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00819e5b0 con 0x7fc00810a180 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.100+0000 7fc007e00700 1 --2- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00819e070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbff8011b60 tx=0x7fbff8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.100+0000 7fc004c00700 1 -- 172.21.15.26:0/1217367601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbff8003b60 con 0x7fc00810a180 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.100+0000 7fc004c00700 1 -- 172.21.15.26:0/1217367601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff801a910 con 0x7fc00810a180 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.100+0000 7fc004c00700 1 -- 172.21.15.26:0/1217367601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbff8024930 con 0x7fc00810a180 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.100+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00819e840 con 0x7fc00810a180 2024-06-13T06:46:36.665 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.101+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00819eca0 con 0x7fc00810a180 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.102+0000 7fc004c00700 1 -- 172.21.15.26:0/1217367601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbff8022020 con 0x7fc00810a180 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.102+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc008063af0 con 0x7fc00810a180 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.102+0000 7fc004c00700 1 --2- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfec0391a0 0x7fbfec03b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.102+0000 7fc004c00700 1 -- 172.21.15.26:0/1217367601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbff804d440 con 0x7fc00810a180 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.103+0000 7fc007400700 1 --2- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfec0391a0 0x7fbfec03b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.103+0000 7fc007400700 1 --2- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfec0391a0 0x7fbfec03b660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbff4006fd0 tx=0x7fbff4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.108+0000 7fc004c00700 1 -- 172.21.15.26:0/1217367601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbff800b3e0 con 0x7fc00810a180 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.339+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc0081a1730 con 0x7fc00810a180 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.339+0000 7fc004c00700 1 -- 172.21.15.26:0/1217367601 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fbff802dde0 con 0x7fc00810a180 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.342+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfec0391a0 msgr2=0x7fbfec03b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:36.666 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.342+0000 7fc00f200700 1 --2- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfec0391a0 0x7fbfec03b660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbff4006fd0 tx=0x7fbff4006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:36.667 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.342+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 msgr2=0x7fc00819e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:36.667 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.342+0000 7fc00f200700 1 --2- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00819e070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbff8011b60 tx=0x7fbff8011c40 comp rx=0 tx=0).stop 2024-06-13T06:46:36.667 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.342+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 shutdown_connections 2024-06-13T06:46:36.667 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.342+0000 7fc00f200700 1 --2- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfec0391a0 0x7fbfec03b660 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:36.667 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.342+0000 7fc00f200700 1 --2- 172.21.15.26:0/1217367601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc00810a180 0x7fc00819e070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:36.667 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.342+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 >> 172.21.15.26:0/1217367601 conn(0x7fc00807d6a0 msgr2=0x7fc008107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:36.667 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.343+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 shutdown_connections 2024-06-13T06:46:36.667 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:35.343+0000 7fc00f200700 1 -- 172.21.15.26:0/1217367601 wait complete. 2024-06-13T06:46:36.689 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-13T06:46:36.689 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-06-13T06:46:36.712 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-13T06:46:36.712 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-06-13T06:46:36.712 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-06-13T06:46:36.712 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-13T06:46:36.719 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph Dashboard is now available at: 2024-06-13T06:46:36.719 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:36.719 INFO:teuthology.orchestra.run.smithi026.stdout: URL: https://smithi026.front.sepia.ceph.com:8443/ 2024-06-13T06:46:36.719 INFO:teuthology.orchestra.run.smithi026.stdout: User: admin 2024-06-13T06:46:36.720 INFO:teuthology.orchestra.run.smithi026.stdout: Password: nnc7k3e8mp 2024-06-13T06:46:36.720 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:36.720 INFO:teuthology.orchestra.run.smithi026.stdout:Saving cluster configuration to /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config directory 2024-06-13T06:46:36.723 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling autotune for osd_memory_target 2024-06-13T06:46:38.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.401+0000 7fe701c00700 1 Processor -- start 2024-06-13T06:46:38.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.401+0000 7fe701c00700 1 -- start start 2024-06-13T06:46:38.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.402+0000 7fe701c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc106bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:38.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.402+0000 7fe701c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc07bdf0 con 0x7fe6fc1067a0 2024-06-13T06:46:38.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.402+0000 7fe6faa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc106bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.403+0000 7fe6faa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc106bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32978/0 (socket says 172.21.15.26:32978) 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.403+0000 7fe6faa00700 1 -- 172.21.15.26:0/781223285 learned_addr learned my addr 172.21.15.26:0/781223285 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.403+0000 7fe6faa00700 1 -- 172.21.15.26:0/781223285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc107100 con 0x7fe6fc1067a0 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.403+0000 7fe6faa00700 1 --2- 172.21.15.26:0/781223285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc106bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0098a0 tx=0x7fe6ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c30069e7982ceb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.404+0000 7fe6f9600700 1 -- 172.21.15.26:0/781223285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6ec004160 con 0x7fe6fc1067a0 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.404+0000 7fe6f9600700 1 -- 172.21.15.26:0/781223285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ec004300 con 0x7fe6fc1067a0 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.404+0000 7fe6f9600700 1 -- 172.21.15.26:0/781223285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6ec011b20 con 0x7fe6fc1067a0 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.405+0000 7fe701c00700 1 -- 172.21.15.26:0/781223285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 msgr2=0x7fe6fc106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.405+0000 7fe701c00700 1 --2- 172.21.15.26:0/781223285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc106bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0098a0 tx=0x7fe6ec009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.405+0000 7fe701c00700 1 -- 172.21.15.26:0/781223285 shutdown_connections 2024-06-13T06:46:38.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.405+0000 7fe701c00700 1 --2- 172.21.15.26:0/781223285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc106bc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.405+0000 7fe701c00700 1 -- 172.21.15.26:0/781223285 >> 172.21.15.26:0/781223285 conn(0x7fe6fc1023c0 msgr2=0x7fe6fc1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.405+0000 7fe701c00700 1 -- 172.21.15.26:0/781223285 shutdown_connections 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.405+0000 7fe701c00700 1 -- 172.21.15.26:0/781223285 wait complete. 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.406+0000 7fe701c00700 1 Processor -- start 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.407+0000 7fe701c00700 1 -- start start 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.407+0000 7fe701c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.407+0000 7fe701c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1081d0 con 0x7fe6fc1067a0 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.407+0000 7fe6faa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.407+0000 7fe6faa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc19e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32994/0 (socket says 172.21.15.26:32994) 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.407+0000 7fe6faa00700 1 -- 172.21.15.26:0/3081757868 learned_addr learned my addr 172.21.15.26:0/3081757868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.408+0000 7fe6faa00700 1 -- 172.21.15.26:0/3081757868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc19e540 con 0x7fe6fc1067a0 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.408+0000 7fe6faa00700 1 --2- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc19e000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0117e0 tx=0x7fe6ec011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:38.373 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.408+0000 7fe6f3400700 1 -- 172.21.15.26:0/3081757868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6ec003ba0 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.408+0000 7fe6f3400700 1 -- 172.21.15.26:0/3081757868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ec01a910 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.408+0000 7fe6f3400700 1 -- 172.21.15.26:0/3081757868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6ec0248f0 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.408+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc19e7d0 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.409+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc19ec30 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.410+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc063af0 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.410+0000 7fe6f3400700 1 -- 172.21.15.26:0/3081757868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe6ec022020 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.410+0000 7fe6f3400700 1 --2- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe6e00391a0 0x7fe6e003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.410+0000 7fe6f3400700 1 -- 172.21.15.26:0/3081757868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe6ec04d700 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.411+0000 7fe6fa000700 1 --2- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe6e00391a0 0x7fe6e003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.411+0000 7fe6fa000700 1 --2- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe6e00391a0 0x7fe6e003b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8006fd0 tx=0x7fe6e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.415+0000 7fe6f3400700 1 -- 172.21.15.26:0/3081757868 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe6ec01ab70 con 0x7fe6fc1067a0 2024-06-13T06:46:38.374 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.645+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe6fc0630a0 con 0x7fe6fc1067a0 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.647+0000 7fe6f3400700 1 -- 172.21.15.26:0/3081757868 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe6ec010380 con 0x7fe6fc1067a0 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.649+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe6e00391a0 msgr2=0x7fe6e003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 --2- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe6e00391a0 0x7fe6e003b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8006fd0 tx=0x7fe6e8006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 msgr2=0x7fe6fc19e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 --2- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc19e000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0117e0 tx=0x7fe6ec011ba0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 shutdown_connections 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 --2- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe6e00391a0 0x7fe6e003b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 --2- 172.21.15.26:0/3081757868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1067a0 0x7fe6fc19e000 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 >> 172.21.15.26:0/3081757868 conn(0x7fe6fc1023c0 msgr2=0x7fe6fc194c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 shutdown_connections 2024-06-13T06:46:38.375 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:37.650+0000 7fe701c00700 1 -- 172.21.15.26:0/3081757868 wait complete. 2024-06-13T06:46:40.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.997+0000 7f29db600700 1 Processor -- start 2024-06-13T06:46:40.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.997+0000 7f29db600700 1 -- start start 2024-06-13T06:46:40.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.998+0000 7f29db600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d41082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:40.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.998+0000 7f29db600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4108820 con 0x7f29d4107ec0 2024-06-13T06:46:40.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.998+0000 7f29d8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d41082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:40.094 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.998+0000 7f29d8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d41082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33008/0 (socket says 172.21.15.26:33008) 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.998+0000 7f29d8c00700 1 -- 172.21.15.26:0/1601341483 learned_addr learned my addr 172.21.15.26:0/1601341483 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.999+0000 7f29d8c00700 1 -- 172.21.15.26:0/1601341483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d4108fd0 con 0x7f29d4107ec0 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.999+0000 7f29d8c00700 1 --2- 172.21.15.26:0/1601341483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d41082e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29c40098a0 tx=0x7f29c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d518aa973870415e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:38.999+0000 7f29d3e00700 1 -- 172.21.15.26:0/1601341483 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29c4004160 con 0x7f29d4107ec0 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.000+0000 7f29d3e00700 1 -- 172.21.15.26:0/1601341483 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c4004300 con 0x7f29d4107ec0 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.000+0000 7f29d3e00700 1 -- 172.21.15.26:0/1601341483 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29c4011b20 con 0x7f29d4107ec0 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.000+0000 7f29db600700 1 -- 172.21.15.26:0/1601341483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 msgr2=0x7f29d41082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.000+0000 7f29db600700 1 --2- 172.21.15.26:0/1601341483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d41082e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29c40098a0 tx=0x7f29c4009bb0 comp rx=0 tx=0).stop 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.000+0000 7f29db600700 1 -- 172.21.15.26:0/1601341483 shutdown_connections 2024-06-13T06:46:40.095 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.000+0000 7f29db600700 1 --2- 172.21.15.26:0/1601341483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d41082e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:40.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.000+0000 7f29db600700 1 -- 172.21.15.26:0/1601341483 >> 172.21.15.26:0/1601341483 conn(0x7f29d4103460 msgr2=0x7f29d41058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.001+0000 7f29db600700 1 -- 172.21.15.26:0/1601341483 shutdown_connections 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.001+0000 7f29db600700 1 -- 172.21.15.26:0/1601341483 wait complete. 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.002+0000 7f29db600700 1 Processor -- start 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.002+0000 7f29db600700 1 -- start start 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.003+0000 7f29db600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.003+0000 7f29db600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f29d410c410 con 0x7f29d4107ec0 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.003+0000 7f29d8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.003+0000 7f29d8c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d419e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33016/0 (socket says 172.21.15.26:33016) 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.003+0000 7f29d8c00700 1 -- 172.21.15.26:0/3544280257 learned_addr learned my addr 172.21.15.26:0/3544280257 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.003+0000 7f29d8c00700 1 -- 172.21.15.26:0/3544280257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d419e550 con 0x7f29d4107ec0 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.004+0000 7f29d8c00700 1 --2- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d419e010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29c4011b60 tx=0x7f29c4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.004+0000 7f29d2000700 1 -- 172.21.15.26:0/3544280257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29c4003b60 con 0x7f29d4107ec0 2024-06-13T06:46:40.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.004+0000 7f29d2000700 1 -- 172.21.15.26:0/3544280257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c401a910 con 0x7f29d4107ec0 2024-06-13T06:46:40.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.004+0000 7f29d2000700 1 -- 172.21.15.26:0/3544280257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29c4024930 con 0x7f29d4107ec0 2024-06-13T06:46:40.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.004+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d419e7e0 con 0x7f29d4107ec0 2024-06-13T06:46:40.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.004+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d419ec40 con 0x7f29d4107ec0 2024-06-13T06:46:40.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.006+0000 7f29d2000700 1 -- 172.21.15.26:0/3544280257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f29c4022020 con 0x7f29d4107ec0 2024-06-13T06:46:40.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.005+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d4063af0 con 0x7f29d4107ec0 2024-06-13T06:46:40.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.006+0000 7f29d2000700 1 --2- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f29b80391a0 0x7f29b803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:40.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.006+0000 7f29d2000700 1 -- 172.21.15.26:0/3544280257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f29c404d420 con 0x7f29d4107ec0 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.006+0000 7f29cbe00700 1 --2- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f29b80391a0 0x7f29b803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.007+0000 7f29cbe00700 1 --2- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f29b80391a0 0x7f29b803b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29bc006fd0 tx=0x7f29bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.012+0000 7f29d2000700 1 -- 172.21.15.26:0/3544280257 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29c400b3e0 con 0x7f29d4107ec0 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.362+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f29d41a1610 con 0x7f29d4107ec0 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.368+0000 7f29d2000700 1 -- 172.21.15.26:0/3544280257 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f29c402de10 con 0x7f29d4107ec0 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.371+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f29b80391a0 msgr2=0x7f29b803b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.371+0000 7f29db600700 1 --2- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f29b80391a0 0x7f29b803b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29bc006fd0 tx=0x7f29bc006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.371+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 msgr2=0x7f29d419e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:40.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.371+0000 7f29db600700 1 --2- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d419e010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29c4011b60 tx=0x7f29c4011c40 comp rx=0 tx=0).stop 2024-06-13T06:46:40.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.371+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 shutdown_connections 2024-06-13T06:46:40.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.371+0000 7f29db600700 1 --2- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f29b80391a0 0x7f29b803b660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:40.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.371+0000 7f29db600700 1 --2- 172.21.15.26:0/3544280257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29d4107ec0 0x7f29d419e010 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:40.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.371+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 >> 172.21.15.26:0/3544280257 conn(0x7f29d4103460 msgr2=0x7f29d4103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:40.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.372+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 shutdown_connections 2024-06-13T06:46:40.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-06-13T06:46:39.372+0000 7f29db600700 1 -- 172.21.15.26:0/3544280257 wait complete. 2024-06-13T06:46:40.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout:Or, if you are only running a single cluster on this host: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: ceph telemetry on 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout:For more information see: 2024-06-13T06:46:40.103 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:40.104 INFO:teuthology.orchestra.run.smithi026.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-13T06:46:40.104 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:46:40.104 INFO:teuthology.orchestra.run.smithi026.stdout:Bootstrap complete. 2024-06-13T06:46:40.215 INFO:tasks.cephadm:Fetching config... 2024-06-13T06:46:40.215 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:46:40.216 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-13T06:46:40.271 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-13T06:46:40.271 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:46:40.271 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-13T06:46:40.320 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-13T06:46:40.321 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:46:40.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-13T06:46:40.435 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-13T06:46:40.436 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:46:40.436 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-13T06:46:40.533 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-13T06:46:40.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7dU9LwRuR25l1pjEPBiXI9gsw2GkjrBMDenBurX/j/fMw6QnJYgxs4WWrtA1zNDy3Ge1lubbHtgr8yRE7YJDMMzU3yw0OGpCZMUvUosZxvz97OMl4C0q6rZJEV9g6n21nzYIBCN5oiTpBoJ6ZZ05iOH2LHZ4yLNw0D6Ut4m8WVF/zUkMsWOOcn6IU1rasrckhlgMl14fZFMyuObaIheccnYnAsdFpC6+6cWImsPmNmaWyLJ+tXNtuvKpFnqDdKowRbXtWvHQMvXsyw1BWKi+gqFhC1q4H5mkAz0yCyWsoZQalt2cQAd+2f/TeLEty9m35HH7qZmSYk2Zhgfd2W6+ok4xfNp5z8I5R7mWwo+11L2m01HRVukZyO5wXVUFWcbkNTEG14+C7THAGEclbl2w1liZ2jWaDNIvzJ16Z+HhsRiZo8EPT5Tg4K+x0TiI+K1lrJwR94u8QwWdf7uiRdpZe7/yPiRFHM+gxGrgxGSeGXhuGThsc6T/V1nHbU3Rz9/E= ceph-559474f0-2950-11ef-bc9c-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-13T06:46:40.616 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:40 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3544280257' entity='client.admin' 2024-06-13T06:46:40.649 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7dU9LwRuR25l1pjEPBiXI9gsw2GkjrBMDenBurX/j/fMw6QnJYgxs4WWrtA1zNDy3Ge1lubbHtgr8yRE7YJDMMzU3yw0OGpCZMUvUosZxvz97OMl4C0q6rZJEV9g6n21nzYIBCN5oiTpBoJ6ZZ05iOH2LHZ4yLNw0D6Ut4m8WVF/zUkMsWOOcn6IU1rasrckhlgMl14fZFMyuObaIheccnYnAsdFpC6+6cWImsPmNmaWyLJ+tXNtuvKpFnqDdKowRbXtWvHQMvXsyw1BWKi+gqFhC1q4H5mkAz0yCyWsoZQalt2cQAd+2f/TeLEty9m35HH7qZmSYk2Zhgfd2W6+ok4xfNp5z8I5R7mWwo+11L2m01HRVukZyO5wXVUFWcbkNTEG14+C7THAGEclbl2w1liZ2jWaDNIvzJ16Z+HhsRiZo8EPT5Tg4K+x0TiI+K1lrJwR94u8QwWdf7uiRdpZe7/yPiRFHM+gxGrgxGSeGXhuGThsc6T/V1nHbU3Rz9/E= ceph-559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:46:40.692 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7dU9LwRuR25l1pjEPBiXI9gsw2GkjrBMDenBurX/j/fMw6QnJYgxs4WWrtA1zNDy3Ge1lubbHtgr8yRE7YJDMMzU3yw0OGpCZMUvUosZxvz97OMl4C0q6rZJEV9g6n21nzYIBCN5oiTpBoJ6ZZ05iOH2LHZ4yLNw0D6Ut4m8WVF/zUkMsWOOcn6IU1rasrckhlgMl14fZFMyuObaIheccnYnAsdFpC6+6cWImsPmNmaWyLJ+tXNtuvKpFnqDdKowRbXtWvHQMvXsyw1BWKi+gqFhC1q4H5mkAz0yCyWsoZQalt2cQAd+2f/TeLEty9m35HH7qZmSYk2Zhgfd2W6+ok4xfNp5z8I5R7mWwo+11L2m01HRVukZyO5wXVUFWcbkNTEG14+C7THAGEclbl2w1liZ2jWaDNIvzJ16Z+HhsRiZo8EPT5Tg4K+x0TiI+K1lrJwR94u8QwWdf7uiRdpZe7/yPiRFHM+gxGrgxGSeGXhuGThsc6T/V1nHbU3Rz9/E= ceph-559474f0-2950-11ef-bc9c-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-13T06:46:40.735 INFO:teuthology.orchestra.run.smithi135.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7dU9LwRuR25l1pjEPBiXI9gsw2GkjrBMDenBurX/j/fMw6QnJYgxs4WWrtA1zNDy3Ge1lubbHtgr8yRE7YJDMMzU3yw0OGpCZMUvUosZxvz97OMl4C0q6rZJEV9g6n21nzYIBCN5oiTpBoJ6ZZ05iOH2LHZ4yLNw0D6Ut4m8WVF/zUkMsWOOcn6IU1rasrckhlgMl14fZFMyuObaIheccnYnAsdFpC6+6cWImsPmNmaWyLJ+tXNtuvKpFnqDdKowRbXtWvHQMvXsyw1BWKi+gqFhC1q4H5mkAz0yCyWsoZQalt2cQAd+2f/TeLEty9m35HH7qZmSYk2Zhgfd2W6+ok4xfNp5z8I5R7mWwo+11L2m01HRVukZyO5wXVUFWcbkNTEG14+C7THAGEclbl2w1liZ2jWaDNIvzJ16Z+HhsRiZo8EPT5Tg4K+x0TiI+K1lrJwR94u8QwWdf7uiRdpZe7/yPiRFHM+gxGrgxGSeGXhuGThsc6T/V1nHbU3Rz9/E= ceph-559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:46:40.746 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7dU9LwRuR25l1pjEPBiXI9gsw2GkjrBMDenBurX/j/fMw6QnJYgxs4WWrtA1zNDy3Ge1lubbHtgr8yRE7YJDMMzU3yw0OGpCZMUvUosZxvz97OMl4C0q6rZJEV9g6n21nzYIBCN5oiTpBoJ6ZZ05iOH2LHZ4yLNw0D6Ut4m8WVF/zUkMsWOOcn6IU1rasrckhlgMl14fZFMyuObaIheccnYnAsdFpC6+6cWImsPmNmaWyLJ+tXNtuvKpFnqDdKowRbXtWvHQMvXsyw1BWKi+gqFhC1q4H5mkAz0yCyWsoZQalt2cQAd+2f/TeLEty9m35HH7qZmSYk2Zhgfd2W6+ok4xfNp5z8I5R7mWwo+11L2m01HRVukZyO5wXVUFWcbkNTEG14+C7THAGEclbl2w1liZ2jWaDNIvzJ16Z+HhsRiZo8EPT5Tg4K+x0TiI+K1lrJwR94u8QwWdf7uiRdpZe7/yPiRFHM+gxGrgxGSeGXhuGThsc6T/V1nHbU3Rz9/E= ceph-559474f0-2950-11ef-bc9c-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-13T06:46:40.787 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7dU9LwRuR25l1pjEPBiXI9gsw2GkjrBMDenBurX/j/fMw6QnJYgxs4WWrtA1zNDy3Ge1lubbHtgr8yRE7YJDMMzU3yw0OGpCZMUvUosZxvz97OMl4C0q6rZJEV9g6n21nzYIBCN5oiTpBoJ6ZZ05iOH2LHZ4yLNw0D6Ut4m8WVF/zUkMsWOOcn6IU1rasrckhlgMl14fZFMyuObaIheccnYnAsdFpC6+6cWImsPmNmaWyLJ+tXNtuvKpFnqDdKowRbXtWvHQMvXsyw1BWKi+gqFhC1q4H5mkAz0yCyWsoZQalt2cQAd+2f/TeLEty9m35HH7qZmSYk2Zhgfd2W6+ok4xfNp5z8I5R7mWwo+11L2m01HRVukZyO5wXVUFWcbkNTEG14+C7THAGEclbl2w1liZ2jWaDNIvzJ16Z+HhsRiZo8EPT5Tg4K+x0TiI+K1lrJwR94u8QwWdf7uiRdpZe7/yPiRFHM+gxGrgxGSeGXhuGThsc6T/V1nHbU3Rz9/E= ceph-559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:46:40.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-13T06:46:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:46:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.591+0000 7fa72a400700 1 -- 172.21.15.26:0/2894094340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 msgr2=0x7fa724074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.591+0000 7fa72a400700 1 --2- 172.21.15.26:0/2894094340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 0x7fa724074d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa714009910 tx=0x7fa714009c20 comp rx=0 tx=0).stop 2024-06-13T06:46:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.592+0000 7fa72a400700 1 -- 172.21.15.26:0/2894094340 shutdown_connections 2024-06-13T06:46:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.592+0000 7fa72a400700 1 --2- 172.21.15.26:0/2894094340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 0x7fa724074d40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.592+0000 7fa72a400700 1 -- 172.21.15.26:0/2894094340 >> 172.21.15.26:0/2894094340 conn(0x7fa7240fd4e0 msgr2=0x7fa7240ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.592+0000 7fa72a400700 1 -- 172.21.15.26:0/2894094340 shutdown_connections 2024-06-13T06:46:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.592+0000 7fa72a400700 1 -- 172.21.15.26:0/2894094340 wait complete. 2024-06-13T06:46:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.593+0000 7fa72a400700 1 Processor -- start 2024-06-13T06:46:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.593+0000 7fa72a400700 1 -- start start 2024-06-13T06:46:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.594+0000 7fa72a400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 0x7fa72419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.594+0000 7fa72a400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa72410e6d0 con 0x7fa724074920 2024-06-13T06:46:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.594+0000 7fa723400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 0x7fa72419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.594+0000 7fa723400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 0x7fa72419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52744/0 (socket says 172.21.15.26:52744) 2024-06-13T06:46:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.594+0000 7fa723400700 1 -- 172.21.15.26:0/319624787 learned_addr learned my addr 172.21.15.26:0/319624787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.594+0000 7fa723400700 1 -- 172.21.15.26:0/319624787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72419e260 con 0x7fa724074920 2024-06-13T06:46:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.595+0000 7fa723400700 1 --2- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 0x7fa72419dd20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa714005950 tx=0x7fa714005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.595+0000 7fa720c00700 1 -- 172.21.15.26:0/319624787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa714026070 con 0x7fa724074920 2024-06-13T06:46:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.595+0000 7fa720c00700 1 -- 172.21.15.26:0/319624787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71400bc30 con 0x7fa724074920 2024-06-13T06:46:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.595+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72419e4f0 con 0x7fa724074920 2024-06-13T06:46:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.595+0000 7fa720c00700 1 -- 172.21.15.26:0/319624787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa714003b30 con 0x7fa724074920 2024-06-13T06:46:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.595+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7241a1180 con 0x7fa724074920 2024-06-13T06:46:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.596+0000 7fa720c00700 1 -- 172.21.15.26:0/319624787 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa714029070 con 0x7fa724074920 2024-06-13T06:46:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.596+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72410ee30 con 0x7fa724074920 2024-06-13T06:46:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.597+0000 7fa720c00700 1 --2- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7080390f0 0x7fa70803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.597+0000 7fa720c00700 1 -- 172.21.15.26:0/319624787 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa71401e070 con 0x7fa724074920 2024-06-13T06:46:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.597+0000 7fa71aa00700 1 --2- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7080390f0 0x7fa70803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:43.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.598+0000 7fa71aa00700 1 --2- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7080390f0 0x7fa70803b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa70c006fd0 tx=0x7fa70c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.603+0000 7fa720c00700 1 -- 172.21.15.26:0/319624787 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa714013070 con 0x7fa724074920 2024-06-13T06:46:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.832+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa72402dca0 con 0x7fa724074920 2024-06-13T06:46:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.836+0000 7fa720c00700 1 -- 172.21.15.26:0/319624787 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa714030070 con 0x7fa724074920 2024-06-13T06:46:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.839+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7080390f0 msgr2=0x7fa70803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.839+0000 7fa72a400700 1 --2- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7080390f0 0x7fa70803b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa70c006fd0 tx=0x7fa70c006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.839+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 msgr2=0x7fa72419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.839+0000 7fa72a400700 1 --2- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 0x7fa72419dd20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa714005950 tx=0x7fa714005030 comp rx=0 tx=0).stop 2024-06-13T06:46:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.840+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 shutdown_connections 2024-06-13T06:46:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.840+0000 7fa72a400700 1 --2- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7080390f0 0x7fa70803b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.840+0000 7fa72a400700 1 --2- 172.21.15.26:0/319624787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa724074920 0x7fa72419dd20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.840+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 >> 172.21.15.26:0/319624787 conn(0x7fa7240fd4e0 msgr2=0x7fa7240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.840+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 shutdown_connections 2024-06-13T06:46:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:43.840+0000 7fa72a400700 1 -- 172.21.15.26:0/319624787 wait complete. 2024-06-13T06:46:45.044 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-13T06:46:45.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-13T06:46:45.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:44 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/319624787' entity='client.admin' 2024-06-13T06:46:45.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:45.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:45.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:45.112 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:45.113 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:45.113 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:45.471 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:46:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.403+0000 7f6bd9000700 1 -- 172.21.15.26:0/2136429336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 msgr2=0x7f6bd4077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.403+0000 7f6bd9000700 1 --2- 172.21.15.26:0/2136429336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 0x7f6bd4077260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009910 tx=0x7f6bc4009c20 comp rx=0 tx=0).stop 2024-06-13T06:46:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.403+0000 7f6bd9000700 1 -- 172.21.15.26:0/2136429336 shutdown_connections 2024-06-13T06:46:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.403+0000 7f6bd9000700 1 --2- 172.21.15.26:0/2136429336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 0x7f6bd4077260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.403+0000 7f6bd9000700 1 -- 172.21.15.26:0/2136429336 >> 172.21.15.26:0/2136429336 conn(0x7f6bd40ff6b0 msgr2=0x7f6bd4101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.404+0000 7f6bd9000700 1 -- 172.21.15.26:0/2136429336 shutdown_connections 2024-06-13T06:46:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.404+0000 7f6bd9000700 1 -- 172.21.15.26:0/2136429336 wait complete. 2024-06-13T06:46:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.405+0000 7f6bd9000700 1 Processor -- start 2024-06-13T06:46:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.405+0000 7f6bd9000700 1 -- start start 2024-06-13T06:46:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.405+0000 7f6bd9000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 0x7f6bd419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.405+0000 7f6bd9000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd410e5e0 con 0x7f6bd4076e40 2024-06-13T06:46:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.405+0000 7f6bd1e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 0x7f6bd419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.405+0000 7f6bd1e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 0x7f6bd419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52766/0 (socket says 172.21.15.26:52766) 2024-06-13T06:46:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.405+0000 7f6bd1e00700 1 -- 172.21.15.26:0/3191082643 learned_addr learned my addr 172.21.15.26:0/3191082643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.406+0000 7f6bd1e00700 1 -- 172.21.15.26:0/3191082643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd419e0c0 con 0x7f6bd4076e40 2024-06-13T06:46:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.406+0000 7f6bd1e00700 1 --2- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 0x7f6bd419db80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4004fa0 tx=0x7f6bc4005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.406+0000 7f6bcaa00700 1 -- 172.21.15.26:0/3191082643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bc4027070 con 0x7f6bd4076e40 2024-06-13T06:46:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.407+0000 7f6bcaa00700 1 -- 172.21.15.26:0/3191082643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc400be40 con 0x7f6bd4076e40 2024-06-13T06:46:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.407+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd419e350 con 0x7f6bd4076e40 2024-06-13T06:46:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.407+0000 7f6bcaa00700 1 -- 172.21.15.26:0/3191082643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bc400f460 con 0x7f6bd4076e40 2024-06-13T06:46:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.407+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd41a1040 con 0x7f6bd4076e40 2024-06-13T06:46:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.408+0000 7f6bcaa00700 1 -- 172.21.15.26:0/3191082643 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6bc402a070 con 0x7f6bd4076e40 2024-06-13T06:46:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.408+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd410ecf0 con 0x7f6bd4076e40 2024-06-13T06:46:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.408+0000 7f6bcaa00700 1 --2- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6bb8039150 0x7f6bb803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.408+0000 7f6bcaa00700 1 -- 172.21.15.26:0/3191082643 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6bc401e030 con 0x7f6bd4076e40 2024-06-13T06:46:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.409+0000 7f6bd1400700 1 --2- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6bb8039150 0x7f6bb803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.409+0000 7f6bd1400700 1 --2- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6bb8039150 0x7f6bb803b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0006fd0 tx=0x7f6bc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:48.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.414+0000 7f6bcaa00700 1 -- 172.21.15.26:0/3191082643 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bc4013070 con 0x7f6bd4076e40 2024-06-13T06:46:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.648+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6bd402ddb0 con 0x7f6bb8039150 2024-06-13T06:46:48.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.655+0000 7f6bcaa00700 1 -- 172.21.15.26:0/3191082643 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6bd402ddb0 con 0x7f6bb8039150 2024-06-13T06:46:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6bb8039150 msgr2=0x7f6bb803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 --2- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6bb8039150 0x7f6bb803b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0006fd0 tx=0x7f6bc0006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 msgr2=0x7f6bd419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 --2- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 0x7f6bd419db80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4004fa0 tx=0x7f6bc4005080 comp rx=0 tx=0).stop 2024-06-13T06:46:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 shutdown_connections 2024-06-13T06:46:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 --2- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6bb8039150 0x7f6bb803b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 --2- 172.21.15.26:0/3191082643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd4076e40 0x7f6bd419db80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 >> 172.21.15.26:0/3191082643 conn(0x7f6bd40ff6b0 msgr2=0x7f6bd4100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.658+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 shutdown_connections 2024-06-13T06:46:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:48.659+0000 7f6bd9000700 1 -- 172.21.15.26:0/3191082643 wait complete. 2024-06-13T06:46:49.539 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi135 2024-06-13T06:46:49.539 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:46:49.539 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.conf 2024-06-13T06:46:49.557 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:46:49.557 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:46:49.614 INFO:tasks.cephadm:Adding host smithi135 to orchestrator... 2024-06-13T06:46:49.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch host add smithi135 2024-06-13T06:46:49.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:49 smithi026 ceph-mon[32041]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:49.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:49 smithi026 ceph-mon[32041]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:49.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:49.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:49.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:49.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:49.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:49 smithi026 ceph-mon[32041]: Updating smithi026:/etc/ceph/ceph.conf 2024-06-13T06:46:50.031 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:46:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:50 smithi026 ceph-mon[32041]: Updating smithi026:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:46:51.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:51 smithi026 ceph-mon[32041]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:46:51.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:51 smithi026 ceph-mon[32041]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:51.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:51 smithi026 ceph-mon[32041]: Updating smithi026:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.client.admin.keyring 2024-06-13T06:46:51.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:51.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:51.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.105+0000 7f4ee5600700 1 -- 172.21.15.26:0/3288097828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 msgr2=0x7f4ee01024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.105+0000 7f4ee5600700 1 --2- 172.21.15.26:0/3288097828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 0x7f4ee01024e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009910 tx=0x7f4ed0009c20 comp rx=0 tx=0).stop 2024-06-13T06:46:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.106+0000 7f4ee5600700 1 -- 172.21.15.26:0/3288097828 shutdown_connections 2024-06-13T06:46:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.106+0000 7f4ee5600700 1 --2- 172.21.15.26:0/3288097828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 0x7f4ee01024e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.106+0000 7f4ee5600700 1 -- 172.21.15.26:0/3288097828 >> 172.21.15.26:0/3288097828 conn(0x7f4ee00fd9d0 msgr2=0x7f4ee00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.106+0000 7f4ee5600700 1 -- 172.21.15.26:0/3288097828 shutdown_connections 2024-06-13T06:46:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.106+0000 7f4ee5600700 1 -- 172.21.15.26:0/3288097828 wait complete. 2024-06-13T06:46:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.107+0000 7f4ee5600700 1 Processor -- start 2024-06-13T06:46:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.107+0000 7f4ee5600700 1 -- start start 2024-06-13T06:46:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.108+0000 7f4ee5600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 0x7f4ee0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.108+0000 7f4ee5600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010c4d0 con 0x7f4ee0102100 2024-06-13T06:46:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.108+0000 7f4edea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 0x7f4ee0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.108+0000 7f4edea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 0x7f4ee0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52786/0 (socket says 172.21.15.26:52786) 2024-06-13T06:46:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.108+0000 7f4edea00700 1 -- 172.21.15.26:0/3007871078 learned_addr learned my addr 172.21.15.26:0/3007871078 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:46:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.109+0000 7f4edea00700 1 -- 172.21.15.26:0/3007871078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee0076680 con 0x7f4ee0102100 2024-06-13T06:46:52.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.109+0000 7f4edea00700 1 --2- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 0x7f4ee0074790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006010 tx=0x7f4ed0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:52.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.109+0000 7f4ed7400700 1 -- 172.21.15.26:0/3007871078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed0026070 con 0x7f4ee0102100 2024-06-13T06:46:52.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.109+0000 7f4ed7400700 1 -- 172.21.15.26:0/3007871078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed000bc30 con 0x7f4ee0102100 2024-06-13T06:46:52.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.109+0000 7f4ed7400700 1 -- 172.21.15.26:0/3007871078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed00039d0 con 0x7f4ee0102100 2024-06-13T06:46:52.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.109+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee0074d30 con 0x7f4ee0102100 2024-06-13T06:46:52.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.110+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee00751f0 con 0x7f4ee0102100 2024-06-13T06:46:52.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.111+0000 7f4ed7400700 1 -- 172.21.15.26:0/3007871078 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4ed0029070 con 0x7f4ee0102100 2024-06-13T06:46:52.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.111+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee01035e0 con 0x7f4ee0102100 2024-06-13T06:46:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.111+0000 7f4ed7400700 1 --2- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ec40390f0 0x7f4ec403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.111+0000 7f4ed7400700 1 -- 172.21.15.26:0/3007871078 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4ed001e070 con 0x7f4ee0102100 2024-06-13T06:46:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.112+0000 7f4ed7e00700 1 --2- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ec40390f0 0x7f4ec403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.112+0000 7f4ed7e00700 1 --2- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ec40390f0 0x7f4ec403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8006fd0 tx=0x7f4ec8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.117+0000 7f4ed7400700 1 -- 172.21.15.26:0/3007871078 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4ed0013070 con 0x7f4ee0102100 2024-06-13T06:46:52.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:52.349+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}) -- 0x7f4ee0075cc0 con 0x7f4ec40390f0 2024-06-13T06:46:54.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:53 smithi026 ceph-mon[32041]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:54.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:53 smithi026 ceph-mon[32041]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:54.666 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:54 smithi026 ceph-mon[32041]: Deploying cephadm binary to smithi135 2024-06-13T06:46:56.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:55 smithi026 ceph-mon[32041]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:57.584 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi135' with addr '172.21.15.135' 2024-06-13T06:46:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.582+0000 7f4ed7400700 1 -- 172.21.15.26:0/3007871078 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4ee0075cc0 con 0x7f4ec40390f0 2024-06-13T06:46:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.586+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ec40390f0 msgr2=0x7f4ec403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.586+0000 7f4ee5600700 1 --2- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ec40390f0 0x7f4ec403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8006fd0 tx=0x7f4ec8006e40 comp rx=0 tx=0).stop 2024-06-13T06:46:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.587+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 msgr2=0x7f4ee0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.587+0000 7f4ee5600700 1 --2- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 0x7f4ee0074790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006010 tx=0x7f4ed0005030 comp rx=0 tx=0).stop 2024-06-13T06:46:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.587+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 shutdown_connections 2024-06-13T06:46:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.587+0000 7f4ee5600700 1 --2- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ec40390f0 0x7f4ec403b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.587+0000 7f4ee5600700 1 --2- 172.21.15.26:0/3007871078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ee0102100 0x7f4ee0074790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.587+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 >> 172.21.15.26:0/3007871078 conn(0x7f4ee00fd9d0 msgr2=0x7f4ee0104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.587+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 shutdown_connections 2024-06-13T06:46:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:46:57.587+0000 7f4ee5600700 1 -- 172.21.15.26:0/3007871078 wait complete. 2024-06-13T06:46:57.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:57 smithi026 ceph-mon[32041]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:46:57.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:57.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:58.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch host ls --format=json 2024-06-13T06:46:58.945 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:46:59.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:58 smithi026 ceph-mon[32041]: Added host smithi135 2024-06-13T06:46:59.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:46:59.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:46:59 smithi026 ceph-mon[32041]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.002+0000 7f6c87200700 1 -- 172.21.15.26:0/1557487162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 msgr2=0x7f6c8810e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.002+0000 7f6c87200700 1 --2- 172.21.15.26:0/1557487162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 0x7f6c8810e7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009910 tx=0x7f6c78009c20 comp rx=0 tx=0).stop 2024-06-13T06:47:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.002+0000 7f6c87200700 1 -- 172.21.15.26:0/1557487162 shutdown_connections 2024-06-13T06:47:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.002+0000 7f6c87200700 1 --2- 172.21.15.26:0/1557487162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 0x7f6c8810e7b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.002+0000 7f6c87200700 1 -- 172.21.15.26:0/1557487162 >> 172.21.15.26:0/1557487162 conn(0x7f6c88077aa0 msgr2=0x7f6c88079ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:01.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.002+0000 7f6c87200700 1 -- 172.21.15.26:0/1557487162 shutdown_connections 2024-06-13T06:47:01.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.002+0000 7f6c87200700 1 -- 172.21.15.26:0/1557487162 wait complete. 2024-06-13T06:47:01.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.003+0000 7f6c87200700 1 Processor -- start 2024-06-13T06:47:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.003+0000 7f6c87200700 1 -- start start 2024-06-13T06:47:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.004+0000 7f6c87200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 0x7f6c8819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.004+0000 7f6c87200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88112920 con 0x7f6c8810e3d0 2024-06-13T06:47:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.004+0000 7f6c85e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 0x7f6c8819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.004+0000 7f6c85e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 0x7f6c8819dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54916/0 (socket says 172.21.15.26:54916) 2024-06-13T06:47:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.004+0000 7f6c85e00700 1 -- 172.21.15.26:0/2746982496 learned_addr learned my addr 172.21.15.26:0/2746982496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:47:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.004+0000 7f6c85e00700 1 -- 172.21.15.26:0/2746982496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8819e260 con 0x7f6c8810e3d0 2024-06-13T06:47:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.005+0000 7f6c85e00700 1 --2- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 0x7f6c8819dd20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c78006010 tx=0x7f6c78005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.005+0000 7f6c7ea00700 1 -- 172.21.15.26:0/2746982496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c78026070 con 0x7f6c8810e3d0 2024-06-13T06:47:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.005+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8819e550 con 0x7f6c8810e3d0 2024-06-13T06:47:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.005+0000 7f6c7ea00700 1 -- 172.21.15.26:0/2746982496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7800bbd0 con 0x7f6c8810e3d0 2024-06-13T06:47:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.005+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c881a25f0 con 0x7f6c8810e3d0 2024-06-13T06:47:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.005+0000 7f6c7ea00700 1 -- 172.21.15.26:0/2746982496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c780039f0 con 0x7f6c8810e3d0 2024-06-13T06:47:01.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.006+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c88109a10 con 0x7f6c8810e3d0 2024-06-13T06:47:01.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.007+0000 7f6c7ea00700 1 -- 172.21.15.26:0/2746982496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6c78029070 con 0x7f6c8810e3d0 2024-06-13T06:47:01.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.007+0000 7f6c7ea00700 1 --2- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c6c0390f0 0x7f6c6c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:01.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.007+0000 7f6c7ea00700 1 -- 172.21.15.26:0/2746982496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c7801e070 con 0x7f6c8810e3d0 2024-06-13T06:47:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.007+0000 7f6c85400700 1 --2- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c6c0390f0 0x7f6c6c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.008+0000 7f6c85400700 1 --2- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c6c0390f0 0x7f6c6c03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006fd0 tx=0x7f6c74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.013+0000 7f6c7ea00700 1 -- 172.21.15.26:0/2746982496 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6c78013070 con 0x7f6c8810e3d0 2024-06-13T06:47:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.250+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c8819f0f0 con 0x7f6c6c0390f0 2024-06-13T06:47:01.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.253+0000 7f6c7ea00700 1 -- 172.21.15.26:0/2746982496 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f6c8819f0f0 con 0x7f6c6c0390f0 2024-06-13T06:47:01.253 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:47:01.253 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}] 2024-06-13T06:47:01.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.256+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c6c0390f0 msgr2=0x7f6c6c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:01.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.256+0000 7f6c87200700 1 --2- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c6c0390f0 0x7f6c6c03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006fd0 tx=0x7f6c74006e40 comp rx=0 tx=0).stop 2024-06-13T06:47:01.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.256+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 msgr2=0x7f6c8819dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:01.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.256+0000 7f6c87200700 1 --2- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 0x7f6c8819dd20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c78006010 tx=0x7f6c78005030 comp rx=0 tx=0).stop 2024-06-13T06:47:01.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.257+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 shutdown_connections 2024-06-13T06:47:01.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.257+0000 7f6c87200700 1 --2- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c6c0390f0 0x7f6c6c03b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:01.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.257+0000 7f6c87200700 1 --2- 172.21.15.26:0/2746982496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c8810e3d0 0x7f6c8819dd20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:01.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.257+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 >> 172.21.15.26:0/2746982496 conn(0x7f6c88077aa0 msgr2=0x7f6c880792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:01.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.257+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 shutdown_connections 2024-06-13T06:47:01.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:01.257+0000 7f6c87200700 1 -- 172.21.15.26:0/2746982496 wait complete. 2024-06-13T06:47:01.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:01 smithi026 ceph-mon[32041]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:01.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:02.018 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-06-13T06:47:02.018 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:47:02.018 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-06-13T06:47:02.036 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:47:02.036 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:47:02.093 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-06-13T06:47:02.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch host add smithi136 2024-06-13T06:47:02.540 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:47:02.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:02 smithi026 ceph-mon[32041]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:47:03.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:03 smithi026 ceph-mon[32041]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.776+0000 7f2e35400700 1 -- 172.21.15.26:0/2212631118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 msgr2=0x7f2e30108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.776+0000 7f2e35400700 1 --2- 172.21.15.26:0/2212631118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 0x7f2e30108170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009910 tx=0x7f2e20009c20 comp rx=0 tx=0).stop 2024-06-13T06:47:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.777+0000 7f2e35400700 1 -- 172.21.15.26:0/2212631118 shutdown_connections 2024-06-13T06:47:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.777+0000 7f2e35400700 1 --2- 172.21.15.26:0/2212631118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 0x7f2e30108170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.777+0000 7f2e35400700 1 -- 172.21.15.26:0/2212631118 >> 172.21.15.26:0/2212631118 conn(0x7f2e30076740 msgr2=0x7f2e30076b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.777+0000 7f2e35400700 1 -- 172.21.15.26:0/2212631118 shutdown_connections 2024-06-13T06:47:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.777+0000 7f2e35400700 1 -- 172.21.15.26:0/2212631118 wait complete. 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-13T06:47:04.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:04 smithi026 ceph-mon[32041]: Updating smithi135:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.778+0000 7f2e35400700 1 Processor -- start 2024-06-13T06:47:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.779+0000 7f2e35400700 1 -- start start 2024-06-13T06:47:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.779+0000 7f2e35400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 0x7f2e3019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.779+0000 7f2e35400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3010c310 con 0x7f2e30107d90 2024-06-13T06:47:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.779+0000 7f2e2fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 0x7f2e3019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.780+0000 7f2e2fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 0x7f2e3019f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43478/0 (socket says 172.21.15.26:43478) 2024-06-13T06:47:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.780+0000 7f2e2fe00700 1 -- 172.21.15.26:0/3911398885 learned_addr learned my addr 172.21.15.26:0/3911398885 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:47:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.780+0000 7f2e2fe00700 1 -- 172.21.15.26:0/3911398885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3019fbb0 con 0x7f2e30107d90 2024-06-13T06:47:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.780+0000 7f2e2fe00700 1 --2- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 0x7f2e3019f670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e20006010 tx=0x7f2e20005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.781+0000 7f2e2cc00700 1 -- 172.21.15.26:0/3911398885 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e20026070 con 0x7f2e30107d90 2024-06-13T06:47:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.781+0000 7f2e2cc00700 1 -- 172.21.15.26:0/3911398885 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e2000bbd0 con 0x7f2e30107d90 2024-06-13T06:47:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.781+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3019fea0 con 0x7f2e30107d90 2024-06-13T06:47:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.781+0000 7f2e2cc00700 1 -- 172.21.15.26:0/3911398885 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e20003910 con 0x7f2e30107d90 2024-06-13T06:47:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.781+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e301997f0 con 0x7f2e30107d90 2024-06-13T06:47:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.782+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e30103440 con 0x7f2e30107d90 2024-06-13T06:47:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.782+0000 7f2e2cc00700 1 -- 172.21.15.26:0/3911398885 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2e20029070 con 0x7f2e30107d90 2024-06-13T06:47:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.782+0000 7f2e2cc00700 1 --2- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e14039150 0x7f2e1403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.783+0000 7f2e2cc00700 1 -- 172.21.15.26:0/3911398885 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e2001e070 con 0x7f2e30107d90 2024-06-13T06:47:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.783+0000 7f2e2f400700 1 --2- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e14039150 0x7f2e1403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:04.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.784+0000 7f2e2f400700 1 --2- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e14039150 0x7f2e1403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006fd0 tx=0x7f2e1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:04.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:04.788+0000 7f2e2cc00700 1 -- 172.21.15.26:0/3911398885 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2e20013070 con 0x7f2e30107d90 2024-06-13T06:47:05.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:05.032+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) -- 0x7f2e3019ad20 con 0x7f2e14039150 2024-06-13T06:47:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:05 smithi026 ceph-mon[32041]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:05 smithi026 ceph-mon[32041]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:47:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:05 smithi026 ceph-mon[32041]: Updating smithi135:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.client.admin.keyring 2024-06-13T06:47:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:07.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:06 smithi026 ceph-mon[32041]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:07.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:06 smithi026 ceph-mon[32041]: Deploying cephadm binary to smithi136 2024-06-13T06:47:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:07 smithi026 ceph-mon[32041]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:09.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.883+0000 7f2e2cc00700 1 -- 172.21.15.26:0/3911398885 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2e3019ad20 con 0x7f2e14039150 2024-06-13T06:47:09.883 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-06-13T06:47:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.887+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e14039150 msgr2=0x7f2e1403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.887+0000 7f2e35400700 1 --2- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e14039150 0x7f2e1403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006fd0 tx=0x7f2e1c006e40 comp rx=0 tx=0).stop 2024-06-13T06:47:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.887+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 msgr2=0x7f2e3019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.887+0000 7f2e35400700 1 --2- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 0x7f2e3019f670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e20006010 tx=0x7f2e20005030 comp rx=0 tx=0).stop 2024-06-13T06:47:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.887+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 shutdown_connections 2024-06-13T06:47:09.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.887+0000 7f2e35400700 1 --2- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e14039150 0x7f2e1403b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:09.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.887+0000 7f2e35400700 1 --2- 172.21.15.26:0/3911398885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e30107d90 0x7f2e3019f670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:09.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.887+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 >> 172.21.15.26:0/3911398885 conn(0x7f2e30076740 msgr2=0x7f2e3010a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:09.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.888+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 shutdown_connections 2024-06-13T06:47:09.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:09.888+0000 7f2e35400700 1 -- 172.21.15.26:0/3911398885 wait complete. 2024-06-13T06:47:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:09 smithi026 ceph-mon[32041]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:10.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch host ls --format=json 2024-06-13T06:47:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:47:11.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:11.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:10 smithi026 ceph-mon[32041]: Added host smithi136 2024-06-13T06:47:11.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:11.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:12.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:11 smithi026 ceph-mon[32041]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:12.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.307+0000 7f04bde00700 1 -- 172.21.15.26:0/2722299715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 msgr2=0x7f04b8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.308+0000 7f04bde00700 1 --2- 172.21.15.26:0/2722299715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 0x7f04b8100830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009910 tx=0x7f04ac009c20 comp rx=0 tx=0).stop 2024-06-13T06:47:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.308+0000 7f04bde00700 1 -- 172.21.15.26:0/2722299715 shutdown_connections 2024-06-13T06:47:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.308+0000 7f04bde00700 1 --2- 172.21.15.26:0/2722299715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 0x7f04b8100830 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.308+0000 7f04bde00700 1 -- 172.21.15.26:0/2722299715 >> 172.21.15.26:0/2722299715 conn(0x7f04b80fbe50 msgr2=0x7f04b80fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.308+0000 7f04bde00700 1 -- 172.21.15.26:0/2722299715 shutdown_connections 2024-06-13T06:47:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.308+0000 7f04bde00700 1 -- 172.21.15.26:0/2722299715 wait complete. 2024-06-13T06:47:13.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.309+0000 7f04bde00700 1 Processor -- start 2024-06-13T06:47:13.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.309+0000 7f04bde00700 1 -- start start 2024-06-13T06:47:13.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.309+0000 7f04bde00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 0x7f04b81013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:13.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.309+0000 7f04bde00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04b810c310 con 0x7f04b8100450 2024-06-13T06:47:13.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.310+0000 7f04bca00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 0x7f04b81013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:13.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.310+0000 7f04bca00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 0x7f04b81013d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55264/0 (socket says 172.21.15.26:55264) 2024-06-13T06:47:13.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.310+0000 7f04bca00700 1 -- 172.21.15.26:0/3619685950 learned_addr learned my addr 172.21.15.26:0/3619685950 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:47:13.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.310+0000 7f04bca00700 1 -- 172.21.15.26:0/3619685950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b81032c0 con 0x7f04b8100450 2024-06-13T06:47:13.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.311+0000 7f04bca00700 1 --2- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 0x7f04b81013d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f04ac006010 tx=0x7f04ac005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:13.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.311+0000 7f04b5600700 1 -- 172.21.15.26:0/3619685950 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04ac026070 con 0x7f04b8100450 2024-06-13T06:47:13.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.311+0000 7f04b5600700 1 -- 172.21.15.26:0/3619685950 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04ac00bc30 con 0x7f04b8100450 2024-06-13T06:47:13.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.311+0000 7f04b5600700 1 -- 172.21.15.26:0/3619685950 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04ac0039d0 con 0x7f04b8100450 2024-06-13T06:47:13.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.311+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b8101970 con 0x7f04b8100450 2024-06-13T06:47:13.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.311+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b8101e30 con 0x7f04b8100450 2024-06-13T06:47:13.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.312+0000 7f04b5600700 1 -- 172.21.15.26:0/3619685950 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f04ac029070 con 0x7f04b8100450 2024-06-13T06:47:13.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.312+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b8063af0 con 0x7f04b8100450 2024-06-13T06:47:13.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.313+0000 7f04b5600700 1 --2- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04a0039150 0x7f04a003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:13.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.313+0000 7f04b5600700 1 -- 172.21.15.26:0/3619685950 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f04ac01e070 con 0x7f04b8100450 2024-06-13T06:47:13.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.313+0000 7f04b7e00700 1 --2- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04a0039150 0x7f04a003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:13.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.314+0000 7f04b7e00700 1 --2- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04a0039150 0x7f04a003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f04a4006fd0 tx=0x7f04a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:13.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.319+0000 7f04b5600700 1 -- 172.21.15.26:0/3619685950 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04ac013070 con 0x7f04b8100450 2024-06-13T06:47:13.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.560+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04b8103530 con 0x7f04a0039150 2024-06-13T06:47:13.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.562+0000 7f04b5600700 1 -- 172.21.15.26:0/3619685950 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f04b8103530 con 0x7f04a0039150 2024-06-13T06:47:13.562 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:47:13.562 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-06-13T06:47:13.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.565+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04a0039150 msgr2=0x7f04a003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:13.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.565+0000 7f04bde00700 1 --2- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04a0039150 0x7f04a003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f04a4006fd0 tx=0x7f04a4006e40 comp rx=0 tx=0).stop 2024-06-13T06:47:13.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.565+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 msgr2=0x7f04b81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:13.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.565+0000 7f04bde00700 1 --2- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 0x7f04b81013d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f04ac006010 tx=0x7f04ac005030 comp rx=0 tx=0).stop 2024-06-13T06:47:13.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.565+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 shutdown_connections 2024-06-13T06:47:13.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.565+0000 7f04bde00700 1 --2- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04a0039150 0x7f04a003b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:13.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.565+0000 7f04bde00700 1 --2- 172.21.15.26:0/3619685950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b8100450 0x7f04b81013d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:13.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.565+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 >> 172.21.15.26:0/3619685950 conn(0x7f04b80fbe50 msgr2=0x7f04b8103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:13.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.566+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 shutdown_connections 2024-06-13T06:47:13.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:13.566+0000 7f04bde00700 1 -- 172.21.15.26:0/3619685950 wait complete. 2024-06-13T06:47:13.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:13 smithi026 ceph-mon[32041]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:13.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:14.320 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-13T06:47:14.320 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd crush tunables default 2024-06-13T06:47:14.784 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:47:14.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:14 smithi026 ceph-mon[32041]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:47:15.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:15 smithi026 ceph-mon[32041]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:16.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.906+0000 7f27e5c00700 1 -- 172.21.15.26:0/1273233533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 msgr2=0x7f27e01024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.906+0000 7f27e5c00700 1 --2- 172.21.15.26:0/1273233533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 0x7f27e01024e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009910 tx=0x7f27d0009c20 comp rx=0 tx=0).stop 2024-06-13T06:47:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.906+0000 7f27e5c00700 1 -- 172.21.15.26:0/1273233533 shutdown_connections 2024-06-13T06:47:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.906+0000 7f27e5c00700 1 --2- 172.21.15.26:0/1273233533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 0x7f27e01024e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.906+0000 7f27e5c00700 1 -- 172.21.15.26:0/1273233533 >> 172.21.15.26:0/1273233533 conn(0x7f27e00fd9d0 msgr2=0x7f27e00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.907+0000 7f27e5c00700 1 -- 172.21.15.26:0/1273233533 shutdown_connections 2024-06-13T06:47:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.907+0000 7f27e5c00700 1 -- 172.21.15.26:0/1273233533 wait complete. 2024-06-13T06:47:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.908+0000 7f27e5c00700 1 Processor -- start 2024-06-13T06:47:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.908+0000 7f27e5c00700 1 -- start start 2024-06-13T06:47:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.909+0000 7f27e5c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 0x7f27e019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.909+0000 7f27e5c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27e010c4d0 con 0x7f27e0102100 2024-06-13T06:47:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.909+0000 7f27dea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 0x7f27e019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:16.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.909+0000 7f27dea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 0x7f27e019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55274/0 (socket says 172.21.15.26:55274) 2024-06-13T06:47:16.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.909+0000 7f27dea00700 1 -- 172.21.15.26:0/542829396 learned_addr learned my addr 172.21.15.26:0/542829396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:47:16.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.909+0000 7f27dea00700 1 -- 172.21.15.26:0/542829396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e019fd60 con 0x7f27e0102100 2024-06-13T06:47:16.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.910+0000 7f27dea00700 1 --2- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 0x7f27e019f820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006010 tx=0x7f27d0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.910+0000 7f27d7400700 1 -- 172.21.15.26:0/542829396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0026070 con 0x7f27e0102100 2024-06-13T06:47:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.910+0000 7f27d7400700 1 -- 172.21.15.26:0/542829396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d000bbd0 con 0x7f27e0102100 2024-06-13T06:47:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.910+0000 7f27d7400700 1 -- 172.21.15.26:0/542829396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0003910 con 0x7f27e0102100 2024-06-13T06:47:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.910+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e01a0050 con 0x7f27e0102100 2024-06-13T06:47:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.910+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01999e0 con 0x7f27e0102100 2024-06-13T06:47:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.911+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e01035e0 con 0x7f27e0102100 2024-06-13T06:47:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.912+0000 7f27d7400700 1 -- 172.21.15.26:0/542829396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f27d0029070 con 0x7f27e0102100 2024-06-13T06:47:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.912+0000 7f27d7400700 1 --2- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f27c4039150 0x7f27c403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.912+0000 7f27d7400700 1 -- 172.21.15.26:0/542829396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27d001e070 con 0x7f27e0102100 2024-06-13T06:47:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.912+0000 7f27de000700 1 --2- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f27c4039150 0x7f27c403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.913+0000 7f27de000700 1 --2- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f27c4039150 0x7f27c403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27cc006fd0 tx=0x7f27cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:16.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:16.918+0000 7f27d7400700 1 -- 172.21.15.26:0/542829396 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27d0050030 con 0x7f27e0102100 2024-06-13T06:47:17.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:16 smithi026 ceph-mon[32041]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-13T06:47:17.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:17.158+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f27e019af10 con 0x7f27e0102100 2024-06-13T06:47:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.049+0000 7f27d7400700 1 -- 172.21.15.26:0/542829396 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f27d0030090 con 0x7f27e0102100 2024-06-13T06:47:18.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:17 smithi026 ceph-mon[32041]: Updating smithi136:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:18.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:17 smithi026 ceph-mon[32041]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:18.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:17 smithi026 ceph-mon[32041]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:47:18.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:17 smithi026 ceph-mon[32041]: Updating smithi136:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.client.admin.keyring 2024-06-13T06:47:18.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:18.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:18.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:18.050 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:17 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/542829396' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-13T06:47:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f27c4039150 msgr2=0x7f27c403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 --2- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f27c4039150 0x7f27c403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27cc006fd0 tx=0x7f27cc006e40 comp rx=0 tx=0).stop 2024-06-13T06:47:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 msgr2=0x7f27e019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 --2- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 0x7f27e019f820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006010 tx=0x7f27d0005030 comp rx=0 tx=0).stop 2024-06-13T06:47:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 shutdown_connections 2024-06-13T06:47:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 --2- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f27c4039150 0x7f27c403b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 --2- 172.21.15.26:0/542829396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e0102100 0x7f27e019f820 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 >> 172.21.15.26:0/542829396 conn(0x7f27e00fd9d0 msgr2=0x7f27e00746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 shutdown_connections 2024-06-13T06:47:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:18.053+0000 7f27e5c00700 1 -- 172.21.15.26:0/542829396 wait complete. 2024-06-13T06:47:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-06-13T06:47:18.847 INFO:tasks.cephadm:Adding mon.a on smithi026 2024-06-13T06:47:18.847 INFO:tasks.cephadm:Adding mon.b on smithi135 2024-06-13T06:47:18.848 INFO:tasks.cephadm:Adding mon.c on smithi136 2024-06-13T06:47:18.848 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch apply mon '3;smithi026:172.21.15.26=a;smithi135:172.21.15.135=b;smithi136:172.21.15.136=c' 2024-06-13T06:47:19.085 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:19 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/542829396' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-13T06:47:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:19 smithi026 ceph-mon[32041]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:19 smithi026 ceph-mon[32041]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:20.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 -- 172.21.15.136:0/1651148055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 msgr2=0x7fe21c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:20.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 --2- 172.21.15.136:0/1651148055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 0x7fe21c101420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009910 tx=0x7fe20c009c20 comp rx=0 tx=0).stop 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 -- 172.21.15.136:0/1651148055 shutdown_connections 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 --2- 172.21.15.136:0/1651148055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 0x7fe21c101420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 -- 172.21.15.136:0/1651148055 >> 172.21.15.136:0/1651148055 conn(0x7fe21c0768f0 msgr2=0x7fe21c076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 -- 172.21.15.136:0/1651148055 shutdown_connections 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 -- 172.21.15.136:0/1651148055 wait complete. 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 Processor -- start 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.532+0000 7fe224386700 1 -- start start 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.533+0000 7fe224386700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 0x7fe21c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.533+0000 7fe224386700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c10b3c0 con 0x7fe21c101040 2024-06-13T06:47:20.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.533+0000 7fe222122700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 0x7fe21c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:20.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.533+0000 7fe222122700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 0x7fe21c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.136:53914/0 (socket says 172.21.15.136:53914) 2024-06-13T06:47:20.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.533+0000 7fe222122700 1 -- 172.21.15.136:0/1767725822 learned_addr learned my addr 172.21.15.136:0/1767725822 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:47:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.534+0000 7fe222122700 1 -- 172.21.15.136:0/1767725822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21c198e10 con 0x7fe21c101040 2024-06-13T06:47:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.534+0000 7fe222122700 1 --2- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 0x7fe21c1988d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006010 tx=0x7fe20c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.535+0000 7fe2137fe700 1 -- 172.21.15.136:0/1767725822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe20c026070 con 0x7fe21c101040 2024-06-13T06:47:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.535+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21c199100 con 0x7fe21c101040 2024-06-13T06:47:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.535+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21c1995c0 con 0x7fe21c101040 2024-06-13T06:47:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.535+0000 7fe2137fe700 1 -- 172.21.15.136:0/1767725822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c00bbd0 con 0x7fe21c101040 2024-06-13T06:47:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.535+0000 7fe2137fe700 1 -- 172.21.15.136:0/1767725822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe20c0038e0 con 0x7fe21c101040 2024-06-13T06:47:20.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.536+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe200005330 con 0x7fe21c101040 2024-06-13T06:47:20.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.537+0000 7fe2137fe700 1 -- 172.21.15.136:0/1767725822 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe20c029070 con 0x7fe21c101040 2024-06-13T06:47:20.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.537+0000 7fe2137fe700 1 --2- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe2080390f0 0x7fe20803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:20.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.537+0000 7fe2137fe700 1 -- 172.21.15.136:0/1767725822 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe20c01e070 con 0x7fe21c101040 2024-06-13T06:47:20.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.537+0000 7fe221921700 1 --2- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe2080390f0 0x7fe20803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:20.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.539+0000 7fe221921700 1 --2- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe2080390f0 0x7fe20803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe218006fd0 tx=0x7fe218006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:20.552 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.553+0000 7fe2137fe700 1 -- 172.21.15.136:0/1767725822 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe20c013070 con 0x7fe21c101040 2024-06-13T06:47:20.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.763+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi135:172.21.15.135=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}) -- 0x7fe200000c80 con 0x7fe2080390f0 2024-06-13T06:47:20.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.778+0000 7fe2137fe700 1 -- 172.21.15.136:0/1767725822 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe200000c80 con 0x7fe2080390f0 2024-06-13T06:47:20.776 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mon update... 2024-06-13T06:47:20.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe2080390f0 msgr2=0x7fe20803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:20.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 --2- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe2080390f0 0x7fe20803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe218006fd0 tx=0x7fe218006e40 comp rx=0 tx=0).stop 2024-06-13T06:47:20.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 msgr2=0x7fe21c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:20.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 --2- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 0x7fe21c1988d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006010 tx=0x7fe20c005030 comp rx=0 tx=0).stop 2024-06-13T06:47:20.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 shutdown_connections 2024-06-13T06:47:20.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 --2- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe2080390f0 0x7fe20803b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:20.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 --2- 172.21.15.136:0/1767725822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe21c101040 0x7fe21c1988d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:20.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 >> 172.21.15.136:0/1767725822 conn(0x7fe21c0768f0 msgr2=0x7fe21c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:20.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 shutdown_connections 2024-06-13T06:47:20.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:20.780+0000 7fe224386700 1 -- 172.21.15.136:0/1767725822 wait complete. 2024-06-13T06:47:21.384 DEBUG:teuthology.orchestra.run.smithi135:mon.b> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.b.service 2024-06-13T06:47:21.387 DEBUG:teuthology.orchestra.run.smithi136:mon.c> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.c.service 2024-06-13T06:47:21.390 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-13T06:47:21.390 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph mon dump -f json 2024-06-13T06:47:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi135:172.21.15.135=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi135:172.21.15.135=b;smithi136:172.21.15.136=c;count:3 2024-06-13T06:47:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:22.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:22.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:22.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:22.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:22.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:22.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:21 smithi026 ceph-mon[32041]: Deploying daemon mon.c on smithi136 2024-06-13T06:47:22.485 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:24.054 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:23 smithi026 ceph-mon[32041]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 -- 172.21.15.136:0/3772954599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 msgr2=0x7f31cc108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 --2- 172.21.15.136:0/3772954599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 0x7f31cc108290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009910 tx=0x7f31c8009c20 comp rx=0 tx=0).stop 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 -- 172.21.15.136:0/3772954599 shutdown_connections 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 --2- 172.21.15.136:0/3772954599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 0x7f31cc108290 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 -- 172.21.15.136:0/3772954599 >> 172.21.15.136:0/3772954599 conn(0x7f31cc078320 msgr2=0x7f31cc078730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 -- 172.21.15.136:0/3772954599 shutdown_connections 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 -- 172.21.15.136:0/3772954599 wait complete. 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 Processor -- start 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 -- start start 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 0x7f31cc19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:25.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.094+0000 7f31d462a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc10c4d0 con 0x7f31cc107eb0 2024-06-13T06:47:25.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.095+0000 7f31d23c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 0x7f31cc19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:25.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.095+0000 7f31d23c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 0x7f31cc19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.136:50958/0 (socket says 172.21.15.136:50958) 2024-06-13T06:47:25.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.095+0000 7f31d23c6700 1 -- 172.21.15.136:0/2853212963 learned_addr learned my addr 172.21.15.136:0/2853212963 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:47:25.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.096+0000 7f31d23c6700 1 -- 172.21.15.136:0/2853212963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc19dbf0 con 0x7f31cc107eb0 2024-06-13T06:47:25.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.096+0000 7f31d23c6700 1 --2- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 0x7f31cc19d6b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005850 tx=0x7f31c8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:25.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.096+0000 7f31c37fe700 1 -- 172.21.15.136:0/2853212963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c8026070 con 0x7f31cc107eb0 2024-06-13T06:47:25.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.096+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31cc19dee0 con 0x7f31cc107eb0 2024-06-13T06:47:25.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.097+0000 7f31c37fe700 1 -- 172.21.15.136:0/2853212963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c800bbd0 con 0x7f31cc107eb0 2024-06-13T06:47:25.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.097+0000 7f31c37fe700 1 -- 172.21.15.136:0/2853212963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c80039b0 con 0x7f31cc107eb0 2024-06-13T06:47:25.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.097+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31cc197830 con 0x7f31cc107eb0 2024-06-13T06:47:25.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.097+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31cc06a070 con 0x7f31cc107eb0 2024-06-13T06:47:25.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.098+0000 7f31c37fe700 1 -- 172.21.15.136:0/2853212963 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f31c8029070 con 0x7f31cc107eb0 2024-06-13T06:47:25.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.098+0000 7f31c37fe700 1 --2- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f31b80390f0 0x7f31b803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:25.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.098+0000 7f31c37fe700 1 -- 172.21.15.136:0/2853212963 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f31c801e070 con 0x7f31cc107eb0 2024-06-13T06:47:25.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.098+0000 7f31d1bc5700 1 --2- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f31b80390f0 0x7f31b803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:25.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.099+0000 7f31d1bc5700 1 --2- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f31b80390f0 0x7f31b803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f31bc006fd0 tx=0x7f31bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:25.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.113+0000 7f31c37fe700 1 -- 172.21.15.136:0/2853212963 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f31c8013070 con 0x7f31cc107eb0 2024-06-13T06:47:25.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.400+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f31cc198d60 con 0x7f31cc107eb0 2024-06-13T06:47:25.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.401+0000 7f31c37fe700 1 -- 172.21.15.136:0/2853212963 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f31c8034090 con 0x7f31cc107eb0 2024-06-13T06:47:25.401 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:47:25.401 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","modified":"2024-06-13T06:45:04.384219Z","created":"2024-06-13T06:45:04.384219Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-13T06:47:25.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.403+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f31b80390f0 msgr2=0x7f31b803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:25.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.403+0000 7f31d462a700 1 --2- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f31b80390f0 0x7f31b803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f31bc006fd0 tx=0x7f31bc006e40 comp rx=0 tx=0).stop 2024-06-13T06:47:25.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.403+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 msgr2=0x7f31cc19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:25.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.403+0000 7f31d462a700 1 --2- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 0x7f31cc19d6b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005850 tx=0x7f31c8005030 comp rx=0 tx=0).stop 2024-06-13T06:47:25.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.403+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 shutdown_connections 2024-06-13T06:47:25.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.403+0000 7f31d462a700 1 --2- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f31b80390f0 0x7f31b803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:25.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.403+0000 7f31d462a700 1 --2- 172.21.15.136:0/2853212963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31cc107eb0 0x7f31cc19d6b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:25.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.403+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 >> 172.21.15.136:0/2853212963 conn(0x7f31cc078320 msgr2=0x7f31cc0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:25.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.404+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 shutdown_connections 2024-06-13T06:47:25.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:25.404+0000 7f31d462a700 1 -- 172.21.15.136:0/2853212963 wait complete. 2024-06-13T06:47:25.404 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-06-13T06:47:25.630 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:25 smithi136 ceph-mon[37229]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-13T06:47:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:26 smithi136 bash[37176]: 6cc9ad3d952baf2177e5d3f3d0ffaca4e825a355eda952df9da78998e763e2d3 2024-06-13T06:47:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:26 smithi136 systemd[1]: Started Ceph mon.c for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:47:27.095 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-13T06:47:27.095 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph mon dump -f json 2024-06-13T06:47:27.331 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:47:30.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:30.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:47:30.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: mon.a calling monitor election 2024-06-13T06:47:30.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:30.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:30.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:30.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:30.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: mon.c calling monitor election 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: monmap epoch 2 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: last_changed 2024-06-13T06:47:25.519130+0000 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:47:30.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: min_mon_release 19 (squid) 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: election_strategy: 1 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: fsmap 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: mgrmap e13: x(active, since 62s) 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: overall HEALTH_OK 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:30.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: mon.a calling monitor election 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: mon.c calling monitor election 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: monmap epoch 2 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: last_changed 2024-06-13T06:47:25.519130+0000 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: min_mon_release 19 (squid) 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: election_strategy: 1 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: fsmap 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: mgrmap e13: x(active, since 62s) 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: overall HEALTH_OK 2024-06-13T06:47:31.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:31.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:31.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.715+0000 7f3b744de700 1 -- 172.21.15.136:0/3234824183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b580035c0 msgr2=0x7f3b58005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:31.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.715+0000 7f3b744de700 1 --2- 172.21.15.136:0/3234824183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b580035c0 0x7f3b58005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c069ef0 tx=0x7f3b6800feb0 comp rx=0 tx=0).stop 2024-06-13T06:47:31.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.715+0000 7f3b744de700 1 -- 172.21.15.136:0/3234824183 shutdown_connections 2024-06-13T06:47:31.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.715+0000 7f3b744de700 1 --2- 172.21.15.136:0/3234824183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b580035c0 0x7f3b58005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:31.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.715+0000 7f3b744de700 1 -- 172.21.15.136:0/3234824183 >> 172.21.15.136:0/3234824183 conn(0x7f3b6c0fc970 msgr2=0x7f3b6c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:31.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.715+0000 7f3b744de700 1 -- 172.21.15.136:0/3234824183 shutdown_connections 2024-06-13T06:47:31.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.715+0000 7f3b744de700 1 -- 172.21.15.136:0/3234824183 wait complete. 2024-06-13T06:47:31.716 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:31 smithi136 ceph-mon[37229]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:31.716 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:31.716 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:31.716 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:31.716 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:31 smithi136 ceph-mon[37229]: Deploying daemon mon.b on smithi135 2024-06-13T06:47:31.716 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.716+0000 7f3b744de700 1 Processor -- start 2024-06-13T06:47:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.716+0000 7f3b744de700 1 -- start start 2024-06-13T06:47:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.716+0000 7f3b744de700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 0x7f3b6c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.716+0000 7f3b744de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b6c19ddb0 0x7f3b6c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.716+0000 7f3b744de700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c10a020 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.716+0000 7f3b744de700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c109ea0 con 0x7f3b580035c0 2024-06-13T06:47:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.717+0000 7f3b7227a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 0x7f3b6c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.717+0000 7f3b7227a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 0x7f3b6c19d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:36072/0 (socket says 172.21.15.136:36072) 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.717+0000 7f3b7227a700 1 -- 172.21.15.136:0/2185233923 learned_addr learned my addr 172.21.15.136:0/2185233923 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.717+0000 7f3b7227a700 1 -- 172.21.15.136:0/2185233923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 msgr2=0x7f3b6c19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.717+0000 7f3b7227a700 1 -- 172.21.15.136:0/2185233923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 msgr2=0x7f3b6c19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.717+0000 7f3b7227a700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 0x7f3b6c19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.717+0000 7f3b7227a700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 0x7f3b6c19d870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.717+0000 7f3b71a79700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b6c19ddb0 0x7f3b6c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.718+0000 7f3b71a79700 1 -- 172.21.15.136:0/2185233923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 msgr2=0x7f3b6c19d870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.718+0000 7f3b71a79700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 0x7f3b6c19d870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.718+0000 7f3b71a79700 1 -- 172.21.15.136:0/2185233923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c19e3f0 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.718+0000 7f3b71a79700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b6c19ddb0 0x7f3b6c197a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00b4d0 tx=0x7f3b5c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.718+0000 7f3b637fe700 1 -- 172.21.15.136:0/2185233923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3b5c011630 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.718+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6c197f70 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.718+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6c198500 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.718+0000 7f3b637fe700 1 -- 172.21.15.136:0/2185233923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c011cb0 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.719+0000 7f3b637fe700 1 -- 172.21.15.136:0/2185233923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3b5c0193b0 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.719+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6c06a6f0 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.720+0000 7f3b637fe700 1 -- 172.21.15.136:0/2185233923 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3b5c018020 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.720+0000 7f3b637fe700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3b50038e90 0x7f3b5003b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.720+0000 7f3b637fe700 1 -- 172.21.15.136:0/2185233923 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3b5c04bc50 con 0x7f3b6c19ddb0 2024-06-13T06:47:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.721+0000 7f3b7227a700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3b50038e90 0x7f3b5003b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.722+0000 7f3b7227a700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3b50038e90 0x7f3b5003b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800fa60 tx=0x7f3b68005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:31.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:31.737+0000 7f3b637fe700 1 -- 172.21.15.136:0/2185233923 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3b5c01f020 con 0x7f3b6c19ddb0 2024-06-13T06:47:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:31 smithi026 ceph-mon[32041]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:31 smithi026 ceph-mon[32041]: Deploying daemon mon.b on smithi135 2024-06-13T06:47:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:32.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.012+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3b6c198f60 con 0x7f3b6c19ddb0 2024-06-13T06:47:32.013 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.013+0000 7f3b637fe700 1 -- 172.21.15.136:0/2185233923 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f3b5c021560 con 0x7f3b6c19ddb0 2024-06-13T06:47:32.013 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:47:32.013 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","modified":"2024-06-13T06:47:25.519130Z","created":"2024-06-13T06:45:04.384219Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-13T06:47:32.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3b50038e90 msgr2=0x7f3b5003b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:32.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3b50038e90 0x7f3b5003b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800fa60 tx=0x7f3b68005c20 comp rx=0 tx=0).stop 2024-06-13T06:47:32.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b6c19ddb0 msgr2=0x7f3b6c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:32.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b6c19ddb0 0x7f3b6c197a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00b4d0 tx=0x7f3b5c00b890 comp rx=0 tx=0).stop 2024-06-13T06:47:32.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 shutdown_connections 2024-06-13T06:47:32.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3b50038e90 0x7f3b5003b350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:32.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b580035c0 0x7f3b6c19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:32.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 --2- 172.21.15.136:0/2185233923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b6c19ddb0 0x7f3b6c197a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:32.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 >> 172.21.15.136:0/2185233923 conn(0x7f3b6c0fc970 msgr2=0x7f3b6c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:32.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 shutdown_connections 2024-06-13T06:47:32.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:32.016+0000 7f3b744de700 1 -- 172.21.15.136:0/2185233923 wait complete. 2024-06-13T06:47:32.016 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-06-13T06:47:32.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:32 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/2185233923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-13T06:47:33.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:32 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/2185233923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-13T06:47:33.585 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-13T06:47:33.585 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph mon dump -f json 2024-06-13T06:47:33.827 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:47:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:33 smithi136 ceph-mon[37229]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:33 smithi026 ceph-mon[32041]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:34.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:34 smithi135 systemd[1]: Started Ceph mon.b for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:47:39.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.698+0000 7fa26dc8c700 1 -- 172.21.15.136:0/1748278746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2540057d0 msgr2=0x7fa254007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:39.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.698+0000 7fa26dc8c700 1 --2- 172.21.15.136:0/1748278746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2540057d0 0x7fa254007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa268100dc0 tx=0x7fa25800fe60 comp rx=0 tx=0).stop 2024-06-13T06:47:39.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.698+0000 7fa26dc8c700 1 -- 172.21.15.136:0/1748278746 shutdown_connections 2024-06-13T06:47:39.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.698+0000 7fa26dc8c700 1 --2- 172.21.15.136:0/1748278746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2540057d0 0x7fa254007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:39.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.698+0000 7fa26dc8c700 1 -- 172.21.15.136:0/1748278746 >> 172.21.15.136:0/1748278746 conn(0x7fa2680fc980 msgr2=0x7fa2680feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:39.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.698+0000 7fa26dc8c700 1 -- 172.21.15.136:0/1748278746 shutdown_connections 2024-06-13T06:47:39.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.699+0000 7fa26dc8c700 1 -- 172.21.15.136:0/1748278746 wait complete. 2024-06-13T06:47:39.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.699+0000 7fa26dc8c700 1 Processor -- start 2024-06-13T06:47:39.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa26dc8c700 1 -- start start 2024-06-13T06:47:39.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa26dc8c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 0x7fa26819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:39.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa26dc8c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 0x7fa268195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:39.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa26dc8c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa268195dd0 0x7fa268196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:39.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa26dc8c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa268107ef0 con 0x7fa268195dd0 2024-06-13T06:47:39.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa26dc8c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa268107d70 con 0x7fa26819bc10 2024-06-13T06:47:39.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa26dc8c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa268108070 con 0x7fa2540057d0 2024-06-13T06:47:39.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa2677fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 0x7fa26819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:39.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa2677fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 0x7fa26819b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:47404/0 (socket says 172.21.15.136:47404) 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.700+0000 7fa2677fe700 1 -- 172.21.15.136:0/3916964707 learned_addr learned my addr 172.21.15.136:0/3916964707 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa2677fe700 1 -- 172.21.15.136:0/3916964707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 msgr2=0x7fa26819b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa2677fe700 1 -- 172.21.15.136:0/3916964707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 msgr2=0x7fa26819b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa2677fe700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 0x7fa26819b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa25effd700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 0x7fa268195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa2677fe700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 0x7fa26819b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa25effd700 1 -- 172.21.15.136:0/3916964707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 msgr2=0x7fa268195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa25effd700 1 -- 172.21.15.136:0/3916964707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 msgr2=0x7fa268195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-13T06:47:39.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa25effd700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 0x7fa268195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-13T06:47:39.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa25effd700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 0x7fa268195890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:47:39.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.701+0000 7fa267fff700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa268195dd0 0x7fa268196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:39.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.702+0000 7fa267fff700 1 -- 172.21.15.136:0/3916964707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 msgr2=0x7fa26819b6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:47:39.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.702+0000 7fa267fff700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 0x7fa26819b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:39.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.702+0000 7fa267fff700 1 -- 172.21.15.136:0/3916964707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 msgr2=0x7fa268195890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:47:39.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.702+0000 7fa267fff700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 0x7fa268195890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:39.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.702+0000 7fa267fff700 1 -- 172.21.15.136:0/3916964707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2681a25a0 con 0x7fa268195dd0 2024-06-13T06:47:39.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.702+0000 7fa267fff700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa268195dd0 0x7fa268196280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa26000bd50 tx=0x7fa26000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:39.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.702+0000 7fa2657fa700 1 -- 172.21.15.136:0/3916964707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260019070 con 0x7fa268195dd0 2024-06-13T06:47:39.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.702+0000 7fa2657fa700 1 -- 172.21.15.136:0/3916964707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26000cd10 con 0x7fa268195dd0 2024-06-13T06:47:39.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.703+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2681a27d0 con 0x7fa268195dd0 2024-06-13T06:47:39.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.703+0000 7fa2657fa700 1 -- 172.21.15.136:0/3916964707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600127e0 con 0x7fa268195dd0 2024-06-13T06:47:39.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.703+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2681a2cc0 con 0x7fa268195dd0 2024-06-13T06:47:39.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.704+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa268063af0 con 0x7fa268195dd0 2024-06-13T06:47:39.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.704+0000 7fa2657fa700 1 -- 172.21.15.136:0/3916964707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa26001c070 con 0x7fa268195dd0 2024-06-13T06:47:39.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.705+0000 7fa2657fa700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa248039580 0x7fa24803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:39.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.705+0000 7fa2657fa700 1 -- 172.21.15.136:0/3916964707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa26004bf70 con 0x7fa268195dd0 2024-06-13T06:47:39.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.705+0000 7fa2677fe700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa248039580 0x7fa24803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:39.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.706+0000 7fa2677fe700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa248039580 0x7fa24803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa25800fa10 tx=0x7fa258005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:39.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:39.722+0000 7fa2657fa700 1 -- 172.21.15.136:0/3916964707 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa260019210 con 0x7fa268195dd0 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: mon.c calling monitor election 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: mon.a calling monitor election 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:40.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: monmap epoch 3 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: last_changed 2024-06-13T06:47:34.585544+0000 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: min_mon_release 19 (squid) 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: election_strategy: 1 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:47:40.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-06-13T06:47:40.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-06-13T06:47:40.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: fsmap 2024-06-13T06:47:40.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:40.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: mgrmap e13: x(active, since 71s) 2024-06-13T06:47:40.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: overall HEALTH_OK 2024-06-13T06:47:40.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:40.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:40.006 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.004+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa26802de60 con 0x7fa268195dd0 2024-06-13T06:47:40.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:40.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: mon.c calling monitor election 2024-06-13T06:47:40.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:47:40.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: mon.a calling monitor election 2024-06-13T06:47:40.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:40.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: monmap epoch 3 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:40.007 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: last_changed 2024-06-13T06:47:34.585544+0000 2024-06-13T06:47:40.008 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:47:40.008 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: min_mon_release 19 (squid) 2024-06-13T06:47:40.008 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: election_strategy: 1 2024-06-13T06:47:40.008 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:47:40.008 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-06-13T06:47:40.008 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-06-13T06:47:40.008 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: fsmap 2024-06-13T06:47:40.008 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:40.009 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: mgrmap e13: x(active, since 71s) 2024-06-13T06:47:40.009 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: overall HEALTH_OK 2024-06-13T06:47:40.009 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:40.009 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:40.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.005+0000 7fa2657fa700 1 -- 172.21.15.136:0/3916964707 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fa260012980 con 0x7fa268195dd0 2024-06-13T06:47:40.009 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-13T06:47:40.009 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":3,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","modified":"2024-06-13T06:47:34.585544Z","created":"2024-06-13T06:45:04.384219Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-13T06:47:40.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa248039580 msgr2=0x7fa24803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:40.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa248039580 0x7fa24803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa25800fa10 tx=0x7fa258005de0 comp rx=0 tx=0).stop 2024-06-13T06:47:40.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa268195dd0 msgr2=0x7fa268196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:40.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa268195dd0 0x7fa268196280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa26000bd50 tx=0x7fa26000be30 comp rx=0 tx=0).stop 2024-06-13T06:47:40.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 shutdown_connections 2024-06-13T06:47:40.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa248039580 0x7fa24803ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:40.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa268195dd0 0x7fa268196280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:40.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2540057d0 0x7fa26819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:40.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 --2- 172.21.15.136:0/3916964707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa26819bc10 0x7fa268195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:40.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 >> 172.21.15.136:0/3916964707 conn(0x7fa2680fc980 msgr2=0x7fa2680fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:40.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 shutdown_connections 2024-06-13T06:47:40.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:40.008+0000 7fa26dc8c700 1 -- 172.21.15.136:0/3916964707 wait complete. 2024-06-13T06:47:40.013 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 3 2024-06-13T06:47:40.497 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-13T06:47:40.498 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph config generate-minimal-conf 2024-06-13T06:47:40.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:40.774 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:40.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:40.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:40 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/3916964707' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-13T06:47:40.775 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:47:41.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:41.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:41.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:41.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:40 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/3916964707' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-13T06:47:41.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: mon.c calling monitor election 2024-06-13T06:47:42.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:47:42.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: mon.a calling monitor election 2024-06-13T06:47:42.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:42.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:47:42.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:42.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:42.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:42.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:42.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:42.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:42.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:42.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:47:42.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: monmap epoch 3 2024-06-13T06:47:42.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:42.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: last_changed 2024-06-13T06:47:34.585544+0000 2024-06-13T06:47:42.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:47:42.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: min_mon_release 19 (squid) 2024-06-13T06:47:42.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: election_strategy: 1 2024-06-13T06:47:42.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:47:42.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-06-13T06:47:42.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: fsmap 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: mgrmap e13: x(active, since 71s) 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: overall HEALTH_OK 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/3916964707' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-13T06:47:42.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: mon.b calling monitor election 2024-06-13T06:47:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: Updating smithi026:/etc/ceph/ceph.conf 2024-06-13T06:47:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-13T06:47:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-13T06:47:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: Updating smithi136:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:42.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: Updating smithi135:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:42.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: mon.b calling monitor election 2024-06-13T06:47:42.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: mon.c calling monitor election 2024-06-13T06:47:42.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: mon.a calling monitor election 2024-06-13T06:47:42.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-13T06:47:42.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: monmap epoch 3 2024-06-13T06:47:42.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:42.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: last_changed 2024-06-13T06:47:34.585544+0000 2024-06-13T06:47:42.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:47:42.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: min_mon_release 19 (squid) 2024-06-13T06:47:42.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: election_strategy: 1 2024-06-13T06:47:42.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:47:42.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-06-13T06:47:42.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-06-13T06:47:42.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: fsmap 2024-06-13T06:47:42.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:42.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: mgrmap e13: x(active, since 73s) 2024-06-13T06:47:42.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: overall HEALTH_OK 2024-06-13T06:47:42.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:42 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: mon.b calling monitor election 2024-06-13T06:47:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: Updating smithi026:/etc/ceph/ceph.conf 2024-06-13T06:47:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-13T06:47:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-13T06:47:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: Updating smithi136:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: Updating smithi135:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: mon.b calling monitor election 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: mon.c calling monitor election 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: mon.a calling monitor election 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: monmap epoch 3 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: last_changed 2024-06-13T06:47:34.585544+0000 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: min_mon_release 19 (squid) 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: election_strategy: 1 2024-06-13T06:47:42.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: fsmap 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: mgrmap e13: x(active, since 73s) 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: overall HEALTH_OK 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:42 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: mon.b calling monitor election 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: Updating smithi026:/etc/ceph/ceph.conf 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: Updating smithi136:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: Updating smithi135:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: mon.b calling monitor election 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: mon.c calling monitor election 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: mon.a calling monitor election 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-13T06:47:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: monmap epoch 3 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: last_changed 2024-06-13T06:47:34.585544+0000 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: created 2024-06-13T06:45:04.384219+0000 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: min_mon_release 19 (squid) 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: election_strategy: 1 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: fsmap 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: mgrmap e13: x(active, since 73s) 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: overall HEALTH_OK 2024-06-13T06:47:42.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.629 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.629 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:42.629 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:42 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.349+0000 7fe8f5400700 1 -- 172.21.15.26:0/3149918032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f0102100 msgr2=0x7fe8f01024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:43.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.349+0000 7fe8f5400700 1 --2- 172.21.15.26:0/3149918032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f0102100 0x7fe8f01024e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009910 tx=0x7fe8e0009c20 comp rx=0 tx=0).stop 2024-06-13T06:47:43.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.349+0000 7fe8f5400700 1 -- 172.21.15.26:0/3149918032 shutdown_connections 2024-06-13T06:47:43.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.349+0000 7fe8f5400700 1 --2- 172.21.15.26:0/3149918032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f0102100 0x7fe8f01024e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:43.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.349+0000 7fe8f5400700 1 -- 172.21.15.26:0/3149918032 >> 172.21.15.26:0/3149918032 conn(0x7fe8f00fd9d0 msgr2=0x7fe8f00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:43.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.349+0000 7fe8f5400700 1 -- 172.21.15.26:0/3149918032 shutdown_connections 2024-06-13T06:47:43.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.349+0000 7fe8f5400700 1 -- 172.21.15.26:0/3149918032 wait complete. 2024-06-13T06:47:43.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.350+0000 7fe8f5400700 1 Processor -- start 2024-06-13T06:47:43.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.350+0000 7fe8f5400700 1 -- start start 2024-06-13T06:47:43.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8f5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8f0102100 0x7fe8f0199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8f5400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f019a380 0x7fe8f019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8f5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f019ec50 0x7fe8f019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8f5400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010c4d0 con 0x7fe8f019a380 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8f5400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010c350 con 0x7fe8f019ec50 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8f5400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010c650 con 0x7fe8f0102100 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8ede00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f019a380 0x7fe8f019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8ede00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f019a380 0x7fe8f019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47556/0 (socket says 172.21.15.26:47556) 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8ede00700 1 -- 172.21.15.26:0/3724490342 learned_addr learned my addr 172.21.15.26:0/3724490342 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:47:43.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8ee800700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8f0102100 0x7fe8f0199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:43.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.351+0000 7fe8ef200700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f019ec50 0x7fe8f019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:43.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.352+0000 7fe8ede00700 1 -- 172.21.15.26:0/3724490342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8f0102100 msgr2=0x7fe8f0199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:43.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.352+0000 7fe8ede00700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8f0102100 0x7fe8f0199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:43.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.352+0000 7fe8ede00700 1 -- 172.21.15.26:0/3724490342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f019ec50 msgr2=0x7fe8f019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:43.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.352+0000 7fe8ede00700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f019ec50 0x7fe8f019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:43.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.352+0000 7fe8ede00700 1 -- 172.21.15.26:0/3724490342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f01a69c0 con 0x7fe8f019a380 2024-06-13T06:47:43.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.352+0000 7fe8ee800700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8f0102100 0x7fe8f0199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:47:43.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.352+0000 7fe8ede00700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f019a380 0x7fe8f019e710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00b780 tx=0x7fe8dc00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:43.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.353+0000 7fe8e7400700 1 -- 172.21.15.26:0/3724490342 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc016070 con 0x7fe8f019a380 2024-06-13T06:47:43.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.353+0000 7fe8e7400700 1 -- 172.21.15.26:0/3724490342 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc0036a0 con 0x7fe8f019a380 2024-06-13T06:47:43.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.353+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f01a6bf0 con 0x7fe8f019a380 2024-06-13T06:47:43.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.353+0000 7fe8e7400700 1 -- 172.21.15.26:0/3724490342 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc011af0 con 0x7fe8f019a380 2024-06-13T06:47:43.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.353+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a7130 con 0x7fe8f019a380 2024-06-13T06:47:43.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.354+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d0005330 con 0x7fe8f019a380 2024-06-13T06:47:43.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.355+0000 7fe8e7400700 1 -- 172.21.15.26:0/3724490342 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe8dc019070 con 0x7fe8f019a380 2024-06-13T06:47:43.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.355+0000 7fe8e7400700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe8d4039580 0x7fe8d403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:43.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.356+0000 7fe8e7400700 1 -- 172.21.15.26:0/3724490342 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe8dc04af80 con 0x7fe8f019a380 2024-06-13T06:47:43.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.356+0000 7fe8ee800700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe8d4039580 0x7fe8d403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:43.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.357+0000 7fe8ee800700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe8d4039580 0x7fe8d403ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0006010 tx=0x7fe8e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:43.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.366+0000 7fe8e7400700 1 -- 172.21.15.26:0/3724490342 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8dc019bb0 con 0x7fe8f019a380 2024-06-13T06:47:43.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: Updating smithi026:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:43.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: Reconfiguring mon.a (unknown last config time)... 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:43.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: Reconfiguring daemon mon.a on smithi026 2024-06-13T06:47:43.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: Updating smithi026:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: Reconfiguring mon.a (unknown last config time)... 2024-06-13T06:47:43.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:43.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:43.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:43.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: Reconfiguring daemon mon.a on smithi026 2024-06-13T06:47:43.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: Updating smithi026:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: Reconfiguring mon.a (unknown last config time)... 2024-06-13T06:47:43.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:43.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:43.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:43.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: Reconfiguring daemon mon.a on smithi026 2024-06-13T06:47:43.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.600+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe8d0005620 con 0x7fe8f019a380 2024-06-13T06:47:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.601+0000 7fe8e7400700 1 -- 172.21.15.26:0/3724490342 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fe8dc01a920 con 0x7fe8f019a380 2024-06-13T06:47:43.601 INFO:teuthology.orchestra.run.smithi026.stdout:# minimal ceph.conf for 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:43.601 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-06-13T06:47:43.601 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:47:43.601 INFO:teuthology.orchestra.run.smithi026.stdout: mon_host = [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-06-13T06:47:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe8d4039580 msgr2=0x7fe8d403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe8d4039580 0x7fe8d403ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0006010 tx=0x7fe8e00058e0 comp rx=0 tx=0).stop 2024-06-13T06:47:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f019a380 msgr2=0x7fe8f019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f019a380 0x7fe8f019e710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00b780 tx=0x7fe8dc00ba90 comp rx=0 tx=0).stop 2024-06-13T06:47:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 shutdown_connections 2024-06-13T06:47:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe8d4039580 0x7fe8d403ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f019ec50 0x7fe8f019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8f0102100 0x7fe8f0199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 --2- 172.21.15.26:0/3724490342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f019a380 0x7fe8f019e710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.604+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 >> 172.21.15.26:0/3724490342 conn(0x7fe8f00fd9d0 msgr2=0x7fe8f00746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.605+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 shutdown_connections 2024-06-13T06:47:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:43.605+0000 7fe8f5400700 1 -- 172.21.15.26:0/3724490342 wait complete. 2024-06-13T06:47:43.972 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:47:43 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:47:43.585+0000 7f683e400700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-06-13T06:47:44.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:44 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3724490342' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:44.405 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-13T06:47:44.405 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:47:44.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2024-06-13T06:47:44.489 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:47:44.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:47:44.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:44 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3724490342' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:44.599 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:44 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3724490342' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:44.618 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:47:44.618 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.conf 2024-06-13T06:47:44.650 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:47:44.650 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:47:44.718 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:47:44.718 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-06-13T06:47:44.745 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:47:44.746 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:47:44.809 INFO:tasks.cephadm:Adding mgr.x on smithi026 2024-06-13T06:47:44.810 INFO:tasks.cephadm:Adding mgr.y on smithi135 2024-06-13T06:47:44.810 INFO:tasks.cephadm:Adding mgr.z on smithi136 2024-06-13T06:47:44.810 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch apply mgr '3;smithi026=x;smithi135=y;smithi136=z' 2024-06-13T06:47:45.075 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:47:45.453 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:45 smithi136 ceph-mon[37229]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:45.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:45 smithi135 ceph-mon[36276]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:45.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:45 smithi026 ceph-mon[32041]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:46.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.596+0000 7f1e94803700 1 -- 172.21.15.136:0/3548666834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e90102b20 msgr2=0x7f1e90104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:46.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.596+0000 7f1e94803700 1 --2- 172.21.15.136:0/3548666834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e90102b20 0x7f1e90104f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009910 tx=0x7f1e80009c20 comp rx=0 tx=0).stop 2024-06-13T06:47:46.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.596+0000 7f1e94803700 1 -- 172.21.15.136:0/3548666834 shutdown_connections 2024-06-13T06:47:46.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.596+0000 7f1e94803700 1 --2- 172.21.15.136:0/3548666834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e90102b20 0x7f1e90104f10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.596+0000 7f1e94803700 1 -- 172.21.15.136:0/3548666834 >> 172.21.15.136:0/3548666834 conn(0x7f1e900fc4d0 msgr2=0x7f1e900fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.596+0000 7f1e94803700 1 -- 172.21.15.136:0/3548666834 shutdown_connections 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.596+0000 7f1e94803700 1 -- 172.21.15.136:0/3548666834 wait complete. 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e94803700 1 Processor -- start 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e94803700 1 -- start start 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e94803700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e90102b20 0x7f1e90102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e94803700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e90100ad0 0x7f1e90100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e94803700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e90101490 0x7f1e901a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e94803700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90108080 con 0x7f1e90100ad0 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e94803700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90107f00 con 0x7f1e90102b20 2024-06-13T06:47:46.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e94803700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90108200 con 0x7f1e90101490 2024-06-13T06:47:46.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e8ed9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e90101490 0x7f1e901a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:46.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e8ed9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e90101490 0x7f1e901a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:34266/0 (socket says 172.21.15.136:34266) 2024-06-13T06:47:46.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e8ed9d700 1 -- 172.21.15.136:0/334448999 learned_addr learned my addr 172.21.15.136:0/334448999 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:47:46.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.597+0000 7f1e8e59c700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e90102b20 0x7f1e90102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:46.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e8ed9d700 1 -- 172.21.15.136:0/334448999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e90102b20 msgr2=0x7f1e90102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:46.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e8ed9d700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e90102b20 0x7f1e90102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:46.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e8dd9b700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e90100ad0 0x7f1e90100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:46.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e8ed9d700 1 -- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e90100ad0 msgr2=0x7f1e90100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:46.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e8ed9d700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e90100ad0 0x7f1e90100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:46.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e8ed9d700 1 -- 172.21.15.136:0/334448999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e901a1ed0 con 0x7f1e90101490 2024-06-13T06:47:46.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e8ed9d700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e90101490 0x7f1e901a1910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800ba50 tx=0x7f1e8800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:46.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e7f7fe700 1 -- 172.21.15.136:0/334448999 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88019070 con 0x7f1e90101490 2024-06-13T06:47:46.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e7f7fe700 1 -- 172.21.15.136:0/334448999 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e88014440 con 0x7f1e90101490 2024-06-13T06:47:46.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e901a21c0 con 0x7f1e90101490 2024-06-13T06:47:46.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e7f7fe700 1 -- 172.21.15.136:0/334448999 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880075b0 con 0x7f1e90101490 2024-06-13T06:47:46.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.598+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e901a27d0 con 0x7f1e90101490 2024-06-13T06:47:46.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.599+0000 7f1e7f7fe700 1 -- 172.21.15.136:0/334448999 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1e8801c070 con 0x7f1e90101490 2024-06-13T06:47:46.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.599+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e90063af0 con 0x7f1e90101490 2024-06-13T06:47:46.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.599+0000 7f1e7f7fe700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e78039580 0x7f1e7803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:46.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.599+0000 7f1e7f7fe700 1 -- 172.21.15.136:0/334448999 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1e8804b750 con 0x7f1e90101490 2024-06-13T06:47:46.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.600+0000 7f1e8e59c700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e78039580 0x7f1e7803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:46.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.601+0000 7f1e8e59c700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e78039580 0x7f1e7803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1e80000c00 tx=0x7f1e8001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:46.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.605+0000 7f1e7f7fe700 1 -- 172.21.15.136:0/334448999 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1e88014980 con 0x7f1e90101490 2024-06-13T06:47:46.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.813+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi135=y;smithi136=z", "target": ["mon-mgr", ""]}) -- 0x7f1e90062960 con 0x7f1e78039580 2024-06-13T06:47:46.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:46.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:46.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:46 smithi136 ceph-mon[37229]: Reconfiguring mon.b (monmap changed)... 2024-06-13T06:47:46.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:46.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:46.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:46.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:46 smithi136 ceph-mon[37229]: Reconfiguring daemon mon.b on smithi135 2024-06-13T06:47:46.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.826+0000 7f1e7f7fe700 1 -- 172.21.15.136:0/334448999 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1e90062960 con 0x7f1e78039580 2024-06-13T06:47:46.826 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mgr update... 2024-06-13T06:47:46.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e78039580 msgr2=0x7f1e7803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:46.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e78039580 0x7f1e7803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1e80000c00 tx=0x7f1e8001a040 comp rx=0 tx=0).stop 2024-06-13T06:47:46.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e90101490 msgr2=0x7f1e901a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:46.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e90101490 0x7f1e901a1910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800ba50 tx=0x7f1e8800bd60 comp rx=0 tx=0).stop 2024-06-13T06:47:46.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 shutdown_connections 2024-06-13T06:47:46.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e78039580 0x7f1e7803ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:46.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e90102b20 0x7f1e90102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:46.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e90101490 0x7f1e901a1910 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:46.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 --2- 172.21.15.136:0/334448999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e90100ad0 0x7f1e90100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:46.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 >> 172.21.15.136:0/334448999 conn(0x7f1e900fc4d0 msgr2=0x7f1e900fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:46.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 shutdown_connections 2024-06-13T06:47:46.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:47:46.828+0000 7f1e94803700 1 -- 172.21.15.136:0/334448999 wait complete. 2024-06-13T06:47:47.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:47.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:47.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:46 smithi026 ceph-mon[32041]: Reconfiguring mon.b (monmap changed)... 2024-06-13T06:47:47.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:47.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:47.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:47.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:46 smithi026 ceph-mon[32041]: Reconfiguring daemon mon.b on smithi135 2024-06-13T06:47:47.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:46 smithi135 ceph-mon[36276]: Reconfiguring mon.b (monmap changed)... 2024-06-13T06:47:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:46 smithi135 ceph-mon[36276]: Reconfiguring daemon mon.b on smithi135 2024-06-13T06:47:47.330 DEBUG:teuthology.orchestra.run.smithi135:mgr.y> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.y.service 2024-06-13T06:47:47.332 DEBUG:teuthology.orchestra.run.smithi136:mgr.z> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.z.service 2024-06-13T06:47:47.334 INFO:tasks.cephadm:Deploying OSDs... 2024-06-13T06:47:47.335 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:47:47.335 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-06-13T06:47:47.396 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-13T06:47:47.396 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-06-13 06:46:45.479995641 +0000 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-06-13 06:41:03.533492723 +0000 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-06-13 06:41:03.533492723 +0000 2024-06-13T06:47:47.452 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-06-13 06:41:03.533492723 +0000 2024-06-13T06:47:47.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-13T06:47:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-06-13T06:47:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-06-13T06:47:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000281574 s, 1.8 MB/s 2024-06-13T06:47:47.537 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-13T06:47:47.599 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-06-13T06:47:47.653 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-13T06:47:47.653 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:47.653 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-06-13T06:47:47.653 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:47.653 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:47.653 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-06-13 06:46:45.541994750 +0000 2024-06-13T06:47:47.653 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-06-13 06:41:04.142483987 +0000 2024-06-13T06:47:47.654 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-06-13 06:41:04.142483987 +0000 2024-06-13T06:47:47.654 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-06-13 06:41:04.142483987 +0000 2024-06-13T06:47:47.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-13T06:47:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-06-13T06:47:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-06-13T06:47:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000278678 s, 1.8 MB/s 2024-06-13T06:47:47.744 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-13T06:47:47.803 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-06-13T06:47:47.858 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-13T06:47:47.858 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:47.859 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-06-13T06:47:47.859 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:47.859 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:47.859 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-06-13 06:46:45.605993830 +0000 2024-06-13T06:47:47.859 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-06-13 06:41:04.746475322 +0000 2024-06-13T06:47:47.859 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-06-13 06:41:04.746475322 +0000 2024-06-13T06:47:47.859 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-06-13 06:41:04.746475322 +0000 2024-06-13T06:47:47.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-13T06:47:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-06-13T06:47:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-06-13T06:47:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000251859 s, 2.0 MB/s 2024-06-13T06:47:47.947 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-13T06:47:48.004 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-06-13T06:47:48.058 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-13T06:47:48.058 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:48.058 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-06-13T06:47:48.059 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:48.059 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:48.059 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-06-13 06:46:45.676992809 +0000 2024-06-13T06:47:48.059 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-06-13 06:41:05.331466930 +0000 2024-06-13T06:47:48.059 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-06-13 06:41:05.331466930 +0000 2024-06-13T06:47:48.059 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-06-13 06:41:05.331466930 +0000 2024-06-13T06:47:48.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-13T06:47:48.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:48.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi135=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:48.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: Saving service mgr spec with placement smithi026=x;smithi135=y;smithi136=z;count:3 2024-06-13T06:47:48.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:48.125 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:48.125 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:48.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:48.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi135=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:48.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: Saving service mgr spec with placement smithi026=x;smithi135=y;smithi136=z;count:3 2024-06-13T06:47:48.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:48.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:48.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:48.134 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-06-13T06:47:48.134 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-06-13T06:47:48.134 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000282586 s, 1.8 MB/s 2024-06-13T06:47:48.137 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-13T06:47:48.193 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-13T06:47:48.194 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-06-13T06:47:48.212 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-13T06:47:48.212 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-13 06:47:34.033080076 +0000 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-13 06:39:31.696316260 +0000 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-13 06:39:31.696316260 +0000 2024-06-13T06:47:48.270 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-13 06:39:31.696316260 +0000 2024-06-13T06:47:48.271 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi135=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: Saving service mgr spec with placement smithi026=x;smithi135=y;smithi136=z;count:3 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:47:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:48.308 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-13T06:47:48.308 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-13T06:47:48.308 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000288968 s, 1.8 MB/s 2024-06-13T06:47:48.310 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-13T06:47:48.367 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-06-13T06:47:48.424 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-13T06:47:48.425 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:48.425 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-06-13T06:47:48.425 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:48.425 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:48.425 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-13 06:47:34.033080076 +0000 2024-06-13T06:47:48.425 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-13 06:39:32.036309115 +0000 2024-06-13T06:47:48.425 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-13 06:39:32.036309115 +0000 2024-06-13T06:47:48.425 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-13 06:39:32.036309115 +0000 2024-06-13T06:47:48.425 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-13T06:47:48.489 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-13T06:47:48.489 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-13T06:47:48.489 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000227374 s, 2.3 MB/s 2024-06-13T06:47:48.490 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-13T06:47:48.548 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-06-13T06:47:48.603 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-13T06:47:48.603 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:48.604 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-13T06:47:48.604 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:48.604 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:48.604 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-13 06:47:34.034080056 +0000 2024-06-13T06:47:48.604 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-13 06:39:32.350302517 +0000 2024-06-13T06:47:48.604 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-13 06:39:32.350302517 +0000 2024-06-13T06:47:48.604 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-13 06:39:32.350302517 +0000 2024-06-13T06:47:48.604 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-13T06:47:48.667 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-13T06:47:48.668 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-13T06:47:48.668 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000224768 s, 2.3 MB/s 2024-06-13T06:47:48.669 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-13T06:47:48.725 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-06-13T06:47:48.781 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-13T06:47:48.781 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:48.781 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-13T06:47:48.781 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:48.781 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:48.782 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-13 06:47:34.034080056 +0000 2024-06-13T06:47:48.782 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-13 06:39:32.673295730 +0000 2024-06-13T06:47:48.782 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-13 06:39:32.673295730 +0000 2024-06-13T06:47:48.782 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-13 06:39:32.673295730 +0000 2024-06-13T06:47:48.782 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-13T06:47:48.844 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-13T06:47:48.844 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-13T06:47:48.844 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000212131 s, 2.4 MB/s 2024-06-13T06:47:48.846 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-13T06:47:48.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:48 smithi136 ceph-mon[37229]: Reconfiguring mon.c (monmap changed)... 2024-06-13T06:47:48.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:48 smithi136 ceph-mon[37229]: Reconfiguring daemon mon.c on smithi136 2024-06-13T06:47:48.902 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-13T06:47:48.902 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-06-13T06:47:48.918 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-13T06:47:48.918 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-06-13 06:47:24.617003919 +0000 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-06-13 06:39:31.709575338 +0000 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-06-13 06:39:31.709575338 +0000 2024-06-13T06:47:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-06-13 06:39:31.709575338 +0000 2024-06-13T06:47:48.974 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-13T06:47:49.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:48 smithi135 ceph-mon[36276]: Reconfiguring mon.c (monmap changed)... 2024-06-13T06:47:49.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:48 smithi135 ceph-mon[36276]: Reconfiguring daemon mon.c on smithi136 2024-06-13T06:47:49.036 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-06-13T06:47:49.037 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-06-13T06:47:49.037 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000185977 s, 2.8 MB/s 2024-06-13T06:47:49.038 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-13T06:47:49.093 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-06-13T06:47:49.148 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-13T06:47:49.148 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:49.148 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-13T06:47:49.148 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:49.148 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:49.148 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-06-13 06:47:24.617003919 +0000 2024-06-13T06:47:49.148 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-06-13 06:39:32.037568433 +0000 2024-06-13T06:47:49.148 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-06-13 06:39:32.037568433 +0000 2024-06-13T06:47:49.149 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-06-13 06:39:32.037568433 +0000 2024-06-13T06:47:49.149 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-13T06:47:49.211 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-06-13T06:47:49.211 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-06-13T06:47:49.211 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000222569 s, 2.3 MB/s 2024-06-13T06:47:49.212 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-13T06:47:49.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:48 smithi026 ceph-mon[32041]: Reconfiguring mon.c (monmap changed)... 2024-06-13T06:47:49.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:48 smithi026 ceph-mon[32041]: Reconfiguring daemon mon.c on smithi136 2024-06-13T06:47:49.268 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-06-13 06:47:24.617003919 +0000 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-06-13 06:39:32.366561508 +0000 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-06-13 06:39:32.366561508 +0000 2024-06-13T06:47:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-06-13 06:39:32.366561508 +0000 2024-06-13T06:47:49.324 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-13T06:47:49.385 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-06-13T06:47:49.385 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-06-13T06:47:49.385 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000232435 s, 2.2 MB/s 2024-06-13T06:47:49.387 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-13T06:47:49.443 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-06-13T06:47:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-13T06:47:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:47:49.499 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-06-13T06:47:49.499 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:47:49.499 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:47:49.499 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-06-13 06:47:24.618003899 +0000 2024-06-13T06:47:49.499 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-06-13 06:39:32.690554689 +0000 2024-06-13T06:47:49.499 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-06-13 06:39:32.690554689 +0000 2024-06-13T06:47:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-06-13 06:39:32.690554689 +0000 2024-06-13T06:47:49.499 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-13T06:47:49.561 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-06-13T06:47:49.561 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-06-13T06:47:49.561 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000180074 s, 2.8 MB/s 2024-06-13T06:47:49.562 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-13T06:47:49.618 INFO:tasks.cephadm:Deploying osd.0 on smithi026 with /dev/vg_nvme/lv_4... 2024-06-13T06:47:49.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-13T06:47:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:50.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:47:50.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:50.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:50.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:49 smithi026 ceph-mon[32041]: Deploying daemon mgr.z on smithi136 2024-06-13T06:47:50.063 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:47:50.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:50.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:50.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:49 smithi136 ceph-mon[37229]: Deploying daemon mgr.z on smithi136 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:50.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:49 smithi135 ceph-mon[36276]: Deploying daemon mgr.z on smithi136 2024-06-13T06:47:51.440 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:51 smithi136 systemd[1]: Started Ceph mgr.z for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:47:51.728 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:51.440+0000 7f3466efd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:47:51.728 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:51.627+0000 7f3466efd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:47:51.729 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:51.728+0000 7f3466efd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:47:52.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:51 smithi026 ceph-mon[32041]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:52.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:52.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:47:52.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:52.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:52.038 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-mon[37229]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:52.038 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.038 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.038 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.038 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:52.038 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:47:52.038 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:52.038 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:52.138 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:51 smithi135 ceph-mon[36276]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:52.139 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.139 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.139 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:52.139 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:52.139 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:47:52.139 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:52.139 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:52.376 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:52 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:52.038+0000 7f3466efd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:47:52.376 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:52 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:52.123+0000 7f3466efd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:47:53.036 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-13T06:47:53.036 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-13T06:47:53.036 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-06-13T06:47:53.036 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-06-13T06:47:53.036 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.038436 s, 273 MB/s 2024-06-13T06:47:53.036 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-06-13T06:47:53.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:52 smithi136 ceph-mon[37229]: Deploying daemon mgr.y on smithi135 2024-06-13T06:47:53.213 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:52 smithi135 ceph-mon[36276]: Deploying daemon mgr.y on smithi135 2024-06-13T06:47:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:52 smithi026 ceph-mon[32041]: Deploying daemon mgr.y on smithi135 2024-06-13T06:47:53.533 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:53 smithi135 systemd[1]: Starting Ceph mgr.y for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:47:53.626 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:53.315+0000 7f3466efd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:47:53.626 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:53.482+0000 7f3466efd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:47:53.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2024-06-13T06:47:53.911 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:53 smithi135 ceph-mon[36276]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:53.911 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:53 smithi135 podman[37643]: 2024-06-13 06:47:53.538719607 +0000 UTC m=+0.016984632 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:47:53.911 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:53 smithi135 podman[37643]: 2024-06-13 06:47:53.651764641 +0000 UTC m=+0.130029694 container create 463b2b5d80ed59d4cd6661e6f1152654c1c58423c45b2de42e519c7ad904caec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-06-13T06:47:53.912 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:53 smithi135 podman[37643]: 2024-06-13 06:47:53.854174878 +0000 UTC m=+0.332439901 container init 463b2b5d80ed59d4cd6661e6f1152654c1c58423c45b2de42e519c7ad904caec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-13T06:47:53.912 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:53 smithi135 podman[37643]: 2024-06-13 06:47:53.856765555 +0000 UTC m=+0.335030568 container start 463b2b5d80ed59d4cd6661e6f1152654c1c58423c45b2de42e519c7ad904caec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-13T06:47:53.912 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:53 smithi135 ceph-mgr[37662]: -- 172.21.15.135:0/2010895063 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc43d885a0 con 0x55fc43d5cc00 2024-06-13T06:47:53.912 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:53 smithi135 bash[37643]: 463b2b5d80ed59d4cd6661e6f1152654c1c58423c45b2de42e519c7ad904caec 2024-06-13T06:47:53.940 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:53 smithi136 ceph-mon[37229]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:53.940 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:53.653+0000 7f3466efd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:47:54.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:53 smithi026 ceph-mon[32041]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:54.190 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:53 smithi135 systemd[1]: Started Ceph mgr.y for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:47:54.190 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:54.008+0000 7fa70397a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:47:54.190 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:54.190+0000 7fa70397a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:47:54.237 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:53.940+0000 7f3466efd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:47:54.237 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:54.032+0000 7f3466efd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:47:54.237 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:54.237+0000 7f3466efd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:47:54.247 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:47:54.532 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:54.292+0000 7fa70397a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:47:54.626 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:54.436+0000 7f3466efd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:47:54.985 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:54.613+0000 7fa70397a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:47:54.985 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:54.701+0000 7fa70397a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:47:55.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:54 smithi026 ceph-mon[32041]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:55.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:55.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-mon[36276]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:55.376 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:55.077+0000 7f3466efd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:47:55.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-mon[37229]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:55.812 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:55.512+0000 7f3466efd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:47:55.812 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:55.616+0000 7f3466efd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:47:55.812 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:55.812+0000 7f3466efd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:47:56.262 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:55.931+0000 7fa70397a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:47:56.262 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:56.095+0000 7fa70397a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:47:56.262 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:56.262+0000 7fa70397a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:47:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 -- 172.21.15.26:0/4248294326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa974074ef0 msgr2=0x7fa974075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:56.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 --2- 172.21.15.26:0/4248294326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa974074ef0 0x7fa974075370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa960009990 tx=0x7fa960009ca0 comp rx=0 tx=0).stop 2024-06-13T06:47:56.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 -- 172.21.15.26:0/4248294326 shutdown_connections 2024-06-13T06:47:56.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 --2- 172.21.15.26:0/4248294326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9740764c0 0x7fa974074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:56.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 --2- 172.21.15.26:0/4248294326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa97410a3b0 0x7fa97410c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:56.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 --2- 172.21.15.26:0/4248294326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa974074ef0 0x7fa974075370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:56.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 -- 172.21.15.26:0/4248294326 >> 172.21.15.26:0/4248294326 conn(0x7fa9740fda40 msgr2=0x7fa9740ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:56.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 -- 172.21.15.26:0/4248294326 shutdown_connections 2024-06-13T06:47:56.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.452+0000 7fa97c400700 1 -- 172.21.15.26:0/4248294326 wait complete. 2024-06-13T06:47:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.453+0000 7fa97c400700 1 Processor -- start 2024-06-13T06:47:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa97c400700 1 -- start start 2024-06-13T06:47:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa97c400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa974074ef0 0x7fa97419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa97c400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9740764c0 0x7fa97419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:56.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa97c400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa97410a3b0 0x7fa9741a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa97c400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa974110330 con 0x7fa9740764c0 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa97c400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741101b0 con 0x7fa974074ef0 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa97c400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741104b0 con 0x7fa97410a3b0 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa979a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa974074ef0 0x7fa97419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa979a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa974074ef0 0x7fa97419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:41136/0 (socket says 172.21.15.26:41136) 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa979a00700 1 -- 172.21.15.26:0/2049000568 learned_addr learned my addr 172.21.15.26:0/2049000568 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.454+0000 7fa97a400700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa97410a3b0 0x7fa9741a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa979000700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9740764c0 0x7fa97419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa979a00700 1 -- 172.21.15.26:0/2049000568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa97410a3b0 msgr2=0x7fa9741a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa979a00700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa97410a3b0 0x7fa9741a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa979a00700 1 -- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9740764c0 msgr2=0x7fa97419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa979a00700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9740764c0 0x7fa97419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa979a00700 1 -- 172.21.15.26:0/2049000568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9741a25d0 con 0x7fa974074ef0 2024-06-13T06:47:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa979000700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9740764c0 0x7fa97419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:47:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa97a400700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa97410a3b0 0x7fa9741a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:47:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa979a00700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa974074ef0 0x7fa97419a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9f0 tx=0x7fa96800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.455+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96800cbd0 con 0x7fa974074ef0 2024-06-13T06:47:56.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.456+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96800cd70 con 0x7fa974074ef0 2024-06-13T06:47:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.456+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9741a28c0 con 0x7fa974074ef0 2024-06-13T06:47:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.456+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968010430 con 0x7fa974074ef0 2024-06-13T06:47:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.456+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741a2e00 con 0x7fa974074ef0 2024-06-13T06:47:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.457+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa9680105d0 con 0x7fa974074ef0 2024-06-13T06:47:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.457+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa974069180 con 0x7fa974074ef0 2024-06-13T06:47:56.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.458+0000 7fa972000700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa95c039330 0x7fa95c03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:56.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.458+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa96804c650 con 0x7fa974074ef0 2024-06-13T06:47:56.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.458+0000 7fa979000700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa95c039330 0x7fa95c03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:56.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.459+0000 7fa979000700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa95c039330 0x7fa95c03b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa96000b8e0 tx=0x7fa96001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:56.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.463+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa968016bd0 con 0x7fa974074ef0 2024-06-13T06:47:56.626 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:56.330+0000 7f3466efd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:47:56.627 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:56.419+0000 7f3466efd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:47:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: Reconfiguring mgr.x (unknown last config time)... 2024-06-13T06:47:56.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:56.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:56.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:56.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-mon[37229]: Reconfiguring daemon mgr.x on smithi026 2024-06-13T06:47:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:56.779+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa9740628d0 con 0x7fa95c039330 2024-06-13T06:47:56.834 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:56.544+0000 7fa70397a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:47:56.834 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:56.634+0000 7fa70397a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:47:56.834 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:56.835+0000 7fa70397a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: Reconfiguring mgr.x (unknown last config time)... 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:56.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:56.836 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:56 smithi135 ceph-mon[36276]: Reconfiguring daemon mgr.x on smithi026 2024-06-13T06:47:56.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:56.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:56.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:56.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: Reconfiguring mgr.x (unknown last config time)... 2024-06-13T06:47:56.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:47:56.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:47:56.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:56.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:56 smithi026 ceph-mon[32041]: Reconfiguring daemon mgr.x on smithi026 2024-06-13T06:47:56.993 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:56.659+0000 7f3466efd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:47:56.993 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:56.788+0000 7f3466efd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:47:56.994 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:56.879+0000 7f3466efd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:47:56.994 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:56.993+0000 7f3466efd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:47:57.283 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:57 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:57.022+0000 7fa70397a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:47:57.626 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:57 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:57.293+0000 7f3466efd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:47:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:57 smithi136 ceph-mon[37229]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:57 smithi136 ceph-mon[37229]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:57 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:57 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:57 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:57.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:57 smithi026 ceph-mon[32041]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:57.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:57 smithi026 ceph-mon[32041]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:57.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:57.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:57.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:58.020 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:57 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:57.610+0000 7fa70397a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:47:58.020 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:58.020+0000 7fa70397a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:47:58.020 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:57 smithi135 ceph-mon[36276]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:58.020 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:57 smithi135 ceph-mon[36276]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:58.021 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:57 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:58.021 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:57 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:58.021 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:57 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:58.282 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:58.115+0000 7fa70397a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:47:58.376 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:57 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:57.891+0000 7f3466efd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:47:58.376 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z[39080]: 2024-06-13T06:47:58.106+0000 7f3466efd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:47:58.628 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:58.315+0000 7fa70397a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:47:58.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:47:58.639+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fa968004da0 con 0x7fa974074ef0 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:58.837+0000 7fa70397a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:58.928+0000 7fa70397a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: Standby manager daemon z started 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:58.928 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:58.929 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:58.929 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:58.929 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:58.929 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:58.929 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:58 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: Standby manager daemon z started 2024-06-13T06:47:59.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-13T06:47:59.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:47:59.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-13T06:47:59.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:47:59.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:59.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:59.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:59.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:58 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: Standby manager daemon z started 2024-06-13T06:47:59.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.? 172.21.15.136:0/2963301272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:58 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:47:59.282 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:59.170+0000 7fa70397a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:47:59.637 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:59.303+0000 7fa70397a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:47:59.637 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:59.398+0000 7fa70397a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:47:59.637 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:59.511+0000 7fa70397a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:47:59.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:59 smithi026 ceph-mon[32041]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:47:59.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:59 smithi026 ceph-mon[32041]: mgrmap e14: x(active, since 90s), standbys: z 2024-06-13T06:47:59.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:47:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-13T06:48:00.033 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:47:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:47:59.813+0000 7fa70397a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:48:00.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:59 smithi135 ceph-mon[36276]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:00.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:59 smithi135 ceph-mon[36276]: mgrmap e14: x(active, since 90s), standbys: z 2024-06-13T06:48:00.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:47:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-13T06:48:00.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:59 smithi136 ceph-mon[37229]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:00.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:59 smithi136 ceph-mon[37229]: mgrmap e14: x(active, since 90s), standbys: z 2024-06-13T06:48:00.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:47:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-13T06:48:00.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:00.651+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mon.2 v2:172.21.15.135:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa968007d10 con 0x7fa974074ef0 2024-06-13T06:48:00.783 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:48:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:48:00.409+0000 7fa70397a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:48:00.783 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 06:48:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y[37658]: 2024-06-13T06:48:00.625+0000 7fa70397a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:48:00.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:00 smithi135 ceph-mon[36276]: Standby manager daemon y started 2024-06-13T06:48:00.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:00 smithi135 ceph-mon[36276]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-13T06:48:00.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:00 smithi135 ceph-mon[36276]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:48:00.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:00 smithi135 ceph-mon[36276]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-13T06:48:00.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:00 smithi135 ceph-mon[36276]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:48:01.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:00 smithi026 ceph-mon[32041]: Standby manager daemon y started 2024-06-13T06:48:01.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:00 smithi026 ceph-mon[32041]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-13T06:48:01.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:00 smithi026 ceph-mon[32041]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:48:01.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:00 smithi026 ceph-mon[32041]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-13T06:48:01.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:00 smithi026 ceph-mon[32041]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:48:01.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:00 smithi136 ceph-mon[37229]: Standby manager daemon y started 2024-06-13T06:48:01.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:00 smithi136 ceph-mon[37229]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-13T06:48:01.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:00 smithi136 ceph-mon[37229]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:48:01.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:00 smithi136 ceph-mon[37229]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-13T06:48:01.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:00 smithi136 ceph-mon[37229]: from='mgr.? 172.21.15.135:0/3102866286' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:48:02.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:01 smithi026 ceph-mon[32041]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:02.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:01 smithi026 ceph-mon[32041]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-06-13T06:48:02.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-13T06:48:02.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:01 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/49788016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]: dispatch 2024-06-13T06:48:02.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:01 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]: dispatch 2024-06-13T06:48:02.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:01 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]': finished 2024-06-13T06:48:02.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:01 smithi026 ceph-mon[32041]: osdmap e5: 1 total, 0 up, 1 in 2024-06-13T06:48:02.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:02.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:01 smithi135 ceph-mon[36276]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:02.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:01 smithi135 ceph-mon[36276]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-06-13T06:48:02.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-13T06:48:02.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:01 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/49788016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]: dispatch 2024-06-13T06:48:02.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:01 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]: dispatch 2024-06-13T06:48:02.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:01 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]': finished 2024-06-13T06:48:02.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:01 smithi135 ceph-mon[36276]: osdmap e5: 1 total, 0 up, 1 in 2024-06-13T06:48:02.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:01 smithi136 ceph-mon[37229]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:01 smithi136 ceph-mon[37229]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-06-13T06:48:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-13T06:48:02.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:01 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/49788016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]: dispatch 2024-06-13T06:48:02.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:01 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]: dispatch 2024-06-13T06:48:02.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:01 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a6b984b-52fb-4d69-9e00-ff59d760e29e"}]': finished 2024-06-13T06:48:02.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:01 smithi136 ceph-mon[37229]: osdmap e5: 1 total, 0 up, 1 in 2024-06-13T06:48:02.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:02.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:02 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2514956932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:03.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:02 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2514956932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:03.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:02 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2514956932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:04.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:03 smithi026 ceph-mon[32041]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:04.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:03 smithi135 ceph-mon[36276]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:04.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:03 smithi136 ceph-mon[37229]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:05 smithi026 ceph-mon[32041]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:06.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:05 smithi135 ceph-mon[36276]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:06.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:05 smithi136 ceph-mon[37229]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:07.879 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:07 smithi026 ceph-mon[32041]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:07 smithi135 ceph-mon[36276]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:07 smithi136 ceph-mon[37229]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:10.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:09 smithi026 ceph-mon[32041]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:10.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:09 smithi135 ceph-mon[36276]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:10.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:09 smithi136 ceph-mon[37229]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:11.929 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:11 smithi136 ceph-mon[37229]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:11.929 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-13T06:48:11.929 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:11 smithi026 ceph-mon[32041]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-13T06:48:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:11 smithi135 ceph-mon[36276]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-13T06:48:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:12 smithi026 ceph-mon[32041]: Deploying daemon osd.0 on smithi026 2024-06-13T06:48:13.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:12 smithi135 ceph-mon[36276]: Deploying daemon osd.0 on smithi026 2024-06-13T06:48:13.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:12 smithi136 ceph-mon[37229]: Deploying daemon osd.0 on smithi026 2024-06-13T06:48:13.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:13 smithi026 ceph-mon[32041]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:14.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:13 smithi135 ceph-mon[36276]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:14.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:13 smithi136 ceph-mon[37229]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:15.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:15 smithi026 ceph-mon[32041]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:15 smithi135 ceph-mon[36276]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:16.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:15 smithi136 ceph-mon[37229]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:18.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:17 smithi026 ceph-mon[32041]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:18.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:17 smithi135 ceph-mon[36276]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:18.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:17 smithi136 ceph-mon[37229]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:20.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:19 smithi026 ceph-mon[32041]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:20.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:19 smithi135 ceph-mon[36276]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:20.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:19 smithi136 ceph-mon[37229]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:21.021 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:21.021 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:21.021 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:21.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:21.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:21.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:21.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:21.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:21.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:22.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:21 smithi136 ceph-mon[37229]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:22.184 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:21 smithi135 ceph-mon[36276]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:21 smithi026 ceph-mon[32041]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:23 smithi136 ceph-mon[37229]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:23 smithi136 ceph-mon[37229]: from='osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:48:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:23 smithi136 ceph-mon[37229]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:48:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:23 smithi026 ceph-mon[32041]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:24.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:24.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:23 smithi026 ceph-mon[32041]: from='osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:48:24.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:23 smithi026 ceph-mon[32041]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:48:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:23 smithi135 ceph-mon[36276]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:23 smithi135 ceph-mon[36276]: from='osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:48:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:23 smithi135 ceph-mon[36276]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:48:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.878+0000 7fa972000700 1 -- 172.21.15.26:0/2049000568 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa9740628d0 con 0x7fa95c039330 2024-06-13T06:48:24.879 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 0 on host 'smithi026' 2024-06-13T06:48:24.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa95c039330 msgr2=0x7fa95c03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa95c039330 0x7fa95c03b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa96000b8e0 tx=0x7fa96001a040 comp rx=0 tx=0).stop 2024-06-13T06:48:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa974074ef0 msgr2=0x7fa97419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa974074ef0 0x7fa97419a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9f0 tx=0x7fa96800ed00 comp rx=0 tx=0).stop 2024-06-13T06:48:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 shutdown_connections 2024-06-13T06:48:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa95c039330 0x7fa95c03b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:24.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa974074ef0 0x7fa97419a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa97410a3b0 0x7fa9741a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.882+0000 7fa97c400700 1 --2- 172.21.15.26:0/2049000568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9740764c0 0x7fa97419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.883+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 >> 172.21.15.26:0/2049000568 conn(0x7fa9740fda40 msgr2=0x7fa9741062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.883+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 shutdown_connections 2024-06-13T06:48:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:48:24.883+0000 7fa97c400700 1 -- 172.21.15.26:0/2049000568 wait complete. 2024-06-13T06:48:25.578 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:25.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-13T06:48:25.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: from='osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:48:25.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: osdmap e6: 1 total, 0 up, 1 in 2024-06-13T06:48:25.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:25.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:48:25.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:25.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:25.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:25.739 DEBUG:teuthology.orchestra.run.smithi026:osd.0> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.0.service 2024-06-13T06:48:25.743 INFO:tasks.cephadm:Deploying osd.1 on smithi135 with /dev/vg_nvme/lv_4... 2024-06-13T06:48:25.744 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-13T06:48:25.770 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:25.770 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-13T06:48:25.770 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: from='osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:48:25.770 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: osdmap e6: 1 total, 0 up, 1 in 2024-06-13T06:48:25.771 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:25.771 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:48:25.771 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:25.771 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:25.771 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-13T06:48:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: from='osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:48:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: osdmap e6: 1 total, 0 up, 1 in 2024-06-13T06:48:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:48:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:25.985 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:48:26.469 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:48:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0[46694]: 2024-06-13T06:48:26.217+0000 7f81fd200700 -1 osd.0 0 waiting for initial osdmap 2024-06-13T06:48:26.469 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:48:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0[46694]: 2024-06-13T06:48:26.231+0000 7f81f3000700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:48:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:26 smithi026 ceph-mon[32041]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:48:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:26 smithi026 ceph-mon[32041]: osdmap e7: 1 total, 0 up, 1 in 2024-06-13T06:48:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:26 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:26 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:26 smithi026 ceph-mon[32041]: from='osd.0 ' entity='osd.0' 2024-06-13T06:48:26.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:26 smithi135 ceph-mon[36276]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:48:26.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:26 smithi135 ceph-mon[36276]: osdmap e7: 1 total, 0 up, 1 in 2024-06-13T06:48:26.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:26 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:26.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:26 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:26.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:26 smithi135 ceph-mon[36276]: from='osd.0 ' entity='osd.0' 2024-06-13T06:48:26.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:26 smithi136 ceph-mon[37229]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:48:26.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:26 smithi136 ceph-mon[37229]: osdmap e7: 1 total, 0 up, 1 in 2024-06-13T06:48:26.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:26 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:26.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:26 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:26.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:26 smithi136 ceph-mon[37229]: from='osd.0 ' entity='osd.0' 2024-06-13T06:48:27.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:27 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:48:27.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:27 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:48:27.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:27 smithi026 ceph-mon[32041]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:27.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:27 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:27.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:27 smithi026 ceph-mon[32041]: osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] boot 2024-06-13T06:48:27.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:27 smithi026 ceph-mon[32041]: osdmap e8: 1 total, 1 up, 1 in 2024-06-13T06:48:27.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:27 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:27 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:48:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:27 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:48:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:27 smithi135 ceph-mon[36276]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:27 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:27 smithi135 ceph-mon[36276]: osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] boot 2024-06-13T06:48:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:27 smithi135 ceph-mon[36276]: osdmap e8: 1 total, 1 up, 1 in 2024-06-13T06:48:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:27 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:27 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:48:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:27 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:48:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:27 smithi136 ceph-mon[37229]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:48:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:27 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:27 smithi136 ceph-mon[37229]: osd.0 [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] boot 2024-06-13T06:48:27.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:27 smithi136 ceph-mon[37229]: osdmap e8: 1 total, 1 up, 1 in 2024-06-13T06:48:27.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:27 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:48:27.933 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-13T06:48:27.933 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-13T06:48:27.933 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-13T06:48:27.933 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-13T06:48:27.933 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113598 s, 923 MB/s 2024-06-13T06:48:27.933 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-13T06:48:28.622 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_4 2024-06-13T06:48:28.896 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:48:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:29 smithi026 ceph-mon[32041]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:29 smithi026 ceph-mon[32041]: osdmap e9: 1 total, 1 up, 1 in 2024-06-13T06:48:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:29 smithi135 ceph-mon[36276]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:29 smithi135 ceph-mon[36276]: osdmap e9: 1 total, 1 up, 1 in 2024-06-13T06:48:29.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:29 smithi136 ceph-mon[37229]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:29.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:29 smithi136 ceph-mon[37229]: osdmap e9: 1 total, 1 up, 1 in 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 -- 172.21.15.135:0/3153492985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52381025f0 msgr2=0x7f5238102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 --2- 172.21.15.135:0/3153492985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52381025f0 0x7f5238102a50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f52340099c0 tx=0x7f5234009cd0 comp rx=0 tx=0).stop 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 -- 172.21.15.135:0/3153492985 shutdown_connections 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 --2- 172.21.15.135:0/3153492985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5238102f90 0x7f5238109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 --2- 172.21.15.135:0/3153492985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52381013f0 0x7f5238101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 --2- 172.21.15.135:0/3153492985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52381025f0 0x7f5238102a50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 -- 172.21.15.135:0/3153492985 >> 172.21.15.135:0/3153492985 conn(0x7f52380fc990 msgr2=0x7f52380fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 -- 172.21.15.135:0/3153492985 shutdown_connections 2024-06-13T06:48:30.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.306+0000 7f52401b2700 1 -- 172.21.15.135:0/3153492985 wait complete. 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f52401b2700 1 Processor -- start 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f52401b2700 1 -- start start 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f52401b2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52381013f0 0x7f52381976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f52401b2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52381025f0 0x7f5238197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f52401b2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5238102f90 0x7f523819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f52401b2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f523810d010 con 0x7f52381025f0 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f52401b2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f523810ce90 con 0x7f5238102f90 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f52401b2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f523810d190 con 0x7f52381013f0 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f523e74f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5238102f90 0x7f523819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:30.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f523e74f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5238102f90 0x7f523819ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53954/0 (socket says 172.21.15.135:53954) 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.307+0000 7f523e74f700 1 -- 172.21.15.135:0/2036417514 learned_addr learned my addr 172.21.15.135:0/2036417514 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523df4e700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52381013f0 0x7f52381976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523d74d700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52381025f0 0x7f5238197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523e74f700 1 -- 172.21.15.135:0/2036417514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52381013f0 msgr2=0x7f52381976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523e74f700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52381013f0 0x7f52381976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523e74f700 1 -- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52381025f0 msgr2=0x7f5238197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523e74f700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52381025f0 0x7f5238197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523e74f700 1 -- 172.21.15.135:0/2036417514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523819f320 con 0x7f5238102f90 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523d74d700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52381025f0 0x7f5238197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:48:30.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f523e74f700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5238102f90 0x7f523819ec80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f523000ada0 tx=0x7f523000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:48:30.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f522effd700 1 -- 172.21.15.135:0/2036417514 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230017020 con 0x7f5238102f90 2024-06-13T06:48:30.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523819f610 con 0x7f5238102f90 2024-06-13T06:48:30.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f522effd700 1 -- 172.21.15.135:0/2036417514 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523000a0d0 con 0x7f5238102f90 2024-06-13T06:48:30.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f522effd700 1 -- 172.21.15.135:0/2036417514 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230005480 con 0x7f5238102f90 2024-06-13T06:48:30.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.308+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523819fba0 con 0x7f5238102f90 2024-06-13T06:48:30.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.309+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5238069180 con 0x7f5238102f90 2024-06-13T06:48:30.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.310+0000 7f522effd700 1 -- 172.21.15.135:0/2036417514 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f5230005620 con 0x7f5238102f90 2024-06-13T06:48:30.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.311+0000 7f522effd700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52240a2b90 0x7f52240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:30.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.311+0000 7f522effd700 1 -- 172.21.15.135:0/2036417514 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f52300cccd0 con 0x7f5238102f90 2024-06-13T06:48:30.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.311+0000 7f523df4e700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52240a2b90 0x7f52240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:30.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.311+0000 7f523df4e700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52240a2b90 0x7f52240a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5238102450 tx=0x7f522800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:48:30.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.315+0000 7f522effd700 1 -- 172.21.15.135:0/2036417514 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5230021020 con 0x7f5238102f90 2024-06-13T06:48:30.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:30.520+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5238062990 con 0x7f52240a2b90 2024-06-13T06:48:31.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:31.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:31.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:31.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:31.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:31.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:31.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:31.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:31.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:31.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:31.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:31.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:31.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:31.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:32 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi026 2024-06-13T06:48:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:32 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi026 to 17094M 2024-06-13T06:48:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:32 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi026 2024-06-13T06:48:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:32 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi026 to 17094M 2024-06-13T06:48:32.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:32 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi026 2024-06-13T06:48:32.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:32 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi026 to 17094M 2024-06-13T06:48:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:33 smithi026 ceph-mon[32041]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:33 smithi026 ceph-mon[32041]: from='client.? 172.21.15.135:0/1873609776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]: dispatch 2024-06-13T06:48:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:33 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]: dispatch 2024-06-13T06:48:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:33 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]': finished 2024-06-13T06:48:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:33 smithi026 ceph-mon[32041]: osdmap e10: 2 total, 1 up, 2 in 2024-06-13T06:48:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:33 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:33 smithi135 ceph-mon[36276]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:33 smithi135 ceph-mon[36276]: from='client.? 172.21.15.135:0/1873609776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]: dispatch 2024-06-13T06:48:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:33 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]: dispatch 2024-06-13T06:48:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:33 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]': finished 2024-06-13T06:48:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:33 smithi135 ceph-mon[36276]: osdmap e10: 2 total, 1 up, 2 in 2024-06-13T06:48:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:33 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:33 smithi136 ceph-mon[37229]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:33 smithi136 ceph-mon[37229]: from='client.? 172.21.15.135:0/1873609776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]: dispatch 2024-06-13T06:48:33.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:33 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]: dispatch 2024-06-13T06:48:33.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:33 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "270b9d62-0c93-48d4-b822-eb2e41ea3bb7"}]': finished 2024-06-13T06:48:33.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:33 smithi136 ceph-mon[37229]: osdmap e10: 2 total, 1 up, 2 in 2024-06-13T06:48:33.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:33 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:34.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:34 smithi135 ceph-mon[36276]: from='client.? 172.21.15.135:0/1701627059' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:34.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:34 smithi136 ceph-mon[37229]: from='client.? 172.21.15.135:0/1701627059' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:35.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:34 smithi026 ceph-mon[32041]: from='client.? 172.21.15.135:0/1701627059' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:35.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:35 smithi135 ceph-mon[36276]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:35.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:35 smithi136 ceph-mon[37229]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:36.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:35 smithi026 ceph-mon[32041]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:37 smithi026 ceph-mon[32041]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:37 smithi135 ceph-mon[36276]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:37 smithi136 ceph-mon[37229]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:39 smithi135 ceph-mon[36276]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:39 smithi136 ceph-mon[37229]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:39 smithi026 ceph-mon[32041]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:40 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-13T06:48:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:40 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:40.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-13T06:48:40.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:41.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-13T06:48:41.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:41 smithi135 ceph-mon[36276]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:41 smithi135 ceph-mon[36276]: Deploying daemon osd.1 on smithi135 2024-06-13T06:48:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:41 smithi136 ceph-mon[37229]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:41 smithi136 ceph-mon[37229]: Deploying daemon osd.1 on smithi135 2024-06-13T06:48:42.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:41 smithi026 ceph-mon[32041]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:42.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:41 smithi026 ceph-mon[32041]: Deploying daemon osd.1 on smithi135 2024-06-13T06:48:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:43 smithi135 ceph-mon[36276]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:43 smithi136 ceph-mon[37229]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:43 smithi026 ceph-mon[32041]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:45.608 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:45 smithi135 ceph-mon[36276]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:45 smithi136 ceph-mon[37229]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:46.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:45 smithi026 ceph-mon[32041]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:47.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:47.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:47.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:47.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:47.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:47.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:47.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:47.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:47.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:47.962 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:47 smithi135 ceph-mon[36276]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:48.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:47 smithi026 ceph-mon[32041]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:48.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:47 smithi136 ceph-mon[37229]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:49.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.005+0000 7f522effd700 1 -- 172.21.15.135:0/2036417514 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5238062990 con 0x7f52240a2b90 2024-06-13T06:48:49.005 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 1 on host 'smithi135' 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52240a2b90 msgr2=0x7f52240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52240a2b90 0x7f52240a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5238102450 tx=0x7f522800a380 comp rx=0 tx=0).stop 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5238102f90 msgr2=0x7f523819ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5238102f90 0x7f523819ec80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f523000ada0 tx=0x7f523000c330 comp rx=0 tx=0).stop 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 shutdown_connections 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52240a2b90 0x7f52240a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5238102f90 0x7f523819ec80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52381013f0 0x7f52381976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 --2- 172.21.15.135:0/2036417514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52381025f0 0x7f5238197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:49.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 >> 172.21.15.135:0/2036417514 conn(0x7f52380fc990 msgr2=0x7f5238105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:49.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 shutdown_connections 2024-06-13T06:48:49.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:48:49.008+0000 7f52401b2700 1 -- 172.21.15.135:0/2036417514 wait complete. 2024-06-13T06:48:49.665 DEBUG:teuthology.orchestra.run.smithi135:osd.1> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.1.service 2024-06-13T06:48:49.668 INFO:tasks.cephadm:Deploying osd.2 on smithi136 with /dev/vg_nvme/lv_4... 2024-06-13T06:48:49.668 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-13T06:48:49.911 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:48:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:49 smithi026 ceph-mon[32041]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:50.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:49 smithi026 ceph-mon[32041]: from='osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:48:50.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:49 smithi026 ceph-mon[32041]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:48:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:49 smithi135 ceph-mon[36276]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:49 smithi135 ceph-mon[36276]: from='osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:48:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:49 smithi135 ceph-mon[36276]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:48:50.309 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:49 smithi136 ceph-mon[37229]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:50.309 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.309 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.310 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.310 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:50.310 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:49 smithi136 ceph-mon[37229]: from='osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:48:50.310 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:49 smithi136 ceph-mon[37229]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:48:51.305 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:51 smithi136 ceph-mon[37229]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-13T06:48:51.305 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:51 smithi136 ceph-mon[37229]: from='osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:48:51.305 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:51 smithi136 ceph-mon[37229]: osdmap e11: 2 total, 1 up, 2 in 2024-06-13T06:48:51.305 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:51.305 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:51 smithi136 ceph-mon[37229]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:48:51.305 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:51 smithi136 ceph-mon[37229]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:51.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:51 smithi026 ceph-mon[32041]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-13T06:48:51.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:51 smithi026 ceph-mon[32041]: from='osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:48:51.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:51 smithi026 ceph-mon[32041]: osdmap e11: 2 total, 1 up, 2 in 2024-06-13T06:48:51.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:51.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:51 smithi026 ceph-mon[32041]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:48:51.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:51 smithi026 ceph-mon[32041]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:51.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:51 smithi135 ceph-mon[36276]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-13T06:48:51.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:51 smithi135 ceph-mon[36276]: from='osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:48:51.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:51 smithi135 ceph-mon[36276]: osdmap e11: 2 total, 1 up, 2 in 2024-06-13T06:48:51.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:51.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:51 smithi135 ceph-mon[36276]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:48:51.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:51 smithi135 ceph-mon[36276]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:51.855 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-13T06:48:51.855 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-13T06:48:51.855 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-06-13T06:48:51.855 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-06-13T06:48:51.855 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00946406 s, 1.1 GB/s 2024-06-13T06:48:51.856 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 2024-06-13T06:48:51.856 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-06-13T06:48:52.126 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:48:51 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1[40842]: 2024-06-13T06:48:51.817+0000 7f5c529a6700 -1 osd.1 0 waiting for initial osdmap 2024-06-13T06:48:52.126 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:48:51 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1[40842]: 2024-06-13T06:48:51.822+0000 7f5c4a791700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:48:52.216 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:52 smithi136 ceph-mon[37229]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:48:52.217 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:52 smithi136 ceph-mon[37229]: osdmap e12: 2 total, 1 up, 2 in 2024-06-13T06:48:52.217 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:52 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.217 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:52 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.217 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:52 smithi136 ceph-mon[37229]: from='osd.1 ' entity='osd.1' 2024-06-13T06:48:52.217 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:52 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.383 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:52 smithi135 ceph-mon[36276]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:48:52.383 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:52 smithi135 ceph-mon[36276]: osdmap e12: 2 total, 1 up, 2 in 2024-06-13T06:48:52.383 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:52 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.383 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:52 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.384 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:52 smithi135 ceph-mon[36276]: from='osd.1 ' entity='osd.1' 2024-06-13T06:48:52.384 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:52 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:52 smithi026 ceph-mon[32041]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:48:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:52 smithi026 ceph-mon[32041]: osdmap e12: 2 total, 1 up, 2 in 2024-06-13T06:48:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:52 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:52 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:52 smithi026 ceph-mon[32041]: from='osd.1 ' entity='osd.1' 2024-06-13T06:48:52.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:52 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:52.511 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-06-13T06:48:52.741 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:48:53.456 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:48:53.456 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:48:53.456 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:53.456 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi135 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] boot 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: osdmap e13: 2 total, 2 up, 2 in 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:53.457 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:53.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:53.458 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:48:53.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi135 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] boot 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: osdmap e13: 2 total, 2 up, 2 in 2024-06-13T06:48:53.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:53.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:53.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:53.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:53.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi135 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: osd.1 [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] boot 2024-06-13T06:48:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: osdmap e13: 2 total, 2 up, 2 in 2024-06-13T06:48:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:48:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:48:54.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.198+0000 7f6a293ba700 1 -- 172.21.15.136:0/3231905843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 msgr2=0x7f6a24102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:54.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.198+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3231905843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 0x7f6a24102900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00b0a0 tx=0x7f6a1c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 -- 172.21.15.136:0/3231905843 shutdown_connections 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3231905843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24104080 0x7f6a2410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3231905843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 0x7f6a24102900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3231905843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a241036e0 0x7f6a24103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 -- 172.21.15.136:0/3231905843 >> 172.21.15.136:0/3231905843 conn(0x7f6a240fda80 msgr2=0x7f6a240ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 -- 172.21.15.136:0/3231905843 shutdown_connections 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 -- 172.21.15.136:0/3231905843 wait complete. 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 Processor -- start 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.199+0000 7f6a293ba700 1 -- start start 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a293ba700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 0x7f6a24198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a293ba700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a241036e0 0x7f6a24198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:54.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a293ba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24104080 0x7f6a2419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a293ba700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e150 con 0x7f6a24104080 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a293ba700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410dfd0 con 0x7f6a241036e0 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a293ba700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e2d0 con 0x7f6a241024e0 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 0x7f6a24198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 0x7f6a24198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:54980/0 (socket says 172.21.15.136:54980) 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 -- 172.21.15.136:0/3252982932 learned_addr learned my addr 172.21.15.136:0/3252982932 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a227fc700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a241036e0 0x7f6a24198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 -- 172.21.15.136:0/3252982932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a241036e0 msgr2=0x7f6a24198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a241036e0 0x7f6a24198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 -- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24104080 msgr2=0x7f6a2419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24104080 0x7f6a2419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 -- 172.21.15.136:0/3252982932 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a241a03f0 con 0x7f6a241024e0 2024-06-13T06:48:54.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a22ffd700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 0x7f6a24198790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c000f80 tx=0x7f6a1c00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:48:54.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a0bfff700 1 -- 172.21.15.136:0/3252982932 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c021030 con 0x7f6a241024e0 2024-06-13T06:48:54.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a0bfff700 1 -- 172.21.15.136:0/3252982932 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c007b30 con 0x7f6a241024e0 2024-06-13T06:48:54.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241a0680 con 0x7f6a241024e0 2024-06-13T06:48:54.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a0bfff700 1 -- 172.21.15.136:0/3252982932 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c005040 con 0x7f6a241024e0 2024-06-13T06:48:54.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.200+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241a0bb0 con 0x7f6a241024e0 2024-06-13T06:48:54.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.201+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a24069180 con 0x7f6a241024e0 2024-06-13T06:48:54.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.202+0000 7f6a0bfff700 1 -- 172.21.15.136:0/3252982932 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f6a1c024070 con 0x7f6a241024e0 2024-06-13T06:48:54.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.203+0000 7f6a0bfff700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a0c0a2b90 0x7f6a0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:54.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.203+0000 7f6a0bfff700 1 -- 172.21.15.136:0/3252982932 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f6a1c00f020 con 0x7f6a241024e0 2024-06-13T06:48:54.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.203+0000 7f6a227fc700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a0c0a2b90 0x7f6a0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:54.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.204+0000 7f6a227fc700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a0c0a2b90 0x7f6a0c0a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a18006fd0 tx=0x7f6a18008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:48:54.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.207+0000 7f6a0bfff700 1 -- 172.21.15.136:0/3252982932 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a1c011390 con 0x7f6a241024e0 2024-06-13T06:48:54.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:48:54.409+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6a240628d0 con 0x7f6a0c0a2b90 2024-06-13T06:48:55.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:55 smithi136 ceph-mon[37229]: osdmap e14: 2 total, 2 up, 2 in 2024-06-13T06:48:55.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:55 smithi136 ceph-mon[37229]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:48:55.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:55 smithi136 ceph-mon[37229]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:55.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:55.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:55.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:55 smithi026 ceph-mon[32041]: osdmap e14: 2 total, 2 up, 2 in 2024-06-13T06:48:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:55 smithi026 ceph-mon[32041]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:48:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:55 smithi026 ceph-mon[32041]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:55.523 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:55 smithi135 ceph-mon[36276]: osdmap e14: 2 total, 2 up, 2 in 2024-06-13T06:48:55.523 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:55 smithi135 ceph-mon[36276]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:48:55.523 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:55 smithi135 ceph-mon[36276]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:55.523 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:55.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:55.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:57.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:57 smithi026 ceph-mon[32041]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:48:57.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:57 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/2850916475' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]: dispatch 2024-06-13T06:48:57.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:57 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]: dispatch 2024-06-13T06:48:57.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:57 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]': finished 2024-06-13T06:48:57.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:57 smithi026 ceph-mon[32041]: osdmap e15: 3 total, 2 up, 3 in 2024-06-13T06:48:57.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:48:57.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:57 smithi135 ceph-mon[36276]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:48:57.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:57 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/2850916475' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]: dispatch 2024-06-13T06:48:57.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:57 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]: dispatch 2024-06-13T06:48:57.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:57 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]': finished 2024-06-13T06:48:57.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:57 smithi135 ceph-mon[36276]: osdmap e15: 3 total, 2 up, 3 in 2024-06-13T06:48:57.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:57 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:48:57.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:57 smithi136 ceph-mon[37229]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:48:57.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:57 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/2850916475' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]: dispatch 2024-06-13T06:48:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:57 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]: dispatch 2024-06-13T06:48:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:57 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f1bc547-0cb8-434f-9467-57da190ce910"}]': finished 2024-06-13T06:48:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:57 smithi136 ceph-mon[37229]: osdmap e15: 3 total, 2 up, 3 in 2024-06-13T06:48:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:57 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:48:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:58 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/765241150' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:58 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/765241150' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:58 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/765241150' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:48:59 smithi026 ceph-mon[32041]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:48:59.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:48:59 smithi135 ceph-mon[36276]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:48:59.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:48:59 smithi136 ceph-mon[37229]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:01.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:01 smithi136 ceph-mon[37229]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:01.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:01 smithi026 ceph-mon[32041]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:01.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:01 smithi135 ceph-mon[36276]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:03.274 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:03 smithi136 ceph-mon[37229]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:03.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:03 smithi026 ceph-mon[32041]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:03.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:03 smithi135 ceph-mon[36276]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:05.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:05 smithi135 ceph-mon[36276]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:05.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-13T06:49:05.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:05.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:05 smithi135 ceph-mon[36276]: Deploying daemon osd.2 on smithi136 2024-06-13T06:49:05.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:05 smithi026 ceph-mon[32041]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:05.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-13T06:49:05.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:05.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:05 smithi026 ceph-mon[32041]: Deploying daemon osd.2 on smithi136 2024-06-13T06:49:05.590 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:05 smithi136 ceph-mon[37229]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:05.590 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-13T06:49:05.590 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:05.590 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:05 smithi136 ceph-mon[37229]: Deploying daemon osd.2 on smithi136 2024-06-13T06:49:07.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:07 smithi026 ceph-mon[32041]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:07.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:07 smithi135 ceph-mon[36276]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:07.603 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:07 smithi136 ceph-mon[37229]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:09.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:09 smithi026 ceph-mon[32041]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:09.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:09 smithi135 ceph-mon[36276]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:09.564 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:09 smithi136 ceph-mon[37229]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:11.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:11.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:11.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:11.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:11.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:11.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:11.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:11.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:11.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:11 smithi026 ceph-mon[32041]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:12.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:11 smithi135 ceph-mon[36276]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:12.082 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:11 smithi136 ceph-mon[37229]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:13.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:13 smithi026 ceph-mon[32041]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:13.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:13.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:13.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:13 smithi135 ceph-mon[36276]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:13.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:13.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:13.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:13 smithi136 ceph-mon[37229]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:13.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:13.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:13.539 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:13.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.595+0000 7f6a0bfff700 1 -- 172.21.15.136:0/3252982932 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6a240628d0 con 0x7f6a0c0a2b90 2024-06-13T06:49:13.595 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 2 on host 'smithi136' 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a0c0a2b90 msgr2=0x7f6a0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a0c0a2b90 0x7f6a0c0a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a18006fd0 tx=0x7f6a18008040 comp rx=0 tx=0).stop 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 msgr2=0x7f6a24198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 0x7f6a24198790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c000f80 tx=0x7f6a1c00be20 comp rx=0 tx=0).stop 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 shutdown_connections 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a0c0a2b90 0x7f6a0c0a5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24104080 0x7f6a2419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a241024e0 0x7f6a24198790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 --2- 172.21.15.136:0/3252982932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a241036e0 0x7f6a24198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:13.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 >> 172.21.15.136:0/3252982932 conn(0x7f6a240fda80 msgr2=0x7f6a24106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:13.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 shutdown_connections 2024-06-13T06:49:13.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:49:13.598+0000 7f6a293ba700 1 -- 172.21.15.136:0/3252982932 wait complete. 2024-06-13T06:49:14.218 DEBUG:teuthology.orchestra.run.smithi136:osd.2> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.2.service 2024-06-13T06:49:14.220 INFO:tasks.cephadm:Deploying osd.3 on smithi026 with /dev/vg_nvme/lv_3... 2024-06-13T06:49:14.220 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-13T06:49:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:14.851 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:14.851 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:14.851 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:14.851 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:14 smithi026 ceph-mon[32041]: from='osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:49:14.852 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:14 smithi026 ceph-mon[32041]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:49:14.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:14 smithi136 ceph-mon[37229]: from='osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:49:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:14 smithi136 ceph-mon[37229]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:49:15.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:15.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:15.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:15.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:14 smithi135 ceph-mon[36276]: from='osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:49:15.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:14 smithi135 ceph-mon[36276]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:49:15.804 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:15 smithi136 ceph-mon[37229]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:15.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:15 smithi136 ceph-mon[37229]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-13T06:49:15.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:15 smithi136 ceph-mon[37229]: osdmap e16: 3 total, 2 up, 3 in 2024-06-13T06:49:15.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:15.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:15 smithi136 ceph-mon[37229]: from='osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:49:15.805 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:15 smithi136 ceph-mon[37229]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:49:15.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:15 smithi026 ceph-mon[32041]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:15.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:15 smithi026 ceph-mon[32041]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-13T06:49:15.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:15 smithi026 ceph-mon[32041]: osdmap e16: 3 total, 2 up, 3 in 2024-06-13T06:49:15.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:15.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:15 smithi026 ceph-mon[32041]: from='osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:49:15.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:15 smithi026 ceph-mon[32041]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:49:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:15 smithi135 ceph-mon[36276]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:15 smithi135 ceph-mon[36276]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-13T06:49:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:15 smithi135 ceph-mon[36276]: osdmap e16: 3 total, 2 up, 3 in 2024-06-13T06:49:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:15 smithi135 ceph-mon[36276]: from='osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:49:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:15 smithi135 ceph-mon[36276]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:49:16.608 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2[42323]: 2024-06-13T06:49:16.142+0000 7f1751dab700 -1 osd.2 0 waiting for initial osdmap 2024-06-13T06:49:16.608 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2[42323]: 2024-06-13T06:49:16.146+0000 7f1749b96700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:49:16.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:49:16.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:49:16.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-mon[37229]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:49:16.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-mon[37229]: osdmap e17: 3 total, 2 up, 3 in 2024-06-13T06:49:16.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:16.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:16.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:16 smithi136 ceph-mon[37229]: from='osd.2 ' entity='osd.2' 2024-06-13T06:49:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:16 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:49:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:16 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:49:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:16 smithi026 ceph-mon[32041]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:49:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:16 smithi026 ceph-mon[32041]: osdmap e17: 3 total, 2 up, 3 in 2024-06-13T06:49:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:16 smithi026 ceph-mon[32041]: from='osd.2 ' entity='osd.2' 2024-06-13T06:49:17.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:16 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:49:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:16 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:49:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:16 smithi135 ceph-mon[36276]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:49:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:16 smithi135 ceph-mon[36276]: osdmap e17: 3 total, 2 up, 3 in 2024-06-13T06:49:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:16 smithi135 ceph-mon[36276]: from='osd.2 ' entity='osd.2' 2024-06-13T06:49:17.671 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-13T06:49:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-13T06:49:17.672 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-06-13T06:49:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-06-13T06:49:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0600057 s, 175 MB/s 2024-06-13T06:49:17.672 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 2024-06-13T06:49:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-06-13T06:49:17.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: pgmap v91: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:17.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:17.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] boot 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: osdmap e18: 3 total, 3 up, 3 in 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:17.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: pgmap v91: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:17.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:17.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] boot 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: osdmap e18: 3 total, 3 up, 3 in 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:17.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:18.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: pgmap v91: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:49:18.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:18.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: osd.2 [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] boot 2024-06-13T06:49:18.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: osdmap e18: 3 total, 3 up, 3 in 2024-06-13T06:49:18.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:49:18.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:18.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:18.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:18.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:18.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:18.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:18.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:18.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2024-06-13T06:49:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:19.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:18 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi136 2024-06-13T06:49:19.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:18 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi136 to 17088M 2024-06-13T06:49:19.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:18 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi136 2024-06-13T06:49:19.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:18 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi136 to 17088M 2024-06-13T06:49:19.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:18 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi136 2024-06-13T06:49:19.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:18 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi136 to 17088M 2024-06-13T06:49:20.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:19 smithi026 ceph-mon[32041]: pgmap v93: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:20.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:49:20.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-13T06:49:20.126 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:19 smithi026 ceph-mon[32041]: osdmap e19: 3 total, 3 up, 3 in 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:19 smithi026 ceph-mon[32041]: osdmap e20: 3 total, 3 up, 3 in 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:19 smithi136 ceph-mon[37229]: pgmap v93: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:19 smithi136 ceph-mon[37229]: osdmap e19: 3 total, 3 up, 3 in 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-13T06:49:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:19 smithi136 ceph-mon[37229]: osdmap e20: 3 total, 3 up, 3 in 2024-06-13T06:49:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:19 smithi135 ceph-mon[36276]: pgmap v93: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:49:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-13T06:49:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:19 smithi135 ceph-mon[36276]: osdmap e19: 3 total, 3 up, 3 in 2024-06-13T06:49:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:49:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-13T06:49:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:19 smithi135 ceph-mon[36276]: osdmap e20: 3 total, 3 up, 3 in 2024-06-13T06:49:20.982 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:49:20 smithi026 sudo[51331]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:49:20.983 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:49:20 smithi026 sudo[51331]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:20.983 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:49:20 smithi026 sudo[51331]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:20.983 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:49:20 smithi026 sudo[51331]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.256+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1333007832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 msgr2=0x7fd5e4107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.256+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1333007832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 0x7fd5e4107ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098f0 tx=0x7fd5d0009c00 comp rx=0 tx=0).stop 2024-06-13T06:49:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.256+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1333007832 shutdown_connections 2024-06-13T06:49:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.256+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1333007832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e41081e0 0x7fd5e410a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.256+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1333007832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4102f80 0x7fd5e4105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.256+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1333007832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 0x7fd5e4107ca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.256+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1333007832 >> 172.21.15.26:0/1333007832 conn(0x7fd5e40fc990 msgr2=0x7fd5e40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.257+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1333007832 shutdown_connections 2024-06-13T06:49:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.257+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1333007832 wait complete. 2024-06-13T06:49:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.258+0000 7fd5e8a00700 1 Processor -- start 2024-06-13T06:49:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.258+0000 7fd5e8a00700 1 -- start start 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.258+0000 7fd5e8a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4102f80 0x7fd5e41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.258+0000 7fd5e8a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 0x7fd5e4198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.258+0000 7fd5e8a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e41081e0 0x7fd5e419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.258+0000 7fd5e8a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e410e150 con 0x7fd5e41081e0 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.258+0000 7fd5e8a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e410dfd0 con 0x7fd5e41058b0 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.258+0000 7fd5e8a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e410e2d0 con 0x7fd5e4102f80 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 0x7fd5e4198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 0x7fd5e4198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36632/0 (socket says 172.21.15.26:36632) 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e1400700 1 -- 172.21.15.26:0/1947384478 learned_addr learned my addr 172.21.15.26:0/1947384478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e2800700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e41081e0 0x7fd5e419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e1e00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4102f80 0x7fd5e41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e2800700 1 -- 172.21.15.26:0/1947384478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4102f80 msgr2=0x7fd5e41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e2800700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4102f80 0x7fd5e41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e2800700 1 -- 172.21.15.26:0/1947384478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 msgr2=0x7fd5e4198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e2800700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 0x7fd5e4198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e2800700 1 -- 172.21.15.26:0/1947384478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e41a0450 con 0x7fd5e41081e0 2024-06-13T06:49:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e1400700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 0x7fd5e4198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:49:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.259+0000 7fd5e1e00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4102f80 0x7fd5e41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:49:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.260+0000 7fd5e2800700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e41081e0 0x7fd5e419fdb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00efc0 tx=0x7fd5dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.260+0000 7fd5daa00700 1 -- 172.21.15.26:0/1947384478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0092e0 con 0x7fd5e41081e0 2024-06-13T06:49:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.260+0000 7fd5daa00700 1 -- 172.21.15.26:0/1947384478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5dc004bb0 con 0x7fd5e41081e0 2024-06-13T06:49:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.260+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e41a06e0 con 0x7fd5e41081e0 2024-06-13T06:49:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.260+0000 7fd5daa00700 1 -- 172.21.15.26:0/1947384478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc004040 con 0x7fd5e41081e0 2024-06-13T06:49:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.260+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41a0c20 con 0x7fd5e41081e0 2024-06-13T06:49:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.261+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e4069180 con 0x7fd5e41081e0 2024-06-13T06:49:21.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.263+0000 7fd5daa00700 1 -- 172.21.15.26:0/1947384478 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd5dc01b030 con 0x7fd5e41081e0 2024-06-13T06:49:21.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.265+0000 7fd5daa00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd5c80a2ca0 0x7fd5c80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:21.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.265+0000 7fd5daa00700 1 -- 172.21.15.26:0/1947384478 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fd5dc0cd700 con 0x7fd5e41081e0 2024-06-13T06:49:21.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.265+0000 7fd5e1e00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd5c80a2ca0 0x7fd5c80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:21.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.266+0000 7fd5e1e00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd5c80a2ca0 0x7fd5c80a5160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4011490 tx=0x7fd5d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:21.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.270+0000 7fd5daa00700 1 -- 172.21.15.26:0/1947384478 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5dc022020 con 0x7fd5e41081e0 2024-06-13T06:49:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:21.521+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd5e4062990 con 0x7fd5c80a2ca0 2024-06-13T06:49:21.752 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:49:21 smithi026 sudo[51360]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:49:21.752 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:49:21 smithi026 sudo[51360]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:21.752 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:49:21 smithi026 sudo[51360]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:21.752 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 06:49:21 smithi026 sudo[51360]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:21.935 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:49:21 smithi135 sudo[44973]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:49:21.936 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:49:21 smithi135 sudo[44973]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:21.936 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:49:21 smithi135 sudo[44973]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:21.936 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:49:21 smithi135 sudo[44973]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:21.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:21 smithi135 ceph-mon[36276]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:21.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:21 smithi135 ceph-mon[36276]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-13T06:49:21.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:21 smithi135 ceph-mon[36276]: osdmap e21: 3 total, 3 up, 3 in 2024-06-13T06:49:21.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:21.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:49:21.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:49:21.937 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:22.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:21 smithi136 ceph-mon[37229]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:22.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:21 smithi136 ceph-mon[37229]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-13T06:49:22.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:21 smithi136 ceph-mon[37229]: osdmap e21: 3 total, 3 up, 3 in 2024-06-13T06:49:22.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:22.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:49:22.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:49:22.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:22.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:21 smithi026 ceph-mon[32041]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:21 smithi026 ceph-mon[32041]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-13T06:49:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:21 smithi026 ceph-mon[32041]: osdmap e21: 3 total, 3 up, 3 in 2024-06-13T06:49:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:49:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:49:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:22.282 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:49:21 smithi135 sudo[44976]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:49:22.283 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:49:21 smithi135 sudo[44976]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:22.283 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:49:21 smithi135 sudo[44976]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:22.283 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 06:49:21 smithi135 sudo[44976]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:22.471 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:22 smithi136 sudo[46392]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:49:22.471 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:22 smithi136 sudo[46392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:22.471 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:22 smithi136 sudo[46392]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:22.471 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:22 smithi136 sudo[46392]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:22.866 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:22 smithi136 sudo[46395]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:49:22.866 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:22 smithi136 sudo[46395]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:22.866 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:22 smithi136 sudo[46395]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:22.866 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 06:49:22 smithi136 sudo[46395]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:23.486 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 sudo[51500]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:49:23.486 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 sudo[51500]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:23.486 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 sudo[51500]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:23.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 sudo[51500]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 sudo[51508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:49:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 sudo[51508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 sudo[51508]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 sudo[51508]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 ceph-mon[32041]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:23 smithi026 ceph-mon[32041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:23 smithi136 ceph-mon[37229]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:23 smithi136 ceph-mon[37229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:24.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:23 smithi135 ceph-mon[36276]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:24.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:24.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:24.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:23 smithi135 ceph-mon[36276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:24.205 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:23 smithi136 sudo[46398]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:49:24.205 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:23 smithi136 sudo[46398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:24.205 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:23 smithi136 sudo[46398]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:24.206 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:23 smithi136 sudo[46398]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 sudo[46401]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:49:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 sudo[46401]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 sudo[46401]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 sudo[46401]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:24.702 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 sudo[44979]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:49:24.703 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 sudo[44979]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:24.703 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 sudo[44979]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:24.703 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 sudo[44979]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:25.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:25.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 sudo[44982]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:49:25.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 sudo[44982]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:49:25.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 sudo[44982]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:49:25.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 sudo[44982]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:49:25.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:25.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:25.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:25.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:49:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:49:26.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:25 smithi026 ceph-mon[32041]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:26.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:25 smithi026 ceph-mon[32041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:26.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:25 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/4021201424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]: dispatch 2024-06-13T06:49:26.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:25 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]: dispatch 2024-06-13T06:49:26.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:25 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]': finished 2024-06-13T06:49:26.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:25 smithi026 ceph-mon[32041]: osdmap e22: 4 total, 3 up, 4 in 2024-06-13T06:49:26.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:25 smithi135 ceph-mon[36276]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:25 smithi135 ceph-mon[36276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:25 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/4021201424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]: dispatch 2024-06-13T06:49:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:25 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]: dispatch 2024-06-13T06:49:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:25 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]': finished 2024-06-13T06:49:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:25 smithi135 ceph-mon[36276]: osdmap e22: 4 total, 3 up, 4 in 2024-06-13T06:49:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:25 smithi136 ceph-mon[37229]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:25 smithi136 ceph-mon[37229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:49:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:25 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/4021201424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]: dispatch 2024-06-13T06:49:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:25 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]: dispatch 2024-06-13T06:49:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:25 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a77d3db-9cde-4c90-86e2-9030466c4753"}]': finished 2024-06-13T06:49:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:25 smithi136 ceph-mon[37229]: osdmap e22: 4 total, 3 up, 4 in 2024-06-13T06:49:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:27.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:26 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/455249938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:49:27.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:26 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/455249938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:49:27.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:26 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/455249938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:49:28.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:27 smithi026 ceph-mon[32041]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:28.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:27 smithi135 ceph-mon[36276]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:27 smithi136 ceph-mon[37229]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:30.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:29 smithi026 ceph-mon[32041]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-06-13T06:49:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:29 smithi135 ceph-mon[36276]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-06-13T06:49:30.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:29 smithi136 ceph-mon[37229]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-06-13T06:49:31.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:31 smithi026 ceph-mon[32041]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-13T06:49:32.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:31 smithi135 ceph-mon[36276]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-13T06:49:32.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:31 smithi136 ceph-mon[37229]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-13T06:49:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:33 smithi026 ceph-mon[32041]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-13T06:49:34.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:33 smithi135 ceph-mon[36276]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-13T06:49:34.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:33 smithi136 ceph-mon[37229]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-13T06:49:35.995 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:35 smithi026 ceph-mon[32041]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-13T06:49:35.995 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-13T06:49:35.995 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:35.995 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:35 smithi026 ceph-mon[32041]: Deploying daemon osd.3 on smithi026 2024-06-13T06:49:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:35 smithi135 ceph-mon[36276]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-13T06:49:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-13T06:49:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:35 smithi135 ceph-mon[36276]: Deploying daemon osd.3 on smithi026 2024-06-13T06:49:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:35 smithi136 ceph-mon[37229]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-13T06:49:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-13T06:49:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:35 smithi136 ceph-mon[37229]: Deploying daemon osd.3 on smithi026 2024-06-13T06:49:38.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:37 smithi026 ceph-mon[32041]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 867 B/s rd, 25 KiB/s wr, 2 op/s 2024-06-13T06:49:38.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:37 smithi135 ceph-mon[36276]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 867 B/s rd, 25 KiB/s wr, 2 op/s 2024-06-13T06:49:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:37 smithi136 ceph-mon[37229]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 867 B/s rd, 25 KiB/s wr, 2 op/s 2024-06-13T06:49:39.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:39 smithi026 ceph-mon[32041]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:49:40.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:39 smithi135 ceph-mon[36276]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:49:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:39 smithi136 ceph-mon[37229]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:49:41.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:41 smithi026 ceph-mon[32041]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:41 smithi135 ceph-mon[36276]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:42.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:41 smithi136 ceph-mon[37229]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:43 smithi026 ceph-mon[32041]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:44.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:43 smithi135 ceph-mon[36276]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:44.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:43 smithi136 ceph-mon[37229]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:45 smithi135 ceph-mon[36276]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:46.066 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:45 smithi026 ceph-mon[32041]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:46.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:45 smithi136 ceph-mon[37229]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:48.226 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:47 smithi026 ceph-mon[32041]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:48.226 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:48.226 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:48.226 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:47 smithi026 ceph-mon[32041]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:49:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:47 smithi135 ceph-mon[36276]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:47 smithi135 ceph-mon[36276]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:49:48.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.333+0000 7fd5daa00700 1 -- 172.21.15.26:0/1947384478 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd5e4062990 con 0x7fd5c80a2ca0 2024-06-13T06:49:48.332 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 3 on host 'smithi026' 2024-06-13T06:49:48.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd5c80a2ca0 msgr2=0x7fd5c80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:48.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd5c80a2ca0 0x7fd5c80a5160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4011490 tx=0x7fd5d4009260 comp rx=0 tx=0).stop 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e41081e0 msgr2=0x7fd5e419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e41081e0 0x7fd5e419fdb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00efc0 tx=0x7fd5dc00c3a0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 shutdown_connections 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd5c80a2ca0 0x7fd5c80a5160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e41081e0 0x7fd5e419fdb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4102f80 0x7fd5e41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 --2- 172.21.15.26:0/1947384478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5e41058b0 0x7fd5e4198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 >> 172.21.15.26:0/1947384478 conn(0x7fd5e40fc990 msgr2=0x7fd5e41039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 shutdown_connections 2024-06-13T06:49:48.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:49:48.336+0000 7fd5e8a00700 1 -- 172.21.15.26:0/1947384478 wait complete. 2024-06-13T06:49:48.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:47 smithi136 ceph-mon[37229]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:48.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:48.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:48.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:47 smithi136 ceph-mon[37229]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:49:49.134 DEBUG:teuthology.orchestra.run.smithi026:osd.3> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.3.service 2024-06-13T06:49:49.139 INFO:tasks.cephadm:Deploying osd.4 on smithi135 with /dev/vg_nvme/lv_3... 2024-06-13T06:49:49.139 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-13T06:49:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:49 smithi135 ceph-mon[36276]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-13T06:49:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:49 smithi135 ceph-mon[36276]: osdmap e23: 4 total, 3 up, 4 in 2024-06-13T06:49:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:49 smithi135 ceph-mon[36276]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:49:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:49 smithi135 ceph-mon[36276]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:49.333 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-mon[32041]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-13T06:49:49.334 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-mon[32041]: osdmap e23: 4 total, 3 up, 4 in 2024-06-13T06:49:49.334 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-mon[32041]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:49:49.334 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:49.334 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-mon[32041]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:49.334 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:49.334 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:49.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:49 smithi136 ceph-mon[37229]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-13T06:49:49.385 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:49 smithi136 ceph-mon[37229]: osdmap e23: 4 total, 3 up, 4 in 2024-06-13T06:49:49.385 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:49 smithi136 ceph-mon[37229]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:49:49.385 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:49.386 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:49 smithi136 ceph-mon[37229]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:49.386 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:49.386 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:49.386 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:49:50.003 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3[53605]: 2024-06-13T06:49:49.708+0000 7fc798a00700 -1 osd.3 0 waiting for initial osdmap 2024-06-13T06:49:50.003 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 06:49:49 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3[53605]: 2024-06-13T06:49:49.735+0000 7fc78e800700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:49:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:50 smithi135 ceph-mon[36276]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:49:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:50 smithi135 ceph-mon[36276]: osdmap e24: 4 total, 3 up, 4 in 2024-06-13T06:49:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:50 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:50 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:50 smithi135 ceph-mon[36276]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' 2024-06-13T06:49:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:50 smithi136 ceph-mon[37229]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:49:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:50 smithi136 ceph-mon[37229]: osdmap e24: 4 total, 3 up, 4 in 2024-06-13T06:49:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:50 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:50.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:50 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:50.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:50 smithi136 ceph-mon[37229]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' 2024-06-13T06:49:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:50 smithi026 ceph-mon[32041]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:49:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:50 smithi026 ceph-mon[32041]: osdmap e24: 4 total, 3 up, 4 in 2024-06-13T06:49:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:50 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:50.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:50 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:50.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:50 smithi026 ceph-mon[32041]: from='osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278]' entity='osd.3' 2024-06-13T06:49:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:51 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:49:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:51 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:49:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:51 smithi135 ceph-mon[36276]: osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] boot 2024-06-13T06:49:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:51 smithi135 ceph-mon[36276]: osdmap e25: 4 total, 4 up, 4 in 2024-06-13T06:49:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:51 smithi135 ceph-mon[36276]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:51.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:51 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:49:51.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:51 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:49:51.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:51.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:51 smithi136 ceph-mon[37229]: osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] boot 2024-06-13T06:49:51.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:51 smithi136 ceph-mon[37229]: osdmap e25: 4 total, 4 up, 4 in 2024-06-13T06:49:51.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:51.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:51 smithi136 ceph-mon[37229]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:51.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:51 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:49:51.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:51 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:49:51.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:51.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:51 smithi026 ceph-mon[32041]: osd.3 [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] boot 2024-06-13T06:49:51.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:51 smithi026 ceph-mon[32041]: osdmap e25: 4 total, 4 up, 4 in 2024-06-13T06:49:51.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:49:51.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:51 smithi026 ceph-mon[32041]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:49:51.449 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-13T06:49:51.450 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-13T06:49:51.450 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-13T06:49:51.450 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-13T06:49:51.450 INFO:teuthology.orchestra.run.smithi135.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133432 s, 786 MB/s 2024-06-13T06:49:51.450 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-13T06:49:52.151 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:52 smithi135 ceph-mon[36276]: osdmap e26: 4 total, 4 up, 4 in 2024-06-13T06:49:52.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:52 smithi135 ceph-mon[36276]: osdmap e27: 4 total, 4 up, 4 in 2024-06-13T06:49:52.178 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_3 2024-06-13T06:49:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:52 smithi136 ceph-mon[37229]: osdmap e26: 4 total, 4 up, 4 in 2024-06-13T06:49:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:52 smithi136 ceph-mon[37229]: osdmap e27: 4 total, 4 up, 4 in 2024-06-13T06:49:52.421 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:49:52.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:52 smithi026 ceph-mon[32041]: osdmap e26: 4 total, 4 up, 4 in 2024-06-13T06:49:52.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:52 smithi026 ceph-mon[32041]: osdmap e27: 4 total, 4 up, 4 in 2024-06-13T06:49:53.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:53 smithi135 ceph-mon[36276]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:49:53.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:53 smithi136 ceph-mon[37229]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:49:53.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:53 smithi026 ceph-mon[32041]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:49:54.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 -- 172.21.15.135:0/3014292014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc404104090 msgr2=0x7fc40410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/3014292014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc404104090 0x7fc40410a940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4000099c0 tx=0x7fc400009cd0 comp rx=0 tx=0).stop 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 -- 172.21.15.135:0/3014292014 shutdown_connections 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/3014292014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc404104090 0x7fc40410a940 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/3014292014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4041024f0 0x7fc404102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/3014292014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4041036f0 0x7fc404103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 -- 172.21.15.135:0/3014292014 >> 172.21.15.135:0/3014292014 conn(0x7fc4040fda90 msgr2=0x7fc4040ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 -- 172.21.15.135:0/3014292014 shutdown_connections 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 -- 172.21.15.135:0/3014292014 wait complete. 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.133+0000 7fc40b6ed700 1 Processor -- start 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 -- start start 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4041024f0 0x7fc404198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:54.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4041036f0 0x7fc404198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc404104090 0x7fc40419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc40410e180 con 0x7fc404104090 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc40410e000 con 0x7fc4041024f0 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc40410e300 con 0x7fc4041036f0 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4041024f0 0x7fc404198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4041024f0 0x7fc404198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:52210/0 (socket says 172.21.15.135:52210) 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 -- 172.21.15.135:0/2350003758 learned_addr learned my addr 172.21.15.135:0/2350003758 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc408c88700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4041036f0 0x7fc404198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 -- 172.21.15.135:0/2350003758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4041036f0 msgr2=0x7fc404198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4041036f0 0x7fc404198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 -- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc404104090 msgr2=0x7fc40419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:54.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc404104090 0x7fc40419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 -- 172.21.15.135:0/2350003758 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4041a0370 con 0x7fc4041024f0 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc408c88700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4041036f0 0x7fc404198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc409489700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4041024f0 0x7fc404198710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b9c0 tx=0x7fc3fc00bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc3fa7fc700 1 -- 172.21.15.135:0/2350003758 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc019070 con 0x7fc4041024f0 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc3fa7fc700 1 -- 172.21.15.135:0/2350003758 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3fc014440 con 0x7fc4041024f0 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4041a0660 con 0x7fc4041024f0 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc3fa7fc700 1 -- 172.21.15.135:0/2350003758 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0125a0 con 0x7fc4041024f0 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.134+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4041a0c70 con 0x7fc4041024f0 2024-06-13T06:49:54.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.135+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc404069180 con 0x7fc4041024f0 2024-06-13T06:49:54.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.136+0000 7fc3fa7fc700 1 -- 172.21.15.135:0/2350003758 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc3fc01c070 con 0x7fc4041024f0 2024-06-13T06:49:54.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.137+0000 7fc3fa7fc700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:54.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.137+0000 7fc3fa7fc700 1 -- 172.21.15.135:0/2350003758 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fc3fc0cd200 con 0x7fc4041024f0 2024-06-13T06:49:54.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.137+0000 7fc408c88700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:54.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.139+0000 7fc408c88700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4005fd0 tx=0x7fc3f4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:54.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.142+0000 7fc3fa7fc700 1 -- 172.21.15.135:0/2350003758 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc3fc09b380 con 0x7fc4041024f0 2024-06-13T06:49:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:54 smithi026 ceph-mon[32041]: osdmap e28: 4 total, 4 up, 4 in 2024-06-13T06:49:54.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:54 smithi135 ceph-mon[36276]: osdmap e28: 4 total, 4 up, 4 in 2024-06-13T06:49:54.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:49:54.355+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc4040628d0 con 0x7fc3f00a2ca0 2024-06-13T06:49:54.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:54 smithi136 ceph-mon[37229]: osdmap e28: 4 total, 4 up, 4 in 2024-06-13T06:49:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: osdmap e29: 4 total, 4 up, 4 in 2024-06-13T06:49:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:49:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:49:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi026 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi026 to 8547M 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:55.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.285 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:55.285 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: osdmap e29: 4 total, 4 up, 4 in 2024-06-13T06:49:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:49:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:49:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:49:55.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi026 2024-06-13T06:49:55.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:55.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:55.379 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi026 to 8547M 2024-06-13T06:49:55.379 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.379 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.379 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:55.379 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.379 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:55.380 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.380 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:55.380 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: osdmap e29: 4 total, 4 up, 4 in 2024-06-13T06:49:55.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:49:55.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:55.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:49:55.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:49:55.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi026 2024-06-13T06:49:55.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi026 to 8547M 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:55.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:49:57.490 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:57 smithi135 ceph-mon[36276]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-06-13T06:49:57.490 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:57 smithi135 ceph-mon[36276]: from='client.? 172.21.15.135:0/3393998748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]: dispatch 2024-06-13T06:49:57.491 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:57 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]: dispatch 2024-06-13T06:49:57.491 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:57 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]': finished 2024-06-13T06:49:57.491 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:57 smithi135 ceph-mon[36276]: osdmap e30: 5 total, 4 up, 5 in 2024-06-13T06:49:57.491 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:57 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:49:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:57 smithi026 ceph-mon[32041]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-06-13T06:49:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:57 smithi026 ceph-mon[32041]: from='client.? 172.21.15.135:0/3393998748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]: dispatch 2024-06-13T06:49:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:57 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]: dispatch 2024-06-13T06:49:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:57 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]': finished 2024-06-13T06:49:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:57 smithi026 ceph-mon[32041]: osdmap e30: 5 total, 4 up, 5 in 2024-06-13T06:49:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:49:57.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:57 smithi136 ceph-mon[37229]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-06-13T06:49:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:57 smithi136 ceph-mon[37229]: from='client.? 172.21.15.135:0/3393998748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]: dispatch 2024-06-13T06:49:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:57 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]: dispatch 2024-06-13T06:49:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:57 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cd55a9b-cc9a-4e67-b486-f1a420368bbc"}]': finished 2024-06-13T06:49:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:57 smithi136 ceph-mon[37229]: osdmap e30: 5 total, 4 up, 5 in 2024-06-13T06:49:57.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:57 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:49:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:58 smithi026 ceph-mon[32041]: from='client.? 172.21.15.135:0/2479397296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:49:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:58 smithi135 ceph-mon[36276]: from='client.? 172.21.15.135:0/2479397296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:49:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:58 smithi136 ceph-mon[37229]: from='client.? 172.21.15.135:0/2479397296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:49:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:49:59 smithi026 ceph-mon[32041]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:49:59.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:49:59 smithi135 ceph-mon[36276]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:49:59.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:49:59 smithi136 ceph-mon[37229]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:50:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:00 smithi026 ceph-mon[32041]: overall HEALTH_OK 2024-06-13T06:50:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:00 smithi135 ceph-mon[36276]: overall HEALTH_OK 2024-06-13T06:50:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:00 smithi136 ceph-mon[37229]: overall HEALTH_OK 2024-06-13T06:50:01.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:01 smithi135 ceph-mon[36276]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-13T06:50:01.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:01 smithi026 ceph-mon[32041]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-13T06:50:01.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:01 smithi136 ceph-mon[37229]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-13T06:50:03.441 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:03 smithi135 ceph-mon[36276]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:03.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:03 smithi136 ceph-mon[37229]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:03 smithi026 ceph-mon[32041]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:05.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:05 smithi135 ceph-mon[36276]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:05.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-13T06:50:05.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:05.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:05 smithi135 ceph-mon[36276]: Deploying daemon osd.4 on smithi135 2024-06-13T06:50:05.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:05 smithi136 ceph-mon[37229]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:05.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-13T06:50:05.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:05.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:05 smithi136 ceph-mon[37229]: Deploying daemon osd.4 on smithi135 2024-06-13T06:50:05.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:05 smithi026 ceph-mon[32041]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:05.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-13T06:50:05.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:05.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:05 smithi026 ceph-mon[32041]: Deploying daemon osd.4 on smithi135 2024-06-13T06:50:07.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:07 smithi135 ceph-mon[36276]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:07.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:07 smithi136 ceph-mon[37229]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:07.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:07 smithi026 ceph-mon[32041]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:09.427 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:09 smithi135 ceph-mon[36276]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:09.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:09 smithi136 ceph-mon[37229]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:09.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:09 smithi026 ceph-mon[32041]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:11.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:11 smithi136 ceph-mon[37229]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:11.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:11.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:11.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:11 smithi026 ceph-mon[32041]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:11 smithi135 ceph-mon[36276]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:13.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:13 smithi136 ceph-mon[37229]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:13.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:13.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:13 smithi026 ceph-mon[32041]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:13.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:13.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:13 smithi135 ceph-mon[36276]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:13.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.244+0000 7fc3fa7fc700 1 -- 172.21.15.135:0/2350003758 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc4040628d0 con 0x7fc3f00a2ca0 2024-06-13T06:50:14.244 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 4 on host 'smithi135' 2024-06-13T06:50:14.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f00a2ca0 msgr2=0x7fc3f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4005fd0 tx=0x7fc3f4005ee0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4041024f0 msgr2=0x7fc404198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4041024f0 0x7fc404198710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b9c0 tx=0x7fc3fc00bd80 comp rx=0 tx=0).stop 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 shutdown_connections 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f00a2ca0 0x7fc3f00a5160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4041024f0 0x7fc404198710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc404104090 0x7fc40419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 --2- 172.21.15.135:0/2350003758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4041036f0 0x7fc404198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 >> 172.21.15.135:0/2350003758 conn(0x7fc4040fda90 msgr2=0x7fc4041069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 shutdown_connections 2024-06-13T06:50:14.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:50:14.247+0000 7fc40b6ed700 1 -- 172.21.15.135:0/2350003758 wait complete. 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:50:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:14 smithi135 ceph-mon[36276]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:50:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:14.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:50:14.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:14 smithi136 ceph-mon[37229]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:50:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:50:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:14 smithi026 ceph-mon[32041]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:50:14.970 DEBUG:teuthology.orchestra.run.smithi135:osd.4> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.4.service 2024-06-13T06:50:14.972 INFO:tasks.cephadm:Deploying osd.5 on smithi136 with /dev/vg_nvme/lv_3... 2024-06-13T06:50:14.973 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-13T06:50:15.214 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:50:15.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:15 smithi135 ceph-mon[36276]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:15.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:15 smithi135 ceph-mon[36276]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-13T06:50:15.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:15 smithi135 ceph-mon[36276]: from='osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:50:15.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:15 smithi135 ceph-mon[36276]: osdmap e31: 5 total, 4 up, 5 in 2024-06-13T06:50:15.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:15.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:15 smithi135 ceph-mon[36276]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:50:15.592 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:15 smithi136 ceph-mon[37229]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:15.592 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:15 smithi136 ceph-mon[37229]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-13T06:50:15.592 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:15 smithi136 ceph-mon[37229]: from='osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:50:15.592 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:15 smithi136 ceph-mon[37229]: osdmap e31: 5 total, 4 up, 5 in 2024-06-13T06:50:15.592 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:15.592 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:15 smithi136 ceph-mon[37229]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:50:15.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:15 smithi026 ceph-mon[32041]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:15 smithi026 ceph-mon[32041]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-13T06:50:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:15 smithi026 ceph-mon[32041]: from='osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:50:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:15 smithi026 ceph-mon[32041]: osdmap e31: 5 total, 4 up, 5 in 2024-06-13T06:50:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:15 smithi026 ceph-mon[32041]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:50:17.239 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 06:50:16 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T06:50:16.949+0000 7f6031953700 -1 osd.4 0 waiting for initial osdmap 2024-06-13T06:50:17.239 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 06:50:16 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T06:50:16.956+0000 7f602973e700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:50:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-13T06:50:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-13T06:50:17.303 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-06-13T06:50:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-06-13T06:50:17.303 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0080968 s, 1.3 GB/s 2024-06-13T06:50:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-06-13T06:50:17.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:17 smithi026 ceph-mon[32041]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:17.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:17 smithi026 ceph-mon[32041]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:50:17.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:17 smithi026 ceph-mon[32041]: osdmap e32: 5 total, 4 up, 5 in 2024-06-13T06:50:17.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:17.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:17.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:17 smithi026 ceph-mon[32041]: from='osd.4 ' entity='osd.4' 2024-06-13T06:50:17.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:17 smithi135 ceph-mon[36276]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:17.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:17 smithi135 ceph-mon[36276]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:50:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:17 smithi135 ceph-mon[36276]: osdmap e32: 5 total, 4 up, 5 in 2024-06-13T06:50:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:17 smithi135 ceph-mon[36276]: from='osd.4 ' entity='osd.4' 2024-06-13T06:50:17.618 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:17 smithi136 ceph-mon[37229]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:17.618 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:17 smithi136 ceph-mon[37229]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:50:17.618 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:17 smithi136 ceph-mon[37229]: osdmap e32: 5 total, 4 up, 5 in 2024-06-13T06:50:17.618 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:17.618 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:17.619 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:17 smithi136 ceph-mon[37229]: from='osd.4 ' entity='osd.4' 2024-06-13T06:50:17.922 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-06-13T06:50:18.163 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] boot 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: osdmap e33: 5 total, 5 up, 5 in 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:18.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:18.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:18.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:18.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:50:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] boot 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: osdmap e33: 5 total, 5 up, 5 in 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:50:18.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:50:18.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:18.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: osd.4 [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] boot 2024-06-13T06:50:18.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: osdmap e33: 5 total, 5 up, 5 in 2024-06-13T06:50:18.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:50:18.506 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.506 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.506 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:18.506 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:18.506 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:18.506 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:18.506 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:18.506 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:19 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi135 2024-06-13T06:50:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:19 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-13T06:50:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:19 smithi026 ceph-mon[32041]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:19 smithi026 ceph-mon[32041]: osdmap e34: 5 total, 5 up, 5 in 2024-06-13T06:50:19.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:19 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi135 2024-06-13T06:50:19.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:19 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-13T06:50:19.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:19 smithi135 ceph-mon[36276]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:19.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:19 smithi135 ceph-mon[36276]: osdmap e34: 5 total, 5 up, 5 in 2024-06-13T06:50:19.562 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:19 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi135 2024-06-13T06:50:19.562 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:19 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-13T06:50:19.562 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:19 smithi136 ceph-mon[37229]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:50:19.562 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:19 smithi136 ceph-mon[37229]: osdmap e34: 5 total, 5 up, 5 in 2024-06-13T06:50:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 -- 172.21.15.136:0/447735877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 msgr2=0x7f2620101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 --2- 172.21.15.136:0/447735877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 0x7f2620101810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2610009990 tx=0x7f2610009ca0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 -- 172.21.15.136:0/447735877 shutdown_connections 2024-06-13T06:50:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 --2- 172.21.15.136:0/447735877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2620102f90 0x7f2620109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 --2- 172.21.15.136:0/447735877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 0x7f2620101810 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 --2- 172.21.15.136:0/447735877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26201025f0 0x7f2620102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 -- 172.21.15.136:0/447735877 >> 172.21.15.136:0/447735877 conn(0x7f26200fc990 msgr2=0x7f26200fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 -- 172.21.15.136:0/447735877 shutdown_connections 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.665+0000 7f2625124700 1 -- 172.21.15.136:0/447735877 wait complete. 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f2625124700 1 Processor -- start 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f2625124700 1 -- start start 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f2625124700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 0x7f2620198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f2625124700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26201025f0 0x7f2620198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f2625124700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2620102f90 0x7f262019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f2625124700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f262010d030 con 0x7f2620102f90 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f2625124700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f262010ceb0 con 0x7f26201025f0 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f2625124700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f262010d1b0 con 0x7f26201013f0 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 0x7f2620198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 0x7f2620198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:45272/0 (socket says 172.21.15.136:45272) 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 -- 172.21.15.136:0/3910211582 learned_addr learned my addr 172.21.15.136:0/3910211582 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 -- 172.21.15.136:0/3910211582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26201025f0 msgr2=0x7f2620198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26201025f0 0x7f2620198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 -- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2620102f90 msgr2=0x7f262019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2620102f90 0x7f262019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 -- 172.21.15.136:0/3910211582 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26201a0390 con 0x7f26201013f0 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.666+0000 7f261ed9d700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 0x7f2620198730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2610005ef0 tx=0x7f2610009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.667+0000 7f2607fff700 1 -- 172.21.15.136:0/3910211582 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610027070 con 0x7f26201013f0 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.667+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26201a0620 con 0x7f26201013f0 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.667+0000 7f2607fff700 1 -- 172.21.15.136:0/3910211582 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2610004210 con 0x7f26201013f0 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.667+0000 7f2607fff700 1 -- 172.21.15.136:0/3910211582 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261000f460 con 0x7f26201013f0 2024-06-13T06:50:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.667+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201a0af0 con 0x7f26201013f0 2024-06-13T06:50:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.668+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26201a0d50 con 0x7f26201013f0 2024-06-13T06:50:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.669+0000 7f2607fff700 1 -- 172.21.15.136:0/3910211582 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f261002a070 con 0x7f26201013f0 2024-06-13T06:50:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.669+0000 7f2607fff700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f26080a2c20 0x7f26080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.670+0000 7f2607fff700 1 -- 172.21.15.136:0/3910211582 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f26100cf470 con 0x7f26201013f0 2024-06-13T06:50:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.670+0000 7f261e59c700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f26080a2c20 0x7f26080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:19.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.671+0000 7f261e59c700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f26080a2c20 0x7f26080a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2614005fd0 tx=0x7f2614005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:19.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.673+0000 7f2607fff700 1 -- 172.21.15.136:0/3910211582 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2610013070 con 0x7f26201013f0 2024-06-13T06:50:19.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:19.874+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f26201a0fe0 con 0x7f26080a2c20 2024-06-13T06:50:20.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:20 smithi136 ceph-mon[37229]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:50:20.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:50:20.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:50:20.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:20.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:20 smithi026 ceph-mon[32041]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:50:20.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:50:20.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:50:20.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:20 smithi135 ceph-mon[36276]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:50:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:50:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:50:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:21.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:21 smithi135 ceph-mon[36276]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:21.596 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:21 smithi136 ceph-mon[37229]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:21 smithi026 ceph-mon[32041]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:23 smithi135 ceph-mon[36276]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:23 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/2840066573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]: dispatch 2024-06-13T06:50:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:23 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]: dispatch 2024-06-13T06:50:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:23 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]': finished 2024-06-13T06:50:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:23 smithi135 ceph-mon[36276]: osdmap e35: 6 total, 5 up, 6 in 2024-06-13T06:50:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:23 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/3568639756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:50:23.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:23 smithi136 ceph-mon[37229]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:23 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/2840066573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]: dispatch 2024-06-13T06:50:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:23 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]: dispatch 2024-06-13T06:50:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:23 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]': finished 2024-06-13T06:50:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:23 smithi136 ceph-mon[37229]: osdmap e35: 6 total, 5 up, 6 in 2024-06-13T06:50:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:23 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/3568639756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:50:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:23 smithi026 ceph-mon[32041]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:23 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/2840066573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]: dispatch 2024-06-13T06:50:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:23 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]: dispatch 2024-06-13T06:50:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:23 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ca38a30-516e-4466-bbdf-4873adf0779a"}]': finished 2024-06-13T06:50:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:23 smithi026 ceph-mon[32041]: osdmap e35: 6 total, 5 up, 6 in 2024-06-13T06:50:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:23 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/3568639756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:50:25.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:25 smithi135 ceph-mon[36276]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:25.554 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:25 smithi026 ceph-mon[32041]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:25.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:25 smithi136 ceph-mon[37229]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:27.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:27 smithi026 ceph-mon[32041]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:27.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:27 smithi135 ceph-mon[36276]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:27.605 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:27 smithi136 ceph-mon[37229]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:29.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:29 smithi136 ceph-mon[37229]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:29.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:29 smithi135 ceph-mon[36276]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:29 smithi026 ceph-mon[32041]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:30.619 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-13T06:50:30.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:30.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:30 smithi136 ceph-mon[37229]: Deploying daemon osd.5 on smithi136 2024-06-13T06:50:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-13T06:50:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:30 smithi026 ceph-mon[32041]: Deploying daemon osd.5 on smithi136 2024-06-13T06:50:30.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:30 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-13T06:50:30.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:30 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:30 smithi135 ceph-mon[36276]: Deploying daemon osd.5 on smithi136 2024-06-13T06:50:31.448 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:31 smithi136 ceph-mon[37229]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:31.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:31 smithi026 ceph-mon[32041]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:31.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:31 smithi135 ceph-mon[36276]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:33.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:33 smithi136 ceph-mon[37229]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:33 smithi026 ceph-mon[32041]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:33.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:33 smithi135 ceph-mon[36276]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:35.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:35 smithi136 ceph-mon[37229]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:35 smithi026 ceph-mon[32041]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:35.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:35 smithi135 ceph-mon[36276]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:36.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:36.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:36.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:36.596 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:36.596 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:36.596 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:36.665 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:36.665 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:36.665 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:37 smithi026 ceph-mon[32041]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:37 smithi135 ceph-mon[36276]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:37 smithi136 ceph-mon[37229]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:39.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.413+0000 7f2607fff700 1 -- 172.21.15.136:0/3910211582 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f26201a0fe0 con 0x7f26080a2c20 2024-06-13T06:50:39.412 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 5 on host 'smithi136' 2024-06-13T06:50:39.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.415+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f26080a2c20 msgr2=0x7f26080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:39.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.415+0000 7f2625124700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f26080a2c20 0x7f26080a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2614005fd0 tx=0x7f2614005d00 comp rx=0 tx=0).stop 2024-06-13T06:50:39.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 msgr2=0x7f2620198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 0x7f2620198730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2610005ef0 tx=0x7f2610009f90 comp rx=0 tx=0).stop 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 shutdown_connections 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f26080a2c20 0x7f26080a50e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2620102f90 0x7f262019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26201013f0 0x7f2620198730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 --2- 172.21.15.136:0/3910211582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26201025f0 0x7f2620198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 >> 172.21.15.136:0/3910211582 conn(0x7f26200fc990 msgr2=0x7f262006a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 shutdown_connections 2024-06-13T06:50:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:50:39.416+0000 7f2625124700 1 -- 172.21.15.136:0/3910211582 wait complete. 2024-06-13T06:50:39.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:39.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:39.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:39.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:50:39.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:50:39.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:39.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:39.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:39.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:39.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:50:39.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:50:39.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:39.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:39.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:40.082 DEBUG:teuthology.orchestra.run.smithi136:osd.5> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.5.service 2024-06-13T06:50:40.084 INFO:tasks.cephadm:Deploying osd.6 on smithi026 with /dev/vg_nvme/lv_2... 2024-06-13T06:50:40.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-13T06:50:40.543 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:40.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:40 smithi136 ceph-mon[37229]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-13T06:50:40.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:40 smithi136 ceph-mon[37229]: from='osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:50:40.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:40 smithi136 ceph-mon[37229]: osdmap e36: 6 total, 5 up, 6 in 2024-06-13T06:50:40.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:40.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:40 smithi136 ceph-mon[37229]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:50:40.627 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 06:50:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T06:50:40.491+0000 7f2fe34d7700 -1 osd.5 0 waiting for initial osdmap 2024-06-13T06:50:40.627 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 06:50:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T06:50:40.501+0000 7f2fdf2ca700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:50:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:40 smithi026 ceph-mon[32041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-13T06:50:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:40 smithi026 ceph-mon[32041]: from='osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:50:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:40 smithi026 ceph-mon[32041]: osdmap e36: 6 total, 5 up, 6 in 2024-06-13T06:50:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:40 smithi026 ceph-mon[32041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:50:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:40 smithi135 ceph-mon[36276]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-13T06:50:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:40 smithi135 ceph-mon[36276]: from='osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:50:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:40 smithi135 ceph-mon[36276]: osdmap e36: 6 total, 5 up, 6 in 2024-06-13T06:50:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:40 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:40 smithi135 ceph-mon[36276]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:50:41.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:41 smithi136 ceph-mon[37229]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:41.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:41 smithi136 ceph-mon[37229]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:50:41.613 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:41 smithi136 ceph-mon[37229]: osdmap e37: 6 total, 5 up, 6 in 2024-06-13T06:50:41.613 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:41.613 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:41.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:41 smithi026 ceph-mon[32041]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:41.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:41 smithi026 ceph-mon[32041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:50:41.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:41 smithi026 ceph-mon[32041]: osdmap e37: 6 total, 5 up, 6 in 2024-06-13T06:50:41.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:41.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:41 smithi135 ceph-mon[36276]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:50:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:41 smithi135 ceph-mon[36276]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:50:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:41 smithi135 ceph-mon[36276]: osdmap e37: 6 total, 5 up, 6 in 2024-06-13T06:50:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:42.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:42 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:50:42.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:42 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:50:42.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:42 smithi026 ceph-mon[32041]: osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] boot 2024-06-13T06:50:42.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:42 smithi026 ceph-mon[32041]: osdmap e38: 6 total, 6 up, 6 in 2024-06-13T06:50:42.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:42 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:42.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:42 smithi026 ceph-mon[32041]: osdmap e39: 6 total, 6 up, 6 in 2024-06-13T06:50:42.704 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:42 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:50:42.704 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:42 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:50:42.704 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:42 smithi136 ceph-mon[37229]: osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] boot 2024-06-13T06:50:42.704 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:42 smithi136 ceph-mon[37229]: osdmap e38: 6 total, 6 up, 6 in 2024-06-13T06:50:42.705 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:42 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:42.705 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:42 smithi136 ceph-mon[37229]: osdmap e39: 6 total, 6 up, 6 in 2024-06-13T06:50:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:42 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:50:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:42 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:50:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:42 smithi135 ceph-mon[36276]: osd.5 [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] boot 2024-06-13T06:50:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:42 smithi135 ceph-mon[36276]: osdmap e38: 6 total, 6 up, 6 in 2024-06-13T06:50:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:42 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:50:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:42 smithi135 ceph-mon[36276]: osdmap e39: 6 total, 6 up, 6 in 2024-06-13T06:50:43.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:43.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:43.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:43.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:43.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:43.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:43 smithi026 ceph-mon[32041]: osdmap e40: 6 total, 6 up, 6 in 2024-06-13T06:50:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-13T06:50:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-13T06:50:43.594 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-06-13T06:50:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-06-13T06:50:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.033516 s, 313 MB/s 2024-06-13T06:50:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:43 smithi135 ceph-mon[36276]: osdmap e40: 6 total, 6 up, 6 in 2024-06-13T06:50:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:50:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:50:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:43 smithi136 ceph-mon[37229]: osdmap e40: 6 total, 6 up, 6 in 2024-06-13T06:50:44.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2024-06-13T06:50:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:45.121 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:44 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi136 2024-06-13T06:50:45.121 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:44 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi136 to 8544M 2024-06-13T06:50:45.121 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:44 smithi026 ceph-mon[32041]: osdmap e41: 6 total, 6 up, 6 in 2024-06-13T06:50:45.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:44 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi136 2024-06-13T06:50:45.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:44 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi136 to 8544M 2024-06-13T06:50:45.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:44 smithi135 ceph-mon[36276]: osdmap e41: 6 total, 6 up, 6 in 2024-06-13T06:50:45.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:44 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi136 2024-06-13T06:50:45.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:44 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi136 to 8544M 2024-06-13T06:50:45.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:44 smithi136 ceph-mon[37229]: osdmap e41: 6 total, 6 up, 6 in 2024-06-13T06:50:46.189 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:45 smithi026 ceph-mon[32041]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:46.189 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:45 smithi026 ceph-mon[32041]: osdmap e42: 6 total, 6 up, 6 in 2024-06-13T06:50:46.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:45 smithi135 ceph-mon[36276]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:45 smithi135 ceph-mon[36276]: osdmap e42: 6 total, 6 up, 6 in 2024-06-13T06:50:46.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:45 smithi136 ceph-mon[37229]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:46.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:45 smithi136 ceph-mon[37229]: osdmap e42: 6 total, 6 up, 6 in 2024-06-13T06:50:47.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 -- 172.21.15.26:0/3125783592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 msgr2=0x7f671c10cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:47.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 --2- 172.21.15.26:0/3125783592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 0x7f671c10cb80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6714009fc0 tx=0x7f671400b330 comp rx=0 tx=0).stop 2024-06-13T06:50:47.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 -- 172.21.15.26:0/3125783592 shutdown_connections 2024-06-13T06:50:47.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 --2- 172.21.15.26:0/3125783592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 0x7f671c10cb80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:47.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 --2- 172.21.15.26:0/3125783592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f671c104730 0x7f671c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:47.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 --2- 172.21.15.26:0/3125783592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f671c105930 0x7f671c105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:47.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 -- 172.21.15.26:0/3125783592 >> 172.21.15.26:0/3125783592 conn(0x7f671c0ffcf0 msgr2=0x7f671c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:47.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 -- 172.21.15.26:0/3125783592 shutdown_connections 2024-06-13T06:50:47.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.400+0000 7f6724600700 1 -- 172.21.15.26:0/3125783592 wait complete. 2024-06-13T06:50:47.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.401+0000 7f6724600700 1 Processor -- start 2024-06-13T06:50:47.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6724600700 1 -- start start 2024-06-13T06:50:47.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6724600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f671c104730 0x7f671c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:47.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6724600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f671c105930 0x7f671c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:47.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6724600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 0x7f671c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:47.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6724600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f671c110350 con 0x7f671c104730 2024-06-13T06:50:47.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6724600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1101d0 con 0x7f671c1062d0 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6724600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1104d0 con 0x7f671c105930 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6721c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f671c104730 0x7f671c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6721c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f671c104730 0x7f671c19a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47652/0 (socket says 172.21.15.26:47652) 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.402+0000 7f6721c00700 1 -- 172.21.15.26:0/1361284831 learned_addr learned my addr 172.21.15.26:0/1361284831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6722600700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 0x7f671c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6721200700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f671c105930 0x7f671c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6721c00700 1 -- 172.21.15.26:0/1361284831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f671c105930 msgr2=0x7f671c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6721c00700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f671c105930 0x7f671c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6721c00700 1 -- 172.21.15.26:0/1361284831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 msgr2=0x7f671c1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6721c00700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 0x7f671c1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6721c00700 1 -- 172.21.15.26:0/1361284831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c1a2500 con 0x7f671c104730 2024-06-13T06:50:47.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6722600700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 0x7f671c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:47.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6721200700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f671c105930 0x7f671c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:47.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.403+0000 7f6721c00700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f671c104730 0x7f671c19a8a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f671000ea20 tx=0x7f671000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:47.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.404+0000 7f671a000700 1 -- 172.21.15.26:0/1361284831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710004220 con 0x7f671c104730 2024-06-13T06:50:47.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.404+0000 7f671a000700 1 -- 172.21.15.26:0/1361284831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67100043c0 con 0x7f671c104730 2024-06-13T06:50:47.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.404+0000 7f671a000700 1 -- 172.21.15.26:0/1361284831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710003c70 con 0x7f671c104730 2024-06-13T06:50:47.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.404+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c1a27f0 con 0x7f671c104730 2024-06-13T06:50:47.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.404+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c1a2ce0 con 0x7f671c104730 2024-06-13T06:50:47.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.405+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c069180 con 0x7f671c104730 2024-06-13T06:50:47.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.407+0000 7f671a000700 1 -- 172.21.15.26:0/1361284831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f671000cd30 con 0x7f671c104730 2024-06-13T06:50:47.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.409+0000 7f671a000700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f67040a2ca0 0x7f67040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:47.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.409+0000 7f671a000700 1 -- 172.21.15.26:0/1361284831 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f67100ce330 con 0x7f671c104730 2024-06-13T06:50:47.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.409+0000 7f6721200700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f67040a2ca0 0x7f67040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:47.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.410+0000 7f6721200700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f67040a2ca0 0x7f67040a5160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6708011490 tx=0x7f6708009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:47.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.414+0000 7f671a000700 1 -- 172.21.15.26:0/1361284831 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f671009c0b0 con 0x7f671c104730 2024-06-13T06:50:47.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:50:47.655+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f671c0628d0 con 0x7f67040a2ca0 2024-06-13T06:50:47.906 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:47 smithi026 ceph-mon[32041]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:50:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:50:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:47 smithi135 ceph-mon[36276]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:50:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:50:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:47 smithi136 ceph-mon[37229]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:50:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:50:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:50:48.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:49.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:48 smithi026 ceph-mon[32041]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:50:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:48 smithi135 ceph-mon[36276]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:50:49.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:48 smithi136 ceph-mon[37229]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:50:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:49 smithi026 ceph-mon[32041]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-13T06:50:50.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:49 smithi135 ceph-mon[36276]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-13T06:50:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:49 smithi136 ceph-mon[37229]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-13T06:50:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:51 smithi026 ceph-mon[32041]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-13T06:50:52.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:51 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3630446156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8f82b9-822e-4b98-8629-b0bd8723c4c1"}]: dispatch 2024-06-13T06:50:52.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:51 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3630446156' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8f82b9-822e-4b98-8629-b0bd8723c4c1"}]': finished 2024-06-13T06:50:52.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:51 smithi026 ceph-mon[32041]: osdmap e43: 7 total, 6 up, 7 in 2024-06-13T06:50:52.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:50:52.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:51 smithi135 ceph-mon[36276]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-13T06:50:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:51 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3630446156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8f82b9-822e-4b98-8629-b0bd8723c4c1"}]: dispatch 2024-06-13T06:50:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:51 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3630446156' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8f82b9-822e-4b98-8629-b0bd8723c4c1"}]': finished 2024-06-13T06:50:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:51 smithi135 ceph-mon[36276]: osdmap e43: 7 total, 6 up, 7 in 2024-06-13T06:50:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:50:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:51 smithi136 ceph-mon[37229]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-13T06:50:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:51 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3630446156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8f82b9-822e-4b98-8629-b0bd8723c4c1"}]: dispatch 2024-06-13T06:50:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:51 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3630446156' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8f82b9-822e-4b98-8629-b0bd8723c4c1"}]': finished 2024-06-13T06:50:52.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:51 smithi136 ceph-mon[37229]: osdmap e43: 7 total, 6 up, 7 in 2024-06-13T06:50:52.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:50:53.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:52 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3770213315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:50:53.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:52 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3770213315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:50:53.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:52 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3770213315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:50:54.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:53 smithi136 ceph-mon[37229]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:50:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:53 smithi026 ceph-mon[32041]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:50:54.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:53 smithi135 ceph-mon[36276]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:50:56.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:55 smithi026 ceph-mon[32041]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-13T06:50:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:55 smithi135 ceph-mon[36276]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-13T06:50:56.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:55 smithi136 ceph-mon[37229]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-13T06:50:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:57 smithi026 ceph-mon[32041]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:50:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:57 smithi135 ceph-mon[36276]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:50:58.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:57 smithi136 ceph-mon[37229]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:51:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:50:59 smithi026 ceph-mon[32041]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:00.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:50:59 smithi135 ceph-mon[36276]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:50:59 smithi136 ceph-mon[37229]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:01 smithi026 ceph-mon[32041]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-13T06:51:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:02.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:01 smithi135 ceph-mon[36276]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-13T06:51:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:01 smithi136 ceph-mon[37229]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-13T06:51:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:03.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:02 smithi026 ceph-mon[32041]: Deploying daemon osd.6 on smithi026 2024-06-13T06:51:03.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:02 smithi135 ceph-mon[36276]: Deploying daemon osd.6 on smithi026 2024-06-13T06:51:03.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:02 smithi136 ceph-mon[37229]: Deploying daemon osd.6 on smithi026 2024-06-13T06:51:04.247 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:03 smithi026 ceph-mon[32041]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:04.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:03 smithi135 ceph-mon[36276]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:03 smithi136 ceph-mon[37229]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:05 smithi026 ceph-mon[32041]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:06.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:05 smithi135 ceph-mon[36276]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:05 smithi136 ceph-mon[37229]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:07 smithi026 ceph-mon[32041]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:08.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:07 smithi135 ceph-mon[36276]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:07 smithi136 ceph-mon[37229]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:10.070 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:09 smithi026 ceph-mon[32041]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:09 smithi135 ceph-mon[36276]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:09 smithi136 ceph-mon[37229]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:11.616 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:11 smithi026 ceph-mon[32041]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:11.616 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:11.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:11.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:11.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:11 smithi136 ceph-mon[37229]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:11.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:11.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:11.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:11 smithi135 ceph-mon[36276]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:13.580 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:13 smithi026 ceph-mon[32041]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:13.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:13 smithi136 ceph-mon[37229]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:13.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:13 smithi135 ceph-mon[36276]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:14 smithi136 ceph-mon[37229]: from='osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:51:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:14 smithi136 ceph-mon[37229]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:51:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:14 smithi026 ceph-mon[32041]: from='osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:51:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:14 smithi026 ceph-mon[32041]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:51:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:14 smithi135 ceph-mon[36276]: from='osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:51:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:14 smithi135 ceph-mon[36276]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:51:15.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.361+0000 7f671a000700 1 -- 172.21.15.26:0/1361284831 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f671c0628d0 con 0x7f67040a2ca0 2024-06-13T06:51:15.362 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 6 on host 'smithi026' 2024-06-13T06:51:15.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.364+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f67040a2ca0 msgr2=0x7f67040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:15.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f67040a2ca0 0x7f67040a5160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6708011490 tx=0x7f6708009260 comp rx=0 tx=0).stop 2024-06-13T06:51:15.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f671c104730 msgr2=0x7f671c19a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:15.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f671c104730 0x7f671c19a8a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f671000ea20 tx=0x7f671000ed30 comp rx=0 tx=0).stop 2024-06-13T06:51:15.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 shutdown_connections 2024-06-13T06:51:15.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f67040a2ca0 0x7f67040a5160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:15.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c1062d0 0x7f671c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:15.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f671c104730 0x7f671c19a8a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:15.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 --2- 172.21.15.26:0/1361284831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f671c105930 0x7f671c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:15.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.365+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 >> 172.21.15.26:0/1361284831 conn(0x7f671c0ffcf0 msgr2=0x7f671c108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:15.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.366+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 shutdown_connections 2024-06-13T06:51:15.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:51:15.366+0000 7f6724600700 1 -- 172.21.15.26:0/1361284831 wait complete. 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: osdmap e44: 7 total, 6 up, 7 in 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: from='osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:15.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-13T06:51:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: osdmap e44: 7 total, 6 up, 7 in 2024-06-13T06:51:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: from='osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:51:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:51:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-13T06:51:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: osdmap e44: 7 total, 6 up, 7 in 2024-06-13T06:51:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: from='osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:51:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:51:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:16.230 DEBUG:teuthology.orchestra.run.smithi026:osd.6> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.6.service 2024-06-13T06:51:16.237 INFO:tasks.cephadm:Deploying osd.7 on smithi135 with /dev/vg_nvme/lv_2... 2024-06-13T06:51:16.237 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-13T06:51:16.483 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:51:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:16 smithi135 ceph-mon[36276]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:51:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:16 smithi135 ceph-mon[36276]: osdmap e45: 7 total, 6 up, 7 in 2024-06-13T06:51:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:16 smithi135 ceph-mon[36276]: from='osd.6 ' entity='osd.6' 2024-06-13T06:51:16.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:16 smithi136 ceph-mon[37229]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:51:16.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:16 smithi136 ceph-mon[37229]: osdmap e45: 7 total, 6 up, 7 in 2024-06-13T06:51:16.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:16.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:16.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:16.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:16 smithi136 ceph-mon[37229]: from='osd.6 ' entity='osd.6' 2024-06-13T06:51:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:16 smithi026 ceph-mon[32041]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:51:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:16 smithi026 ceph-mon[32041]: osdmap e45: 7 total, 6 up, 7 in 2024-06-13T06:51:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:16.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:16.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:16 smithi026 ceph-mon[32041]: from='osd.6 ' entity='osd.6' 2024-06-13T06:51:17.494 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:17 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:51:17.494 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:17 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:51:17.494 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:17 smithi026 ceph-mon[32041]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:17.494 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:17.494 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:17 smithi026 ceph-mon[32041]: osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] boot 2024-06-13T06:51:17.494 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:17 smithi026 ceph-mon[32041]: osdmap e46: 7 total, 7 up, 7 in 2024-06-13T06:51:17.494 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:17 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:51:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:17 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:51:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:17 smithi135 ceph-mon[36276]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:17 smithi135 ceph-mon[36276]: osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] boot 2024-06-13T06:51:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:17 smithi135 ceph-mon[36276]: osdmap e46: 7 total, 7 up, 7 in 2024-06-13T06:51:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:17.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:17 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:51:17.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:17 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:51:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:17 smithi136 ceph-mon[37229]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:51:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:17 smithi136 ceph-mon[37229]: osd.6 [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] boot 2024-06-13T06:51:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:17 smithi136 ceph-mon[37229]: osdmap e46: 7 total, 7 up, 7 in 2024-06-13T06:51:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:51:18.425 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-13T06:51:18.426 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-13T06:51:18.426 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-13T06:51:18.426 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-13T06:51:18.426 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114816 s, 913 MB/s 2024-06-13T06:51:18.426 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-13T06:51:19.127 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:19 smithi135 ceph-mon[36276]: osdmap e47: 7 total, 7 up, 7 in 2024-06-13T06:51:19.127 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:19 smithi135 ceph-mon[36276]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:19.154 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_2 2024-06-13T06:51:19.305 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:19 smithi026 ceph-mon[32041]: osdmap e47: 7 total, 7 up, 7 in 2024-06-13T06:51:19.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:19 smithi026 ceph-mon[32041]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:19.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:19 smithi136 ceph-mon[37229]: osdmap e47: 7 total, 7 up, 7 in 2024-06-13T06:51:19.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:19 smithi136 ceph-mon[37229]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:19.396 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.006+0000 7f50b60c4700 1 -- 172.21.15.135:0/3442291683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b01040c0 msgr2=0x7f50b010a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.006+0000 7f50b60c4700 1 --2- 172.21.15.135:0/3442291683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b01040c0 0x7f50b010a970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50a800b0d0 tx=0x7f50a800b3e0 comp rx=0 tx=0).stop 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.006+0000 7f50b60c4700 1 -- 172.21.15.135:0/3442291683 shutdown_connections 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.006+0000 7f50b60c4700 1 --2- 172.21.15.135:0/3442291683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b01040c0 0x7f50b010a970 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.006+0000 7f50b60c4700 1 --2- 172.21.15.135:0/3442291683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50b0102520 0x7f50b0102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.006+0000 7f50b60c4700 1 --2- 172.21.15.135:0/3442291683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50b0103720 0x7f50b0103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.006+0000 7f50b60c4700 1 -- 172.21.15.135:0/3442291683 >> 172.21.15.135:0/3442291683 conn(0x7f50b00fdae0 msgr2=0x7f50b00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.006+0000 7f50b60c4700 1 -- 172.21.15.135:0/3442291683 shutdown_connections 2024-06-13T06:51:21.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 -- 172.21.15.135:0/3442291683 wait complete. 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 Processor -- start 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 -- start start 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b0102520 0x7f50b0076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50b0103720 0x7f50b0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50b01040c0 0x7f50b0074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f50b010e180 con 0x7f50b0103720 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f50b010e000 con 0x7f50b0102520 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50b60c4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f50b010e300 con 0x7f50b01040c0 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.007+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b0102520 0x7f50b0076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b0102520 0x7f50b0076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:50740/0 (socket says 172.21.15.135:50740) 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50af7fe700 1 -- 172.21.15.135:0/4151646235 learned_addr learned my addr 172.21.15.135:0/4151646235 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-13T06:51:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50affff700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50b01040c0 0x7f50b0074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50af7fe700 1 -- 172.21.15.135:0/4151646235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50b01040c0 msgr2=0x7f50b0074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50af7fe700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50b01040c0 0x7f50b0074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50af7fe700 1 -- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50b0103720 msgr2=0x7f50b0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50af7fe700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50b0103720 0x7f50b0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50af7fe700 1 -- 172.21.15.135:0/4151646235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b0075300 con 0x7f50b0102520 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50af7fe700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b0102520 0x7f50b0076230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c970 tx=0x7f50a000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50acff9700 1 -- 172.21.15.135:0/4151646235 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0004220 con 0x7f50b0102520 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b00755f0 con 0x7f50b0102520 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50acff9700 1 -- 172.21.15.135:0/4151646235 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50a0007d80 con 0x7f50b0102520 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b01a71c0 con 0x7f50b0102520 2024-06-13T06:51:21.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.008+0000 7f50acff9700 1 -- 172.21.15.135:0/4151646235 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0003e50 con 0x7f50b0102520 2024-06-13T06:51:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.009+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b0069180 con 0x7f50b0102520 2024-06-13T06:51:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.011+0000 7f50acff9700 1 -- 172.21.15.135:0/4151646235 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f50a00078a0 con 0x7f50b0102520 2024-06-13T06:51:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.011+0000 7f50acff9700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f50980a2e40 0x7f50980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.012+0000 7f50acff9700 1 -- 172.21.15.135:0/4151646235 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f50a00ce280 con 0x7f50b0102520 2024-06-13T06:51:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.012+0000 7f50aeffd700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f50980a2e40 0x7f50980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:21.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.012+0000 7f50aeffd700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f50980a2e40 0x7f50980a5300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009770 tx=0x7f50a4006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:21.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.015+0000 7f50acff9700 1 -- 172.21.15.135:0/4151646235 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f50a0014030 con 0x7f50b0102520 2024-06-13T06:51:21.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:21.217+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f50b00628d0 con 0x7f50980a2e40 2024-06-13T06:51:21.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:21 smithi026 ceph-mon[32041]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:21.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:51:21.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:51:21.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:21.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:21 smithi135 ceph-mon[36276]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:21.588 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:51:21.588 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:51:21.588 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:21.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:21 smithi136 ceph-mon[37229]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:21.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:51:21.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:51:21.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:22.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:22 smithi135 ceph-mon[36276]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:22 smithi136 ceph-mon[37229]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:22 smithi026 ceph-mon[32041]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi026 2024-06-13T06:51:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi026 to 5698M 2024-06-13T06:51:23.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:23.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:23.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:23.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:23.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:23.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi026 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi026 to 5698M 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:23.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:23.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:23.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi026 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi026 to 5698M 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:23.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:23.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:23.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:24 smithi026 ceph-mon[32041]: from='client.? 172.21.15.135:0/1848453109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]: dispatch 2024-06-13T06:51:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:24 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]: dispatch 2024-06-13T06:51:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:24 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]': finished 2024-06-13T06:51:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:24 smithi026 ceph-mon[32041]: osdmap e48: 8 total, 7 up, 8 in 2024-06-13T06:51:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:24.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:24 smithi135 ceph-mon[36276]: from='client.? 172.21.15.135:0/1848453109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]: dispatch 2024-06-13T06:51:24.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:24 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]: dispatch 2024-06-13T06:51:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:24 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]': finished 2024-06-13T06:51:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:24 smithi135 ceph-mon[36276]: osdmap e48: 8 total, 7 up, 8 in 2024-06-13T06:51:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:24.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:24 smithi136 ceph-mon[37229]: from='client.? 172.21.15.135:0/1848453109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]: dispatch 2024-06-13T06:51:24.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:24 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]: dispatch 2024-06-13T06:51:24.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:24 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9756cfaa-3ffb-42d6-9154-98f4d4aceb64"}]': finished 2024-06-13T06:51:24.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:24 smithi136 ceph-mon[37229]: osdmap e48: 8 total, 7 up, 8 in 2024-06-13T06:51:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:25 smithi026 ceph-mon[32041]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:25 smithi026 ceph-mon[32041]: from='client.? 172.21.15.135:0/837999678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:51:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:25 smithi135 ceph-mon[36276]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:25 smithi135 ceph-mon[36276]: from='client.? 172.21.15.135:0/837999678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:51:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:25 smithi136 ceph-mon[37229]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:25 smithi136 ceph-mon[37229]: from='client.? 172.21.15.135:0/837999678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:51:27.716 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:27 smithi135 ceph-mon[36276]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:27.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:27 smithi026 ceph-mon[32041]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:27 smithi136 ceph-mon[37229]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:29 smithi026 ceph-mon[32041]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:29.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:29 smithi135 ceph-mon[36276]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:29.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:29 smithi136 ceph-mon[37229]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:31.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:31 smithi026 ceph-mon[32041]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:31.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:31 smithi135 ceph-mon[36276]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:31.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:31 smithi136 ceph-mon[37229]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:32.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-13T06:51:32.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:32.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:32 smithi135 ceph-mon[36276]: Deploying daemon osd.7 on smithi135 2024-06-13T06:51:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-13T06:51:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:32 smithi026 ceph-mon[32041]: Deploying daemon osd.7 on smithi135 2024-06-13T06:51:32.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-13T06:51:32.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:32.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:32 smithi136 ceph-mon[37229]: Deploying daemon osd.7 on smithi135 2024-06-13T06:51:33.500 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:33 smithi135 ceph-mon[36276]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:33 smithi026 ceph-mon[32041]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:33 smithi136 ceph-mon[37229]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:35.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:35 smithi135 ceph-mon[36276]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:35 smithi026 ceph-mon[32041]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:35.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:35 smithi136 ceph-mon[37229]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:37 smithi026 ceph-mon[32041]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:37 smithi135 ceph-mon[36276]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:37 smithi136 ceph-mon[37229]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:37.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:37.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:39.503 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:39 smithi135 ceph-mon[36276]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:39.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:39 smithi026 ceph-mon[32041]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:39 smithi136 ceph-mon[37229]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:41.461 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 7 on host 'smithi135' 2024-06-13T06:51:41.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.461+0000 7f50acff9700 1 -- 172.21.15.135:0/4151646235 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f50b00628d0 con 0x7f50980a2e40 2024-06-13T06:51:41.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f50980a2e40 msgr2=0x7f50980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f50980a2e40 0x7f50980a5300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009770 tx=0x7f50a4006cd0 comp rx=0 tx=0).stop 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b0102520 msgr2=0x7f50b0076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b0102520 0x7f50b0076230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c970 tx=0x7f50a000cc80 comp rx=0 tx=0).stop 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 shutdown_connections 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f50980a2e40 0x7f50980a5300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50b0102520 0x7f50b0076230 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50b01040c0 0x7f50b0074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 --2- 172.21.15.135:0/4151646235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50b0103720 0x7f50b0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 >> 172.21.15.135:0/4151646235 conn(0x7f50b00fdae0 msgr2=0x7f50b0106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 shutdown_connections 2024-06-13T06:51:41.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:51:41.464+0000 7f50b60c4700 1 -- 172.21.15.135:0/4151646235 wait complete. 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:51:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:41.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:41.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:51:41.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:51:41.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:41.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:42.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:42.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:42.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:42.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:42.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:42.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:42.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:51:42.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:51:42.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:42.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:42.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:42.130 DEBUG:teuthology.orchestra.run.smithi135:osd.7> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.7.service 2024-06-13T06:51:42.132 INFO:tasks.cephadm:Deploying osd.8 on smithi136 with /dev/vg_nvme/lv_2... 2024-06-13T06:51:42.133 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-13T06:51:42.376 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:51:42.638 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 06:51:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T06:51:42.634+0000 7f0be4215700 -1 osd.7 0 waiting for initial osdmap 2024-06-13T06:51:42.826 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:42 smithi136 ceph-mon[37229]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-13T06:51:42.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:42 smithi136 ceph-mon[37229]: from='osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:51:42.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:42 smithi136 ceph-mon[37229]: osdmap e49: 8 total, 7 up, 8 in 2024-06-13T06:51:42.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:42 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:42.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:42 smithi136 ceph-mon[37229]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:51:42.962 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 06:51:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T06:51:42.643+0000 7f0be0008700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:51:42.965 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:42 smithi135 ceph-mon[36276]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-13T06:51:42.966 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:42 smithi135 ceph-mon[36276]: from='osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:51:42.966 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:42 smithi135 ceph-mon[36276]: osdmap e49: 8 total, 7 up, 8 in 2024-06-13T06:51:42.966 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:42 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:42.966 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:42 smithi135 ceph-mon[36276]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:51:43.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:42 smithi026 ceph-mon[32041]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-13T06:51:43.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:42 smithi026 ceph-mon[32041]: from='osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:51:43.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:42 smithi026 ceph-mon[32041]: osdmap e49: 8 total, 7 up, 8 in 2024-06-13T06:51:43.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:42 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:43.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:42 smithi026 ceph-mon[32041]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:51:43.865 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:43 smithi136 ceph-mon[37229]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:43.865 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:43 smithi136 ceph-mon[37229]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:51:43.865 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:43 smithi136 ceph-mon[37229]: osdmap e50: 8 total, 7 up, 8 in 2024-06-13T06:51:43.865 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:43.865 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:43 smithi026 ceph-mon[32041]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:43 smithi026 ceph-mon[32041]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:51:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:43 smithi026 ceph-mon[32041]: osdmap e50: 8 total, 7 up, 8 in 2024-06-13T06:51:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:43 smithi135 ceph-mon[36276]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:43 smithi135 ceph-mon[36276]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:51:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:43 smithi135 ceph-mon[36276]: osdmap e50: 8 total, 7 up, 8 in 2024-06-13T06:51:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:44.465 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-13T06:51:44.465 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-13T06:51:44.465 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-06-13T06:51:44.465 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-06-13T06:51:44.465 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00845022 s, 1.2 GB/s 2024-06-13T06:51:44.465 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-06-13T06:51:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:44 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:51:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:44 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:51:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:44 smithi135 ceph-mon[36276]: osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] boot 2024-06-13T06:51:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:44 smithi135 ceph-mon[36276]: osdmap e51: 8 total, 8 up, 8 in 2024-06-13T06:51:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:44 smithi135 ceph-mon[36276]: osdmap e52: 8 total, 8 up, 8 in 2024-06-13T06:51:44.851 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:44 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:51:44.851 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:44 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:51:44.851 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:44 smithi136 ceph-mon[37229]: osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] boot 2024-06-13T06:51:44.851 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:44 smithi136 ceph-mon[37229]: osdmap e51: 8 total, 8 up, 8 in 2024-06-13T06:51:44.852 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:44.852 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:44 smithi136 ceph-mon[37229]: osdmap e52: 8 total, 8 up, 8 in 2024-06-13T06:51:45.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:44 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:51:45.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:44 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:51:45.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:44 smithi026 ceph-mon[32041]: osd.7 [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] boot 2024-06-13T06:51:45.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:44 smithi026 ceph-mon[32041]: osdmap e51: 8 total, 8 up, 8 in 2024-06-13T06:51:45.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:51:45.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:44 smithi026 ceph-mon[32041]: osdmap e52: 8 total, 8 up, 8 in 2024-06-13T06:51:45.213 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-06-13T06:51:45.451 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:51:46.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: osdmap e53: 8 total, 8 up, 8 in 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: osdmap e53: 8 total, 8 up, 8 in 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:46.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: osdmap e53: 8 total, 8 up, 8 in 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 -- 172.21.15.136:0/3122945454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141025b0 msgr2=0x7f5614102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 --2- 172.21.15.136:0/3122945454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141025b0 0x7f5614102a10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5610009990 tx=0x7f5610009ca0 comp rx=0 tx=0).stop 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 -- 172.21.15.136:0/3122945454 shutdown_connections 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 --2- 172.21.15.136:0/3122945454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5614102f50 0x7f5614109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 --2- 172.21.15.136:0/3122945454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56141013b0 0x7f56141017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 --2- 172.21.15.136:0/3122945454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141025b0 0x7f5614102a10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 -- 172.21.15.136:0/3122945454 >> 172.21.15.136:0/3122945454 conn(0x7f56140fc930 msgr2=0x7f56140fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 -- 172.21.15.136:0/3122945454 shutdown_connections 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.117+0000 7f561c100700 1 -- 172.21.15.136:0/3122945454 wait complete. 2024-06-13T06:51:47.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561c100700 1 Processor -- start 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561c100700 1 -- start start 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561c100700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141013b0 0x7f561406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561c100700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56141025b0 0x7f561419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561c100700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5614102f50 0x7f561419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561c100700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f561410d030 con 0x7f5614102f50 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561c100700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f561410ceb0 con 0x7f56141025b0 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561c100700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f561410d1b0 con 0x7f56141013b0 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f5619e9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141013b0 0x7f561406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f5619e9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141013b0 0x7f561406b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:42510/0 (socket says 172.21.15.136:42510) 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f5619e9c700 1 -- 172.21.15.136:0/1407730887 learned_addr learned my addr 172.21.15.136:0/1407730887 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.118+0000 7f561969b700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56141025b0 0x7f561419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:47.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f5619e9c700 1 -- 172.21.15.136:0/1407730887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56141025b0 msgr2=0x7f561419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f5619e9c700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56141025b0 0x7f561419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f5619e9c700 1 -- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5614102f50 msgr2=0x7f561419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f5619e9c700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5614102f50 0x7f561419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f5619e9c700 1 -- 172.21.15.136:0/1407730887 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56141a03e0 con 0x7f56141013b0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f5619e9c700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141013b0 0x7f561406b7c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f560400b500 tx=0x7f560400b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f560affd700 1 -- 172.21.15.136:0/1407730887 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604015020 con 0x7f56141013b0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f560affd700 1 -- 172.21.15.136:0/1407730887 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5604011d10 con 0x7f56141013b0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56141a06d0 con 0x7f56141013b0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f560affd700 1 -- 172.21.15.136:0/1407730887 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560400f2e0 con 0x7f56141013b0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.119+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56141a0ce0 con 0x7f56141013b0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.120+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5614069180 con 0x7f56141013b0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.122+0000 7f560affd700 1 -- 172.21.15.136:0/1407730887 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5604018020 con 0x7f56141013b0 2024-06-13T06:51:47.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.122+0000 7f560affd700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f56000a2ca0 0x7f56000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:47.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.122+0000 7f560affd700 1 -- 172.21.15.136:0/1407730887 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f56040ce0e0 con 0x7f56141013b0 2024-06-13T06:51:47.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.123+0000 7f561969b700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f56000a2ca0 0x7f56000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:47.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.124+0000 7f561969b700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f56000a2ca0 0x7f56000a5160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f56100039c0 tx=0x7f5610005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:47.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.126+0000 7f560affd700 1 -- 172.21.15.136:0/1407730887 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f560401f020 con 0x7f56141013b0 2024-06-13T06:51:47.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:46 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi135 2024-06-13T06:51:47.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:46 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-13T06:51:47.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:46 smithi026 ceph-mon[32041]: osdmap e54: 8 total, 8 up, 8 in 2024-06-13T06:51:47.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:46 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi135 2024-06-13T06:51:47.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:46 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-13T06:51:47.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:46 smithi135 ceph-mon[36276]: osdmap e54: 8 total, 8 up, 8 in 2024-06-13T06:51:47.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:51:47.330+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f56140628d0 con 0x7f56000a2ca0 2024-06-13T06:51:47.330 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:46 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi135 2024-06-13T06:51:47.330 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:46 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-13T06:51:47.330 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:46 smithi136 ceph-mon[37229]: osdmap e54: 8 total, 8 up, 8 in 2024-06-13T06:51:48.192 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:47 smithi136 ceph-mon[37229]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:48.192 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:47 smithi136 ceph-mon[37229]: osdmap e55: 8 total, 8 up, 8 in 2024-06-13T06:51:48.192 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:51:48.192 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:51:48.192 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:47 smithi026 ceph-mon[32041]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:47 smithi026 ceph-mon[32041]: osdmap e55: 8 total, 8 up, 8 in 2024-06-13T06:51:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:51:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:51:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:47 smithi135 ceph-mon[36276]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:47 smithi135 ceph-mon[36276]: osdmap e55: 8 total, 8 up, 8 in 2024-06-13T06:51:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:51:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:51:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:49.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:48 smithi026 ceph-mon[32041]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:49.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:48 smithi136 ceph-mon[37229]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:49.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:48 smithi135 ceph-mon[36276]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:49 smithi026 ceph-mon[32041]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:50.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:49 smithi135 ceph-mon[36276]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:49 smithi136 ceph-mon[37229]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:51.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:50 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/2935774286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]: dispatch 2024-06-13T06:51:51.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:50 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]: dispatch 2024-06-13T06:51:51.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:50 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]': finished 2024-06-13T06:51:51.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:50 smithi026 ceph-mon[32041]: osdmap e56: 9 total, 8 up, 9 in 2024-06-13T06:51:51.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:50 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:51:51.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:50 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/3670779557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:51:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:50 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/2935774286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]: dispatch 2024-06-13T06:51:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:50 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]: dispatch 2024-06-13T06:51:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:50 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]': finished 2024-06-13T06:51:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:50 smithi135 ceph-mon[36276]: osdmap e56: 9 total, 8 up, 9 in 2024-06-13T06:51:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:50 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:51:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:50 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/3670779557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:51:51.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:50 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/2935774286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]: dispatch 2024-06-13T06:51:51.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:50 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]: dispatch 2024-06-13T06:51:51.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:50 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ef2c367-8eae-4f63-9ea8-5af0a4129fd9"}]': finished 2024-06-13T06:51:51.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:50 smithi136 ceph-mon[37229]: osdmap e56: 9 total, 8 up, 9 in 2024-06-13T06:51:51.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:50 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:51:51.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:50 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/3670779557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:51:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:51 smithi026 ceph-mon[32041]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:51 smithi135 ceph-mon[36276]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:51 smithi136 ceph-mon[37229]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:54.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:53 smithi136 ceph-mon[37229]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:53 smithi026 ceph-mon[32041]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:54.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:53 smithi135 ceph-mon[36276]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:56.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:55 smithi136 ceph-mon[37229]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:56.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:55 smithi026 ceph-mon[32041]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:56.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:55 smithi135 ceph-mon[36276]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:57 smithi026 ceph-mon[32041]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-13T06:51:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:58.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:57 smithi135 ceph-mon[36276]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:57 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-13T06:51:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:57 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:58.343 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:57 smithi136 ceph-mon[37229]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:51:58.343 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:57 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-13T06:51:58.344 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:57 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:58 smithi026 ceph-mon[32041]: Deploying daemon osd.8 on smithi136 2024-06-13T06:51:59.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:58 smithi135 ceph-mon[36276]: Deploying daemon osd.8 on smithi136 2024-06-13T06:51:59.368 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:58 smithi136 ceph-mon[37229]: Deploying daemon osd.8 on smithi136 2024-06-13T06:52:00.103 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:51:59 smithi136 ceph-mon[37229]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:51:59 smithi026 ceph-mon[32041]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:00.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:51:59 smithi135 ceph-mon[36276]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:02.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:01 smithi136 ceph-mon[37229]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:01 smithi026 ceph-mon[32041]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:02.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:01 smithi135 ceph-mon[36276]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:04.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:03 smithi136 ceph-mon[37229]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:04.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:03 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:04.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:03 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:04.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:03 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:03 smithi026 ceph-mon[32041]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:03 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:03 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:03 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:03 smithi135 ceph-mon[36276]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:03 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:03 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:03 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:06.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:06 smithi136 ceph-mon[37229]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:06 smithi026 ceph-mon[32041]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:06.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:06 smithi135 ceph-mon[36276]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:07.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.428+0000 7f560affd700 1 -- 172.21.15.136:0/1407730887 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f56140628d0 con 0x7f56000a2ca0 2024-06-13T06:52:07.427 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 8 on host 'smithi136' 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f56000a2ca0 msgr2=0x7f56000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f56000a2ca0 0x7f56000a5160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f56100039c0 tx=0x7f5610005fb0 comp rx=0 tx=0).stop 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141013b0 msgr2=0x7f561406b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141013b0 0x7f561406b7c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f560400b500 tx=0x7f560400b8c0 comp rx=0 tx=0).stop 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 shutdown_connections 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f56000a2ca0 0x7f56000a5160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5614102f50 0x7f561419fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56141013b0 0x7f561406b7c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 --2- 172.21.15.136:0/1407730887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56141025b0 0x7f561419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:07.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 >> 172.21.15.136:0/1407730887 conn(0x7f56140fc930 msgr2=0x7f56141058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:07.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 shutdown_connections 2024-06-13T06:52:07.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:52:07.429+0000 7f561c100700 1 -- 172.21.15.136:0/1407730887 wait complete. 2024-06-13T06:52:07.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:07.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:07.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:07.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:07.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:07.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:07.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:52:07.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:52:07.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:07.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:07.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:08.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:08.102 DEBUG:teuthology.orchestra.run.smithi136:osd.8> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.8.service 2024-06-13T06:52:08.104 INFO:tasks.cephadm:Deploying osd.9 on smithi026 with /dev/vg_nvme/lv_1... 2024-06-13T06:52:08.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-13T06:52:08.568 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:09.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:09 smithi135 ceph-mon[36276]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-13T06:52:09.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:09 smithi135 ceph-mon[36276]: from='osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:52:09.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:09 smithi135 ceph-mon[36276]: osdmap e57: 9 total, 8 up, 9 in 2024-06-13T06:52:09.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:09 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:09.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:09 smithi135 ceph-mon[36276]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:52:09.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:09 smithi136 ceph-mon[37229]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-13T06:52:09.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:09 smithi136 ceph-mon[37229]: from='osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:52:09.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:09 smithi136 ceph-mon[37229]: osdmap e57: 9 total, 8 up, 9 in 2024-06-13T06:52:09.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:09 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:09.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:09 smithi136 ceph-mon[37229]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:52:09.377 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 06:52:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T06:52:09.017+0000 7fb8f0620700 -1 osd.8 0 waiting for initial osdmap 2024-06-13T06:52:09.377 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 06:52:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T06:52:09.027+0000 7fb8ec413700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:52:09.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:09 smithi026 ceph-mon[32041]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-13T06:52:09.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:09 smithi026 ceph-mon[32041]: from='osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:52:09.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:09 smithi026 ceph-mon[32041]: osdmap e57: 9 total, 8 up, 9 in 2024-06-13T06:52:09.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:09 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:09.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:09 smithi026 ceph-mon[32041]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:52:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:10 smithi026 ceph-mon[32041]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:10 smithi026 ceph-mon[32041]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:52:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:10 smithi026 ceph-mon[32041]: osdmap e58: 9 total, 8 up, 9 in 2024-06-13T06:52:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:10 smithi026 ceph-mon[32041]: osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] boot 2024-06-13T06:52:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:10 smithi026 ceph-mon[32041]: osdmap e59: 9 total, 9 up, 9 in 2024-06-13T06:52:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:10 smithi135 ceph-mon[36276]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:10 smithi135 ceph-mon[36276]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:52:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:10 smithi135 ceph-mon[36276]: osdmap e58: 9 total, 8 up, 9 in 2024-06-13T06:52:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:10 smithi135 ceph-mon[36276]: osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] boot 2024-06-13T06:52:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:10 smithi135 ceph-mon[36276]: osdmap e59: 9 total, 9 up, 9 in 2024-06-13T06:52:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:10.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:10 smithi136 ceph-mon[37229]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:52:10.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:10 smithi136 ceph-mon[37229]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:52:10.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:10 smithi136 ceph-mon[37229]: osdmap e58: 9 total, 8 up, 9 in 2024-06-13T06:52:10.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:10.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:10 smithi136 ceph-mon[37229]: osd.8 [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] boot 2024-06-13T06:52:10.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:10 smithi136 ceph-mon[37229]: osdmap e59: 9 total, 9 up, 9 in 2024-06-13T06:52:10.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:52:11.227 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:11 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:52:11.227 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:11 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:52:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:11 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:52:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:11 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:52:11.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:11 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:52:11.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:11 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:52:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-13T06:52:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-13T06:52:11.595 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-06-13T06:52:11.596 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-06-13T06:52:11.596 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0371899 s, 282 MB/s 2024-06-13T06:52:11.596 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: osdmap e60: 9 total, 9 up, 9 in 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi136 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi136 to 5696M 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:12.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:12.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: osdmap e60: 9 total, 9 up, 9 in 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi136 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi136 to 5696M 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: osdmap e60: 9 total, 9 up, 9 in 2024-06-13T06:52:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi136 2024-06-13T06:52:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:12.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi136 to 5696M 2024-06-13T06:52:12.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:12.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:12.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:12.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2024-06-13T06:52:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:13.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:13 smithi136 ceph-mon[37229]: osdmap e61: 9 total, 9 up, 9 in 2024-06-13T06:52:13.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:13 smithi136 ceph-mon[37229]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:13.412 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:13 smithi026 ceph-mon[32041]: osdmap e61: 9 total, 9 up, 9 in 2024-06-13T06:52:13.412 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:13 smithi026 ceph-mon[32041]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:13.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:13 smithi135 ceph-mon[36276]: osdmap e61: 9 total, 9 up, 9 in 2024-06-13T06:52:13.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:13 smithi135 ceph-mon[36276]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:14 smithi026 ceph-mon[32041]: osdmap e62: 9 total, 9 up, 9 in 2024-06-13T06:52:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:14 smithi136 ceph-mon[37229]: osdmap e62: 9 total, 9 up, 9 in 2024-06-13T06:52:14.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:14 smithi135 ceph-mon[36276]: osdmap e62: 9 total, 9 up, 9 in 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 -- 172.21.15.26:0/2561195953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 msgr2=0x7f9c1c10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 --2- 172.21.15.26:0/2561195953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 0x7f9c1c10a990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009fc0 tx=0x7f9c1400b330 comp rx=0 tx=0).stop 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 -- 172.21.15.26:0/2561195953 shutdown_connections 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 --2- 172.21.15.26:0/2561195953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 0x7f9c1c10a990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 --2- 172.21.15.26:0/2561195953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c1c102540 0x7f9c1c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 --2- 172.21.15.26:0/2561195953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c1c103740 0x7f9c1c103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 -- 172.21.15.26:0/2561195953 >> 172.21.15.26:0/2561195953 conn(0x7f9c1c0fdae0 msgr2=0x7f9c1c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 -- 172.21.15.26:0/2561195953 shutdown_connections 2024-06-13T06:52:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.333+0000 7f9c22600700 1 -- 172.21.15.26:0/2561195953 wait complete. 2024-06-13T06:52:15.335 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:15 smithi026 ceph-mon[32041]: osdmap e63: 9 total, 9 up, 9 in 2024-06-13T06:52:15.335 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:15 smithi026 ceph-mon[32041]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:15.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.335+0000 7f9c22600700 1 Processor -- start 2024-06-13T06:52:15.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.335+0000 7f9c22600700 1 -- start start 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.335+0000 7f9c22600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c1c102540 0x7f9c1c076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.335+0000 7f9c22600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c1c103740 0x7f9c1c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.335+0000 7f9c22600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 0x7f9c1c074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.335+0000 7f9c22600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10e160 con 0x7f9c1c102540 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.335+0000 7f9c22600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10dfe0 con 0x7f9c1c1040e0 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.335+0000 7f9c22600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10e2e0 con 0x7f9c1c103740 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c1c103740 0x7f9c1c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c1c103740 0x7f9c1c074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37414/0 (socket says 172.21.15.26:37414) 2024-06-13T06:52:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 -- 172.21.15.26:0/2156855735 learned_addr learned my addr 172.21.15.26:0/2156855735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1b400700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c1c102540 0x7f9c1c076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1be00700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 0x7f9c1c074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 -- 172.21.15.26:0/2156855735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 msgr2=0x7f9c1c074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 0x7f9c1c074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 -- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c1c102540 msgr2=0x7f9c1c076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c1c102540 0x7f9c1c076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 -- 172.21.15.26:0/2156855735 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c075670 con 0x7f9c1c103740 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1be00700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 0x7f9c1c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:52:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.336+0000 7f9c1aa00700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c1c103740 0x7f9c1c074880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800e9a0 tx=0x7f9c0800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.337+0000 7f9c13e00700 1 -- 172.21.15.26:0/2156855735 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800cbd0 con 0x7f9c1c103740 2024-06-13T06:52:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.337+0000 7f9c13e00700 1 -- 172.21.15.26:0/2156855735 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c0800cd70 con 0x7f9c1c103740 2024-06-13T06:52:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.337+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c1a2840 con 0x7f9c1c103740 2024-06-13T06:52:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.337+0000 7f9c13e00700 1 -- 172.21.15.26:0/2156855735 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08018750 con 0x7f9c1c103740 2024-06-13T06:52:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.337+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c1a2d50 con 0x7f9c1c103740 2024-06-13T06:52:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.338+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c069180 con 0x7f9c1c103740 2024-06-13T06:52:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.341+0000 7f9c13e00700 1 -- 172.21.15.26:0/2156855735 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9c08010430 con 0x7f9c1c103740 2024-06-13T06:52:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.342+0000 7f9c13e00700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c000a2c20 0x7f9c000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.342+0000 7f9c13e00700 1 -- 172.21.15.26:0/2156855735 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f9c080ce9f0 con 0x7f9c1c103740 2024-06-13T06:52:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.342+0000 7f9c1b400700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c000a2c20 0x7f9c000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.343+0000 7f9c1b400700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c000a2c20 0x7f9c000a50e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009750 tx=0x7f9c0c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:15.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.346+0000 7f9c13e00700 1 -- 172.21.15.26:0/2156855735 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c0809c260 con 0x7f9c1c103740 2024-06-13T06:52:15.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:15 smithi136 ceph-mon[37229]: osdmap e63: 9 total, 9 up, 9 in 2024-06-13T06:52:15.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:15 smithi136 ceph-mon[37229]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:15.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:15 smithi135 ceph-mon[36276]: osdmap e63: 9 total, 9 up, 9 in 2024-06-13T06:52:15.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:15 smithi135 ceph-mon[36276]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:15.587+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9c1c0628d0 con 0x7f9c000a2c20 2024-06-13T06:52:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:16 smithi136 ceph-mon[37229]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:52:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:52:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:16 smithi026 ceph-mon[32041]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:52:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:52:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:16 smithi135 ceph-mon[36276]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:52:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:52:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:17 smithi026 ceph-mon[32041]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-13T06:52:17.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:17 smithi136 ceph-mon[37229]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-13T06:52:17.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:17 smithi135 ceph-mon[36276]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-13T06:52:19.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:19 smithi135 ceph-mon[36276]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-13T06:52:19.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:19 smithi136 ceph-mon[37229]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-13T06:52:19.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:19 smithi026 ceph-mon[32041]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-13T06:52:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:20 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3335998512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47d5b8cf-ab62-44a6-b713-84d52661c442"}]: dispatch 2024-06-13T06:52:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:20 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3335998512' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47d5b8cf-ab62-44a6-b713-84d52661c442"}]': finished 2024-06-13T06:52:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:20 smithi135 ceph-mon[36276]: osdmap e64: 10 total, 9 up, 10 in 2024-06-13T06:52:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:20 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3335998512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47d5b8cf-ab62-44a6-b713-84d52661c442"}]: dispatch 2024-06-13T06:52:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:20 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3335998512' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47d5b8cf-ab62-44a6-b713-84d52661c442"}]': finished 2024-06-13T06:52:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:20 smithi136 ceph-mon[37229]: osdmap e64: 10 total, 9 up, 10 in 2024-06-13T06:52:20.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:20 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3335998512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47d5b8cf-ab62-44a6-b713-84d52661c442"}]: dispatch 2024-06-13T06:52:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:20 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3335998512' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47d5b8cf-ab62-44a6-b713-84d52661c442"}]': finished 2024-06-13T06:52:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:20 smithi026 ceph-mon[32041]: osdmap e64: 10 total, 9 up, 10 in 2024-06-13T06:52:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:21.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:21 smithi135 ceph-mon[36276]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-13T06:52:21.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:21 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/492929551' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:52:21.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:21 smithi136 ceph-mon[37229]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-13T06:52:21.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:21 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/492929551' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:52:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:21 smithi026 ceph-mon[32041]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-13T06:52:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:21 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/492929551' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:52:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:23 smithi135 ceph-mon[36276]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-13T06:52:23.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:23 smithi136 ceph-mon[37229]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-13T06:52:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:23 smithi026 ceph-mon[32041]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-13T06:52:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:25 smithi026 ceph-mon[32041]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:52:25.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:25 smithi135 ceph-mon[36276]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:52:25.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:25 smithi136 ceph-mon[37229]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:52:27.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:27 smithi026 ceph-mon[32041]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:27.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:27 smithi136 ceph-mon[37229]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:27 smithi135 ceph-mon[36276]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:29.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:29 smithi026 ceph-mon[32041]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:29.585 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:29 smithi135 ceph-mon[36276]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:29.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:29 smithi136 ceph-mon[37229]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-13T06:52:30.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:30.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:30 smithi136 ceph-mon[37229]: Deploying daemon osd.9 on smithi026 2024-06-13T06:52:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-13T06:52:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:30 smithi026 ceph-mon[32041]: Deploying daemon osd.9 on smithi026 2024-06-13T06:52:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:30 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-13T06:52:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:30 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:30 smithi135 ceph-mon[36276]: Deploying daemon osd.9 on smithi026 2024-06-13T06:52:31.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:31 smithi026 ceph-mon[32041]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:31.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:31 smithi136 ceph-mon[37229]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:31.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:31 smithi135 ceph-mon[36276]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:33.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:33 smithi026 ceph-mon[32041]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:33.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:33 smithi136 ceph-mon[37229]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:33.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:33 smithi135 ceph-mon[36276]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:35.496 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:35 smithi026 ceph-mon[32041]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:35.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:35 smithi136 ceph-mon[37229]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:35.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:35 smithi135 ceph-mon[36276]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:37.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:37 smithi026 ceph-mon[32041]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:37.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:37 smithi136 ceph-mon[37229]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:37.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:37 smithi135 ceph-mon[36276]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:39.745 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:39 smithi026 ceph-mon[32041]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:39.745 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:39.745 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:39.746 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:39 smithi135 ceph-mon[36276]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:39 smithi136 ceph-mon[37229]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:39.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:39.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:39.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:41.736 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:41 smithi026 ceph-mon[32041]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:41.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:41 smithi135 ceph-mon[36276]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:41 smithi136 ceph-mon[37229]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:42.632 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:42 smithi026 ceph-mon[32041]: from='osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:52:42.632 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:42 smithi026 ceph-mon[32041]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:52:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:42 smithi135 ceph-mon[36276]: from='osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:52:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:42 smithi135 ceph-mon[36276]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:52:42.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:42 smithi136 ceph-mon[37229]: from='osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:52:42.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:42 smithi136 ceph-mon[37229]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:52:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:43 smithi135 ceph-mon[36276]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:43 smithi135 ceph-mon[36276]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-13T06:52:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:43 smithi135 ceph-mon[36276]: from='osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:52:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:43 smithi135 ceph-mon[36276]: osdmap e65: 10 total, 9 up, 10 in 2024-06-13T06:52:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:43 smithi135 ceph-mon[36276]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:52:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:43 smithi136 ceph-mon[37229]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:43 smithi136 ceph-mon[37229]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-13T06:52:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:43 smithi136 ceph-mon[37229]: from='osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:52:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:43 smithi136 ceph-mon[37229]: osdmap e65: 10 total, 9 up, 10 in 2024-06-13T06:52:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:43 smithi136 ceph-mon[37229]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:52:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:43.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:43.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:43 smithi026 ceph-mon[32041]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:43.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:43 smithi026 ceph-mon[32041]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-13T06:52:43.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:43 smithi026 ceph-mon[32041]: from='osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:52:43.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:43 smithi026 ceph-mon[32041]: osdmap e65: 10 total, 9 up, 10 in 2024-06-13T06:52:43.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:43.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:43 smithi026 ceph-mon[32041]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-06-13T06:52:43.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:43.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:44.249 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 9 on host 'smithi026' 2024-06-13T06:52:44.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.248+0000 7f9c13e00700 1 -- 172.21.15.26:0/2156855735 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9c1c0628d0 con 0x7f9c000a2c20 2024-06-13T06:52:44.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c000a2c20 msgr2=0x7f9c000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c000a2c20 0x7f9c000a50e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009750 tx=0x7f9c0c009310 comp rx=0 tx=0).stop 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c1c103740 msgr2=0x7f9c1c074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c1c103740 0x7f9c1c074880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800e9a0 tx=0x7f9c0800ecb0 comp rx=0 tx=0).stop 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 shutdown_connections 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c000a2c20 0x7f9c000a50e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c1040e0 0x7f9c1c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c1c102540 0x7f9c1c076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 --2- 172.21.15.26:0/2156855735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c1c103740 0x7f9c1c074880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 >> 172.21.15.26:0/2156855735 conn(0x7f9c1c0fdae0 msgr2=0x7f9c1c106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 shutdown_connections 2024-06-13T06:52:44.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:52:44.252+0000 7f9c22600700 1 -- 172.21.15.26:0/2156855735 wait complete. 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: osdmap e66: 10 total, 9 up, 10 in 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:44.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:44.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:52:44.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: osdmap e66: 10 total, 9 up, 10 in 2024-06-13T06:52:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:45.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-06-13T06:52:45.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: osdmap e66: 10 total, 9 up, 10 in 2024-06-13T06:52:45.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:45.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:45.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:45.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:45.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:45.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:45.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:45.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:45.375 DEBUG:teuthology.orchestra.run.smithi026:osd.9> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.9.service 2024-06-13T06:52:45.381 INFO:tasks.cephadm:Deploying osd.10 on smithi135 with /dev/vg_nvme/lv_1... 2024-06-13T06:52:45.381 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-13T06:52:45.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:45 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:52:45.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:45 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:52:45.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:45 smithi026 ceph-mon[32041]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:45.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:45 smithi026 ceph-mon[32041]: osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] boot 2024-06-13T06:52:45.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:45 smithi026 ceph-mon[32041]: osdmap e67: 10 total, 10 up, 10 in 2024-06-13T06:52:45.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:45.626 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:52:45.748 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:45 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:52:45.748 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:45 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:52:45.748 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:45 smithi135 ceph-mon[36276]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:45.748 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:45 smithi135 ceph-mon[36276]: osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] boot 2024-06-13T06:52:45.748 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:45 smithi135 ceph-mon[36276]: osdmap e67: 10 total, 10 up, 10 in 2024-06-13T06:52:45.748 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:45 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:52:45.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:45 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:52:45.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:45 smithi136 ceph-mon[37229]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:52:45.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:45 smithi136 ceph-mon[37229]: osd.9 [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] boot 2024-06-13T06:52:45.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:45 smithi136 ceph-mon[37229]: osdmap e67: 10 total, 10 up, 10 in 2024-06-13T06:52:45.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:52:47.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:47 smithi026 ceph-mon[32041]: osdmap e68: 10 total, 10 up, 10 in 2024-06-13T06:52:47.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:47 smithi026 ceph-mon[32041]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:47.287 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:47 smithi135 ceph-mon[36276]: osdmap e68: 10 total, 10 up, 10 in 2024-06-13T06:52:47.287 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:47 smithi135 ceph-mon[36276]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:47.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:47 smithi136 ceph-mon[37229]: osdmap e68: 10 total, 10 up, 10 in 2024-06-13T06:52:47.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:47 smithi136 ceph-mon[37229]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-13T06:52:47.753 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-13T06:52:47.753 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-13T06:52:47.753 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-13T06:52:47.753 INFO:teuthology.orchestra.run.smithi135.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124202 s, 844 MB/s 2024-06-13T06:52:47.753 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 2024-06-13T06:52:47.753 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-13T06:52:48.429 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_1 2024-06-13T06:52:48.669 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:52:49.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:49 smithi135 ceph-mon[36276]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:49.566 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:49 smithi026 ceph-mon[32041]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:49.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:49 smithi136 ceph-mon[37229]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:50.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 -- 172.21.15.135:0/1915537942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 msgr2=0x7fd9a4107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:50.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 --2- 172.21.15.135:0/1915537942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 0x7fd9a4107ca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd994009990 tx=0x7fd994009ca0 comp rx=0 tx=0).stop 2024-06-13T06:52:50.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 -- 172.21.15.135:0/1915537942 shutdown_connections 2024-06-13T06:52:50.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 --2- 172.21.15.135:0/1915537942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a41081e0 0x7fd9a410a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 --2- 172.21.15.135:0/1915537942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9a4102f80 0x7fd9a4105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 --2- 172.21.15.135:0/1915537942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 0x7fd9a4107ca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 -- 172.21.15.135:0/1915537942 >> 172.21.15.135:0/1915537942 conn(0x7fd9a40fc9b0 msgr2=0x7fd9a40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 -- 172.21.15.135:0/1915537942 shutdown_connections 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.376+0000 7fd9ac686700 1 -- 172.21.15.135:0/1915537942 wait complete. 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9ac686700 1 Processor -- start 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9ac686700 1 -- start start 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9ac686700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9a4102f80 0x7fd9a4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9ac686700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 0x7fd9a4198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9ac686700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a41081e0 0x7fd9a419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:50.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9ac686700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a410e1c0 con 0x7fd9a41081e0 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9ac686700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a410e040 con 0x7fd9a41058b0 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9ac686700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a410e340 con 0x7fd9a4102f80 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 0x7fd9a4198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 0x7fd9a4198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:43382/0 (socket says 172.21.15.135:43382) 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 -- 172.21.15.135:0/613466609 learned_addr learned my addr 172.21.15.135:0/613466609 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9aa422700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9a4102f80 0x7fd9a4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 -- 172.21.15.135:0/613466609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9a4102f80 msgr2=0x7fd9a4198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9a4102f80 0x7fd9a4198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 -- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a41081e0 msgr2=0x7fd9a419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a41081e0 0x7fd9a419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 -- 172.21.15.135:0/613466609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a41a0470 con 0x7fd9a41058b0 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.377+0000 7fd9a9c21700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 0x7fd9a4198d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd994005910 tx=0x7fd9940047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.378+0000 7fd99b7fe700 1 -- 172.21.15.135:0/613466609 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994027070 con 0x7fd9a41058b0 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.378+0000 7fd99b7fe700 1 -- 172.21.15.135:0/613466609 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd994022470 con 0x7fd9a41058b0 2024-06-13T06:52:50.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.378+0000 7fd99b7fe700 1 -- 172.21.15.135:0/613466609 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99400f6a0 con 0x7fd9a41058b0 2024-06-13T06:52:50.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.378+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a41a0700 con 0x7fd9a41058b0 2024-06-13T06:52:50.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.378+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41a0c30 con 0x7fd9a41058b0 2024-06-13T06:52:50.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.378+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a4069180 con 0x7fd9a41058b0 2024-06-13T06:52:50.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.380+0000 7fd99b7fe700 1 -- 172.21.15.135:0/613466609 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd99402a070 con 0x7fd9a41058b0 2024-06-13T06:52:50.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.380+0000 7fd99b7fe700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9900a2ca0 0x7fd9900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:50.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.380+0000 7fd99b7fe700 1 -- 172.21.15.135:0/613466609 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7fd9940cf950 con 0x7fd9a41058b0 2024-06-13T06:52:50.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.381+0000 7fd9aa422700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9900a2ca0 0x7fd9900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:50.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.381+0000 7fd9aa422700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9900a2ca0 0x7fd9900a5160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:50.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.384+0000 7fd99b7fe700 1 -- 172.21.15.135:0/613466609 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd994013070 con 0x7fd9a41058b0 2024-06-13T06:52:50.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:52:50.587+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd9a40628d0 con 0x7fd9900a2ca0 2024-06-13T06:52:51.474 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi026 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.475 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi026 to 4273M 2024-06-13T06:52:51.476 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.476 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:51.476 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:51.476 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:52:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:52:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi026 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi026 to 4273M 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:51.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:52:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:52:51.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi026 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi026 to 4273M 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:51.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:51.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:52:53.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:53 smithi136 ceph-mon[37229]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:53.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:53 smithi026 ceph-mon[32041]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:53.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:53 smithi135 ceph-mon[36276]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:54 smithi136 ceph-mon[37229]: from='client.? 172.21.15.135:0/1373005002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]: dispatch 2024-06-13T06:52:54.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:54 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]: dispatch 2024-06-13T06:52:54.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:54 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]': finished 2024-06-13T06:52:54.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:54 smithi136 ceph-mon[37229]: osdmap e69: 11 total, 10 up, 11 in 2024-06-13T06:52:54.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:52:54.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:54 smithi136 ceph-mon[37229]: from='client.? 172.21.15.135:0/2549247172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:52:54.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:54 smithi026 ceph-mon[32041]: from='client.? 172.21.15.135:0/1373005002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]: dispatch 2024-06-13T06:52:54.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:54 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]: dispatch 2024-06-13T06:52:54.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:54 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]': finished 2024-06-13T06:52:54.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:54 smithi026 ceph-mon[32041]: osdmap e69: 11 total, 10 up, 11 in 2024-06-13T06:52:54.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:52:54.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:54 smithi026 ceph-mon[32041]: from='client.? 172.21.15.135:0/2549247172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:52:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:54 smithi135 ceph-mon[36276]: from='client.? 172.21.15.135:0/1373005002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]: dispatch 2024-06-13T06:52:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:54 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]: dispatch 2024-06-13T06:52:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:54 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aed06586-2329-4eb7-8d42-e8c06e20313f"}]': finished 2024-06-13T06:52:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:54 smithi135 ceph-mon[36276]: osdmap e69: 11 total, 10 up, 11 in 2024-06-13T06:52:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:52:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:54 smithi135 ceph-mon[36276]: from='client.? 172.21.15.135:0/2549247172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:52:55.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:55 smithi026 ceph-mon[32041]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:55.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:55 smithi135 ceph-mon[36276]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:55.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:55 smithi136 ceph-mon[37229]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:57.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:57 smithi026 ceph-mon[32041]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:57.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:57 smithi135 ceph-mon[36276]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:57 smithi136 ceph-mon[37229]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:52:59 smithi026 ceph-mon[32041]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:59.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:52:59 smithi135 ceph-mon[36276]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:52:59.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:52:59 smithi136 ceph-mon[37229]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:01.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:01 smithi026 ceph-mon[32041]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:01.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-13T06:53:01.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:01.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:01 smithi026 ceph-mon[32041]: Deploying daemon osd.10 on smithi135 2024-06-13T06:53:01.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:01 smithi135 ceph-mon[36276]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:01.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-13T06:53:01.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:01.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:01 smithi135 ceph-mon[36276]: Deploying daemon osd.10 on smithi135 2024-06-13T06:53:01.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:01 smithi136 ceph-mon[37229]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:01.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-13T06:53:01.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:01.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:01 smithi136 ceph-mon[37229]: Deploying daemon osd.10 on smithi135 2024-06-13T06:53:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:03 smithi026 ceph-mon[32041]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:03.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:03 smithi135 ceph-mon[36276]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:03.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:03 smithi136 ceph-mon[37229]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:05.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:05 smithi026 ceph-mon[32041]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:05.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:05 smithi135 ceph-mon[36276]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:05.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:05 smithi136 ceph-mon[37229]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:07 smithi135 ceph-mon[36276]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:07 smithi136 ceph-mon[37229]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:08.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:07 smithi026 ceph-mon[32041]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:09 smithi026 ceph-mon[32041]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:09 smithi135 ceph-mon[36276]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:09 smithi136 ceph-mon[37229]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:10.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.429+0000 7fd99b7fe700 1 -- 172.21.15.135:0/613466609 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd9a40628d0 con 0x7fd9900a2ca0 2024-06-13T06:53:10.429 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 10 on host 'smithi135' 2024-06-13T06:53:10.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9900a2ca0 msgr2=0x7fd9900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:10.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9900a2ca0 0x7fd9900a5160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).stop 2024-06-13T06:53:10.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 msgr2=0x7fd9a4198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:10.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 0x7fd9a4198d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd994005910 tx=0x7fd9940047b0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 shutdown_connections 2024-06-13T06:53:10.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9900a2ca0 0x7fd9900a5160 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9a41081e0 0x7fd9a419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9a4102f80 0x7fd9a4198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 --2- 172.21.15.135:0/613466609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9a41058b0 0x7fd9a4198d50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 >> 172.21.15.135:0/613466609 conn(0x7fd9a40fc9b0 msgr2=0x7fd9a4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:10.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 shutdown_connections 2024-06-13T06:53:10.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-13T06:53:10.432+0000 7fd9ac686700 1 -- 172.21.15.135:0/613466609 wait complete. 2024-06-13T06:53:11.082 DEBUG:teuthology.orchestra.run.smithi135:osd.10> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.10.service 2024-06-13T06:53:11.084 INFO:tasks.cephadm:Deploying osd.11 on smithi136 with /dev/vg_nvme/lv_1... 2024-06-13T06:53:11.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-13T06:53:11.330 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:53:11.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:11 smithi136 ceph-mon[37229]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:11.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:11 smithi136 ceph-mon[37229]: from='osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:53:11.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:11 smithi136 ceph-mon[37229]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:53:11.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.641 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:11 smithi135 ceph-mon[36276]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:11.641 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.641 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.641 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:11 smithi135 ceph-mon[36276]: from='osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:53:11.641 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:11 smithi135 ceph-mon[36276]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:53:11.641 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.641 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:11 smithi026 ceph-mon[32041]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:11 smithi026 ceph-mon[32041]: from='osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:53:11.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:11 smithi026 ceph-mon[32041]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:53:11.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:11.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:12.722 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:12 smithi135 ceph-mon[36276]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-13T06:53:12.723 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:12 smithi135 ceph-mon[36276]: from='osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:53:12.723 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:12 smithi135 ceph-mon[36276]: osdmap e70: 11 total, 10 up, 11 in 2024-06-13T06:53:12.723 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:12.723 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:12 smithi135 ceph-mon[36276]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:53:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:12 smithi026 ceph-mon[32041]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-13T06:53:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:12 smithi026 ceph-mon[32041]: from='osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:53:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:12 smithi026 ceph-mon[32041]: osdmap e70: 11 total, 10 up, 11 in 2024-06-13T06:53:12.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:12.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:12 smithi026 ceph-mon[32041]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:53:12.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:12 smithi136 ceph-mon[37229]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-13T06:53:12.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:12 smithi136 ceph-mon[37229]: from='osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:53:12.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:12 smithi136 ceph-mon[37229]: osdmap e70: 11 total, 10 up, 11 in 2024-06-13T06:53:12.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:12.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:12 smithi136 ceph-mon[37229]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-13T06:53:13.400 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 06:53:13 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T06:53:13.126+0000 7f0b3c875700 -1 osd.10 0 waiting for initial osdmap 2024-06-13T06:53:13.400 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 06:53:13 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T06:53:13.138+0000 7f0b34660700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:53:13.517 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-13T06:53:13.517 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-13T06:53:13.517 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-06-13T06:53:13.517 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-06-13T06:53:13.517 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00975576 s, 1.1 GB/s 2024-06-13T06:53:13.517 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-06-13T06:53:13.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:13 smithi136 ceph-mon[37229]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:13.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:13 smithi136 ceph-mon[37229]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:53:13.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:13 smithi136 ceph-mon[37229]: osdmap e71: 11 total, 10 up, 11 in 2024-06-13T06:53:13.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:13.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:13.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:13 smithi136 ceph-mon[37229]: from='osd.10 ' entity='osd.10' 2024-06-13T06:53:13.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:13 smithi135 ceph-mon[36276]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:13.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:13 smithi135 ceph-mon[36276]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:53:13.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:13 smithi135 ceph-mon[36276]: osdmap e71: 11 total, 10 up, 11 in 2024-06-13T06:53:13.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:13.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:13.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:13 smithi135 ceph-mon[36276]: from='osd.10 ' entity='osd.10' 2024-06-13T06:53:13.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:13 smithi026 ceph-mon[32041]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:53:13.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:13 smithi026 ceph-mon[32041]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-13T06:53:13.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:13 smithi026 ceph-mon[32041]: osdmap e71: 11 total, 10 up, 11 in 2024-06-13T06:53:13.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:13.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:13.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:13 smithi026 ceph-mon[32041]: from='osd.10 ' entity='osd.10' 2024-06-13T06:53:14.232 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-06-13T06:53:14.470 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:53:14.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:14 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:53:14.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:14 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:53:14.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:14 smithi136 ceph-mon[37229]: osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] boot 2024-06-13T06:53:14.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:14 smithi136 ceph-mon[37229]: osdmap e72: 11 total, 11 up, 11 in 2024-06-13T06:53:14.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:14.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:14 smithi136 ceph-mon[37229]: osdmap e73: 11 total, 11 up, 11 in 2024-06-13T06:53:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:14 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:53:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:14 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:53:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:14 smithi026 ceph-mon[32041]: osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] boot 2024-06-13T06:53:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:14 smithi026 ceph-mon[32041]: osdmap e72: 11 total, 11 up, 11 in 2024-06-13T06:53:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:14 smithi026 ceph-mon[32041]: osdmap e73: 11 total, 11 up, 11 in 2024-06-13T06:53:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:14 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:53:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:14 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:53:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:14 smithi135 ceph-mon[36276]: osd.10 [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] boot 2024-06-13T06:53:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:14 smithi135 ceph-mon[36276]: osdmap e72: 11 total, 11 up, 11 in 2024-06-13T06:53:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:53:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:14 smithi135 ceph-mon[36276]: osdmap e73: 11 total, 11 up, 11 in 2024-06-13T06:53:15.687 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 571 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi135 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:15.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:15.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:15.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:15.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:15 smithi136 ceph-mon[37229]: osdmap e74: 11 total, 11 up, 11 in 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 571 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi135 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-13T06:53:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:15.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:15.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:15.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:15.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:15.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:15 smithi026 ceph-mon[32041]: osdmap e74: 11 total, 11 up, 11 in 2024-06-13T06:53:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 571 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi135 2024-06-13T06:53:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:15.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:15 smithi135 ceph-mon[36276]: osdmap e74: 11 total, 11 up, 11 in 2024-06-13T06:53:16.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 -- 172.21.15.136:0/836463743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c1025f0 msgr2=0x7f772c102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:16.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 --2- 172.21.15.136:0/836463743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c1025f0 0x7f772c102a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f771c009990 tx=0x7f771c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 -- 172.21.15.136:0/836463743 shutdown_connections 2024-06-13T06:53:16.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 --2- 172.21.15.136:0/836463743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f772c1013f0 0x7f772c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 --2- 172.21.15.136:0/836463743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f772c102f90 0x7f772c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 --2- 172.21.15.136:0/836463743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c1025f0 0x7f772c102a50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 -- 172.21.15.136:0/836463743 >> 172.21.15.136:0/836463743 conn(0x7f772c0fc990 msgr2=0x7f772c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 -- 172.21.15.136:0/836463743 shutdown_connections 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.227+0000 7f77323c5700 1 -- 172.21.15.136:0/836463743 wait complete. 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f77323c5700 1 Processor -- start 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f77323c5700 1 -- start start 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f77323c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f772c1013f0 0x7f772c06acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f77323c5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f772c1025f0 0x7f772c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f77323c5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c102f90 0x7f772c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f77323c5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f772c10d0d0 con 0x7f772c1013f0 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f77323c5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f772c10cf50 con 0x7f772c1025f0 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f77323c5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f772c10d250 con 0x7f772c102f90 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f7730962700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c102f90 0x7f772c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f7730962700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c102f90 0x7f772c1a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:51162/0 (socket says 172.21.15.136:51162) 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f7730962700 1 -- 172.21.15.136:0/501503089 learned_addr learned my addr 172.21.15.136:0/501503089 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f772b7fe700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f772c1025f0 0x7f772c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f7730962700 1 -- 172.21.15.136:0/501503089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f772c1025f0 msgr2=0x7f772c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f7730962700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f772c1025f0 0x7f772c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f7730962700 1 -- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f772c1013f0 msgr2=0x7f772c06acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f772bfff700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f772c1013f0 0x7f772c06acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f7730962700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f772c1013f0 0x7f772c06acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.228+0000 7f7730962700 1 -- 172.21.15.136:0/501503089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c1a2740 con 0x7f772c102f90 2024-06-13T06:53:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.229+0000 7f7730962700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c102f90 0x7f772c1a1f60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f772400b970 tx=0x7f772400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.229+0000 7f77297fa700 1 -- 172.21.15.136:0/501503089 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724019070 con 0x7f772c102f90 2024-06-13T06:53:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.229+0000 7f77297fa700 1 -- 172.21.15.136:0/501503089 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f772400ce10 con 0x7f772c102f90 2024-06-13T06:53:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.229+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c1a2a30 con 0x7f772c102f90 2024-06-13T06:53:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.229+0000 7f77297fa700 1 -- 172.21.15.136:0/501503089 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724007370 con 0x7f772c102f90 2024-06-13T06:53:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.229+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c1a2fa0 con 0x7f772c102f90 2024-06-13T06:53:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.230+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772c069180 con 0x7f772c102f90 2024-06-13T06:53:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.231+0000 7f77297fa700 1 -- 172.21.15.136:0/501503089 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f772401c070 con 0x7f772c102f90 2024-06-13T06:53:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.232+0000 7f77297fa700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77180a2c20 0x7f77180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.232+0000 7f77297fa700 1 -- 172.21.15.136:0/501503089 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f77240cddc0 con 0x7f772c102f90 2024-06-13T06:53:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.232+0000 7f772bfff700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77180a2c20 0x7f77180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.233+0000 7f772bfff700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77180a2c20 0x7f77180a50e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7714006fd0 tx=0x7f7714008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:16.236 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.236+0000 7f77297fa700 1 -- 172.21.15.136:0/501503089 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f772409b370 con 0x7f772c102f90 2024-06-13T06:53:16.442 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:16.442+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f772c0628d0 con 0x7f77180a2c20 2024-06-13T06:53:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:16 smithi026 ceph-mon[32041]: osdmap e75: 11 total, 11 up, 11 in 2024-06-13T06:53:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:53:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:53:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:16 smithi135 ceph-mon[36276]: osdmap e75: 11 total, 11 up, 11 in 2024-06-13T06:53:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:53:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:53:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:17.310 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:16 smithi136 ceph-mon[37229]: osdmap e75: 11 total, 11 up, 11 in 2024-06-13T06:53:17.310 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:53:17.310 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:53:17.310 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:18.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:17 smithi136 ceph-mon[37229]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 572 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:18.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:17 smithi136 ceph-mon[37229]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:53:18.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:17 smithi136 ceph-mon[37229]: osdmap e76: 11 total, 11 up, 11 in 2024-06-13T06:53:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:17 smithi026 ceph-mon[32041]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 572 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:17 smithi026 ceph-mon[32041]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:53:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:17 smithi026 ceph-mon[32041]: osdmap e76: 11 total, 11 up, 11 in 2024-06-13T06:53:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:17 smithi135 ceph-mon[36276]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 572 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:17 smithi135 ceph-mon[36276]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:53:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:17 smithi135 ceph-mon[36276]: osdmap e76: 11 total, 11 up, 11 in 2024-06-13T06:53:20.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:19 smithi026 ceph-mon[32041]: pgmap v271: 1 pgs: 1 remapped+peering; 577 KiB data, 572 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:20.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:19 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/3351390890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]: dispatch 2024-06-13T06:53:20.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:19 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]: dispatch 2024-06-13T06:53:20.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:19 smithi026 ceph-mon[32041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]': finished 2024-06-13T06:53:20.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:19 smithi026 ceph-mon[32041]: osdmap e77: 12 total, 11 up, 12 in 2024-06-13T06:53:20.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:20.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:19 smithi026 ceph-mon[32041]: from='client.? 172.21.15.136:0/2331687589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:53:20.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:19 smithi135 ceph-mon[36276]: pgmap v271: 1 pgs: 1 remapped+peering; 577 KiB data, 572 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:19 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/3351390890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]: dispatch 2024-06-13T06:53:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:19 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]: dispatch 2024-06-13T06:53:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:19 smithi135 ceph-mon[36276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]': finished 2024-06-13T06:53:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:19 smithi135 ceph-mon[36276]: osdmap e77: 12 total, 11 up, 12 in 2024-06-13T06:53:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:19 smithi135 ceph-mon[36276]: from='client.? 172.21.15.136:0/2331687589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:53:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:19 smithi136 ceph-mon[37229]: pgmap v271: 1 pgs: 1 remapped+peering; 577 KiB data, 572 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:19 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/3351390890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]: dispatch 2024-06-13T06:53:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:19 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]: dispatch 2024-06-13T06:53:20.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:19 smithi136 ceph-mon[37229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca783667-d631-4e49-bf3a-5baf754f95fa"}]': finished 2024-06-13T06:53:20.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:19 smithi136 ceph-mon[37229]: osdmap e77: 12 total, 11 up, 12 in 2024-06-13T06:53:20.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:20.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:19 smithi136 ceph-mon[37229]: from='client.? 172.21.15.136:0/2331687589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:53:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:21 smithi026 ceph-mon[32041]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-13T06:53:22.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:21 smithi135 ceph-mon[36276]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-13T06:53:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:21 smithi136 ceph-mon[37229]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-13T06:53:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:23 smithi026 ceph-mon[32041]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-06-13T06:53:24.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:23 smithi135 ceph-mon[36276]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-06-13T06:53:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:23 smithi136 ceph-mon[37229]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-06-13T06:53:26.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:25 smithi026 ceph-mon[32041]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:53:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:25 smithi135 ceph-mon[36276]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:53:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:25 smithi136 ceph-mon[37229]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:53:27.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:27 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-13T06:53:27.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:27 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:27.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:27 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-13T06:53:27.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:27 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:27.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:27 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-13T06:53:27.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:27 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:28 smithi136 ceph-mon[37229]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-13T06:53:28.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:28 smithi136 ceph-mon[37229]: Deploying daemon osd.11 on smithi136 2024-06-13T06:53:28.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:28 smithi135 ceph-mon[36276]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-13T06:53:28.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:28 smithi135 ceph-mon[36276]: Deploying daemon osd.11 on smithi136 2024-06-13T06:53:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:28 smithi026 ceph-mon[32041]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-13T06:53:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:28 smithi026 ceph-mon[32041]: Deploying daemon osd.11 on smithi136 2024-06-13T06:53:30.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:30 smithi136 ceph-mon[37229]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:53:30.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:30 smithi135 ceph-mon[36276]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:53:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:30 smithi026 ceph-mon[32041]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-13T06:53:32.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:32 smithi135 ceph-mon[36276]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-13T06:53:32.336 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:32 smithi136 ceph-mon[37229]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-13T06:53:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:32 smithi026 ceph-mon[32041]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-13T06:53:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:33 smithi136 ceph-mon[37229]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:33.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:33 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:33.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:33 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:33.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:33 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:33 smithi026 ceph-mon[32041]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:33 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:33 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:33 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:33 smithi135 ceph-mon[36276]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:33 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:33 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:33 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:35.855 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:35 smithi136 ceph-mon[37229]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:36.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:35 smithi026 ceph-mon[32041]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:35 smithi135 ceph-mon[36276]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:36.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.631+0000 7f77297fa700 1 -- 172.21.15.136:0/501503089 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f772c0628d0 con 0x7f77180a2c20 2024-06-13T06:53:36.631 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 11 on host 'smithi136' 2024-06-13T06:53:36.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.633+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77180a2c20 msgr2=0x7f77180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:36.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.633+0000 7f77323c5700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77180a2c20 0x7f77180a50e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7714006fd0 tx=0x7f7714008040 comp rx=0 tx=0).stop 2024-06-13T06:53:36.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.633+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c102f90 msgr2=0x7f772c1a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:36.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.633+0000 7f77323c5700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c102f90 0x7f772c1a1f60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f772400b970 tx=0x7f772400bc80 comp rx=0 tx=0).stop 2024-06-13T06:53:36.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.634+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 shutdown_connections 2024-06-13T06:53:36.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.634+0000 7f77323c5700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77180a2c20 0x7f77180a50e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.634+0000 7f77323c5700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f772c102f90 0x7f772c1a1f60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.634+0000 7f77323c5700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f772c1013f0 0x7f772c06acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.634+0000 7f77323c5700 1 --2- 172.21.15.136:0/501503089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f772c1025f0 0x7f772c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.634+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 >> 172.21.15.136:0/501503089 conn(0x7f772c0fc990 msgr2=0x7f772c105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:36.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.634+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 shutdown_connections 2024-06-13T06:53:36.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:36.634+0000 7f77323c5700 1 -- 172.21.15.136:0/501503089 wait complete. 2024-06-13T06:53:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:36.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:53:36.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:53:36.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:36.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:36.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:37.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:37.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:37.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:53:37.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:53:37.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:37.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:37.296 DEBUG:teuthology.orchestra.run.smithi136:osd.11> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.11.service 2024-06-13T06:53:37.298 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-13T06:53:37.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd stat -f json 2024-06-13T06:53:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:37 smithi136 ceph-mon[37229]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:37 smithi136 ceph-mon[37229]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-13T06:53:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:37 smithi136 ceph-mon[37229]: from='osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:53:37.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:37 smithi136 ceph-mon[37229]: osdmap e78: 12 total, 11 up, 12 in 2024-06-13T06:53:37.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:37.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:37 smithi136 ceph-mon[37229]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:53:38.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:37 smithi026 ceph-mon[32041]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:38.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:37 smithi026 ceph-mon[32041]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-13T06:53:38.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:37 smithi026 ceph-mon[32041]: from='osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:53:38.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:37 smithi026 ceph-mon[32041]: osdmap e78: 12 total, 11 up, 12 in 2024-06-13T06:53:38.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:38.124 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:37 smithi026 ceph-mon[32041]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:53:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:37 smithi135 ceph-mon[36276]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:37 smithi135 ceph-mon[36276]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-13T06:53:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:37 smithi135 ceph-mon[36276]: from='osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:53:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:37 smithi135 ceph-mon[36276]: osdmap e78: 12 total, 11 up, 12 in 2024-06-13T06:53:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:37 smithi135 ceph-mon[36276]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-13T06:53:38.595 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 06:53:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T06:53:38.336+0000 7f4fc42b5700 -1 osd.11 0 waiting for initial osdmap 2024-06-13T06:53:38.595 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 06:53:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T06:53:38.351+0000 7f4fbc0a0700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:53:38.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:38 smithi136 ceph-mon[37229]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:53:38.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:38 smithi136 ceph-mon[37229]: osdmap e79: 12 total, 11 up, 12 in 2024-06-13T06:53:38.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:38 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:38.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:38 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:38.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:38 smithi136 ceph-mon[37229]: from='osd.11 ' entity='osd.11' 2024-06-13T06:53:39.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:38 smithi026 ceph-mon[32041]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:53:39.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:38 smithi026 ceph-mon[32041]: osdmap e79: 12 total, 11 up, 12 in 2024-06-13T06:53:39.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:38 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:39.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:38 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:39.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:38 smithi026 ceph-mon[32041]: from='osd.11 ' entity='osd.11' 2024-06-13T06:53:39.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:38 smithi135 ceph-mon[36276]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-13T06:53:39.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:38 smithi135 ceph-mon[36276]: osdmap e79: 12 total, 11 up, 12 in 2024-06-13T06:53:39.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:38 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:39.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:38 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:39.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:38 smithi135 ceph-mon[36276]: from='osd.11 ' entity='osd.11' 2024-06-13T06:53:39.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:39 smithi026 ceph-mon[32041]: purged_snaps scrub starts 2024-06-13T06:53:39.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:39 smithi026 ceph-mon[32041]: purged_snaps scrub ok 2024-06-13T06:53:39.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:39 smithi026 ceph-mon[32041]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:39.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:39.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:39 smithi026 ceph-mon[32041]: osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] boot 2024-06-13T06:53:39.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:39 smithi026 ceph-mon[32041]: osdmap e80: 12 total, 12 up, 12 in 2024-06-13T06:53:39.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:40.119 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:39 smithi136 ceph-mon[37229]: purged_snaps scrub starts 2024-06-13T06:53:40.120 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:39 smithi136 ceph-mon[37229]: purged_snaps scrub ok 2024-06-13T06:53:40.120 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:39 smithi136 ceph-mon[37229]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:40.120 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:40.120 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:39 smithi136 ceph-mon[37229]: osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] boot 2024-06-13T06:53:40.120 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:39 smithi136 ceph-mon[37229]: osdmap e80: 12 total, 12 up, 12 in 2024-06-13T06:53:40.120 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 -- 172.21.15.26:0/317315818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 msgr2=0x7f2b841030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 --2- 172.21.15.26:0/317315818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 0x7f2b841030d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009990 tx=0x7f2b74009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 -- 172.21.15.26:0/317315818 shutdown_connections 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 --2- 172.21.15.26:0/317315818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b84105fe0 0x7f2b84106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 --2- 172.21.15.26:0/317315818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 0x7f2b841030d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 --2- 172.21.15.26:0/317315818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b84103610 0x7f2b84105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 -- 172.21.15.26:0/317315818 >> 172.21.15.26:0/317315818 conn(0x7f2b840fc8c0 msgr2=0x7f2b840fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 -- 172.21.15.26:0/317315818 shutdown_connections 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.227+0000 7f2b83200700 1 -- 172.21.15.26:0/317315818 wait complete. 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.228+0000 7f2b83200700 1 Processor -- start 2024-06-13T06:53:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.228+0000 7f2b83200700 1 -- start start 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b83200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 0x7f2b8419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b83200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b84103610 0x7f2b8419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b83200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b84105fe0 0x7f2b841a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b83200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841145b0 con 0x7f2b84100ca0 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b83200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2b84114430 con 0x7f2b84105fe0 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b83200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b84114730 con 0x7f2b84103610 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b81e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 0x7f2b8419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b79400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b84103610 0x7f2b8419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b81e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 0x7f2b8419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59634/0 (socket says 172.21.15.26:59634) 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b81e00700 1 -- 172.21.15.26:0/208354501 learned_addr learned my addr 172.21.15.26:0/208354501 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b82800700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b84105fe0 0x7f2b841a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b79400700 1 -- 172.21.15.26:0/208354501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b84105fe0 msgr2=0x7f2b841a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b79400700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b84105fe0 0x7f2b841a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b79400700 1 -- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 msgr2=0x7f2b8419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.229+0000 7f2b79400700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 0x7f2b8419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.230+0000 7f2b79400700 1 -- 172.21.15.26:0/208354501 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b841a6840 con 0x7f2b84103610 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.230+0000 7f2b79400700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b84103610 0x7f2b8419f170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00ea20 tx=0x7f2b6c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.230+0000 7f2b7b400700 1 -- 172.21.15.26:0/208354501 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00cbd0 con 0x7f2b84103610 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.230+0000 7f2b7b400700 1 -- 172.21.15.26:0/208354501 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c00cd70 con 0x7f2b84103610 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.230+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b841a6b30 con 0x7f2b84103610 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.230+0000 7f2b7b400700 1 -- 172.21.15.26:0/208354501 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c018660 con 0x7f2b84103610 2024-06-13T06:53:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.230+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b841a7070 con 0x7f2b84103610 2024-06-13T06:53:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.232+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b84069180 con 0x7f2b84103610 2024-06-13T06:53:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.233+0000 7f2b7b400700 1 -- 172.21.15.26:0/208354501 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2b6c018800 con 0x7f2b84103610 2024-06-13T06:53:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.235+0000 7f2b7b400700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b680a2c20 0x7f2b680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.235+0000 7f2b7b400700 1 -- 172.21.15.26:0/208354501 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f2b6c0cee60 con 0x7f2b84103610 2024-06-13T06:53:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.235+0000 7f2b81e00700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b680a2c20 0x7f2b680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.236+0000 7f2b81e00700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b680a2c20 0x7f2b680a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400b2e0 tx=0x7f2b7400b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.239+0000 7f2b7b400700 1 -- 172.21.15.26:0/208354501 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2b6c09c1c0 con 0x7f2b84103610 2024-06-13T06:53:40.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:39 smithi135 ceph-mon[36276]: purged_snaps scrub starts 2024-06-13T06:53:40.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:39 smithi135 ceph-mon[36276]: purged_snaps scrub ok 2024-06-13T06:53:40.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:39 smithi135 ceph-mon[36276]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:53:40.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:40.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:39 smithi135 ceph-mon[36276]: osd.11 [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] boot 2024-06-13T06:53:40.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:39 smithi135 ceph-mon[36276]: osdmap e80: 12 total, 12 up, 12 in 2024-06-13T06:53:40.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:53:40.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.471+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2b840630a0 con 0x7f2b84103610 2024-06-13T06:53:40.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.471+0000 7f2b7b400700 1 -- 172.21.15.26:0/208354501 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f2b6c09bac0 con 0x7f2b84103610 2024-06-13T06:53:40.471 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:53:40.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b680a2c20 msgr2=0x7f2b680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b680a2c20 0x7f2b680a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400b2e0 tx=0x7f2b7400b880 comp rx=0 tx=0).stop 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b84103610 msgr2=0x7f2b8419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b84103610 0x7f2b8419f170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00ea20 tx=0x7f2b6c00ed30 comp rx=0 tx=0).stop 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 shutdown_connections 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b680a2c20 0x7f2b680a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b84105fe0 0x7f2b841a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b84100ca0 0x7f2b8419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 --2- 172.21.15.26:0/208354501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b84103610 0x7f2b8419f170 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 >> 172.21.15.26:0/208354501 conn(0x7f2b840fc8c0 msgr2=0x7f2b84104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 shutdown_connections 2024-06-13T06:53:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:40.476+0000 7f2b83200700 1 -- 172.21.15.26:0/208354501 wait complete. 2024-06-13T06:53:41.431 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1718261619,"num_in_osds":12,"osd_in_since":1718261599,"num_remapped_pgs":0} 2024-06-13T06:53:41.432 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: osdmap e81: 12 total, 12 up, 12 in 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/208354501' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: Detected new or changed devices on smithi136 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi136 to 4272M 2024-06-13T06:53:41.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:41.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:41.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: osdmap e81: 12 total, 12 up, 12 in 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/208354501' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: Detected new or changed devices on smithi136 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi136 to 4272M 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:41.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: osdmap e81: 12 total, 12 up, 12 in 2024-06-13T06:53:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/208354501' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-13T06:53:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: Detected new or changed devices on smithi136 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi136 to 4272M 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:41.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:43.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:43 smithi136 ceph-mon[37229]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:43.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:43 smithi026 ceph-mon[32041]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:43 smithi135 ceph-mon[36276]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:44.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 -- 172.21.15.26:0/1299346210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac107840 msgr2=0x7f7fac10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:44.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 --2- 172.21.15.26:0/1299346210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac107840 0x7f7fac10f3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400b0d0 tx=0x7f7fa400b3e0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 -- 172.21.15.26:0/1299346210 shutdown_connections 2024-06-13T06:53:44.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 --2- 172.21.15.26:0/1299346210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac107840 0x7f7fac10f3c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 --2- 172.21.15.26:0/1299346210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fac102740 0x7f7fac102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 --2- 172.21.15.26:0/1299346210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fac1030f0 0x7f7fac107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 -- 172.21.15.26:0/1299346210 >> 172.21.15.26:0/1299346210 conn(0x7f7fac0795f0 msgr2=0x7f7fac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:44.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 -- 172.21.15.26:0/1299346210 shutdown_connections 2024-06-13T06:53:44.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.306+0000 7f7fb4600700 1 -- 172.21.15.26:0/1299346210 wait complete. 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.307+0000 7f7fb4600700 1 Processor -- start 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.307+0000 7f7fb4600700 1 -- start start 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb4600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac102740 0x7f7fac19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb4600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fac1030f0 0x7f7fac19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb4600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fac107840 0x7f7fac19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb4600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac1127b0 con 0x7f7fac102740 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb4600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac112630 con 0x7f7fac1030f0 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb4600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac112930 con 0x7f7fac107840 2024-06-13T06:53:44.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb2600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fac107840 0x7f7fac19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fac1030f0 0x7f7fac19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb1c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac102740 0x7f7fac19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb2600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fac107840 0x7f7fac19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38824/0 (socket says 172.21.15.26:38824) 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fac1030f0 0x7f7fac19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:60292/0 (socket says 172.21.15.26:60292) 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb2600700 1 -- 172.21.15.26:0/2397264713 learned_addr learned my addr 172.21.15.26:0/2397264713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb2600700 1 -- 172.21.15.26:0/2397264713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fac1030f0 msgr2=0x7f7fac19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb2600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fac1030f0 0x7f7fac19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb2600700 1 -- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac102740 msgr2=0x7f7fac19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb2600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac102740 0x7f7fac19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb2600700 1 -- 172.21.15.26:0/2397264713 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac19fd80 con 0x7f7fac107840 2024-06-13T06:53:44.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.308+0000 7f7fb1200700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fac1030f0 0x7f7fac19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:44.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.309+0000 7f7fb2600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fac107840 0x7f7fac19f640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d010 tx=0x7f7fa40079a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:44.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.309+0000 7f7faa000700 1 -- 172.21.15.26:0/2397264713 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4003c00 con 0x7f7fac107840 2024-06-13T06:53:44.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.309+0000 7f7faa000700 1 -- 172.21.15.26:0/2397264713 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa4022070 con 0x7f7fac107840 2024-06-13T06:53:44.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.309+0000 7f7faa000700 1 -- 172.21.15.26:0/2397264713 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa401b6c0 con 0x7f7fac107840 2024-06-13T06:53:44.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.309+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac1a0010 con 0x7f7fac107840 2024-06-13T06:53:44.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.309+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac109300 con 0x7f7fac107840 2024-06-13T06:53:44.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.310+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fac103bd0 con 0x7f7fac107840 2024-06-13T06:53:44.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.313+0000 7f7faa000700 1 -- 172.21.15.26:0/2397264713 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7fa4009ba0 con 0x7f7fac107840 2024-06-13T06:53:44.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.314+0000 7f7faa000700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7f940a2c20 0x7f7f940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:44.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.314+0000 7f7faa000700 1 -- 172.21.15.26:0/2397264713 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7fa40cfca0 con 0x7f7fac107840 2024-06-13T06:53:44.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.314+0000 7f7fb1c00700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7f940a2c20 0x7f7f940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:44.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.315+0000 7f7fb1c00700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7f940a2c20 0x7f7f940a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009c60 tx=0x7f7fa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:44.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.316+0000 7f7faa000700 1 -- 172.21.15.26:0/2397264713 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7fa401a3c0 con 0x7f7fac107840 2024-06-13T06:53:44.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.549+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fac19c0d0 con 0x7f7fac107840 2024-06-13T06:53:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.551+0000 7f7faa000700 1 -- 172.21.15.26:0/2397264713 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18457 (secure 0 0 0) 0x7f7fa409d090 con 0x7f7fac107840 2024-06-13T06:53:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:53:44.552 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:53:40.342959+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:53:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7f940a2c20 msgr2=0x7f7f940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7f940a2c20 0x7f7f940a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009c60 tx=0x7f7fa0009260 comp rx=0 tx=0).stop 2024-06-13T06:53:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fac107840 msgr2=0x7f7fac19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fac107840 0x7f7fac19f640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d010 tx=0x7f7fa40079a0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 shutdown_connections 2024-06-13T06:53:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7f940a2c20 0x7f7f940a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fac107840 0x7f7fac19f640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac102740 0x7f7fac19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 --2- 172.21.15.26:0/2397264713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fac1030f0 0x7f7fac19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.554+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 >> 172.21.15.26:0/2397264713 conn(0x7f7fac0795f0 msgr2=0x7f7fac100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.555+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 shutdown_connections 2024-06-13T06:53:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:44.555+0000 7f7fb4600700 1 -- 172.21.15.26:0/2397264713 wait complete. 2024-06-13T06:53:45.472 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-13T06:49:18.857067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-13T06:53:45.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-13T06:53:45.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:45 smithi136 ceph-mon[37229]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:45.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:45 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2397264713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:53:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:45 smithi026 ceph-mon[32041]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:45 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2397264713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:53:45.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:45 smithi135 ceph-mon[36276]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:45.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:45 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2397264713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:53:45.935 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:47.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:47 smithi136 ceph-mon[37229]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:47.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:47 smithi026 ceph-mon[32041]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:47.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:47 smithi135 ceph-mon[36276]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.266+0000 7fe3fe400700 1 -- 172.21.15.26:0/302499606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f8106af0 msgr2=0x7fe3f8100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.266+0000 7fe3fe400700 1 --2- 172.21.15.26:0/302499606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f8106af0 0x7fe3f8100cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009990 tx=0x7fe3e4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.266+0000 7fe3fe400700 1 -- 172.21.15.26:0/302499606 shutdown_connections 2024-06-13T06:53:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.266+0000 7fe3fe400700 1 --2- 172.21.15.26:0/302499606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f81012e0 0x7fe3f8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.266+0000 7fe3fe400700 1 --2- 172.21.15.26:0/302499606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f8106140 0x7fe3f8106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.266+0000 7fe3fe400700 1 --2- 172.21.15.26:0/302499606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f8106af0 0x7fe3f8100cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.266+0000 7fe3fe400700 1 -- 172.21.15.26:0/302499606 >> 172.21.15.26:0/302499606 conn(0x7fe3f8076640 msgr2=0x7fe3f8076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.266+0000 7fe3fe400700 1 -- 172.21.15.26:0/302499606 shutdown_connections 2024-06-13T06:53:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.267+0000 7fe3fe400700 1 -- 172.21.15.26:0/302499606 wait complete. 2024-06-13T06:53:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.268+0000 7fe3fe400700 1 Processor -- start 2024-06-13T06:53:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.268+0000 7fe3fe400700 1 -- start start 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fe400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f81012e0 0x7fe3f819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fe400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f8106140 0x7fe3f819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fe400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f8106af0 0x7fe3f81a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fe400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8118e40 con 0x7fe3f8106140 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fe400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8118cc0 con 0x7fe3f8106af0 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fe400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8118fc0 con 0x7fe3f81012e0 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3f7e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f8106140 0x7fe3f819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fd000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f81012e0 0x7fe3f819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fd000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f81012e0 0x7fe3f819f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38840/0 (socket says 172.21.15.26:38840) 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fd000700 1 -- 172.21.15.26:0/4183616682 learned_addr learned my addr 172.21.15.26:0/4183616682 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:53:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fda00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f8106af0 0x7fe3f81a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fd000700 1 -- 172.21.15.26:0/4183616682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f8106af0 msgr2=0x7fe3f81a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fd000700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f8106af0 0x7fe3f81a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fd000700 1 -- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f8106140 msgr2=0x7fe3f819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fd000700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f8106140 0x7fe3f819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.269+0000 7fe3fd000700 1 -- 172.21.15.26:0/4183616682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f81a42c0 con 0x7fe3f81012e0 2024-06-13T06:53:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.270+0000 7fe3fd000700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f81012e0 0x7fe3f819f2b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00e970 tx=0x7fe3ec00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.270+0000 7fe3f5600700 1 -- 172.21.15.26:0/4183616682 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec004220 con 0x7fe3f81012e0 2024-06-13T06:53:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.270+0000 7fe3f5600700 1 -- 172.21.15.26:0/4183616682 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3ec0043c0 con 0x7fe3f81012e0 2024-06-13T06:53:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.270+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f81a45b0 con 0x7fe3f81012e0 2024-06-13T06:53:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.270+0000 7fe3f5600700 1 -- 172.21.15.26:0/4183616682 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec003dc0 con 0x7fe3f81012e0 2024-06-13T06:53:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.271+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f81abe30 con 0x7fe3f81012e0 2024-06-13T06:53:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.272+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f8069180 con 0x7fe3f81012e0 2024-06-13T06:53:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.274+0000 7fe3f5600700 1 -- 172.21.15.26:0/4183616682 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe3ec043ae0 con 0x7fe3f81012e0 2024-06-13T06:53:48.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.275+0000 7fe3f5600700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3e00a2ca0 0x7fe3e00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:48.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.275+0000 7fe3f5600700 1 -- 172.21.15.26:0/4183616682 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe3ec0cecb0 con 0x7fe3f81012e0 2024-06-13T06:53:48.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.275+0000 7fe3f7e00700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3e00a2ca0 0x7fe3e00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:48.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.276+0000 7fe3f7e00700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3e00a2ca0 0x7fe3e00a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400b2e0 tx=0x7fe3e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:48.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.278+0000 7fe3f5600700 1 -- 172.21.15.26:0/4183616682 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe3ec09c010 con 0x7fe3f81012e0 2024-06-13T06:53:48.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.515+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe3f81ac000 con 0x7fe3f81012e0 2024-06-13T06:53:48.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.516+0000 7fe3f5600700 1 -- 172.21.15.26:0/4183616682 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fe3ec09b910 con 0x7fe3f81012e0 2024-06-13T06:53:48.516 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-06-13T06:53:48.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3e00a2ca0 msgr2=0x7fe3e00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:48.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3e00a2ca0 0x7fe3e00a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400b2e0 tx=0x7fe3e4005960 comp rx=0 tx=0).stop 2024-06-13T06:53:48.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f81012e0 msgr2=0x7fe3f819f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:48.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f81012e0 0x7fe3f819f2b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00e970 tx=0x7fe3ec00ec80 comp rx=0 tx=0).stop 2024-06-13T06:53:48.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 shutdown_connections 2024-06-13T06:53:48.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3e00a2ca0 0x7fe3e00a5160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f8106af0 0x7fe3f81a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f81012e0 0x7fe3f819f2b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 --2- 172.21.15.26:0/4183616682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3f8106140 0x7fe3f819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:48.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 >> 172.21.15.26:0/4183616682 conn(0x7fe3f8076640 msgr2=0x7fe3f8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:48.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.520+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 shutdown_connections 2024-06-13T06:53:48.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:48.521+0000 7fe3fe400700 1 -- 172.21.15.26:0/4183616682 wait complete. 2024-06-13T06:53:49.584 INFO:tasks.cephadm:Adding mds.a on smithi026 2024-06-13T06:53:49.584 INFO:tasks.cephadm:Adding mds.d on smithi026 2024-06-13T06:53:49.584 INFO:tasks.cephadm:Adding mds.g on smithi026 2024-06-13T06:53:49.584 INFO:tasks.cephadm:Adding mds.j on smithi026 2024-06-13T06:53:49.585 INFO:tasks.cephadm:Adding mds.b on smithi135 2024-06-13T06:53:49.585 INFO:tasks.cephadm:Adding mds.e on smithi135 2024-06-13T06:53:49.585 INFO:tasks.cephadm:Adding mds.h on smithi135 2024-06-13T06:53:49.585 INFO:tasks.cephadm:Adding mds.k on smithi135 2024-06-13T06:53:49.585 INFO:tasks.cephadm:Adding mds.c on smithi136 2024-06-13T06:53:49.585 INFO:tasks.cephadm:Adding mds.f on smithi136 2024-06-13T06:53:49.585 INFO:tasks.cephadm:Adding mds.i on smithi136 2024-06-13T06:53:49.585 INFO:tasks.cephadm:Adding mds.l on smithi136 2024-06-13T06:53:49.585 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph orch apply mds all '12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l' 2024-06-13T06:53:49.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:49 smithi136 ceph-mon[37229]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:49.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:49 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/4183616682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:53:49.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:49 smithi026 ceph-mon[32041]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:49.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:49 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/4183616682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:53:49.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:49 smithi135 ceph-mon[36276]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:49.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:49 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/4183616682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:53:49.833 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:53:51.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:51 smithi136 ceph-mon[37229]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 -- 172.21.15.136:0/1698345220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86081081e0 msgr2=0x7f860810a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 --2- 172.21.15.136:0/1698345220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86081081e0 0x7f860810a650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f860000b0a0 tx=0x7f860000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 -- 172.21.15.136:0/1698345220 shutdown_connections 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 --2- 172.21.15.136:0/1698345220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86081081e0 0x7f860810a650 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 --2- 172.21.15.136:0/1698345220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8608102f80 0x7f8608105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 --2- 172.21.15.136:0/1698345220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86081058b0 0x7f8608107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 -- 172.21.15.136:0/1698345220 >> 172.21.15.136:0/1698345220 conn(0x7f86080fc990 msgr2=0x7f86080fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 -- 172.21.15.136:0/1698345220 shutdown_connections 2024-06-13T06:53:51.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.602+0000 7f860c9be700 1 -- 172.21.15.136:0/1698345220 wait complete. 2024-06-13T06:53:51.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860c9be700 1 Processor -- start 2024-06-13T06:53:51.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860c9be700 1 -- start start 2024-06-13T06:53:51.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860c9be700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8608102f80 0x7f86081987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:51.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860c9be700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86081058b0 0x7f8608198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:51.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860c9be700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86081081e0 0x7f860819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860c9be700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f860810e1a0 con 0x7f86081081e0 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860c9be700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f860810e020 con 0x7f86081058b0 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860c9be700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f860810e320 con 0x7f8608102f80 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860659c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8608102f80 0x7f86081987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860659c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8608102f80 0x7f86081987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:45418/0 (socket says 172.21.15.136:45418) 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860659c700 1 -- 172.21.15.136:0/3377608330 learned_addr learned my addr 172.21.15.136:0/3377608330 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f85fdd9b700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86081058b0 0x7f8608198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860659c700 1 -- 172.21.15.136:0/3377608330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86081058b0 msgr2=0x7f8608198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860659c700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86081058b0 0x7f8608198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860659c700 1 -- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86081081e0 msgr2=0x7f860819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860659c700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86081081e0 0x7f860819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f860659c700 1 -- 172.21.15.136:0/3377608330 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86081a0440 con 0x7f8608102f80 2024-06-13T06:53:51.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.603+0000 7f85fdd9b700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86081058b0 0x7f8608198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:53:51.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.604+0000 7f860659c700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8608102f80 0x7f86081987e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b560 tx=0x7f85f800b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:51.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.604+0000 7f85fffff700 1 -- 172.21.15.136:0/3377608330 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8015020 con 0x7f8608102f80 2024-06-13T06:53:51.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.604+0000 7f85fffff700 1 -- 172.21.15.136:0/3377608330 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f8011d10 con 0x7f8608102f80 2024-06-13T06:53:51.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.604+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86081a0730 con 0x7f8608102f80 2024-06-13T06:53:51.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.604+0000 7f85fffff700 1 -- 172.21.15.136:0/3377608330 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f800f2e0 con 0x7f8608102f80 2024-06-13T06:53:51.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.604+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a0cf0 con 0x7f8608102f80 2024-06-13T06:53:51.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.605+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8608069180 con 0x7f8608102f80 2024-06-13T06:53:51.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.606+0000 7f85fffff700 1 -- 172.21.15.136:0/3377608330 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f85f8018020 con 0x7f8608102f80 2024-06-13T06:53:51.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.606+0000 7f85fffff700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85f40a2c20 0x7f85f40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:51.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.607+0000 7f85fffff700 1 -- 172.21.15.136:0/3377608330 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f85f80ce3b0 con 0x7f8608102f80 2024-06-13T06:53:51.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.607+0000 7f85fdd9b700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85f40a2c20 0x7f85f40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:51.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.608+0000 7f85fdd9b700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85f40a2c20 0x7f85f40a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f85f0007900 tx=0x7f85f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:51.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.611+0000 7f85fffff700 1 -- 172.21.15.136:0/3377608330 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f85f801f020 con 0x7f8608102f80 2024-06-13T06:53:51.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:51 smithi026 ceph-mon[32041]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:51.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:51 smithi135 ceph-mon[36276]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:51.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.816+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}) -- 0x7f8608062ad0 con 0x7f85f40a2c20 2024-06-13T06:53:51.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.834+0000 7f85fffff700 1 -- 172.21.15.136:0/3377608330 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8608062ad0 con 0x7f85f40a2c20 2024-06-13T06:53:51.833 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mds.all update... 2024-06-13T06:53:51.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85f40a2c20 msgr2=0x7f85f40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85f40a2c20 0x7f85f40a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f85f0007900 tx=0x7f85f0008040 comp rx=0 tx=0).stop 2024-06-13T06:53:51.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8608102f80 msgr2=0x7f86081987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8608102f80 0x7f86081987e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b560 tx=0x7f85f800b920 comp rx=0 tx=0).stop 2024-06-13T06:53:51.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 shutdown_connections 2024-06-13T06:53:51.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85f40a2c20 0x7f85f40a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86081081e0 0x7f860819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8608102f80 0x7f86081987e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 --2- 172.21.15.136:0/3377608330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86081058b0 0x7f8608198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.836+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 >> 172.21.15.136:0/3377608330 conn(0x7f86080fc990 msgr2=0x7f86081038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:51.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.837+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 shutdown_connections 2024-06-13T06:53:51.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-13T06:53:51.837+0000 7f860c9be700 1 -- 172.21.15.136:0/3377608330 wait complete. 2024-06-13T06:53:52.604 DEBUG:teuthology.orchestra.run.smithi026:mds.a> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.a.service 2024-06-13T06:53:52.609 DEBUG:teuthology.orchestra.run.smithi026:mds.d> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.d.service 2024-06-13T06:53:52.655 DEBUG:teuthology.orchestra.run.smithi026:mds.g> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.g.service 2024-06-13T06:53:52.702 DEBUG:teuthology.orchestra.run.smithi026:mds.j> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.j.service 2024-06-13T06:53:52.747 DEBUG:teuthology.orchestra.run.smithi135:mds.b> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.b.service 2024-06-13T06:53:52.749 DEBUG:teuthology.orchestra.run.smithi135:mds.e> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.e.service 2024-06-13T06:53:52.792 DEBUG:teuthology.orchestra.run.smithi135:mds.h> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.h.service 2024-06-13T06:53:52.836 DEBUG:teuthology.orchestra.run.smithi135:mds.k> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.k.service 2024-06-13T06:53:52.879 DEBUG:teuthology.orchestra.run.smithi136:mds.c> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.c.service 2024-06-13T06:53:52.881 DEBUG:teuthology.orchestra.run.smithi136:mds.f> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.f.service 2024-06-13T06:53:52.924 DEBUG:teuthology.orchestra.run.smithi136:mds.i> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.i.service 2024-06-13T06:53:52.966 DEBUG:teuthology.orchestra.run.smithi136:mds.l> sudo journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.l.service 2024-06-13T06:53:53.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:52 smithi026 ceph-mon[32041]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:53:53.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:52 smithi026 ceph-mon[32041]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-06-13T06:53:53.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:52 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:53.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:52 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:53.009 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-13T06:53:53.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:53:53.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:52 smithi135 ceph-mon[36276]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:53:53.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:52 smithi135 ceph-mon[36276]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-06-13T06:53:53.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:52 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:53.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:52 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:53.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:52 smithi136 ceph-mon[37229]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:53:53.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:52 smithi136 ceph-mon[37229]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-06-13T06:53:53.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:52 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:53.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:52 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:53.466 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:53 smithi026 ceph-mon[32041]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:53:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:53:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:54.123 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:53 smithi136 ceph-mon[37229]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:54.123 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:54.123 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:54.123 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:54.123 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:53:54.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:53:54.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:53 smithi135 ceph-mon[36276]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:53:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:53:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:55.100 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:54 smithi136 ceph-mon[37229]: Deploying daemon mds.l on smithi136 2024-06-13T06:53:55.173 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:54 smithi026 ceph-mon[32041]: Deploying daemon mds.l on smithi136 2024-06-13T06:53:55.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:54 smithi135 ceph-mon[36276]: Deploying daemon mds.l on smithi136 2024-06-13T06:53:55.376 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 systemd[1]: Starting Ceph mds.l for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:53:55.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.720+0000 7fc414200700 1 -- 172.21.15.26:0/1686319265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40c1046b0 msgr2=0x7fc40c104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:55.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.720+0000 7fc414200700 1 --2- 172.21.15.26:0/1686319265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40c1046b0 0x7fc40c104b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80098f0 tx=0x7fc3f8009c00 comp rx=0 tx=0).stop 2024-06-13T06:53:55.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.721+0000 7fc414200700 1 -- 172.21.15.26:0/1686319265 shutdown_connections 2024-06-13T06:53:55.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.721+0000 7fc414200700 1 --2- 172.21.15.26:0/1686319265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c105070 0x7fc40c111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.721+0000 7fc414200700 1 --2- 172.21.15.26:0/1686319265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc40c10a6b0 0x7fc40c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.721+0000 7fc414200700 1 --2- 172.21.15.26:0/1686319265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40c1046b0 0x7fc40c104b30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.721+0000 7fc414200700 1 -- 172.21.15.26:0/1686319265 >> 172.21.15.26:0/1686319265 conn(0x7fc40c1001a0 msgr2=0x7fc40c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:55.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.721+0000 7fc414200700 1 -- 172.21.15.26:0/1686319265 shutdown_connections 2024-06-13T06:53:55.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.721+0000 7fc414200700 1 -- 172.21.15.26:0/1686319265 wait complete. 2024-06-13T06:53:55.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.722+0000 7fc414200700 1 Processor -- start 2024-06-13T06:53:55.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.722+0000 7fc414200700 1 -- start start 2024-06-13T06:53:55.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc414200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc40c1046b0 0x7fc40c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc414200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40c105070 0x7fc40c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc414200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c10a6b0 0x7fc40c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc414200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c1149e0 con 0x7fc40c10a6b0 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc414200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c114860 con 0x7fc40c105070 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc414200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c114b60 con 0x7fc40c1046b0 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc411800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc40c1046b0 0x7fc40c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc411800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc40c1046b0 0x7fc40c19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55814/0 (socket says 172.21.15.26:55814) 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc411800700 1 -- 172.21.15.26:0/2143110083 learned_addr learned my addr 172.21.15.26:0/2143110083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc410e00700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40c105070 0x7fc40c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc412200700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c10a6b0 0x7fc40c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc411800700 1 -- 172.21.15.26:0/2143110083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40c105070 msgr2=0x7fc40c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc411800700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40c105070 0x7fc40c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc411800700 1 -- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c10a6b0 msgr2=0x7fc40c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc411800700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c10a6b0 0x7fc40c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.723+0000 7fc411800700 1 -- 172.21.15.26:0/2143110083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c19fe80 con 0x7fc40c1046b0 2024-06-13T06:53:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.724+0000 7fc411800700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc40c1046b0 0x7fc40c19ae70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc40000e9a0 tx=0x7fc40000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.724+0000 7fc40a000700 1 -- 172.21.15.26:0/2143110083 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40000ca10 con 0x7fc40c1046b0 2024-06-13T06:53:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.724+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c1a0170 con 0x7fc40c1046b0 2024-06-13T06:53:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.724+0000 7fc40a000700 1 -- 172.21.15.26:0/2143110083 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc40000cbb0 con 0x7fc40c1046b0 2024-06-13T06:53:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.724+0000 7fc40a000700 1 -- 172.21.15.26:0/2143110083 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400018680 con 0x7fc40c1046b0 2024-06-13T06:53:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.724+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c1a79f0 con 0x7fc40c1046b0 2024-06-13T06:53:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.726+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c069180 con 0x7fc40c1046b0 2024-06-13T06:53:55.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.728+0000 7fc40a000700 1 -- 172.21.15.26:0/2143110083 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc400010430 con 0x7fc40c1046b0 2024-06-13T06:53:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.729+0000 7fc40a000700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f40a2ca0 0x7fc3f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.729+0000 7fc40a000700 1 -- 172.21.15.26:0/2143110083 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4000cf100 con 0x7fc40c1046b0 2024-06-13T06:53:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.729+0000 7fc410e00700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f40a2ca0 0x7fc3f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.730+0000 7fc410e00700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f40a2ca0 0x7fc3f40a5160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80098c0 tx=0x7fc3f8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:55.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:55.731+0000 7fc40a000700 1 -- 172.21.15.26:0/2143110083 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc40009c460 con 0x7fc40c1046b0 2024-06-13T06:53:55.776 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 podman[67609]: 2024-06-13 06:53:55.412461721 +0000 UTC m=+0.018036719 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:53:55.776 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 podman[67609]: 2024-06-13 06:53:55.533283301 +0000 UTC m=+0.138858329 container create f687d66e3dc9527555db7e91e063559ebf69dfa15a6b41c04970da68a43f564e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-13T06:53:55.776 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 podman[67609]: 2024-06-13 06:53:55.777550003 +0000 UTC m=+0.383124989 container init f687d66e3dc9527555db7e91e063559ebf69dfa15a6b41c04970da68a43f564e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-13T06:53:56.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.015+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc40c02dca0 con 0x7fc40c1046b0 2024-06-13T06:53:56.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.016+0000 7fc40a000700 1 -- 172.21.15.26:0/2143110083 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fc40009bd60 con 0x7fc40c1046b0 2024-06-13T06:53:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:53:56.015 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-13T06:53:56.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f40a2ca0 msgr2=0x7fc3f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:56.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f40a2ca0 0x7fc3f40a5160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80098c0 tx=0x7fc3f8019040 comp rx=0 tx=0).stop 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc40c1046b0 msgr2=0x7fc40c19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc40c1046b0 0x7fc40c19ae70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc40000e9a0 tx=0x7fc40000ecb0 comp rx=0 tx=0).stop 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 shutdown_connections 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc3f40a2ca0 0x7fc3f40a5160 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c10a6b0 0x7fc40c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc40c1046b0 0x7fc40c19ae70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 --2- 172.21.15.26:0/2143110083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40c105070 0x7fc40c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 >> 172.21.15.26:0/2143110083 conn(0x7fc40c1001a0 msgr2=0x7fc40c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:56.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 shutdown_connections 2024-06-13T06:53:56.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:53:56.019+0000 7fc414200700 1 -- 172.21.15.26:0/2143110083 wait complete. 2024-06-13T06:53:56.020 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2024-06-13T06:53:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:55 smithi136 ceph-mon[37229]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:55 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6832/2047884215,v1:172.21.15.136:6833/2047884215] up:boot 2024-06-13T06:53:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:55 smithi136 ceph-mon[37229]: fsmap 1 up:standby 2024-06-13T06:53:56.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:53:56.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:53:56.127 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 podman[67609]: 2024-06-13 06:53:55.780316904 +0000 UTC m=+0.385891891 container start f687d66e3dc9527555db7e91e063559ebf69dfa15a6b41c04970da68a43f564e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-13T06:53:56.127 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 ceph-mds[67628]: -- 172.21.15.136:0/1909952343 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640f1119a40 con 0x5640f10ed000 2024-06-13T06:53:56.127 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l[67624]: starting mds.l at 2024-06-13T06:53:56.127 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 bash[67609]: f687d66e3dc9527555db7e91e063559ebf69dfa15a6b41c04970da68a43f564e 2024-06-13T06:53:56.127 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 06:53:55 smithi136 systemd[1]: Started Ceph mds.l for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:53:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:55 smithi026 ceph-mon[32041]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:55 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6832/2047884215,v1:172.21.15.136:6833/2047884215] up:boot 2024-06-13T06:53:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:55 smithi026 ceph-mon[32041]: fsmap 1 up:standby 2024-06-13T06:53:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:53:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:53:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:55 smithi135 ceph-mon[36276]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:55 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6832/2047884215,v1:172.21.15.136:6833/2047884215] up:boot 2024-06-13T06:53:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:55 smithi135 ceph-mon[36276]: fsmap 1 up:standby 2024-06-13T06:53:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:53:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:53:57.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:53:57.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:53:57.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:57.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:56 smithi026 ceph-mon[32041]: Deploying daemon mds.k on smithi135 2024-06-13T06:53:57.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:56 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2143110083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:53:57.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:53:57.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:53:57.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:57.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:56 smithi135 ceph-mon[36276]: Deploying daemon mds.k on smithi135 2024-06-13T06:53:57.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:56 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2143110083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:53:57.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:53:57.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:53:57.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:53:57.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:57.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:56 smithi136 ceph-mon[37229]: Deploying daemon mds.k on smithi135 2024-06-13T06:53:57.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:56 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2143110083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:53:58.124 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:57 smithi135 ceph-mon[36276]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:57 smithi026 ceph-mon[32041]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:57 smithi136 ceph-mon[37229]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:53:58.509 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 systemd[1]: Starting Ceph mds.k for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:53:58.774 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 podman[65608]: 2024-06-13 06:53:58.413004281 +0000 UTC m=+0.016911661 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:53:58.774 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 podman[65608]: 2024-06-13 06:53:58.531476965 +0000 UTC m=+0.135384330 container create 2bd02c9f1caa7660cabadd093600c4ef2a116e5a0e85c104b61e6801e531c86f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-13T06:53:58.774 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 podman[65608]: 2024-06-13 06:53:58.77568422 +0000 UTC m=+0.379591593 container init 2bd02c9f1caa7660cabadd093600c4ef2a116e5a0e85c104b61e6801e531c86f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-13T06:53:58.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:53:59.033 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 podman[65608]: 2024-06-13 06:53:58.779031954 +0000 UTC m=+0.382939320 container start 2bd02c9f1caa7660cabadd093600c4ef2a116e5a0e85c104b61e6801e531c86f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:53:59.033 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 ceph-mds[65627]: -- 172.21.15.135:0/2068095479 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612c6477a40 con 0x5612c644ac00 2024-06-13T06:53:59.033 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k[65623]: starting mds.k at 2024-06-13T06:53:59.033 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 bash[65608]: 2bd02c9f1caa7660cabadd093600c4ef2a116e5a0e85c104b61e6801e531c86f 2024-06-13T06:53:59.033 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 06:53:58 smithi135 systemd[1]: Started Ceph mds.k for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:53:59.313 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: Deploying daemon mds.j on smithi026 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.135:6832/2065578555,v1:172.21.15.135:6833/2065578555] up:boot 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: fsmap 2 up:standby 2024-06-13T06:54:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:53:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: Deploying daemon mds.j on smithi026 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.135:6832/2065578555,v1:172.21.15.135:6833/2065578555] up:boot 2024-06-13T06:54:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: fsmap 2 up:standby 2024-06-13T06:54:00.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:53:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-13T06:54:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: Deploying daemon mds.j on smithi026 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.135:6832/2065578555,v1:172.21.15.135:6833/2065578555] up:boot 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: fsmap 2 up:standby 2024-06-13T06:54:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:53:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-13T06:54:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 -- 172.21.15.26:0/3533030429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a0108ac0 msgr2=0x7f99a0108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 --2- 172.21.15.26:0/3533030429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a0108ac0 0x7f99a0108f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9988009990 tx=0x7f9988009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 -- 172.21.15.26:0/3533030429 shutdown_connections 2024-06-13T06:54:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 --2- 172.21.15.26:0/3533030429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0109480 0x7f99a0115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 --2- 172.21.15.26:0/3533030429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a010eac0 0x7f99a010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 --2- 172.21.15.26:0/3533030429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a0108ac0 0x7f99a0108f40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 -- 172.21.15.26:0/3533030429 >> 172.21.15.26:0/3533030429 conn(0x7f99a0075c60 msgr2=0x7f99a01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 -- 172.21.15.26:0/3533030429 shutdown_connections 2024-06-13T06:54:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.944+0000 7f99a6400700 1 -- 172.21.15.26:0/3533030429 wait complete. 2024-06-13T06:54:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.946+0000 7f99a6400700 1 Processor -- start 2024-06-13T06:54:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.946+0000 7f99a6400700 1 -- start start 2024-06-13T06:54:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f99a6400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a0108ac0 0x7f99a019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f99a6400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0109480 0x7f99a019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f99a6400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a010eac0 0x7f99a01a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f99a6400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0118e10 con 0x7f99a0108ac0 2024-06-13T06:54:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f99a6400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0118c90 con 0x7f99a010eac0 2024-06-13T06:54:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f99a6400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0118f90 con 0x7f99a0109480 2024-06-13T06:54:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f9996a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0109480 0x7f99a019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f9996a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0109480 0x7f99a019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55836/0 (socket says 172.21.15.26:55836) 2024-06-13T06:54:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f9996a00700 1 -- 172.21.15.26:0/3402814312 learned_addr learned my addr 172.21.15.26:0/3402814312 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f999fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a010eac0 0x7f99a01a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f999f400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a0108ac0 0x7f99a019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f999fe00700 1 -- 172.21.15.26:0/3402814312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0109480 msgr2=0x7f99a019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f999fe00700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0109480 0x7f99a019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f999fe00700 1 -- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a0108ac0 msgr2=0x7f99a019f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f999fe00700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a0108ac0 0x7f99a019f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.947+0000 7f999fe00700 1 -- 172.21.15.26:0/3402814312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a01a4230 con 0x7f99a010eac0 2024-06-13T06:54:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.948+0000 7f999fe00700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a010eac0 0x7f99a01a3af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f999800e970 tx=0x7f999800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.948+0000 7f999cc00700 1 -- 172.21.15.26:0/3402814312 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998004220 con 0x7f99a010eac0 2024-06-13T06:54:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.948+0000 7f999cc00700 1 -- 172.21.15.26:0/3402814312 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9998004d10 con 0x7f99a010eac0 2024-06-13T06:54:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.948+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a01a4520 con 0x7f99a010eac0 2024-06-13T06:54:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.948+0000 7f999cc00700 1 -- 172.21.15.26:0/3402814312 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998010700 con 0x7f99a010eac0 2024-06-13T06:54:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.948+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a01abda0 con 0x7f99a010eac0 2024-06-13T06:54:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.950+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a0069180 con 0x7f99a010eac0 2024-06-13T06:54:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.951+0000 7f999cc00700 1 -- 172.21.15.26:0/3402814312 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9998004750 con 0x7f99a010eac0 2024-06-13T06:54:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.952+0000 7f999cc00700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f99840a2ca0 0x7f99840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.953+0000 7f999cc00700 1 -- 172.21.15.26:0/3402814312 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f99980ce8c0 con 0x7f99a010eac0 2024-06-13T06:54:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.953+0000 7f999f400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f99840a2ca0 0x7f99840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.953+0000 7f999f400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f99840a2ca0 0x7f99840a5160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f999000f540 tx=0x7f999000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:01.956+0000 7f999cc00700 1 -- 172.21.15.26:0/3402814312 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f999809bc20 con 0x7f99a010eac0 2024-06-13T06:54:02.021 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:01 smithi026 ceph-mon[32041]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.237+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99a01a0580 con 0x7f99a010eac0 2024-06-13T06:54:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.238+0000 7f999cc00700 1 -- 172.21.15.26:0/3402814312 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f999809b520 con 0x7f99a010eac0 2024-06-13T06:54:02.238 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:54:02.238 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-13T06:54:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.242+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f99840a2ca0 msgr2=0x7f99840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f99840a2ca0 0x7f99840a5160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f999000f540 tx=0x7f999000b040 comp rx=0 tx=0).stop 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a010eac0 msgr2=0x7f99a01a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a010eac0 0x7f99a01a3af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f999800e970 tx=0x7f999800ec80 comp rx=0 tx=0).stop 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 shutdown_connections 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f99840a2ca0 0x7f99840a5160 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a010eac0 0x7f99a01a3af0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a0108ac0 0x7f99a019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 --2- 172.21.15.26:0/3402814312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99a0109480 0x7f99a019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 >> 172.21.15.26:0/3402814312 conn(0x7f99a0075c60 msgr2=0x7f99a0076180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 shutdown_connections 2024-06-13T06:54:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:02.243+0000 7f99a6400700 1 -- 172.21.15.26:0/3402814312 wait complete. 2024-06-13T06:54:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2024-06-13T06:54:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:01 smithi135 ceph-mon[36276]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:01 smithi136 ceph-mon[37229]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:03.230 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:02 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3402814312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:03.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:02 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3402814312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:03.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:02 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3402814312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:03 smithi026 ceph-mon[32041]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:03 smithi135 ceph-mon[36276]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:03 smithi136 ceph-mon[37229]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:04.752 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jun 13 06:54:04 smithi026 systemd[1]: Started Ceph mds.j for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:05.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: Deploying daemon mds.d on smithi026 2024-06-13T06:54:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.26:6834/153334101,v1:172.21.15.26:6835/153334101] up:boot 2024-06-13T06:54:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: fsmap 3 up:standby 2024-06-13T06:54:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: Deploying daemon mds.d on smithi026 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.26:6834/153334101,v1:172.21.15.26:6835/153334101] up:boot 2024-06-13T06:54:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: fsmap 3 up:standby 2024-06-13T06:54:06.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-13T06:54:06.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:06.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: Deploying daemon mds.d on smithi026 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.26:6834/153334101,v1:172.21.15.26:6835/153334101] up:boot 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: fsmap 3 up:standby 2024-06-13T06:54:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-13T06:54:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:07.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:07 smithi026 ceph-mon[32041]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:07 smithi135 ceph-mon[36276]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:08.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:07 smithi136 ceph-mon[37229]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.712+0000 7f97efa00700 1 -- 172.21.15.26:0/3870646156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 msgr2=0x7f97e806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.712+0000 7f97efa00700 1 --2- 172.21.15.26:0/3870646156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 0x7f97e806a1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f97d800b0a0 tx=0x7f97d800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.713+0000 7f97efa00700 1 -- 172.21.15.26:0/3870646156 shutdown_connections 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.713+0000 7f97efa00700 1 --2- 172.21.15.26:0/3870646156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 0x7f97e806a1a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.713+0000 7f97efa00700 1 --2- 172.21.15.26:0/3870646156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e8104ed0 0x7f97e8105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.713+0000 7f97efa00700 1 --2- 172.21.15.26:0/3870646156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97e806a6e0 0x7f97e806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.713+0000 7f97efa00700 1 -- 172.21.15.26:0/3870646156 >> 172.21.15.26:0/3870646156 conn(0x7f97e8078340 msgr2=0x7f97e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.713+0000 7f97efa00700 1 -- 172.21.15.26:0/3870646156 shutdown_connections 2024-06-13T06:54:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.713+0000 7f97efa00700 1 -- 172.21.15.26:0/3870646156 wait complete. 2024-06-13T06:54:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.714+0000 7f97efa00700 1 Processor -- start 2024-06-13T06:54:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.714+0000 7f97efa00700 1 -- start start 2024-06-13T06:54:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.714+0000 7f97efa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e806a6e0 0x7f97e819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.714+0000 7f97efa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97e8104ed0 0x7f97e819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.714+0000 7f97efa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 0x7f97e81a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.714+0000 7f97efa00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8118dc0 con 0x7f97e806a6e0 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.714+0000 7f97efa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8118c40 con 0x7f97e81073c0 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.714+0000 7f97efa00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8118f40 con 0x7f97e8104ed0 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97ed000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e806a6e0 0x7f97e819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97ed000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e806a6e0 0x7f97e819f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38540/0 (socket says 172.21.15.26:38540) 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97ed000700 1 -- 172.21.15.26:0/3211966179 learned_addr learned my addr 172.21.15.26:0/3211966179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97eda00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 0x7f97e81a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97dfe00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97e8104ed0 0x7f97e819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:09.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97ed000700 1 -- 172.21.15.26:0/3211966179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97e8104ed0 msgr2=0x7f97e819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:09.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97ed000700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97e8104ed0 0x7f97e819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:09.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97ed000700 1 -- 172.21.15.26:0/3211966179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 msgr2=0x7f97e81a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97ed000700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 0x7f97e81a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97ed000700 1 -- 172.21.15.26:0/3211966179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e81a4260 con 0x7f97e806a6e0 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.715+0000 7f97eda00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 0x7f97e81a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.716+0000 7f97ed000700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e806a6e0 0x7f97e819f250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97d800b070 tx=0x7f97d8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.716+0000 7f97e6000700 1 -- 172.21.15.26:0/3211966179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8021070 con 0x7f97e806a6e0 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.716+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e81a44f0 con 0x7f97e806a6e0 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.716+0000 7f97e6000700 1 -- 172.21.15.26:0/3211966179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d8003c00 con 0x7f97e806a6e0 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.716+0000 7f97e6000700 1 -- 172.21.15.26:0/3211966179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8004750 con 0x7f97e806a6e0 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.716+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e81abdd0 con 0x7f97e806a6e0 2024-06-13T06:54:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.717+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e8069180 con 0x7f97e806a6e0 2024-06-13T06:54:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.720+0000 7f97e6000700 1 -- 172.21.15.26:0/3211966179 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f97d8007980 con 0x7f97e806a6e0 2024-06-13T06:54:09.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.721+0000 7f97e6000700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97cc0a2c20 0x7f97cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:09.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.721+0000 7f97e6000700 1 -- 172.21.15.26:0/3211966179 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f97d800f020 con 0x7f97e806a6e0 2024-06-13T06:54:09.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.721+0000 7f97dfe00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97cc0a2c20 0x7f97cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:09.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.722+0000 7f97dfe00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97cc0a2c20 0x7f97cc0a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97e806a470 tx=0x7f97d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:09.726+0000 7f97e6000700 1 -- 172.21.15.26:0/3211966179 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f97d802a020 con 0x7f97e806a6e0 2024-06-13T06:54:09.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:09 smithi026 ceph-mon[32041]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:09.801 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 06:54:09 smithi026 podman[73189]: 2024-06-13 06:54:09.760682952 +0000 UTC m=+1.000679432 container start 7b56f2264536c6f3d099c6aaf6bcca20e7b2049226f691d528b04b96f0d3091f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-13T06:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.013+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97e81abfa0 con 0x7f97e806a6e0 2024-06-13T06:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.014+0000 7f97e6000700 1 -- 172.21.15.26:0/3211966179 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f97d809d490 con 0x7f97e806a6e0 2024-06-13T06:54:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:54:10.014 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-06-13T06:54:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.017+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97cc0a2c20 msgr2=0x7f97cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.017+0000 7f97efa00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97cc0a2c20 0x7f97cc0a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97e806a470 tx=0x7f97d4009260 comp rx=0 tx=0).stop 2024-06-13T06:54:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e806a6e0 msgr2=0x7f97e819f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e806a6e0 0x7f97e819f250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97d800b070 tx=0x7f97d8004c80 comp rx=0 tx=0).stop 2024-06-13T06:54:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 shutdown_connections 2024-06-13T06:54:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97cc0a2c20 0x7f97cc0a50e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97e81073c0 0x7f97e81a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e806a6e0 0x7f97e819f250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 --2- 172.21.15.26:0/3211966179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97e8104ed0 0x7f97e819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 >> 172.21.15.26:0/3211966179 conn(0x7f97e8078340 msgr2=0x7f97e8100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 shutdown_connections 2024-06-13T06:54:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:10.018+0000 7f97efa00700 1 -- 172.21.15.26:0/3211966179 wait complete. 2024-06-13T06:54:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 4 2024-06-13T06:54:10.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:09 smithi135 ceph-mon[36276]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:10.068 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 06:54:09 smithi026 ceph-mds[73238]: -- 172.21.15.26:0/1693037083 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556651a27a40 con 0x5566519fa800 2024-06-13T06:54:10.068 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 06:54:09 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-d[73212]: starting mds.d at 2024-06-13T06:54:10.068 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 06:54:09 smithi026 bash[73189]: 7b56f2264536c6f3d099c6aaf6bcca20e7b2049226f691d528b04b96f0d3091f 2024-06-13T06:54:10.068 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 06:54:09 smithi026 systemd[1]: Started Ceph mds.d for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:10.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:09 smithi136 ceph-mon[37229]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:10.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:10 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3211966179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:10.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:10.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:10.658 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:10.658 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:10.658 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:10.986 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:10 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3211966179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:10.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:10.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:10.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:10.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:10.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:10.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:11.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:11.029 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:10 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3211966179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:11.029 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:11.029 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:11.029 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:11.030 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:11.030 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:11.030 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:11 smithi026 ceph-mon[32041]: Deploying daemon mds.e on smithi135 2024-06-13T06:54:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:11 smithi026 ceph-mon[32041]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:11 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.26:6836/69365853,v1:172.21.15.26:6837/69365853] up:boot 2024-06-13T06:54:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:11 smithi026 ceph-mon[32041]: fsmap 4 up:standby 2024-06-13T06:54:12.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-13T06:54:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:11 smithi135 ceph-mon[36276]: Deploying daemon mds.e on smithi135 2024-06-13T06:54:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:11 smithi135 ceph-mon[36276]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:11 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.26:6836/69365853,v1:172.21.15.26:6837/69365853] up:boot 2024-06-13T06:54:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:11 smithi135 ceph-mon[36276]: fsmap 4 up:standby 2024-06-13T06:54:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-13T06:54:12.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:11 smithi136 ceph-mon[37229]: Deploying daemon mds.e on smithi135 2024-06-13T06:54:12.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:11 smithi136 ceph-mon[37229]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:12.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:11 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.26:6836/69365853,v1:172.21.15.26:6837/69365853] up:boot 2024-06-13T06:54:12.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:11 smithi136 ceph-mon[37229]: fsmap 4 up:standby 2024-06-13T06:54:12.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-13T06:54:12.446 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 systemd[1]: Starting Ceph mds.e for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:54:12.721 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 podman[65922]: 2024-06-13 06:54:12.350207701 +0000 UTC m=+0.017305400 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:54:12.721 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 podman[65922]: 2024-06-13 06:54:12.461450253 +0000 UTC m=+0.128547983 container create 35b31f9a25c97c33667d50fb342ce1d5f6ab344d4fcdcdbb6a6893ace5f2aa51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-13T06:54:12.721 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 podman[65922]: 2024-06-13 06:54:12.722456713 +0000 UTC m=+0.389554399 container init 35b31f9a25c97c33667d50fb342ce1d5f6ab344d4fcdcdbb6a6893ace5f2aa51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-13T06:54:12.845 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:13.033 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 podman[65922]: 2024-06-13 06:54:12.72537092 +0000 UTC m=+0.392468613 container start 35b31f9a25c97c33667d50fb342ce1d5f6ab344d4fcdcdbb6a6893ace5f2aa51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-06-13T06:54:13.033 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 ceph-mds[65941]: -- 172.21.15.135:0/558046939 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596f7279a40 con 0x5596f724d000 2024-06-13T06:54:13.033 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e[65937]: starting mds.e at 2024-06-13T06:54:13.033 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 bash[65922]: 35b31f9a25c97c33667d50fb342ce1d5f6ab344d4fcdcdbb6a6893ace5f2aa51 2024-06-13T06:54:13.033 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 06:54:12 smithi135 systemd[1]: Started Ceph mds.e for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:14.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: Deploying daemon mds.f on smithi136 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.135:6834/1212410310,v1:172.21.15.135:6835/1212410310] up:boot 2024-06-13T06:54:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: fsmap 5 up:standby 2024-06-13T06:54:14.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-13T06:54:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:14.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:14.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:14.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:14.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: Deploying daemon mds.f on smithi136 2024-06-13T06:54:14.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.135:6834/1212410310,v1:172.21.15.135:6835/1212410310] up:boot 2024-06-13T06:54:14.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: fsmap 5 up:standby 2024-06-13T06:54:14.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: Deploying daemon mds.f on smithi136 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.135:6834/1212410310,v1:172.21.15.135:6835/1212410310] up:boot 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: fsmap 5 up:standby 2024-06-13T06:54:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-13T06:54:15.312 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 systemd[1]: Starting Ceph mds.f for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:54:15.312 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 podman[67925]: 2024-06-13 06:54:15.21563743 +0000 UTC m=+0.017366453 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:54:15.572 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 podman[67925]: 2024-06-13 06:54:15.329821414 +0000 UTC m=+0.131550480 container create 39599f86b7a559e58f6ef46d5f71bf2f4e83cd20f7aa8940a2c3d1f44ef6a1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-13T06:54:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.859+0000 7fb660400700 1 -- 172.21.15.26:0/1419364697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658074fe0 msgr2=0x7fb65810e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.859+0000 7fb660400700 1 --2- 172.21.15.26:0/1419364697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658074fe0 0x7fb65810e950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb644009990 tx=0x7fb644009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.859+0000 7fb660400700 1 -- 172.21.15.26:0/1419364697 shutdown_connections 2024-06-13T06:54:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.859+0000 7fb660400700 1 --2- 172.21.15.26:0/1419364697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6580746c0 0x7fb658074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.859+0000 7fb660400700 1 --2- 172.21.15.26:0/1419364697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65810ee90 0x7fb658111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.859+0000 7fb660400700 1 --2- 172.21.15.26:0/1419364697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb658074fe0 0x7fb65810e950 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.859+0000 7fb660400700 1 -- 172.21.15.26:0/1419364697 >> 172.21.15.26:0/1419364697 conn(0x7fb6580795f0 msgr2=0x7fb658079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.859+0000 7fb660400700 1 -- 172.21.15.26:0/1419364697 shutdown_connections 2024-06-13T06:54:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.860+0000 7fb660400700 1 -- 172.21.15.26:0/1419364697 wait complete. 2024-06-13T06:54:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.861+0000 7fb660400700 1 Processor -- start 2024-06-13T06:54:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.861+0000 7fb660400700 1 -- start start 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.861+0000 7fb660400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6580746c0 0x7fb65819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.861+0000 7fb660400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb658074fe0 0x7fb65819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.861+0000 7fb660400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb65810ee90 0x7fb65819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.861+0000 7fb660400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581149c0 con 0x7fb6580746c0 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.861+0000 7fb660400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb658114840 con 0x7fb65810ee90 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.861+0000 7fb660400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb658114b40 con 0x7fb658074fe0 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb658074fe0 0x7fb65819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65da00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6580746c0 0x7fb65819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb658074fe0 0x7fb65819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44968/0 (socket says 172.21.15.26:44968) 2024-06-13T06:54:15.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65d000700 1 -- 172.21.15.26:0/1310966185 learned_addr learned my addr 172.21.15.26:0/1310966185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65e400700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb65810ee90 0x7fb65819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65d000700 1 -- 172.21.15.26:0/1310966185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb65810ee90 msgr2=0x7fb65819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65d000700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb65810ee90 0x7fb65819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65d000700 1 -- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6580746c0 msgr2=0x7fb65819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65d000700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6580746c0 0x7fb65819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65d000700 1 -- 172.21.15.26:0/1310966185 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65819fdb0 con 0x7fb658074fe0 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.862+0000 7fb65e400700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb65810ee90 0x7fb65819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.863+0000 7fb65d000700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb658074fe0 0x7fb65819b330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb644000c00 tx=0x7fb64400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.863+0000 7fb656000700 1 -- 172.21.15.26:0/1310966185 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644027070 con 0x7fb658074fe0 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.863+0000 7fb656000700 1 -- 172.21.15.26:0/1310966185 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb644003d70 con 0x7fb658074fe0 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.863+0000 7fb656000700 1 -- 172.21.15.26:0/1310966185 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64400f6c0 con 0x7fb658074fe0 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.863+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6581a0040 con 0x7fb658074fe0 2024-06-13T06:54:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.863+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a7920 con 0x7fb658074fe0 2024-06-13T06:54:15.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.864+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb658069180 con 0x7fb658074fe0 2024-06-13T06:54:15.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.866+0000 7fb656000700 1 -- 172.21.15.26:0/1310966185 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb64402a070 con 0x7fb658074fe0 2024-06-13T06:54:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.868+0000 7fb656000700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb6400a2ca0 0x7fb6400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.868+0000 7fb656000700 1 -- 172.21.15.26:0/1310966185 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb6440cfc90 con 0x7fb658074fe0 2024-06-13T06:54:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.868+0000 7fb65da00700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb6400a2ca0 0x7fb6400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.869+0000 7fb65da00700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb6400a2ca0 0x7fb6400a5160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00ac50 tx=0x7fb64c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:15.872+0000 7fb656000700 1 -- 172.21.15.26:0/1310966185 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb64405e850 con 0x7fb658074fe0 2024-06-13T06:54:15.876 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 podman[67925]: 2024-06-13 06:54:15.573957213 +0000 UTC m=+0.375686236 container init 39599f86b7a559e58f6ef46d5f71bf2f4e83cd20f7aa8940a2c3d1f44ef6a1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-13T06:54:15.876 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 podman[67925]: 2024-06-13 06:54:15.576518562 +0000 UTC m=+0.378247587 container start 39599f86b7a559e58f6ef46d5f71bf2f4e83cd20f7aa8940a2c3d1f44ef6a1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-13T06:54:15.876 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-mds[67943]: -- 172.21.15.136:0/2539638014 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564696caba40 con 0x564696c7ec00 2024-06-13T06:54:15.876 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f[67939]: starting mds.f at 2024-06-13T06:54:15.877 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 bash[67925]: 39599f86b7a559e58f6ef46d5f71bf2f4e83cd20f7aa8940a2c3d1f44ef6a1ac 2024-06-13T06:54:15.877 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 06:54:15 smithi136 systemd[1]: Started Ceph mds.f for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.154+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6581a7af0 con 0x7fb658074fe0 2024-06-13T06:54:16.154 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:15 smithi026 ceph-mon[32041]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:16.154 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.154 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.154 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.154 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:16.154 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:16.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.154+0000 7fb656000700 1 -- 172.21.15.26:0/1310966185 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7fb644013070 con 0x7fb658074fe0 2024-06-13T06:54:16.155 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:54:16.155 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-06-13T06:54:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.158+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb6400a2ca0 msgr2=0x7fb6400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.158+0000 7fb660400700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb6400a2ca0 0x7fb6400a5160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00ac50 tx=0x7fb64c00a380 comp rx=0 tx=0).stop 2024-06-13T06:54:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.158+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb658074fe0 msgr2=0x7fb65819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.158+0000 7fb660400700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb658074fe0 0x7fb65819b330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb644000c00 tx=0x7fb64400bab0 comp rx=0 tx=0).stop 2024-06-13T06:54:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.159+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 shutdown_connections 2024-06-13T06:54:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.159+0000 7fb660400700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb6400a2ca0 0x7fb6400a5160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.159+0000 7fb660400700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb65810ee90 0x7fb65819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.159+0000 7fb660400700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6580746c0 0x7fb65819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.159+0000 7fb660400700 1 --2- 172.21.15.26:0/1310966185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb658074fe0 0x7fb65819b330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.159+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 >> 172.21.15.26:0/1310966185 conn(0x7fb6580795f0 msgr2=0x7fb6580ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.159+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 shutdown_connections 2024-06-13T06:54:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:16.159+0000 7fb660400700 1 -- 172.21.15.26:0/1310966185 wait complete. 2024-06-13T06:54:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-06-13T06:54:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:15 smithi135 ceph-mon[36276]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-mon[37229]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:17.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:16 smithi136 ceph-mon[37229]: Deploying daemon mds.c on smithi136 2024-06-13T06:54:17.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:16 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6834/582335478,v1:172.21.15.136:6835/582335478] up:boot 2024-06-13T06:54:17.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:16 smithi136 ceph-mon[37229]: fsmap 6 up:standby 2024-06-13T06:54:17.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:16 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T06:54:17.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:16 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1310966185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:16 smithi026 ceph-mon[32041]: Deploying daemon mds.c on smithi136 2024-06-13T06:54:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:16 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6834/582335478,v1:172.21.15.136:6835/582335478] up:boot 2024-06-13T06:54:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:16 smithi026 ceph-mon[32041]: fsmap 6 up:standby 2024-06-13T06:54:17.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:16 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T06:54:17.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:16 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1310966185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:16 smithi135 ceph-mon[36276]: Deploying daemon mds.c on smithi136 2024-06-13T06:54:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:16 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6834/582335478,v1:172.21.15.136:6835/582335478] up:boot 2024-06-13T06:54:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:16 smithi135 ceph-mon[36276]: fsmap 6 up:standby 2024-06-13T06:54:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:16 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T06:54:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:16 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1310966185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:17 smithi026 ceph-mon[32041]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:18.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:17 smithi135 ceph-mon[36276]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:18.332 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:17 smithi136 ceph-mon[37229]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:18.332 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 systemd[1]: Starting Ceph mds.c for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:54:18.623 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 podman[68228]: 2024-06-13 06:54:18.235630428 +0000 UTC m=+0.016820319 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:54:18.623 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 podman[68228]: 2024-06-13 06:54:18.353169036 +0000 UTC m=+0.134358927 container create 0a3f5b1f765a2fdd694d5d3c21f0801443cb3954da3a94819d665ec16a41b9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True) 2024-06-13T06:54:18.623 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 podman[68228]: 2024-06-13 06:54:18.625056469 +0000 UTC m=+0.406246359 container init 0a3f5b1f765a2fdd694d5d3c21f0801443cb3954da3a94819d665ec16a41b9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5) 2024-06-13T06:54:18.876 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 podman[68228]: 2024-06-13 06:54:18.628009347 +0000 UTC m=+0.409199239 container start 0a3f5b1f765a2fdd694d5d3c21f0801443cb3954da3a94819d665ec16a41b9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:54:18.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 ceph-mds[68247]: -- 172.21.15.136:0/3526547906 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d044e5a40 con 0x556d044b8c00 2024-06-13T06:54:18.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c[68243]: starting mds.c at 2024-06-13T06:54:18.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 bash[68228]: 0a3f5b1f765a2fdd694d5d3c21f0801443cb3954da3a94819d665ec16a41b9d6 2024-06-13T06:54:18.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 06:54:18 smithi136 systemd[1]: Started Ceph mds.c for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:19.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:20.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: Deploying daemon mds.h on smithi135 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6836/1512917649,v1:172.21.15.136:6837/1512917649] up:boot 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: fsmap 7 up:standby 2024-06-13T06:54:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:19 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T06:54:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: Deploying daemon mds.h on smithi135 2024-06-13T06:54:20.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6836/1512917649,v1:172.21.15.136:6837/1512917649] up:boot 2024-06-13T06:54:20.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: fsmap 7 up:standby 2024-06-13T06:54:20.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:19 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T06:54:20.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:20.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: Deploying daemon mds.h on smithi135 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6836/1512917649,v1:172.21.15.136:6837/1512917649] up:boot 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: fsmap 7 up:standby 2024-06-13T06:54:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:19 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T06:54:21.432 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 systemd[1]: Starting Ceph mds.h for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:54:21.709 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 podman[66227]: 2024-06-13 06:54:21.335681673 +0000 UTC m=+0.017060142 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:54:21.709 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 podman[66227]: 2024-06-13 06:54:21.457826885 +0000 UTC m=+0.139205349 container create f27642e54dd87fd11822fa7e1ed127ca749b157af35961a7314eb44fe4fa3343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-13T06:54:21.709 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 podman[66227]: 2024-06-13 06:54:21.710720128 +0000 UTC m=+0.392098609 container init f27642e54dd87fd11822fa7e1ed127ca749b157af35961a7314eb44fe4fa3343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-13T06:54:22.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:21 smithi135 ceph-mon[36276]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:22.033 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 podman[66227]: 2024-06-13 06:54:21.713422028 +0000 UTC m=+0.394800491 container start f27642e54dd87fd11822fa7e1ed127ca749b157af35961a7314eb44fe4fa3343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-13T06:54:22.033 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 ceph-mds[66247]: -- 172.21.15.135:0/4253469012 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55811f743a40 con 0x55811f716c00 2024-06-13T06:54:22.033 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h[66243]: starting mds.h at 2024-06-13T06:54:22.033 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 bash[66227]: f27642e54dd87fd11822fa7e1ed127ca749b157af35961a7314eb44fe4fa3343 2024-06-13T06:54:22.033 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 06:54:21 smithi135 systemd[1]: Started Ceph mds.h for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:22.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.063+0000 7fe435e00700 1 -- 172.21.15.26:0/2168445734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe430104f80 msgr2=0x7fe430111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:22.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.063+0000 7fe435e00700 1 --2- 172.21.15.26:0/2168445734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe430104f80 0x7fe430111490 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe42800b0a0 tx=0x7fe42800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.063+0000 7fe435e00700 1 -- 172.21.15.26:0/2168445734 shutdown_connections 2024-06-13T06:54:22.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.063+0000 7fe435e00700 1 --2- 172.21.15.26:0/2168445734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe430104f80 0x7fe430111490 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.063+0000 7fe435e00700 1 --2- 172.21.15.26:0/2168445734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43010a650 0x7fe43010aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.063+0000 7fe435e00700 1 --2- 172.21.15.26:0/2168445734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4301045c0 0x7fe430104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.063+0000 7fe435e00700 1 -- 172.21.15.26:0/2168445734 >> 172.21.15.26:0/2168445734 conn(0x7fe4301000b0 msgr2=0x7fe4301024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:22.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.064+0000 7fe435e00700 1 -- 172.21.15.26:0/2168445734 shutdown_connections 2024-06-13T06:54:22.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:21 smithi026 ceph-mon[32041]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.064+0000 7fe435e00700 1 -- 172.21.15.26:0/2168445734 wait complete. 2024-06-13T06:54:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.065+0000 7fe435e00700 1 Processor -- start 2024-06-13T06:54:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.066+0000 7fe435e00700 1 -- start start 2024-06-13T06:54:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.066+0000 7fe435e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4301045c0 0x7fe4301a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.066+0000 7fe435e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe430104f80 0x7fe4301a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.066+0000 7fe435e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43010a650 0x7fe43019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.066+0000 7fe435e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe430114960 con 0x7fe43010a650 2024-06-13T06:54:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.066+0000 7fe435e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4301147e0 con 0x7fe430104f80 2024-06-13T06:54:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.066+0000 7fe435e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe430114ae0 con 0x7fe4301045c0 2024-06-13T06:54:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe430104f80 0x7fe4301a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe430104f80 0x7fe4301a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49906/0 (socket says 172.21.15.26:49906) 2024-06-13T06:54:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 -- 172.21.15.26:0/2836705505 learned_addr learned my addr 172.21.15.26:0/2836705505 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe434a00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4301045c0 0x7fe4301a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe435400700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43010a650 0x7fe43019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 -- 172.21.15.26:0/2836705505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4301045c0 msgr2=0x7fe4301a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4301045c0 0x7fe4301a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 -- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43010a650 msgr2=0x7fe43019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43010a650 0x7fe43019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 -- 172.21.15.26:0/2836705505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43019b260 con 0x7fe430104f80 2024-06-13T06:54:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.067+0000 7fe42fe00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe430104f80 0x7fe4301a0e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00e9c0 tx=0x7fe41c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.068+0000 7fe42d600700 1 -- 172.21.15.26:0/2836705505 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c00cbd0 con 0x7fe430104f80 2024-06-13T06:54:22.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.068+0000 7fe42d600700 1 -- 172.21.15.26:0/2836705505 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c00cd70 con 0x7fe430104f80 2024-06-13T06:54:22.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.068+0000 7fe42d600700 1 -- 172.21.15.26:0/2836705505 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c010430 con 0x7fe430104f80 2024-06-13T06:54:22.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.068+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43019b550 con 0x7fe430104f80 2024-06-13T06:54:22.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.068+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4301a77c0 con 0x7fe430104f80 2024-06-13T06:54:22.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.069+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe430069180 con 0x7fe430104f80 2024-06-13T06:54:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.071+0000 7fe42d600700 1 -- 172.21.15.26:0/2836705505 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe41c004750 con 0x7fe430104f80 2024-06-13T06:54:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.072+0000 7fe42d600700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4180a2c20 0x7fe4180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.073+0000 7fe42d600700 1 -- 172.21.15.26:0/2836705505 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe41c0ceee0 con 0x7fe430104f80 2024-06-13T06:54:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.073+0000 7fe434a00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4180a2c20 0x7fe4180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.073+0000 7fe434a00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4180a2c20 0x7fe4180a50e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe4301095a0 tx=0x7fe424009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:22.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.077+0000 7fe42d600700 1 -- 172.21.15.26:0/2836705505 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe41c09c240 con 0x7fe430104f80 2024-06-13T06:54:22.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:21 smithi136 ceph-mon[37229]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:22.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.349+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4301a79f0 con 0x7fe430104f80 2024-06-13T06:54:22.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.350+0000 7fe42d600700 1 -- 172.21.15.26:0/2836705505 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7fe41c09bb40 con 0x7fe430104f80 2024-06-13T06:54:22.350 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:54:22.351 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-13T06:54:22.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4180a2c20 msgr2=0x7fe4180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4180a2c20 0x7fe4180a50e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe4301095a0 tx=0x7fe424009260 comp rx=0 tx=0).stop 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe430104f80 msgr2=0x7fe4301a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe430104f80 0x7fe4301a0e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00e9c0 tx=0x7fe41c00ecd0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 shutdown_connections 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4180a2c20 0x7fe4180a50e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe43010a650 0x7fe43019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4301045c0 0x7fe4301a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 --2- 172.21.15.26:0/2836705505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe430104f80 0x7fe4301a0e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 >> 172.21.15.26:0/2836705505 conn(0x7fe4301000b0 msgr2=0x7fe430101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 shutdown_connections 2024-06-13T06:54:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:22.354+0000 7fe435e00700 1 -- 172.21.15.26:0/2836705505 wait complete. 2024-06-13T06:54:22.355 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.135:6836/363278001,v1:172.21.15.135:6837/363278001] up:boot 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: fsmap 8 up:standby 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:22.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:22.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: Deploying daemon mds.i on smithi136 2024-06-13T06:54:22.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:22 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2836705505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:23.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.135:6836/363278001,v1:172.21.15.135:6837/363278001] up:boot 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: fsmap 8 up:standby 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: Deploying daemon mds.i on smithi136 2024-06-13T06:54:23.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:22 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2836705505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:23.285 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.135:6836/363278001,v1:172.21.15.135:6837/363278001] up:boot 2024-06-13T06:54:23.285 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: fsmap 8 up:standby 2024-06-13T06:54:23.285 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T06:54:23.285 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:23.286 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:23.286 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:23.286 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:23.286 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:23.286 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:23.286 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: Deploying daemon mds.i on smithi136 2024-06-13T06:54:23.286 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:22 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2836705505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:23 smithi136 ceph-mon[37229]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:24.127 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 systemd[1]: Starting Ceph mds.i for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:54:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:23 smithi026 ceph-mon[32041]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:23 smithi135 ceph-mon[36276]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:24.499 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 podman[68532]: 2024-06-13 06:54:24.152175469 +0000 UTC m=+0.017228675 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:54:24.499 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 podman[68532]: 2024-06-13 06:54:24.269997412 +0000 UTC m=+0.135050611 container create 2de7eedbed6cce50fd5e48ff87fe52cc0bb5daf1168e3ea0cd5782f84954ac55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-13T06:54:24.500 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 podman[68532]: 2024-06-13 06:54:24.49819982 +0000 UTC m=+0.363253029 container init 2de7eedbed6cce50fd5e48ff87fe52cc0bb5daf1168e3ea0cd5782f84954ac55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:54:24.876 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 podman[68532]: 2024-06-13 06:54:24.501016031 +0000 UTC m=+0.366069230 container start 2de7eedbed6cce50fd5e48ff87fe52cc0bb5daf1168e3ea0cd5782f84954ac55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-13T06:54:24.877 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 ceph-mds[68551]: -- 172.21.15.136:0/672457635 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1857afa40 con 0x55d185783000 2024-06-13T06:54:24.877 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: starting mds.i at 2024-06-13T06:54:24.877 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 bash[68532]: 2de7eedbed6cce50fd5e48ff87fe52cc0bb5daf1168e3ea0cd5782f84954ac55 2024-06-13T06:54:24.877 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 06:54:24 smithi136 systemd[1]: Started Ceph mds.i for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:25.342 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: Deploying daemon mds.g on smithi026 2024-06-13T06:54:25.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:boot 2024-06-13T06:54:25.724 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: fsmap 9 up:standby 2024-06-13T06:54:25.724 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: Deploying daemon mds.g on smithi026 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:boot 2024-06-13T06:54:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: fsmap 9 up:standby 2024-06-13T06:54:26.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T06:54:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: Deploying daemon mds.g on smithi026 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:boot 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: fsmap 9 up:standby 2024-06-13T06:54:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T06:54:27.033 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:27.833 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:27 smithi026 ceph-mon[32041]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:28.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:27 smithi135 ceph-mon[36276]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:27 smithi136 ceph-mon[37229]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:28.208 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:27 smithi026 systemd[1]: Starting Ceph mds.g for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:54:29.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:28 smithi026 podman[73957]: 2024-06-13 06:54:28.750785728 +0000 UTC m=+0.034527495 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:54:29.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:28 smithi026 podman[73957]: 2024-06-13 06:54:28.90629304 +0000 UTC m=+0.190034844 container create e8a84a9ef3c5701505ced762c37272798e029bb437ce1eb21f29a6b964d77326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-13T06:54:29.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:29 smithi026 podman[73957]: 2024-06-13 06:54:29.184397337 +0000 UTC m=+0.468139120 container init e8a84a9ef3c5701505ced762c37272798e029bb437ce1eb21f29a6b964d77326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T06:54:29.502 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:29 smithi026 podman[73957]: 2024-06-13 06:54:29.195025058 +0000 UTC m=+0.478766836 container start e8a84a9ef3c5701505ced762c37272798e029bb437ce1eb21f29a6b964d77326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-06-13T06:54:29.502 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-mds[73989]: -- 172.21.15.26:0/1549327325 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d1101ba40 con 0x562d10feec00 2024-06-13T06:54:29.502 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: starting mds.g at 2024-06-13T06:54:29.502 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:29 smithi026 bash[73957]: e8a84a9ef3c5701505ced762c37272798e029bb437ce1eb21f29a6b964d77326 2024-06-13T06:54:29.502 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:54:29 smithi026 systemd[1]: Started Ceph mds.g for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-mon[32041]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:29.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:29.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:29.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 -- 172.21.15.26:0/2235678447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7ac1030f0 msgr2=0x7fb7ac107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 --2- 172.21.15.26:0/2235678447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7ac1030f0 0x7fb7ac107110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb798009920 tx=0x7fb798009c30 comp rx=0 tx=0).stop 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 -- 172.21.15.26:0/2235678447 shutdown_connections 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 --2- 172.21.15.26:0/2235678447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7ac107840 0x7fb7ac10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 --2- 172.21.15.26:0/2235678447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ac102740 0x7fb7ac102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 --2- 172.21.15.26:0/2235678447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7ac1030f0 0x7fb7ac107110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 -- 172.21.15.26:0/2235678447 >> 172.21.15.26:0/2235678447 conn(0x7fb7ac0795f0 msgr2=0x7fb7ac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 -- 172.21.15.26:0/2235678447 shutdown_connections 2024-06-13T06:54:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.012+0000 7fb7b3000700 1 -- 172.21.15.26:0/2235678447 wait complete. 2024-06-13T06:54:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.014+0000 7fb7b3000700 1 Processor -- start 2024-06-13T06:54:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.014+0000 7fb7b3000700 1 -- start start 2024-06-13T06:54:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.015+0000 7fb7b3000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ac102740 0x7fb7ac074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.015+0000 7fb7b3000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7ac1030f0 0x7fb7ac074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.015+0000 7fb7b3000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7ac107840 0x7fb7ac075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.015+0000 7fb7b3000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac1127a0 con 0x7fb7ac102740 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.015+0000 7fb7b3000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac112620 con 0x7fb7ac1030f0 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.015+0000 7fb7b3000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac112920 con 0x7fb7ac107840 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.015+0000 7fb7abe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ac102740 0x7fb7ac074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7abe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ac102740 0x7fb7ac074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60756/0 (socket says 172.21.15.26:60756) 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7abe00700 1 -- 172.21.15.26:0/3680514889 learned_addr learned my addr 172.21.15.26:0/3680514889 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7ab400700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7ac1030f0 0x7fb7ac074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7b1000700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7ac107840 0x7fb7ac075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7abe00700 1 -- 172.21.15.26:0/3680514889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7ac107840 msgr2=0x7fb7ac075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7abe00700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7ac107840 0x7fb7ac075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7abe00700 1 -- 172.21.15.26:0/3680514889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7ac1030f0 msgr2=0x7fb7ac074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7abe00700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7ac1030f0 0x7fb7ac074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.016+0000 7fb7abe00700 1 -- 172.21.15.26:0/3680514889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac1a71a0 con 0x7fb7ac102740 2024-06-13T06:54:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.017+0000 7fb7abe00700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ac102740 0x7fb7ac074790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00e9c0 tx=0x7fb79c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.017+0000 7fb7a8c00700 1 -- 172.21.15.26:0/3680514889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c004a30 con 0x7fb7ac102740 2024-06-13T06:54:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.017+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ac1a7490 con 0x7fb7ac102740 2024-06-13T06:54:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.017+0000 7fb7a8c00700 1 -- 172.21.15.26:0/3680514889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79c00cca0 con 0x7fb7ac102740 2024-06-13T06:54:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.017+0000 7fb7a8c00700 1 -- 172.21.15.26:0/3680514889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c005610 con 0x7fb7ac102740 2024-06-13T06:54:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.017+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ac1a79f0 con 0x7fb7ac102740 2024-06-13T06:54:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.019+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ac069180 con 0x7fb7ac102740 2024-06-13T06:54:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.021+0000 7fb7a8c00700 1 -- 172.21.15.26:0/3680514889 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb79c004cf0 con 0x7fb7ac102740 2024-06-13T06:54:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.023+0000 7fb7a8c00700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7900a2ca0 0x7fb7900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.023+0000 7fb7a8c00700 1 -- 172.21.15.26:0/3680514889 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb79c0cecd0 con 0x7fb7ac102740 2024-06-13T06:54:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.024+0000 7fb7ab400700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7900a2ca0 0x7fb7900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.025+0000 7fb7ab400700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7900a2ca0 0x7fb7900a5160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098f0 tx=0x7fb798019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:30.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.032+0000 7fb7a8c00700 1 -- 172.21.15.26:0/3680514889 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb79c09c030 con 0x7fb7ac102740 2024-06-13T06:54:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:29 smithi135 ceph-mon[36276]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:30.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:29 smithi136 ceph-mon[37229]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:30.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:30.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:30.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:30.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:30.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:30.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:30.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.336+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7ac02dca0 con 0x7fb7ac102740 2024-06-13T06:54:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.338+0000 7fb7a8c00700 1 -- 172.21.15.26:0/3680514889 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7fb79c09b930 con 0x7fb7ac102740 2024-06-13T06:54:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:54:30.337 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-06-13T06:54:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.340+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7900a2ca0 msgr2=0x7fb7900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.340+0000 7fb7b3000700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7900a2ca0 0x7fb7900a5160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098f0 tx=0x7fb798019040 comp rx=0 tx=0).stop 2024-06-13T06:54:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.340+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ac102740 msgr2=0x7fb7ac074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.340+0000 7fb7b3000700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ac102740 0x7fb7ac074790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00e9c0 tx=0x7fb79c00ecd0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.341+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 shutdown_connections 2024-06-13T06:54:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.341+0000 7fb7b3000700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7900a2ca0 0x7fb7900a5160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.341+0000 7fb7b3000700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7ac107840 0x7fb7ac075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.341+0000 7fb7b3000700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ac102740 0x7fb7ac074790 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.341+0000 7fb7b3000700 1 --2- 172.21.15.26:0/3680514889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7ac1030f0 0x7fb7ac074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.341+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 >> 172.21.15.26:0/3680514889 conn(0x7fb7ac0795f0 msgr2=0x7fb7ac100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.341+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 shutdown_connections 2024-06-13T06:54:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:30.341+0000 7fb7b3000700 1 -- 172.21.15.26:0/3680514889 wait complete. 2024-06-13T06:54:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 11 2024-06-13T06:54:30.934 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:30 smithi135 ceph-mon[36276]: Deploying daemon mds.a on smithi026 2024-06-13T06:54:30.934 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:30 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] up:boot 2024-06-13T06:54:30.934 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:30 smithi135 ceph-mon[36276]: fsmap 10 up:standby 2024-06-13T06:54:30.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:30 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T06:54:30.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:30 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3680514889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:31.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:30 smithi026 ceph-mon[32041]: Deploying daemon mds.a on smithi026 2024-06-13T06:54:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:30 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] up:boot 2024-06-13T06:54:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:30 smithi026 ceph-mon[32041]: fsmap 10 up:standby 2024-06-13T06:54:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T06:54:31.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:30 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3680514889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:31.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:30 smithi136 ceph-mon[37229]: Deploying daemon mds.a on smithi026 2024-06-13T06:54:31.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:30 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] up:boot 2024-06-13T06:54:31.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:30 smithi136 ceph-mon[37229]: fsmap 10 up:standby 2024-06-13T06:54:31.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T06:54:31.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:30 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3680514889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:31.946 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:31 smithi135 ceph-mon[36276]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:31.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:31 smithi026 ceph-mon[32041]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:32.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:31 smithi136 ceph-mon[37229]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:33.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:33.758 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:33 smithi026 ceph-mon[32041]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:34.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:33 smithi135 ceph-mon[36276]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:34.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:33 smithi136 ceph-mon[37229]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:34.495 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 06:54:34 smithi026 podman[74349]: 2024-06-13 06:54:34.206929469 +0000 UTC m=+0.145724711 container create 80963fba2169601c989395b0ee9ea8f6d8280df5b9393ab9454fd9ae8089e52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True) 2024-06-13T06:54:34.495 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 06:54:34 smithi026 podman[74349]: 2024-06-13 06:54:34.495095439 +0000 UTC m=+0.433890714 container init 80963fba2169601c989395b0ee9ea8f6d8280df5b9393ab9454fd9ae8089e52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-13T06:54:34.695 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:34.752 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 06:54:34 smithi026 podman[74349]: 2024-06-13 06:54:34.507080421 +0000 UTC m=+0.445875636 container start 80963fba2169601c989395b0ee9ea8f6d8280df5b9393ab9454fd9ae8089e52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-13T06:54:34.752 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 06:54:34 smithi026 ceph-mds[74386]: -- 172.21.15.26:0/2765499453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed1862ba40 con 0x55ed185ff000 2024-06-13T06:54:34.752 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 06:54:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a[74382]: starting mds.a at 2024-06-13T06:54:34.752 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 06:54:34 smithi026 bash[74349]: 80963fba2169601c989395b0ee9ea8f6d8280df5b9393ab9454fd9ae8089e52f 2024-06-13T06:54:34.752 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 06:54:34 smithi026 systemd[1]: Started Ceph mds.a for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.26:6840/3560957238,v1:172.21.15.26:6841/3560957238] up:boot 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: fsmap 11 up:standby 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:35.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:35.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:35.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:35 smithi026 ceph-mon[32041]: Deploying daemon mds.b on smithi135 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.26:6840/3560957238,v1:172.21.15.26:6841/3560957238] up:boot 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: fsmap 11 up:standby 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:35 smithi135 ceph-mon[36276]: Deploying daemon mds.b on smithi135 2024-06-13T06:54:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.26:6840/3560957238,v1:172.21.15.26:6841/3560957238] up:boot 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: fsmap 11 up:standby 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:35 smithi136 ceph-mon[37229]: Deploying daemon mds.b on smithi135 2024-06-13T06:54:37.033 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:36 smithi135 systemd[1]: Starting Ceph mds.b for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T06:54:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 -- 172.21.15.26:0/4137726856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 msgr2=0x7f75381115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 --2- 172.21.15.26:0/4137726856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 0x7f75381115a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f753000b0a0 tx=0x7f753000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 -- 172.21.15.26:0/4137726856 shutdown_connections 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 --2- 172.21.15.26:0/4137726856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753810a6d0 0x7f753810aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 --2- 172.21.15.26:0/4137726856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 0x7f75381115a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 --2- 172.21.15.26:0/4137726856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75381046d0 0x7f7538104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 -- 172.21.15.26:0/4137726856 >> 172.21.15.26:0/4137726856 conn(0x7f75381001c0 msgr2=0x7f75381025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 -- 172.21.15.26:0/4137726856 shutdown_connections 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.206+0000 7f7540600700 1 -- 172.21.15.26:0/4137726856 wait complete. 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.207+0000 7f7540600700 1 Processor -- start 2024-06-13T06:54:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.207+0000 7f7540600700 1 -- start start 2024-06-13T06:54:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f7540600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75381046d0 0x7f753819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f7540600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 0x7f753819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f7540600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753810a6d0 0x7f753819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f7540600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7538114ab0 con 0x7f75381046d0 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f7540600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7538114930 con 0x7f753810a6d0 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f7540600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7538114c30 con 0x7f7538105090 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f753dc00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75381046d0 0x7f753819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f753d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 0x7f753819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f753d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 0x7f753819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50486/0 (socket says 172.21.15.26:50486) 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f753d200700 1 -- 172.21.15.26:0/3519637256 learned_addr learned my addr 172.21.15.26:0/3519637256 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.208+0000 7f753e600700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753810a6d0 0x7f753819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f753d200700 1 -- 172.21.15.26:0/3519637256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753810a6d0 msgr2=0x7f753819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f753d200700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753810a6d0 0x7f753819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f753d200700 1 -- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75381046d0 msgr2=0x7f753819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f753d200700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75381046d0 0x7f753819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f753d200700 1 -- 172.21.15.26:0/3519637256 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753819fe00 con 0x7f7538105090 2024-06-13T06:54:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f753d200700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 0x7f753819b380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f752400e9a0 tx=0x7f752400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f7536000700 1 -- 172.21.15.26:0/3519637256 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400ca10 con 0x7f7538105090 2024-06-13T06:54:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75381a00f0 con 0x7f7538105090 2024-06-13T06:54:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f7536000700 1 -- 172.21.15.26:0/3519637256 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f752400cbb0 con 0x7f7538105090 2024-06-13T06:54:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.209+0000 7f7536000700 1 -- 172.21.15.26:0/3519637256 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524018680 con 0x7f7538105090 2024-06-13T06:54:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.210+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75381a7970 con 0x7f7538105090 2024-06-13T06:54:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.211+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7538069180 con 0x7f7538105090 2024-06-13T06:54:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.213+0000 7f7536000700 1 -- 172.21.15.26:0/3519637256 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7524010430 con 0x7f7538105090 2024-06-13T06:54:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.214+0000 7f7536000700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f75200a2c20 0x7f75200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.214+0000 7f7536000700 1 -- 172.21.15.26:0/3519637256 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f75240cee80 con 0x7f7538105090 2024-06-13T06:54:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.214+0000 7f753dc00700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f75200a2c20 0x7f75200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.215+0000 7f753dc00700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f75200a2c20 0x7f75200a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f752c009be0 tx=0x7f752c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:37.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.218+0000 7f7536000700 1 -- 172.21.15.26:0/3519637256 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f752409c1e0 con 0x7f7538105090 2024-06-13T06:54:37.372 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:37 smithi135 podman[66543]: 2024-06-13 06:54:36.995597991 +0000 UTC m=+0.016896417 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:54:37.372 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:37 smithi135 podman[66543]: 2024-06-13 06:54:37.12962901 +0000 UTC m=+0.150927423 container create 9e7a3d12d3b61765aee5df85890a8287a8bec8e45ed03d3dbd1be7878b57ecf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:54:37.373 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:37 smithi135 podman[66543]: 2024-06-13 06:54:37.37394822 +0000 UTC m=+0.395246647 container init 9e7a3d12d3b61765aee5df85890a8287a8bec8e45ed03d3dbd1be7878b57ecf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-13T06:54:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.492+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f753802dca0 con 0x7f7538105090 2024-06-13T06:54:37.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.493+0000 7f7536000700 1 -- 172.21.15.26:0/3519637256 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f752409bae0 con 0x7f7538105090 2024-06-13T06:54:37.493 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:54:37.493 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-06-13T06:54:37.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f75200a2c20 msgr2=0x7f75200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f75200a2c20 0x7f75200a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f752c009be0 tx=0x7f752c009260 comp rx=0 tx=0).stop 2024-06-13T06:54:37.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 msgr2=0x7f753819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 0x7f753819b380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f752400e9a0 tx=0x7f752400ecb0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 shutdown_connections 2024-06-13T06:54:37.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f75200a2c20 0x7f75200a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753810a6d0 0x7f753819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75381046d0 0x7f753819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 --2- 172.21.15.26:0/3519637256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538105090 0x7f753819b380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.496+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 >> 172.21.15.26:0/3519637256 conn(0x7f75381001c0 msgr2=0x7f75381006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:37.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.497+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 shutdown_connections 2024-06-13T06:54:37.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:37.497+0000 7f7540600700 1 -- 172.21.15.26:0/3519637256 wait complete. 2024-06-13T06:54:37.498 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 12 2024-06-13T06:54:37.693 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:37 smithi135 podman[66543]: 2024-06-13 06:54:37.376597091 +0000 UTC m=+0.397895508 container start 9e7a3d12d3b61765aee5df85890a8287a8bec8e45ed03d3dbd1be7878b57ecf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-13T06:54:37.693 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-mds[66562]: -- 172.21.15.135:0/2734995445 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649eb9c7a40 con 0x5649eb99ac00 2024-06-13T06:54:37.693 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: starting mds.b at 2024-06-13T06:54:37.693 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:37 smithi135 bash[66543]: 9e7a3d12d3b61765aee5df85890a8287a8bec8e45ed03d3dbd1be7878b57ecf6 2024-06-13T06:54:37.693 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 06:54:37 smithi135 systemd[1]: Started Ceph mds.b for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T06:54:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:37 smithi026 ceph-mon[32041]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:37 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3519637256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:37.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:37.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:37 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-mon[36276]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3519637256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:37 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:37 smithi136 ceph-mon[37229]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:37 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3519637256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:37 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:38.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:38 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:boot 2024-06-13T06:54:38.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:38 smithi135 ceph-mon[36276]: fsmap 12 up:standby 2024-06-13T06:54:38.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:38 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T06:54:38.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:38 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:38.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:38 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:39.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:38 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:boot 2024-06-13T06:54:39.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:38 smithi026 ceph-mon[32041]: fsmap 12 up:standby 2024-06-13T06:54:39.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:38 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T06:54:39.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:38 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:39.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:38 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:39.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:38 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:boot 2024-06-13T06:54:39.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:38 smithi136 ceph-mon[37229]: fsmap 12 up:standby 2024-06-13T06:54:39.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:38 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T06:54:39.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:38 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:39.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:38 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:39 smithi026 ceph-mon[32041]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:39 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:39 smithi135 ceph-mon[36276]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:39 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:39 smithi136 ceph-mon[37229]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:39 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:40.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:42.031 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:41 smithi136 ceph-mon[37229]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:42.031 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.031 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.031 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:42.031 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:42.031 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.031 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:41 smithi135 ceph-mon[36276]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:42.070 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:41 smithi026 ceph-mon[32041]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:42.071 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.071 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.071 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:42.071 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:42.071 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:42.071 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 -- 172.21.15.26:0/4126873757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c1030f0 msgr2=0x7f4d5c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 --2- 172.21.15.26:0/4126873757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c1030f0 0x7f4d5c107110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0098f0 tx=0x7f4d4c009c00 comp rx=0 tx=0).stop 2024-06-13T06:54:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 -- 172.21.15.26:0/4126873757 shutdown_connections 2024-06-13T06:54:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 --2- 172.21.15.26:0/4126873757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d5c102740 0x7f4d5c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 --2- 172.21.15.26:0/4126873757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c107840 0x7f4d5c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 --2- 172.21.15.26:0/4126873757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c1030f0 0x7f4d5c107110 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 -- 172.21.15.26:0/4126873757 >> 172.21.15.26:0/4126873757 conn(0x7f4d5c0795f0 msgr2=0x7f4d5c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 -- 172.21.15.26:0/4126873757 shutdown_connections 2024-06-13T06:54:43.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.335+0000 7f4d64800700 1 -- 172.21.15.26:0/4126873757 wait complete. 2024-06-13T06:54:43.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.337+0000 7f4d64800700 1 Processor -- start 2024-06-13T06:54:43.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.337+0000 7f4d64800700 1 -- start start 2024-06-13T06:54:43.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.337+0000 7f4d64800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c102740 0x7f4d5c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:43.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.337+0000 7f4d64800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1030f0 0x7f4d5c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:43.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.337+0000 7f4d64800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d5c107840 0x7f4d5c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:43.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.337+0000 7f4d64800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c112750 con 0x7f4d5c1030f0 2024-06-13T06:54:43.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.337+0000 7f4d64800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1125d0 con 0x7f4d5c107840 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.337+0000 7f4d64800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1128d0 con 0x7f4d5c102740 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d61400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1030f0 0x7f4d5c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d61e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c102740 0x7f4d5c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d61400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1030f0 0x7f4d5c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40794/0 (socket says 172.21.15.26:40794) 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d61400700 1 -- 172.21.15.26:0/1577084425 learned_addr learned my addr 172.21.15.26:0/1577084425 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d62800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d5c107840 0x7f4d5c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d62800700 1 -- 172.21.15.26:0/1577084425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c102740 msgr2=0x7f4d5c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d62800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c102740 0x7f4d5c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d62800700 1 -- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1030f0 msgr2=0x7f4d5c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d62800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1030f0 0x7f4d5c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d62800700 1 -- 172.21.15.26:0/1577084425 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c1a7120 con 0x7f4d5c107840 2024-06-13T06:54:43.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d61400700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1030f0 0x7f4d5c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:43.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.338+0000 7f4d61e00700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c102740 0x7f4d5c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:43.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.339+0000 7f4d62800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d5c107840 0x7f4d5c075210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800efc0 tx=0x7f4d5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:43.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.339+0000 7f4d56a00700 1 -- 172.21.15.26:0/1577084425 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d580092e0 con 0x7f4d5c107840 2024-06-13T06:54:43.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.339+0000 7f4d56a00700 1 -- 172.21.15.26:0/1577084425 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d58004bb0 con 0x7f4d5c107840 2024-06-13T06:54:43.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.339+0000 7f4d56a00700 1 -- 172.21.15.26:0/1577084425 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58004280 con 0x7f4d5c107840 2024-06-13T06:54:43.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.339+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c1a73b0 con 0x7f4d5c107840 2024-06-13T06:54:43.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.339+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c1a7910 con 0x7f4d5c107840 2024-06-13T06:54:43.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.341+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d5c103bd0 con 0x7f4d5c107840 2024-06-13T06:54:43.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.342+0000 7f4d56a00700 1 -- 172.21.15.26:0/1577084425 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4d5801b030 con 0x7f4d5c107840 2024-06-13T06:54:43.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.343+0000 7f4d56a00700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d440a2ca0 0x7f4d440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:43.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.344+0000 7f4d56a00700 1 -- 172.21.15.26:0/1577084425 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4d580ce630 con 0x7f4d5c107840 2024-06-13T06:54:43.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.344+0000 7f4d61e00700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d440a2ca0 0x7f4d440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:43.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.345+0000 7f4d61e00700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d440a2ca0 0x7f4d440a5160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009750 tx=0x7f4d50009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:43.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.348+0000 7f4d56a00700 1 -- 172.21.15.26:0/1577084425 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4d58022020 con 0x7f4d5c107840 2024-06-13T06:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.647+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d5c02dca0 con 0x7f4d5c107840 2024-06-13T06:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.648+0000 7f4d56a00700 1 -- 172.21.15.26:0/1577084425 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7f4d58010040 con 0x7f4d5c107840 2024-06-13T06:54:43.648 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:54:43.648 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-06-13T06:54:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.652+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d440a2ca0 msgr2=0x7f4d440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.652+0000 7f4d64800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d440a2ca0 0x7f4d440a5160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009750 tx=0x7f4d50009310 comp rx=0 tx=0).stop 2024-06-13T06:54:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.652+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d5c107840 msgr2=0x7f4d5c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.652+0000 7f4d64800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d5c107840 0x7f4d5c075210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800efc0 tx=0x7f4d5800c3a0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.653+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 shutdown_connections 2024-06-13T06:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.653+0000 7f4d64800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d440a2ca0 0x7f4d440a5160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.653+0000 7f4d64800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d5c107840 0x7f4d5c075210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.653+0000 7f4d64800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d5c102740 0x7f4d5c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.653+0000 7f4d64800700 1 --2- 172.21.15.26:0/1577084425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c1030f0 0x7f4d5c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.653+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 >> 172.21.15.26:0/1577084425 conn(0x7f4d5c0795f0 msgr2=0x7f4d5c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.653+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 shutdown_connections 2024-06-13T06:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:43.653+0000 7f4d64800700 1 -- 172.21.15.26:0/1577084425 wait complete. 2024-06-13T06:54:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 13 2024-06-13T06:54:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:43 smithi026 ceph-mon[32041]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:43 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:43 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1577084425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:44.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:43 smithi136 ceph-mon[37229]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:44.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:43 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:43 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1577084425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:43 smithi135 ceph-mon[36276]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:43 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:43 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1577084425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:45.060 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-13T06:54:45.499 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:45 smithi026 ceph-mon[32041]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:46.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:45 smithi135 ceph-mon[36276]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:46.381 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:45 smithi136 ceph-mon[37229]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:47.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:47 smithi026 ceph-mon[32041]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:47.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:47.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:47.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:47 smithi135 ceph-mon[36276]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:47.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:47.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:47.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:47 smithi136 ceph-mon[37229]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:47.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:47.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:47.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 -- 172.21.15.26:0/3922038736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 msgr2=0x7fb40c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 --2- 172.21.15.26:0/3922038736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 0x7fb40c06ab40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009990 tx=0x7fb3f8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 -- 172.21.15.26:0/3922038736 shutdown_connections 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 --2- 172.21.15.26:0/3922038736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb40c104ed0 0x7fb40c105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 --2- 172.21.15.26:0/3922038736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb40c1073c0 0x7fb40c06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 --2- 172.21.15.26:0/3922038736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 0x7fb40c06ab40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 -- 172.21.15.26:0/3922038736 >> 172.21.15.26:0/3922038736 conn(0x7fb40c078340 msgr2=0x7fb40c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 -- 172.21.15.26:0/3922038736 shutdown_connections 2024-06-13T06:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.909+0000 7fb411600700 1 -- 172.21.15.26:0/3922038736 wait complete. 2024-06-13T06:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.910+0000 7fb411600700 1 Processor -- start 2024-06-13T06:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.910+0000 7fb411600700 1 -- start start 2024-06-13T06:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb411600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 0x7fb40c19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb411600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb40c104ed0 0x7fb40c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb411600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb40c1073c0 0x7fb40c1a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb411600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c118d70 con 0x7fb40c1073c0 2024-06-13T06:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb411600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c118bf0 con 0x7fb40c104ed0 2024-06-13T06:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb411600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c118ef0 con 0x7fb40c06a6e0 2024-06-13T06:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb40aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 0x7fb40c19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb40b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb40c1073c0 0x7fb40c1a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:47.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb40aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 0x7fb40c19f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48176/0 (socket says 172.21.15.26:48176) 2024-06-13T06:54:47.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.911+0000 7fb40aa00700 1 -- 172.21.15.26:0/1619760109 learned_addr learned my addr 172.21.15.26:0/1619760109 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:47.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.912+0000 7fb40a000700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb40c104ed0 0x7fb40c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.912+0000 7fb40aa00700 1 -- 172.21.15.26:0/1619760109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb40c104ed0 msgr2=0x7fb40c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.912+0000 7fb40aa00700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb40c104ed0 0x7fb40c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.912+0000 7fb40aa00700 1 -- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb40c1073c0 msgr2=0x7fb40c1a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.912+0000 7fb40aa00700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb40c1073c0 0x7fb40c1a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.912+0000 7fb40aa00700 1 -- 172.21.15.26:0/1619760109 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1a4230 con 0x7fb40c06a6e0 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.912+0000 7fb40aa00700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 0x7fb40c19f220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00d980 tx=0x7fb3fc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.912+0000 7fb403400700 1 -- 172.21.15.26:0/1619760109 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc016070 con 0x7fb40c06a6e0 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.913+0000 7fb403400700 1 -- 172.21.15.26:0/1619760109 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3fc004d10 con 0x7fb40c06a6e0 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.913+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1a4520 con 0x7fb40c06a6e0 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.913+0000 7fb403400700 1 -- 172.21.15.26:0/1619760109 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0054e0 con 0x7fb40c06a6e0 2024-06-13T06:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.913+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1abda0 con 0x7fb40c06a6e0 2024-06-13T06:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.914+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c069180 con 0x7fb40c06a6e0 2024-06-13T06:54:47.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.916+0000 7fb403400700 1 -- 172.21.15.26:0/1619760109 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb3fc004750 con 0x7fb40c06a6e0 2024-06-13T06:54:47.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.917+0000 7fb403400700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3f00a2c20 0x7fb3f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:47.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.917+0000 7fb403400700 1 -- 172.21.15.26:0/1619760109 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb3fc0ce8e0 con 0x7fb40c06a6e0 2024-06-13T06:54:47.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.918+0000 7fb40a000700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3f00a2c20 0x7fb3f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:47.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.918+0000 7fb40a000700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3f00a2c20 0x7fb3f00a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800b2e0 tx=0x7fb3f8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:47.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:47.922+0000 7fb403400700 1 -- 172.21.15.26:0/1619760109 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3fc01f020 con 0x7fb40c06a6e0 2024-06-13T06:54:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.153+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb40c1abf70 con 0x7fb40c06a6e0 2024-06-13T06:54:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.154+0000 7fb403400700 1 -- 172.21.15.26:0/1619760109 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fb3fc09bc40 con 0x7fb40c06a6e0 2024-06-13T06:54:48.154 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:54:48.154 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-06-13T06:54:48.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T06:54:48.154 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-06-13T06:54:48.154 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-06-13T06:54:48.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T06:54:48.154 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-06-13T06:54:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.157+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3f00a2c20 msgr2=0x7fb3f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.157+0000 7fb411600700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3f00a2c20 0x7fb3f00a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800b2e0 tx=0x7fb3f8005960 comp rx=0 tx=0).stop 2024-06-13T06:54:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.157+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 msgr2=0x7fb40c19f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.157+0000 7fb411600700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 0x7fb40c19f220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00d980 tx=0x7fb3fc00dc90 comp rx=0 tx=0).stop 2024-06-13T06:54:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.158+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 shutdown_connections 2024-06-13T06:54:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.158+0000 7fb411600700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3f00a2c20 0x7fb3f00a50e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.158+0000 7fb411600700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb40c1073c0 0x7fb40c1a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.158+0000 7fb411600700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb40c06a6e0 0x7fb40c19f220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.158+0000 7fb411600700 1 --2- 172.21.15.26:0/1619760109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb40c104ed0 0x7fb40c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.158+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 >> 172.21.15.26:0/1619760109 conn(0x7fb40c078340 msgr2=0x7fb40c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.158+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 shutdown_connections 2024-06-13T06:54:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:48.158+0000 7fb411600700 1 -- 172.21.15.26:0/1619760109 wait complete. 2024-06-13T06:54:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:48 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1619760109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-13T06:54:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:48 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1619760109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-13T06:54:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:48 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1619760109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-13T06:54:49.156 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-13T06:54:49.156 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-13T06:54:49.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:49 smithi026 ceph-mon[32041]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:49.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:49 smithi135 ceph-mon[36276]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:49.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:49 smithi136 ceph-mon[37229]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:49.630 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:51.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:51 smithi026 ceph-mon[32041]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:51.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:51 smithi135 ceph-mon[36276]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:51 smithi136 ceph-mon[37229]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.094+0000 7f22a8400700 1 -- 172.21.15.26:0/2906827425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01085f0 msgr2=0x7f22a01089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.094+0000 7f22a8400700 1 --2- 172.21.15.26:0/2906827425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01085f0 0x7f22a01089d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2294009990 tx=0x7f2294009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.095+0000 7f22a8400700 1 -- 172.21.15.26:0/2906827425 shutdown_connections 2024-06-13T06:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.095+0000 7f22a8400700 1 --2- 172.21.15.26:0/2906827425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22a0102ea0 0x7f22a010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.095+0000 7f22a8400700 1 --2- 172.21.15.26:0/2906827425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01085f0 0x7f22a01089d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.095+0000 7f22a8400700 1 --2- 172.21.15.26:0/2906827425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a01024e0 0x7f22a0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.095+0000 7f22a8400700 1 -- 172.21.15.26:0/2906827425 >> 172.21.15.26:0/2906827425 conn(0x7f22a0079640 msgr2=0x7f22a0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.095+0000 7f22a8400700 1 -- 172.21.15.26:0/2906827425 shutdown_connections 2024-06-13T06:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.095+0000 7f22a8400700 1 -- 172.21.15.26:0/2906827425 wait complete. 2024-06-13T06:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.096+0000 7f22a8400700 1 Processor -- start 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.096+0000 7f22a8400700 1 -- start start 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.096+0000 7f22a8400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01024e0 0x7f22a019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.096+0000 7f22a8400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22a0102ea0 0x7f22a019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.096+0000 7f22a8400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a01085f0 0x7f22a019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.096+0000 7f22a8400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0112800 con 0x7f22a01024e0 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.096+0000 7f22a8400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0112680 con 0x7f22a0102ea0 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.096+0000 7f22a8400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0112980 con 0x7f22a01085f0 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01024e0 0x7f22a019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01024e0 0x7f22a019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40818/0 (socket says 172.21.15.26:40818) 2024-06-13T06:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5a00700 1 -- 172.21.15.26:0/3647294725 learned_addr learned my addr 172.21.15.26:0/3647294725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a6400700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a01085f0 0x7f22a019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5000700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22a0102ea0 0x7f22a019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5a00700 1 -- 172.21.15.26:0/3647294725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a01085f0 msgr2=0x7f22a019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5a00700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a01085f0 0x7f22a019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5a00700 1 -- 172.21.15.26:0/3647294725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22a0102ea0 msgr2=0x7f22a019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5a00700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22a0102ea0 0x7f22a019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.097+0000 7f22a5a00700 1 -- 172.21.15.26:0/3647294725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a019ff80 con 0x7f22a01024e0 2024-06-13T06:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.098+0000 7f22a5a00700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01024e0 0x7f22a019af70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f229400b820 tx=0x7f229400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.098+0000 7f229e000700 1 -- 172.21.15.26:0/3647294725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22940056d0 con 0x7f22a01024e0 2024-06-13T06:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.098+0000 7f229e000700 1 -- 172.21.15.26:0/3647294725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2294003d70 con 0x7f22a01024e0 2024-06-13T06:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.098+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a01a0210 con 0x7f22a01024e0 2024-06-13T06:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.098+0000 7f229e000700 1 -- 172.21.15.26:0/3647294725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229400f460 con 0x7f22a01024e0 2024-06-13T06:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.098+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a01a7af0 con 0x7f22a01024e0 2024-06-13T06:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.099+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a0069180 con 0x7f22a01024e0 2024-06-13T06:54:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.101+0000 7f229e000700 1 -- 172.21.15.26:0/3647294725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2294009d10 con 0x7f22a01024e0 2024-06-13T06:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.103+0000 7f229e000700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22880a2e40 0x7f22880a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.103+0000 7f229e000700 1 -- 172.21.15.26:0/3647294725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f22940cfa50 con 0x7f22a01024e0 2024-06-13T06:54:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.103+0000 7f22a5000700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22880a2e40 0x7f22880a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.104+0000 7f22a5000700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22880a2e40 0x7f22880a5300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f22a010f7e0 tx=0x7f228c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.107+0000 7f229e000700 1 -- 172.21.15.26:0/3647294725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2294013070 con 0x7f22a01024e0 2024-06-13T06:54:52.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:52.341+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f22a01a7cc0 con 0x7f22a01024e0 2024-06-13T06:54:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.296+0000 7f229e000700 1 -- 172.21.15.26:0/3647294725 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f229409ce40 con 0x7f22a01024e0 2024-06-13T06:54:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.302+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22880a2e40 msgr2=0x7f22880a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.302+0000 7f22a8400700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22880a2e40 0x7f22880a5300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f22a010f7e0 tx=0x7f228c009260 comp rx=0 tx=0).stop 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.302+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01024e0 msgr2=0x7f22a019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.302+0000 7f22a8400700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01024e0 0x7f22a019af70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f229400b820 tx=0x7f229400bab0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.303+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 shutdown_connections 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.303+0000 7f22a8400700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22880a2e40 0x7f22880a5300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.303+0000 7f22a8400700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a01085f0 0x7f22a019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.303+0000 7f22a8400700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22a01024e0 0x7f22a019af70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.303+0000 7f22a8400700 1 --2- 172.21.15.26:0/3647294725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22a0102ea0 0x7f22a019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.303+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 >> 172.21.15.26:0/3647294725 conn(0x7f22a0079640 msgr2=0x7f22a010d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:53.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.304+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 shutdown_connections 2024-06-13T06:54:53.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:53.304+0000 7f22a8400700 1 -- 172.21.15.26:0/3647294725 wait complete. 2024-06-13T06:54:53.309 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-06-13T06:54:53.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:53 smithi136 ceph-mon[37229]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:53.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:53 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3647294725' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:54:53.735 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:53 smithi026 ceph-mon[32041]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:53.735 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:53 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3647294725' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:54:53.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:53 smithi135 ceph-mon[36276]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:53.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:53 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3647294725' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:54:54.320 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-13T06:54:54.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:54 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3647294725' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-13T06:54:54.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:54 smithi026 ceph-mon[32041]: osdmap e82: 12 total, 12 up, 12 in 2024-06-13T06:54:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:54 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3647294725' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-13T06:54:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:54 smithi136 ceph-mon[37229]: osdmap e82: 12 total, 12 up, 12 in 2024-06-13T06:54:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:54 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3647294725' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-13T06:54:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:54 smithi135 ceph-mon[36276]: osdmap e82: 12 total, 12 up, 12 in 2024-06-13T06:54:55.420 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:55 smithi026 ceph-mon[32041]: pgmap v325: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:55.420 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:55 smithi026 ceph-mon[32041]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:54:55.420 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:55 smithi026 ceph-mon[32041]: osdmap e83: 12 total, 12 up, 12 in 2024-06-13T06:54:55.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:55 smithi136 ceph-mon[37229]: pgmap v325: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:55.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:55 smithi136 ceph-mon[37229]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:54:55.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:55 smithi136 ceph-mon[37229]: osdmap e83: 12 total, 12 up, 12 in 2024-06-13T06:54:55.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:55 smithi135 ceph-mon[36276]: pgmap v325: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:55.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:55 smithi135 ceph-mon[36276]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:54:55.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:55 smithi135 ceph-mon[36276]: osdmap e83: 12 total, 12 up, 12 in 2024-06-13T06:54:56.471 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:56 smithi026 ceph-mon[32041]: osdmap e84: 12 total, 12 up, 12 in 2024-06-13T06:54:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:56 smithi136 ceph-mon[37229]: osdmap e84: 12 total, 12 up, 12 in 2024-06-13T06:54:56.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:56 smithi135 ceph-mon[36276]: osdmap e84: 12 total, 12 up, 12 in 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.021+0000 7f4b71000700 1 -- 172.21.15.26:0/2846099677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c1030f0 msgr2=0x7f4b6c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.021+0000 7f4b71000700 1 --2- 172.21.15.26:0/2846099677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c1030f0 0x7f4b6c107110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009920 tx=0x7f4b54009c30 comp rx=0 tx=0).stop 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.021+0000 7f4b71000700 1 -- 172.21.15.26:0/2846099677 shutdown_connections 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.021+0000 7f4b71000700 1 --2- 172.21.15.26:0/2846099677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b6c107840 0x7f4b6c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.021+0000 7f4b71000700 1 --2- 172.21.15.26:0/2846099677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c102740 0x7f4b6c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.021+0000 7f4b71000700 1 --2- 172.21.15.26:0/2846099677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c1030f0 0x7f4b6c107110 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.021+0000 7f4b71000700 1 -- 172.21.15.26:0/2846099677 >> 172.21.15.26:0/2846099677 conn(0x7f4b6c0795f0 msgr2=0x7f4b6c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.022+0000 7f4b71000700 1 -- 172.21.15.26:0/2846099677 shutdown_connections 2024-06-13T06:54:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.022+0000 7f4b71000700 1 -- 172.21.15.26:0/2846099677 wait complete. 2024-06-13T06:54:57.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.023+0000 7f4b71000700 1 Processor -- start 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.023+0000 7f4b71000700 1 -- start start 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.023+0000 7f4b71000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c102740 0x7f4b6c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.023+0000 7f4b71000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b6c1030f0 0x7f4b6c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.023+0000 7f4b71000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c107840 0x7f4b6c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.023+0000 7f4b71000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1127c0 con 0x7f4b6c102740 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.023+0000 7f4b71000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c112640 con 0x7f4b6c1030f0 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.023+0000 7f4b71000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c112940 con 0x7f4b6c107840 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b6a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c107840 0x7f4b6c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b63e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b6c1030f0 0x7f4b6c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b6a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c107840 0x7f4b6c19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49578/0 (socket says 172.21.15.26:49578) 2024-06-13T06:54:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b6a800700 1 -- 172.21.15.26:0/1895609108 learned_addr learned my addr 172.21.15.26:0/1895609108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:54:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b69e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c102740 0x7f4b6c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b6a800700 1 -- 172.21.15.26:0/1895609108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b6c1030f0 msgr2=0x7f4b6c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b6a800700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b6c1030f0 0x7f4b6c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b6a800700 1 -- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c102740 msgr2=0x7f4b6c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b6a800700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c102740 0x7f4b6c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.024+0000 7f4b6a800700 1 -- 172.21.15.26:0/1895609108 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c19fe70 con 0x7f4b6c107840 2024-06-13T06:54:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.025+0000 7f4b6a800700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c107840 0x7f4b6c19f730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400c110 tx=0x7f4b6400c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.025+0000 7f4b62a00700 1 -- 172.21.15.26:0/1895609108 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b640092e0 con 0x7f4b6c107840 2024-06-13T06:54:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.025+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c1a0100 con 0x7f4b6c107840 2024-06-13T06:54:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.025+0000 7f4b62a00700 1 -- 172.21.15.26:0/1895609108 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b64004bb0 con 0x7f4b6c107840 2024-06-13T06:54:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.025+0000 7f4b62a00700 1 -- 172.21.15.26:0/1895609108 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64007d10 con 0x7f4b6c107840 2024-06-13T06:54:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.025+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1a79e0 con 0x7f4b6c107840 2024-06-13T06:54:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.026+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c069180 con 0x7f4b6c107840 2024-06-13T06:54:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.029+0000 7f4b62a00700 1 -- 172.21.15.26:0/1895609108 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4b6401a020 con 0x7f4b6c107840 2024-06-13T06:54:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.030+0000 7f4b62a00700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b500a2c20 0x7f4b500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.030+0000 7f4b62a00700 1 -- 172.21.15.26:0/1895609108 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f4b640cf300 con 0x7f4b6c107840 2024-06-13T06:54:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.030+0000 7f4b69e00700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b500a2c20 0x7f4b500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.031+0000 7f4b69e00700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b500a2c20 0x7f4b500a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009c60 tx=0x7f4b5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:57.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.032+0000 7f4b62a00700 1 -- 172.21.15.26:0/1895609108 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4b64010040 con 0x7f4b6c107840 2024-06-13T06:54:57.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.262+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f4b6c1a7bb0 con 0x7f4b6c107840 2024-06-13T06:54:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.873+0000 7f4b62a00700 1 -- 172.21.15.26:0/1895609108 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) ==== 171+0+0 (secure 0 0 0) 0x7f4b6409c4f0 con 0x7f4b6c107840 2024-06-13T06:54:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.881+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b500a2c20 msgr2=0x7f4b500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.881+0000 7f4b71000700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b500a2c20 0x7f4b500a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009c60 tx=0x7f4b5c009260 comp rx=0 tx=0).stop 2024-06-13T06:54:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.881+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c107840 msgr2=0x7f4b6c19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.881+0000 7f4b71000700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c107840 0x7f4b6c19f730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400c110 tx=0x7f4b6400c420 comp rx=0 tx=0).stop 2024-06-13T06:54:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.882+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 shutdown_connections 2024-06-13T06:54:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.882+0000 7f4b71000700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b500a2c20 0x7f4b500a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.882+0000 7f4b71000700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c107840 0x7f4b6c19f730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.882+0000 7f4b71000700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c102740 0x7f4b6c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.882+0000 7f4b71000700 1 --2- 172.21.15.26:0/1895609108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b6c1030f0 0x7f4b6c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.882+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 >> 172.21.15.26:0/1895609108 conn(0x7f4b6c0795f0 msgr2=0x7f4b6c100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.883+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 shutdown_connections 2024-06-13T06:54:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:54:57.884+0000 7f4b71000700 1 -- 172.21.15.26:0/1895609108 wait complete. 2024-06-13T06:54:57.891 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-06-13T06:54:58.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:57 smithi136 ceph-mon[37229]: pgmap v328: 33 pgs: 15 active+clean, 3 creating+peering, 15 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:58.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:57 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1895609108' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:54:58.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:57 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:54:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:57 smithi026 ceph-mon[32041]: pgmap v328: 33 pgs: 15 active+clean, 3 creating+peering, 15 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:57 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1895609108' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:54:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:57 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:54:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:57 smithi135 ceph-mon[36276]: pgmap v328: 33 pgs: 15 active+clean, 3 creating+peering, 15 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:54:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:57 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1895609108' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:54:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:57 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:54:58.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-13T06:54:59.224 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:58 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-13T06:54:59.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:58 smithi026 ceph-mon[32041]: osdmap e85: 12 total, 12 up, 12 in 2024-06-13T06:54:59.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:58 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-13T06:54:59.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:58 smithi135 ceph-mon[36276]: osdmap e85: 12 total, 12 up, 12 in 2024-06-13T06:54:59.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:58 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-13T06:54:59.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:58 smithi136 ceph-mon[37229]: osdmap e85: 12 total, 12 up, 12 in 2024-06-13T06:55:00.184 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:59 smithi135 ceph-mon[36276]: pgmap v330: 97 pgs: 27 active+clean, 9 creating+peering, 61 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:00.184 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:59 smithi135 ceph-mon[36276]: osdmap e86: 12 total, 12 up, 12 in 2024-06-13T06:55:00.184 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:54:59 smithi135 ceph-mon[36276]: osdmap e87: 12 total, 12 up, 12 in 2024-06-13T06:55:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:59 smithi026 ceph-mon[32041]: pgmap v330: 97 pgs: 27 active+clean, 9 creating+peering, 61 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:59 smithi026 ceph-mon[32041]: osdmap e86: 12 total, 12 up, 12 in 2024-06-13T06:55:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:54:59 smithi026 ceph-mon[32041]: osdmap e87: 12 total, 12 up, 12 in 2024-06-13T06:55:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:59 smithi136 ceph-mon[37229]: pgmap v330: 97 pgs: 27 active+clean, 9 creating+peering, 61 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:59 smithi136 ceph-mon[37229]: osdmap e86: 12 total, 12 up, 12 in 2024-06-13T06:55:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:54:59 smithi136 ceph-mon[37229]: osdmap e87: 12 total, 12 up, 12 in 2024-06-13T06:55:01.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 -- 172.21.15.26:0/1164974543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f249410eec0 msgr2=0x7f24941112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 --2- 172.21.15.26:0/1164974543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f249410eec0 0x7f24941112b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f248c00b0a0 tx=0x7f248c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 -- 172.21.15.26:0/1164974543 shutdown_connections 2024-06-13T06:55:01.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 --2- 172.21.15.26:0/1164974543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f249410eec0 0x7f24941112b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 --2- 172.21.15.26:0/1164974543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24940746c0 0x7f2494074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 --2- 172.21.15.26:0/1164974543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2494074fe0 0x7f249410e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 -- 172.21.15.26:0/1164974543 >> 172.21.15.26:0/1164974543 conn(0x7f24940795e0 msgr2=0x7f24940799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:01.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 -- 172.21.15.26:0/1164974543 shutdown_connections 2024-06-13T06:55:01.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.585+0000 7f2498c00700 1 -- 172.21.15.26:0/1164974543 wait complete. 2024-06-13T06:55:01.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.587+0000 7f2498c00700 1 Processor -- start 2024-06-13T06:55:01.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.587+0000 7f2498c00700 1 -- start start 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2498c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24940746c0 0x7f249419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2498c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2494074fe0 0x7f249419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2498c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f249410eec0 0x7f249419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2498c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2494114a60 con 0x7f2494074fe0 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2498c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24941148e0 con 0x7f24940746c0 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2498c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2494114be0 con 0x7f249410eec0 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2491e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24940746c0 0x7f249419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2491e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24940746c0 0x7f249419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:58686/0 (socket says 172.21.15.26:58686) 2024-06-13T06:55:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2491e00700 1 -- 172.21.15.26:0/1242204964 learned_addr learned my addr 172.21.15.26:0/1242204964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:01.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2491400700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2494074fe0 0x7f249419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.588+0000 7f2492800700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f249410eec0 0x7f249419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f2491e00700 1 -- 172.21.15.26:0/1242204964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f249410eec0 msgr2=0x7f249419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f2491e00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f249410eec0 0x7f249419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f2491e00700 1 -- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2494074fe0 msgr2=0x7f249419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f2491e00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2494074fe0 0x7f249419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f2491e00700 1 -- 172.21.15.26:0/1242204964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249419fe20 con 0x7f24940746c0 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f2491e00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24940746c0 0x7f249419ae10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f248400d980 tx=0x7f248400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f248aa00700 1 -- 172.21.15.26:0/1242204964 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484016070 con 0x7f24940746c0 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f248aa00700 1 -- 172.21.15.26:0/1242204964 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2484004d10 con 0x7f24940746c0 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.589+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24941a0110 con 0x7f24940746c0 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.590+0000 7f248aa00700 1 -- 172.21.15.26:0/1242204964 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484005670 con 0x7f24940746c0 2024-06-13T06:55:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.590+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941a7990 con 0x7f24940746c0 2024-06-13T06:55:01.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.591+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2494069180 con 0x7f24940746c0 2024-06-13T06:55:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.593+0000 7f248aa00700 1 -- 172.21.15.26:0/1242204964 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2484004750 con 0x7f24940746c0 2024-06-13T06:55:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.594+0000 7f248aa00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24780a2ca0 0x7f24780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.594+0000 7f248aa00700 1 -- 172.21.15.26:0/1242204964 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7f24840cecb0 con 0x7f24940746c0 2024-06-13T06:55:01.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.594+0000 7f2491400700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24780a2ca0 0x7f24780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.595+0000 7f2491400700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24780a2ca0 0x7f24780a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2494110a30 tx=0x7f24800073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:01.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.598+0000 7f248aa00700 1 -- 172.21.15.26:0/1242204964 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f248401f020 con 0x7f24940746c0 2024-06-13T06:55:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.878+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f249419c170 con 0x7f24940746c0 2024-06-13T06:55:01.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.907+0000 7f248aa00700 1 -- 172.21.15.26:0/1242204964 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f248409bd20 con 0x7f24940746c0 2024-06-13T06:55:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.912+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24780a2ca0 msgr2=0x7f24780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.912+0000 7f2498c00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24780a2ca0 0x7f24780a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2494110a30 tx=0x7f24800073d0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.912+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24940746c0 msgr2=0x7f249419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.912+0000 7f2498c00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24940746c0 0x7f249419ae10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f248400d980 tx=0x7f248400dc90 comp rx=0 tx=0).stop 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.913+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 shutdown_connections 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.913+0000 7f2498c00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24780a2ca0 0x7f24780a5160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.913+0000 7f2498c00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24940746c0 0x7f249419ae10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.913+0000 7f2498c00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f249410eec0 0x7f249419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.913+0000 7f2498c00700 1 --2- 172.21.15.26:0/1242204964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2494074fe0 0x7f249419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.913+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 >> 172.21.15.26:0/1242204964 conn(0x7f24940795e0 msgr2=0x7f24940ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.913+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 shutdown_connections 2024-06-13T06:55:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:01.913+0000 7f2498c00700 1 -- 172.21.15.26:0/1242204964 wait complete. 2024-06-13T06:55:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-06-13T06:55:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:01 smithi026 ceph-mon[32041]: pgmap v333: 97 pgs: 52 active+clean, 12 creating+peering, 33 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:01 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1242204964' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:55:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:01 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:55:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:01 smithi026 ceph-mon[32041]: daemon mds.g assigned to filesystem cephfs as rank 0 2024-06-13T06:55:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:01 smithi135 ceph-mon[36276]: pgmap v333: 97 pgs: 52 active+clean, 12 creating+peering, 33 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:01 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1242204964' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:55:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:01 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:55:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:01 smithi135 ceph-mon[36276]: daemon mds.g assigned to filesystem cephfs as rank 0 2024-06-13T06:55:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:01 smithi136 ceph-mon[37229]: pgmap v333: 97 pgs: 52 active+clean, 12 creating+peering, 33 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:01 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1242204964' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:55:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:01 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:55:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:01 smithi136 ceph-mon[37229]: daemon mds.g assigned to filesystem cephfs as rank 0 2024-06-13T06:55:02.789 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-13T06:55:02.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-13T06:55:03.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:02 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-13T06:55:03.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:02 smithi026 ceph-mon[32041]: osdmap e88: 12 total, 12 up, 12 in 2024-06-13T06:55:03.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:02 smithi026 ceph-mon[32041]: fsmap cephfs:1 {0=g=up:creating} 11 up:standby 2024-06-13T06:55:03.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:02 smithi026 ceph-mon[32041]: daemon mds.g is now active in filesystem cephfs as rank 0 2024-06-13T06:55:03.276 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:03.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:02 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-13T06:55:03.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:02 smithi135 ceph-mon[36276]: osdmap e88: 12 total, 12 up, 12 in 2024-06-13T06:55:03.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:02 smithi135 ceph-mon[36276]: fsmap cephfs:1 {0=g=up:creating} 11 up:standby 2024-06-13T06:55:03.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:02 smithi135 ceph-mon[36276]: daemon mds.g is now active in filesystem cephfs as rank 0 2024-06-13T06:55:03.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:02 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-13T06:55:03.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:02 smithi136 ceph-mon[37229]: osdmap e88: 12 total, 12 up, 12 in 2024-06-13T06:55:03.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:02 smithi136 ceph-mon[37229]: fsmap cephfs:1 {0=g=up:creating} 11 up:standby 2024-06-13T06:55:03.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:02 smithi136 ceph-mon[37229]: daemon mds.g is now active in filesystem cephfs as rank 0 2024-06-13T06:55:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:03 smithi026 ceph-mon[32041]: pgmap v335: 97 pgs: 67 active+clean, 12 creating+peering, 18 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:03 smithi026 ceph-mon[32041]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-13T06:55:04.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:03 smithi026 ceph-mon[32041]: Cluster is now healthy 2024-06-13T06:55:04.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:03 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] up:active 2024-06-13T06:55:04.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:03 smithi026 ceph-mon[32041]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:04.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:03 smithi026 ceph-mon[32041]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:03 smithi135 ceph-mon[36276]: pgmap v335: 97 pgs: 67 active+clean, 12 creating+peering, 18 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:03 smithi135 ceph-mon[36276]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-13T06:55:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:03 smithi135 ceph-mon[36276]: Cluster is now healthy 2024-06-13T06:55:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:03 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] up:active 2024-06-13T06:55:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:03 smithi135 ceph-mon[36276]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:03 smithi135 ceph-mon[36276]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:03 smithi136 ceph-mon[37229]: pgmap v335: 97 pgs: 67 active+clean, 12 creating+peering, 18 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:03 smithi136 ceph-mon[37229]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-13T06:55:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:03 smithi136 ceph-mon[37229]: Cluster is now healthy 2024-06-13T06:55:04.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:03 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] up:active 2024-06-13T06:55:04.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:03 smithi136 ceph-mon[37229]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:04.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:03 smithi136 ceph-mon[37229]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:05.002 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:55:04 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: 2024-06-13T06:55:04.667+0000 7f4b66e00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-13T06:55:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.624+0000 7fa086e00700 1 -- 172.21.15.26:0/1010233179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 msgr2=0x7fa080101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.624+0000 7fa086e00700 1 --2- 172.21.15.26:0/1010233179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 0x7fa080101550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa074009990 tx=0x7fa074009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.625+0000 7fa086e00700 1 -- 172.21.15.26:0/1010233179 shutdown_connections 2024-06-13T06:55:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.625+0000 7fa086e00700 1 --2- 172.21.15.26:0/1010233179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 0x7fa080101550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.625+0000 7fa086e00700 1 --2- 172.21.15.26:0/1010233179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa08010ed20 0x7fa080111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.625+0000 7fa086e00700 1 --2- 172.21.15.26:0/1010233179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa080101b20 0x7fa08010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.625+0000 7fa086e00700 1 -- 172.21.15.26:0/1010233179 >> 172.21.15.26:0/1010233179 conn(0x7fa0800fcdb0 msgr2=0x7fa0800ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.625+0000 7fa086e00700 1 -- 172.21.15.26:0/1010233179 shutdown_connections 2024-06-13T06:55:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.625+0000 7fa086e00700 1 -- 172.21.15.26:0/1010233179 wait complete. 2024-06-13T06:55:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.626+0000 7fa086e00700 1 Processor -- start 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.626+0000 7fa086e00700 1 -- start start 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.626+0000 7fa086e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 0x7fa0801a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.626+0000 7fa086e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa080101b20 0x7fa0801a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.626+0000 7fa086e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa08010ed20 0x7fa08019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.626+0000 7fa086e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0801148d0 con 0x7fa080101b20 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.626+0000 7fa086e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa080114750 con 0x7fa080101170 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.626+0000 7fa086e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa080114a50 con 0x7fa08010ed20 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 0x7fa0801a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa080101b20 0x7fa0801a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:05.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 0x7fa0801a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:52508/0 (socket says 172.21.15.26:52508) 2024-06-13T06:55:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085a00700 1 -- 172.21.15.26:0/2552440621 learned_addr learned my addr 172.21.15.26:0/2552440621 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa086400700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa08010ed20 0x7fa08019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085a00700 1 -- 172.21.15.26:0/2552440621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa08010ed20 msgr2=0x7fa08019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085a00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa08010ed20 0x7fa08019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085a00700 1 -- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa080101b20 msgr2=0x7fa0801a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085a00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa080101b20 0x7fa0801a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.627+0000 7fa085a00700 1 -- 172.21.15.26:0/2552440621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08019b410 con 0x7fa080101170 2024-06-13T06:55:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.628+0000 7fa085a00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 0x7fa0801a0a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa074000c00 tx=0x7fa07400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.628+0000 7fa07e000700 1 -- 172.21.15.26:0/2552440621 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074027070 con 0x7fa080101170 2024-06-13T06:55:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.628+0000 7fa07e000700 1 -- 172.21.15.26:0/2552440621 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa074003d70 con 0x7fa080101170 2024-06-13T06:55:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.628+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08019b6a0 con 0x7fa080101170 2024-06-13T06:55:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.628+0000 7fa07e000700 1 -- 172.21.15.26:0/2552440621 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07400f460 con 0x7fa080101170 2024-06-13T06:55:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.628+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0801a79e0 con 0x7fa080101170 2024-06-13T06:55:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.629+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08019ba00 con 0x7fa080101170 2024-06-13T06:55:05.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.631+0000 7fa07e000700 1 -- 172.21.15.26:0/2552440621 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa07402a070 con 0x7fa080101170 2024-06-13T06:55:05.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.633+0000 7fa07e000700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa0680a2ca0 0x7fa0680a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:05.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.633+0000 7fa07e000700 1 -- 172.21.15.26:0/2552440621 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa0740cfe50 con 0x7fa080101170 2024-06-13T06:55:05.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.633+0000 7fa085000700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa0680a2ca0 0x7fa0680a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:05.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.634+0000 7fa085000700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa0680a2ca0 0x7fa0680a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa08019c280 tx=0x7fa06c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:05.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.635+0000 7fa07e000700 1 -- 172.21.15.26:0/2552440621 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa074013070 con 0x7fa080101170 2024-06-13T06:55:05.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.881+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fa080069180 con 0x7fa080101170 2024-06-13T06:55:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.939+0000 7fa07e000700 1 -- 172.21.15.26:0/2552440621 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v89) ==== 157+0+0 (secure 0 0 0) 0x7fa07409cef0 con 0x7fa080101170 2024-06-13T06:55:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.942+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa0680a2ca0 msgr2=0x7fa0680a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.942+0000 7fa086e00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa0680a2ca0 0x7fa0680a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa08019c280 tx=0x7fa06c009260 comp rx=0 tx=0).stop 2024-06-13T06:55:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 msgr2=0x7fa0801a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 0x7fa0801a0a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa074000c00 tx=0x7fa07400bab0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 shutdown_connections 2024-06-13T06:55:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa0680a2ca0 0x7fa0680a5160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa080101170 0x7fa0801a0a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa08010ed20 0x7fa08019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 --2- 172.21.15.26:0/2552440621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa080101b20 0x7fa0801a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 >> 172.21.15.26:0/2552440621 conn(0x7fa0800fcdb0 msgr2=0x7fa08010ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 shutdown_connections 2024-06-13T06:55:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:05.943+0000 7fa086e00700 1 -- 172.21.15.26:0/2552440621 wait complete. 2024-06-13T06:55:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:05 smithi026 ceph-mon[32041]: pgmap v336: 97 pgs: 97 active+clean; 578 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-13T06:55:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:05 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2552440621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:55:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:05 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:55:06.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:05 smithi135 ceph-mon[36276]: pgmap v336: 97 pgs: 97 active+clean; 578 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-13T06:55:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:05 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2552440621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:55:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:05 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:55:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:05 smithi136 ceph-mon[37229]: pgmap v336: 97 pgs: 97 active+clean; 578 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-13T06:55:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:05 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2552440621' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:55:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:05 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:55:06.839 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-13T06:55:07.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:06 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-13T06:55:07.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:06 smithi026 ceph-mon[32041]: osdmap e89: 12 total, 12 up, 12 in 2024-06-13T06:55:07.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:06 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-13T06:55:07.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:06 smithi135 ceph-mon[36276]: osdmap e89: 12 total, 12 up, 12 in 2024-06-13T06:55:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:07.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:06 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-13T06:55:07.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:06 smithi136 ceph-mon[37229]: osdmap e89: 12 total, 12 up, 12 in 2024-06-13T06:55:08.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:07 smithi026 ceph-mon[32041]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-13T06:55:08.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:07 smithi135 ceph-mon[36276]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-13T06:55:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:07 smithi136 ceph-mon[37229]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-13T06:55:09.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 -- 172.21.15.26:0/3374880199 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 msgr2=0x7f77bc10f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:09.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 --2- 172.21.15.26:0/3374880199 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 0x7f77bc10f3d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77b40099c0 tx=0x7f77b4009cd0 comp rx=0 tx=0).stop 2024-06-13T06:55:09.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 -- 172.21.15.26:0/3374880199 shutdown_connections 2024-06-13T06:55:09.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 --2- 172.21.15.26:0/3374880199 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 0x7f77bc10f3d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:09.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 --2- 172.21.15.26:0/3374880199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc06a3d0 0x7f77bc06a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:09.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 --2- 172.21.15.26:0/3374880199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77bc06ad80 0x7f77bc107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:09.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 -- 172.21.15.26:0/3374880199 >> 172.21.15.26:0/3374880199 conn(0x7f77bc078330 msgr2=0x7f77bc078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:09.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 -- 172.21.15.26:0/3374880199 shutdown_connections 2024-06-13T06:55:09.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.655+0000 7f77c3800700 1 -- 172.21.15.26:0/3374880199 wait complete. 2024-06-13T06:55:09.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.656+0000 7f77c3800700 1 Processor -- start 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c3800700 1 -- start start 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c3800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77bc06a3d0 0x7f77bc1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c3800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc06ad80 0x7f77bc1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c3800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 0x7f77bc19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c3800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc112790 con 0x7f77bc06ad80 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c3800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc112610 con 0x7f77bc107850 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c3800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc112910 con 0x7f77bc06a3d0 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77bbe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc06ad80 0x7f77bc1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77bbe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc06ad80 0x7f77bc1a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32804/0 (socket says 172.21.15.26:32804) 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77bbe00700 1 -- 172.21.15.26:0/3342683601 learned_addr learned my addr 172.21.15.26:0/3342683601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c0e00700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77bc06a3d0 0x7f77bc1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.657+0000 7f77c1800700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 0x7f77bc19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.658+0000 7f77bbe00700 1 -- 172.21.15.26:0/3342683601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77bc06a3d0 msgr2=0x7f77bc1a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.658+0000 7f77bbe00700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77bc06a3d0 0x7f77bc1a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.658+0000 7f77bbe00700 1 -- 172.21.15.26:0/3342683601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 msgr2=0x7f77bc19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.658+0000 7f77bbe00700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 0x7f77bc19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.658+0000 7f77bbe00700 1 -- 172.21.15.26:0/3342683601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc19b250 con 0x7f77bc06ad80 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.658+0000 7f77c0e00700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77bc06a3d0 0x7f77bc1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.658+0000 7f77c1800700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 0x7f77bc19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.658+0000 7f77bbe00700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc06ad80 0x7f77bc1a0e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77a800e9c0 tx=0x7f77a800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.659+0000 7f77b9600700 1 -- 172.21.15.26:0/3342683601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a800cbd0 con 0x7f77bc06ad80 2024-06-13T06:55:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.659+0000 7f77b9600700 1 -- 172.21.15.26:0/3342683601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a800ee60 con 0x7f77bc06ad80 2024-06-13T06:55:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.659+0000 7f77b9600700 1 -- 172.21.15.26:0/3342683601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8005630 con 0x7f77bc06ad80 2024-06-13T06:55:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.659+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc19b540 con 0x7f77bc06ad80 2024-06-13T06:55:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.659+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc1a77b0 con 0x7f77bc06ad80 2024-06-13T06:55:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.660+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc050a00 con 0x7f77bc06ad80 2024-06-13T06:55:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.662+0000 7f77b9600700 1 -- 172.21.15.26:0/3342683601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f77a8004750 con 0x7f77bc06ad80 2024-06-13T06:55:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.665+0000 7f77b9600700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77a40a2d40 0x7f77a40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.665+0000 7f77b9600700 1 -- 172.21.15.26:0/3342683601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f77a80cf4c0 con 0x7f77bc06ad80 2024-06-13T06:55:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.665+0000 7f77c0e00700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77a40a2d40 0x7f77a40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.666+0000 7f77c0e00700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77a40a2d40 0x7f77a40a5200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77bc107260 tx=0x7f77b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.668+0000 7f77b9600700 1 -- 172.21.15.26:0/3342683601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f77a8053850 con 0x7f77bc06ad80 2024-06-13T06:55:09.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:09.902+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f77bc1a7a10 con 0x7f77bc06ad80 2024-06-13T06:55:10.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:10 smithi136 ceph-mon[37229]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:55:10.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:10 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3342683601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:55:10.888 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:10 smithi135 ceph-mon[36276]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:55:10.888 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:10 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3342683601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:55:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.977+0000 7f77b9600700 1 -- 172.21.15.26:0/3342683601 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v91) ==== 228+0+0 (secure 0 0 0) 0x7f77a809c390 con 0x7f77bc06ad80 2024-06-13T06:55:10.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:10 smithi026 ceph-mon[32041]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:55:10.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:10 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3342683601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:55:10.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.982+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77a40a2d40 msgr2=0x7f77a40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:10.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.982+0000 7f77c3800700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77a40a2d40 0x7f77a40a5200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77bc107260 tx=0x7f77b000b040 comp rx=0 tx=0).stop 2024-06-13T06:55:10.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.982+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc06ad80 msgr2=0x7f77bc1a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:10.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.982+0000 7f77c3800700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc06ad80 0x7f77bc1a0e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77a800e9c0 tx=0x7f77a800ecd0 comp rx=0 tx=0).stop 2024-06-13T06:55:10.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.984+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 shutdown_connections 2024-06-13T06:55:10.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.984+0000 7f77c3800700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77a40a2d40 0x7f77a40a5200 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:10.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.984+0000 7f77c3800700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77bc107850 0x7f77bc19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:10.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.984+0000 7f77c3800700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77bc06a3d0 0x7f77bc1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:10.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.984+0000 7f77c3800700 1 --2- 172.21.15.26:0/3342683601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc06ad80 0x7f77bc1a0e20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:10.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.984+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 >> 172.21.15.26:0/3342683601 conn(0x7f77bc078330 msgr2=0x7f77bc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:10.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.985+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 shutdown_connections 2024-06-13T06:55:10.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:10.985+0000 7f77c3800700 1 -- 172.21.15.26:0/3342683601 wait complete. 2024-06-13T06:55:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data_ec' created 2024-06-13T06:55:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:11 smithi026 ceph-mon[32041]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:55:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:11 smithi026 ceph-mon[32041]: osdmap e90: 12 total, 12 up, 12 in 2024-06-13T06:55:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:11 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3342683601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-13T06:55:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:11 smithi026 ceph-mon[32041]: osdmap e91: 12 total, 12 up, 12 in 2024-06-13T06:55:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:11 smithi135 ceph-mon[36276]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:55:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:11 smithi135 ceph-mon[36276]: osdmap e90: 12 total, 12 up, 12 in 2024-06-13T06:55:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:11 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3342683601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-13T06:55:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:11 smithi135 ceph-mon[36276]: osdmap e91: 12 total, 12 up, 12 in 2024-06-13T06:55:11.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-13T06:55:11.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:11 smithi136 ceph-mon[37229]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:55:11.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:11 smithi136 ceph-mon[37229]: osdmap e90: 12 total, 12 up, 12 in 2024-06-13T06:55:11.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:11 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3342683601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-13T06:55:11.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:11 smithi136 ceph-mon[37229]: osdmap e91: 12 total, 12 up, 12 in 2024-06-13T06:55:12.345 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:13.062 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:12 smithi136 ceph-mon[37229]: osdmap e92: 12 total, 12 up, 12 in 2024-06-13T06:55:13.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:12 smithi026 ceph-mon[32041]: osdmap e92: 12 total, 12 up, 12 in 2024-06-13T06:55:13.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:12 smithi135 ceph-mon[36276]: osdmap e92: 12 total, 12 up, 12 in 2024-06-13T06:55:14.182 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:13 smithi026 ceph-mon[32041]: pgmap v344: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:55:14.182 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:13 smithi026 ceph-mon[32041]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:55:14.182 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:13 smithi026 ceph-mon[32041]: osdmap e93: 12 total, 12 up, 12 in 2024-06-13T06:55:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:13 smithi135 ceph-mon[36276]: pgmap v344: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:55:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:13 smithi135 ceph-mon[36276]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:55:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:13 smithi135 ceph-mon[36276]: osdmap e93: 12 total, 12 up, 12 in 2024-06-13T06:55:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:13 smithi136 ceph-mon[37229]: pgmap v344: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:55:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:13 smithi136 ceph-mon[37229]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:55:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:13 smithi136 ceph-mon[37229]: osdmap e93: 12 total, 12 up, 12 in 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 -- 172.21.15.26:0/216387169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae880746c0 msgr2=0x7fae88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 --2- 172.21.15.26:0/216387169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae880746c0 0x7fae88074aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae780098f0 tx=0x7fae78009c00 comp rx=0 tx=0).stop 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 -- 172.21.15.26:0/216387169 shutdown_connections 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 --2- 172.21.15.26:0/216387169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae880746c0 0x7fae88074aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 --2- 172.21.15.26:0/216387169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8810eec0 0x7fae881112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 --2- 172.21.15.26:0/216387169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae88074fe0 0x7fae8810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 -- 172.21.15.26:0/216387169 >> 172.21.15.26:0/216387169 conn(0x7fae88079560 msgr2=0x7fae88079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 -- 172.21.15.26:0/216387169 shutdown_connections 2024-06-13T06:55:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.729+0000 7fae8d200700 1 -- 172.21.15.26:0/216387169 wait complete. 2024-06-13T06:55:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.730+0000 7fae8d200700 1 Processor -- start 2024-06-13T06:55:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.730+0000 7fae8d200700 1 -- start start 2024-06-13T06:55:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae8d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae880746c0 0x7fae8819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae8d200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae88074fe0 0x7fae8819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae8d200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8810eec0 0x7fae8819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae8d200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae881149d0 con 0x7fae8810eec0 2024-06-13T06:55:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae8d200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae88114850 con 0x7fae88074fe0 2024-06-13T06:55:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae8d200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae88114b50 con 0x7fae880746c0 2024-06-13T06:55:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae85e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae880746c0 0x7fae8819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae85400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae88074fe0 0x7fae8819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae86800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8810eec0 0x7fae8819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae85e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae880746c0 0x7fae8819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:46092/0 (socket says 172.21.15.26:46092) 2024-06-13T06:55:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae85e00700 1 -- 172.21.15.26:0/3665331426 learned_addr learned my addr 172.21.15.26:0/3665331426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae85400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae88074fe0 0x7fae8819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:40502/0 (socket says 172.21.15.26:40502) 2024-06-13T06:55:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.731+0000 7fae86800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8810eec0 0x7fae8819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43308/0 (socket says 172.21.15.26:43308) 2024-06-13T06:55:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.732+0000 7fae85e00700 1 -- 172.21.15.26:0/3665331426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae88074fe0 msgr2=0x7fae8819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.732+0000 7fae85e00700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae88074fe0 0x7fae8819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.732+0000 7fae85e00700 1 -- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8810eec0 msgr2=0x7fae8819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.732+0000 7fae85e00700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8810eec0 0x7fae8819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.732+0000 7fae85e00700 1 -- 172.21.15.26:0/3665331426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8819fe80 con 0x7fae880746c0 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.732+0000 7fae86800700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8810eec0 0x7fae8819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.732+0000 7fae85400700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae88074fe0 0x7fae8819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.732+0000 7fae85e00700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae880746c0 0x7fae8819aec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fae78009230 tx=0x7fae7800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.733+0000 7fae7ea00700 1 -- 172.21.15.26:0/3665331426 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78005180 con 0x7fae880746c0 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.733+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae881a0110 con 0x7fae880746c0 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.733+0000 7fae7ea00700 1 -- 172.21.15.26:0/3665331426 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae7800b560 con 0x7fae880746c0 2024-06-13T06:55:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.733+0000 7fae7ea00700 1 -- 172.21.15.26:0/3665331426 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78003a70 con 0x7fae880746c0 2024-06-13T06:55:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.733+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881a79f0 con 0x7fae880746c0 2024-06-13T06:55:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.734+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae88075db0 con 0x7fae880746c0 2024-06-13T06:55:14.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.738+0000 7fae7ea00700 1 -- 172.21.15.26:0/3665331426 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fae78029060 con 0x7fae880746c0 2024-06-13T06:55:14.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.739+0000 7fae7ea00700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae6c0a2c20 0x7fae6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:14.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.739+0000 7fae7ea00700 1 -- 172.21.15.26:0/3665331426 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8770+0+0 (secure 0 0 0) 0x7fae780d0110 con 0x7fae880746c0 2024-06-13T06:55:14.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.739+0000 7fae85400700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae6c0a2c20 0x7fae6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:14.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.740+0000 7fae7ea00700 1 -- 172.21.15.26:0/3665331426 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae7805d850 con 0x7fae880746c0 2024-06-13T06:55:14.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.740+0000 7fae85400700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae6c0a2c20 0x7fae6c0a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae88110af0 tx=0x7fae74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:14.996+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fae881a7bc0 con 0x7fae880746c0 2024-06-13T06:55:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.016+0000 7fae7ea00700 1 -- 172.21.15.26:0/3665331426 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v94) ==== 169+0+0 (secure 0 0 0) 0x7fae78013030 con 0x7fae880746c0 2024-06-13T06:55:16.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.019+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae6c0a2c20 msgr2=0x7fae6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:16.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.019+0000 7fae8d200700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae6c0a2c20 0x7fae6c0a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae88110af0 tx=0x7fae74009260 comp rx=0 tx=0).stop 2024-06-13T06:55:16.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.019+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae880746c0 msgr2=0x7fae8819aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:16.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.019+0000 7fae8d200700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae880746c0 0x7fae8819aec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fae78009230 tx=0x7fae7800fa70 comp rx=0 tx=0).stop 2024-06-13T06:55:16.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.019+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 shutdown_connections 2024-06-13T06:55:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.019+0000 7fae8d200700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae6c0a2c20 0x7fae6c0a50e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.019+0000 7fae8d200700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8810eec0 0x7fae8819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.019+0000 7fae8d200700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae880746c0 0x7fae8819aec0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.020+0000 7fae8d200700 1 --2- 172.21.15.26:0/3665331426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae88074fe0 0x7fae8819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.020+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 >> 172.21.15.26:0/3665331426 conn(0x7fae88079560 msgr2=0x7fae881008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.020+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 shutdown_connections 2024-06-13T06:55:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:16.020+0000 7fae8d200700 1 -- 172.21.15.26:0/3665331426 wait complete. 2024-06-13T06:55:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:set pool 4 allow_ec_overwrites to true 2024-06-13T06:55:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:16 smithi026 ceph-mon[32041]: pgmap v346: 129 pgs: 6 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:16 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3665331426' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:55:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:16 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:55:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:16 smithi135 ceph-mon[36276]: pgmap v346: 129 pgs: 6 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:16 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3665331426' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:55:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:16 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:55:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:16 smithi136 ceph-mon[37229]: pgmap v346: 129 pgs: 6 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:16 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3665331426' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:55:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:16 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:55:16.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-13T06:55:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:17 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-13T06:55:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:17 smithi135 ceph-mon[36276]: osdmap e94: 12 total, 12 up, 12 in 2024-06-13T06:55:17.302 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:17.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:17 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-13T06:55:17.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:17 smithi136 ceph-mon[37229]: osdmap e94: 12 total, 12 up, 12 in 2024-06-13T06:55:17.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:17 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-13T06:55:17.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:17 smithi026 ceph-mon[32041]: osdmap e94: 12 total, 12 up, 12 in 2024-06-13T06:55:18.108 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:18 smithi026 ceph-mon[32041]: pgmap v348: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:18 smithi135 ceph-mon[36276]: pgmap v348: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:18 smithi136 ceph-mon[37229]: pgmap v348: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.734+0000 7f8c1b400700 1 -- 172.21.15.26:0/2776211402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141084e0 msgr2=0x7f8c141088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.734+0000 7f8c1b400700 1 --2- 172.21.15.26:0/2776211402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141084e0 0x7f8c141088c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c080099c0 tx=0x7f8c08009cd0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.735+0000 7f8c1b400700 1 -- 172.21.15.26:0/2776211402 shutdown_connections 2024-06-13T06:55:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.735+0000 7f8c1b400700 1 --2- 172.21.15.26:0/2776211402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c1406ab10 0x7f8c1410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.735+0000 7f8c1b400700 1 --2- 172.21.15.26:0/2776211402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141084e0 0x7f8c141088c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.735+0000 7f8c1b400700 1 --2- 172.21.15.26:0/2776211402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c1406a170 0x7f8c1406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.735+0000 7f8c1b400700 1 -- 172.21.15.26:0/2776211402 >> 172.21.15.26:0/2776211402 conn(0x7f8c140782c0 msgr2=0x7f8c140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.735+0000 7f8c1b400700 1 -- 172.21.15.26:0/2776211402 shutdown_connections 2024-06-13T06:55:19.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.735+0000 7f8c1b400700 1 -- 172.21.15.26:0/2776211402 wait complete. 2024-06-13T06:55:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.736+0000 7f8c1b400700 1 Processor -- start 2024-06-13T06:55:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.736+0000 7f8c1b400700 1 -- start start 2024-06-13T06:55:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c1b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1406a170 0x7f8c1419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c1b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c1406ab10 0x7f8c1419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c1b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c141084e0 0x7f8c14198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c1b400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14112770 con 0x7f8c1406a170 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c1b400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141125f0 con 0x7f8c141084e0 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c1b400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141128f0 con 0x7f8c1406ab10 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c18a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1406a170 0x7f8c1419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c18a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1406a170 0x7f8c1419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43334/0 (socket says 172.21.15.26:43334) 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c18a00700 1 -- 172.21.15.26:0/829647027 learned_addr learned my addr 172.21.15.26:0/829647027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c13e00700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c1406ab10 0x7f8c1419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.737+0000 7f8c19400700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c141084e0 0x7f8c14198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c18a00700 1 -- 172.21.15.26:0/829647027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c1406ab10 msgr2=0x7f8c1419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c18a00700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c1406ab10 0x7f8c1419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c18a00700 1 -- 172.21.15.26:0/829647027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c141084e0 msgr2=0x7f8c14198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c18a00700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c141084e0 0x7f8c14198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c18a00700 1 -- 172.21.15.26:0/829647027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c141991d0 con 0x7f8c1406a170 2024-06-13T06:55:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c18a00700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1406a170 0x7f8c1419e860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c08006010 tx=0x7f8c08003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c11600700 1 -- 172.21.15.26:0/829647027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08003e50 con 0x7f8c1406a170 2024-06-13T06:55:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c14199460 con 0x7f8c1406a170 2024-06-13T06:55:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c11600700 1 -- 172.21.15.26:0/829647027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c080055a0 con 0x7f8c1406a170 2024-06-13T06:55:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.738+0000 7f8c11600700 1 -- 172.21.15.26:0/829647027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800fe30 con 0x7f8c1406a170 2024-06-13T06:55:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.739+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141a5730 con 0x7f8c1406a170 2024-06-13T06:55:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.740+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c14103b90 con 0x7f8c1406a170 2024-06-13T06:55:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.742+0000 7f8c11600700 1 -- 172.21.15.26:0/829647027 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c0800f4f0 con 0x7f8c1406a170 2024-06-13T06:55:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.743+0000 7f8c11600700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8bfc0a2dc0 0x7f8bfc0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.743+0000 7f8c11600700 1 -- 172.21.15.26:0/829647027 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7f8c080cfff0 con 0x7f8c1406a170 2024-06-13T06:55:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.744+0000 7f8c13e00700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8bfc0a2dc0 0x7f8bfc0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.744+0000 7f8c13e00700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8bfc0a2dc0 0x7f8bfc0a5280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8c140788a0 tx=0x7f8c00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:19.748+0000 7f8c11600700 1 -- 172.21.15.26:0/829647027 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c08013070 con 0x7f8c1406a170 2024-06-13T06:55:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:20.035+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f8c14199f50 con 0x7f8c1406a170 2024-06-13T06:55:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:20 smithi135 ceph-mon[36276]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:20 smithi136 ceph-mon[37229]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:20 smithi026 ceph-mon[32041]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:21.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.045+0000 7f8c11600700 1 -- 172.21.15.26:0/829647027 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f8c0809cda0 con 0x7f8c1406a170 2024-06-13T06:55:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.049+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8bfc0a2dc0 msgr2=0x7f8bfc0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.049+0000 7f8c1b400700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8bfc0a2dc0 0x7f8bfc0a5280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8c140788a0 tx=0x7f8c00009260 comp rx=0 tx=0).stop 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.049+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1406a170 msgr2=0x7f8c1419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.049+0000 7f8c1b400700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1406a170 0x7f8c1419e860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c08006010 tx=0x7f8c08003a90 comp rx=0 tx=0).stop 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.050+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 shutdown_connections 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.050+0000 7f8c1b400700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8bfc0a2dc0 0x7f8bfc0a5280 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.050+0000 7f8c1b400700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c141084e0 0x7f8c14198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.050+0000 7f8c1b400700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1406a170 0x7f8c1419e860 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.050+0000 7f8c1b400700 1 --2- 172.21.15.26:0/829647027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c1406ab10 0x7f8c1419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.050+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 >> 172.21.15.26:0/829647027 conn(0x7f8c140782c0 msgr2=0x7f8c140ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.050+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 shutdown_connections 2024-06-13T06:55:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:21.050+0000 7f8c1b400700 1 -- 172.21.15.26:0/829647027 wait complete. 2024-06-13T06:55:21.051 INFO:teuthology.orchestra.run.smithi026.stderr:added data pool 4 to fsmap 2024-06-13T06:55:21.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:21 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/829647027' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-13T06:55:21.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:21 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/829647027' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-13T06:55:21.484 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:21 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/829647027' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-13T06:55:21.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:55:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:22 smithi026 ceph-mon[32041]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:22 smithi026 ceph-mon[32041]: osdmap e95: 12 total, 12 up, 12 in 2024-06-13T06:55:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:22 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/829647027' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-13T06:55:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:22 smithi026 ceph-mon[32041]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:22 smithi136 ceph-mon[37229]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:22 smithi136 ceph-mon[37229]: osdmap e95: 12 total, 12 up, 12 in 2024-06-13T06:55:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:22 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/829647027' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-13T06:55:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:22 smithi136 ceph-mon[37229]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:22.450 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:22 smithi135 ceph-mon[36276]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:22 smithi135 ceph-mon[36276]: osdmap e95: 12 total, 12 up, 12 in 2024-06-13T06:55:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:22 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/829647027' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-13T06:55:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:22 smithi135 ceph-mon[36276]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:55:23.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:23 smithi026 ceph-mon[32041]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:23.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:23 smithi026 ceph-mon[32041]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-13T06:55:23.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:23 smithi026 ceph-mon[32041]: Cluster is now healthy 2024-06-13T06:55:23.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:23 smithi136 ceph-mon[37229]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:23.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:23 smithi136 ceph-mon[37229]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-13T06:55:23.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:23 smithi136 ceph-mon[37229]: Cluster is now healthy 2024-06-13T06:55:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:23 smithi135 ceph-mon[36276]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:23 smithi135 ceph-mon[36276]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-13T06:55:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:23 smithi135 ceph-mon[36276]: Cluster is now healthy 2024-06-13T06:55:24.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.823+0000 7f125ec00700 1 -- 172.21.15.26:0/1218300379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 msgr2=0x7f12581088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.823+0000 7f125ec00700 1 --2- 172.21.15.26:0/1218300379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 0x7f12581088e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1248009990 tx=0x7f1248009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.823+0000 7f125ec00700 1 -- 172.21.15.26:0/1218300379 shutdown_connections 2024-06-13T06:55:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.823+0000 7f125ec00700 1 --2- 172.21.15.26:0/1218300379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 0x7f12581088e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.823+0000 7f125ec00700 1 --2- 172.21.15.26:0/1218300379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f125806ab30 0x7f125810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.823+0000 7f125ec00700 1 --2- 172.21.15.26:0/1218300379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f125806a170 0x7f125806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.823+0000 7f125ec00700 1 -- 172.21.15.26:0/1218300379 >> 172.21.15.26:0/1218300379 conn(0x7f1258078340 msgr2=0x7f1258078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.824+0000 7f125ec00700 1 -- 172.21.15.26:0/1218300379 shutdown_connections 2024-06-13T06:55:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.824+0000 7f125ec00700 1 -- 172.21.15.26:0/1218300379 wait complete. 2024-06-13T06:55:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.825+0000 7f125ec00700 1 Processor -- start 2024-06-13T06:55:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.825+0000 7f125ec00700 1 -- start start 2024-06-13T06:55:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.825+0000 7f125ec00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f125806a170 0x7f1258101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.825+0000 7f125ec00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f125806ab30 0x7f12581019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.825+0000 7f125ec00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 0x7f1258101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.825+0000 7f125ec00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12581127b0 con 0x7f125806ab30 2024-06-13T06:55:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.825+0000 7f125ec00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1258112630 con 0x7f1258108500 2024-06-13T06:55:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.825+0000 7f125ec00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1258112930 con 0x7f125806a170 2024-06-13T06:55:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f125806ab30 0x7f12581019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f125cc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 0x7f1258101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f125806a170 0x7f1258101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f125cc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 0x7f1258101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46064/0 (socket says 172.21.15.26:46064) 2024-06-13T06:55:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f125806ab30 0x7f12581019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36484/0 (socket says 172.21.15.26:36484) 2024-06-13T06:55:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f125cc00700 1 -- 172.21.15.26:0/1249313445 learned_addr learned my addr 172.21.15.26:0/1249313445 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257e00700 1 -- 172.21.15.26:0/1249313445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 msgr2=0x7f1258101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257e00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 0x7f1258101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257e00700 1 -- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f125806ab30 msgr2=0x7f12581019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257e00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f125806ab30 0x7f12581019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257e00700 1 -- 172.21.15.26:0/1249313445 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12581a7230 con 0x7f125806a170 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f125cc00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 0x7f1258101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.826+0000 7f1257e00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f125806a170 0x7f1258101460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1248000c00 tx=0x7f124800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.827+0000 7f1254c00700 1 -- 172.21.15.26:0/1249313445 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248026070 con 0x7f125806a170 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.827+0000 7f1254c00700 1 -- 172.21.15.26:0/1249313445 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1248003d70 con 0x7f125806a170 2024-06-13T06:55:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.827+0000 7f1254c00700 1 -- 172.21.15.26:0/1249313445 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248021410 con 0x7f125806a170 2024-06-13T06:55:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.827+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12581a74c0 con 0x7f125806a170 2024-06-13T06:55:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.827+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12581a7910 con 0x7f125806a170 2024-06-13T06:55:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.828+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125810a9e0 con 0x7f125806a170 2024-06-13T06:55:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.831+0000 7f1254c00700 1 -- 172.21.15.26:0/1249313445 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1248029070 con 0x7f125806a170 2024-06-13T06:55:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.832+0000 7f1254c00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f123c0a2c20 0x7f123c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.832+0000 7f1254c00700 1 -- 172.21.15.26:0/1249313445 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f12480d03a0 con 0x7f125806a170 2024-06-13T06:55:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.832+0000 7f1257400700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f123c0a2c20 0x7f123c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.833+0000 7f1257400700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f123c0a2c20 0x7f123c0a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1258078a40 tx=0x7f1244009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:24.834+0000 7f1254c00700 1 -- 172.21.15.26:0/1249313445 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1248030020 con 0x7f125806a170 2024-06-13T06:55:25.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.113+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1258069180 con 0x7f125806a170 2024-06-13T06:55:25.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.114+0000 7f1254c00700 1 -- 172.21.15.26:0/1249313445 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11798 (secure 0 0 0) 0x7f1248013070 con 0x7f125806a170 2024-06-13T06:55:25.114 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:55:25.114 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T06:55:20.037369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24517,"qdb_cluster":[24517]},"id":1}]} 2024-06-13T06:55:25.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.117+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f123c0a2c20 msgr2=0x7f123c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.117+0000 7f125ec00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f123c0a2c20 0x7f123c0a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1258078a40 tx=0x7f1244009260 comp rx=0 tx=0).stop 2024-06-13T06:55:25.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.117+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f125806a170 msgr2=0x7f1258101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.117+0000 7f125ec00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f125806a170 0x7f1258101460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1248000c00 tx=0x7f124800bab0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.118+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 shutdown_connections 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.118+0000 7f125ec00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f123c0a2c20 0x7f123c0a50e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.118+0000 7f125ec00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1258108500 0x7f1258101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.118+0000 7f125ec00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f125806a170 0x7f1258101460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.118+0000 7f125ec00700 1 --2- 172.21.15.26:0/1249313445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f125806ab30 0x7f12581019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.118+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 >> 172.21.15.26:0/1249313445 conn(0x7f1258078340 msgr2=0x7f1258100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.118+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 shutdown_connections 2024-06-13T06:55:25.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:25.118+0000 7f125ec00700 1 -- 172.21.15.26:0/1249313445 wait complete. 2024-06-13T06:55:25.119 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-06-13T06:55:25.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:25 smithi026 ceph-mon[32041]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:25.500 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:25 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1249313445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:25.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:25 smithi136 ceph-mon[37229]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:25.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:25 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1249313445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:25 smithi135 ceph-mon[36276]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:25 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1249313445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:26.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:55:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:27.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:27 smithi026 ceph-mon[32041]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:27.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:27 smithi136 ceph-mon[37229]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:27 smithi135 ceph-mon[36276]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:28.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-06-13T06:55:28.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-13T06:55:28.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-13T06:55:28.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-06-13T06:55:28.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-06-13T06:55:28.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-06-13T06:55:28.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-13T06:55:28.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-13T06:55:28.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-13T06:55:28.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]: dispatch 2024-06-13T06:55:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-13T06:55:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-13T06:55:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]: dispatch 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 -- 172.21.15.26:0/2879318785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 msgr2=0x7fdb7410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 --2- 172.21.15.26:0/2879318785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 0x7fdb7410f320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00b0d0 tx=0x7fdb6c00b3e0 comp rx=0 tx=0).stop 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 -- 172.21.15.26:0/2879318785 shutdown_connections 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 --2- 172.21.15.26:0/2879318785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb741084e0 0x7fdb741088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 --2- 172.21.15.26:0/2879318785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 0x7fdb7410f320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 --2- 172.21.15.26:0/2879318785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb7406a170 0x7fdb7406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 -- 172.21.15.26:0/2879318785 >> 172.21.15.26:0/2879318785 conn(0x7fdb740782c0 msgr2=0x7fdb740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 -- 172.21.15.26:0/2879318785 shutdown_connections 2024-06-13T06:55:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.822+0000 7fdb79000700 1 -- 172.21.15.26:0/2879318785 wait complete. 2024-06-13T06:55:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.824+0000 7fdb79000700 1 Processor -- start 2024-06-13T06:55:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.824+0000 7fdb79000700 1 -- start start 2024-06-13T06:55:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.824+0000 7fdb79000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb7406a170 0x7fdb7419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.824+0000 7fdb79000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 0x7fdb7419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb79000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb741084e0 0x7fdb7419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb79000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741127b0 con 0x7fdb7406ab10 2024-06-13T06:55:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb79000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74112630 con 0x7fdb7406a170 2024-06-13T06:55:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb79000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74112930 con 0x7fdb741084e0 2024-06-13T06:55:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb71400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 0x7fdb7419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb71e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb7406a170 0x7fdb7419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb71e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb7406a170 0x7fdb7419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46086/0 (socket says 172.21.15.26:46086) 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb71400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 0x7fdb7419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36500/0 (socket says 172.21.15.26:36500) 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb71e00700 1 -- 172.21.15.26:0/423685703 learned_addr learned my addr 172.21.15.26:0/423685703 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb72800700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb741084e0 0x7fdb7419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb72800700 1 -- 172.21.15.26:0/423685703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb7406a170 msgr2=0x7fdb7419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb72800700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb7406a170 0x7fdb7419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb72800700 1 -- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 msgr2=0x7fdb7419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb72800700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 0x7fdb7419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb72800700 1 -- 172.21.15.26:0/423685703 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7419fdc0 con 0x7fdb741084e0 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb71e00700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb7406a170 0x7fdb7419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.825+0000 7fdb71400700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 0x7fdb7419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.826+0000 7fdb72800700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb741084e0 0x7fdb7419f680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00d010 tx=0x7fdb6c009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.826+0000 7fdb6aa00700 1 -- 172.21.15.26:0/423685703 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c003a00 con 0x7fdb741084e0 2024-06-13T06:55:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.826+0000 7fdb6aa00700 1 -- 172.21.15.26:0/423685703 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c022070 con 0x7fdb741084e0 2024-06-13T06:55:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.826+0000 7fdb6aa00700 1 -- 172.21.15.26:0/423685703 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c01b6c0 con 0x7fdb741084e0 2024-06-13T06:55:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.826+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb741a0050 con 0x7fdb741084e0 2024-06-13T06:55:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.826+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741a7930 con 0x7fdb741084e0 2024-06-13T06:55:28.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.827+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb74103b90 con 0x7fdb741084e0 2024-06-13T06:55:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.830+0000 7fdb6aa00700 1 -- 172.21.15.26:0/423685703 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdb6c04ed10 con 0x7fdb741084e0 2024-06-13T06:55:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.831+0000 7fdb6aa00700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb580a2ca0 0x7fdb580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.831+0000 7fdb6aa00700 1 -- 172.21.15.26:0/423685703 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7fdb6c0cffe0 con 0x7fdb741084e0 2024-06-13T06:55:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.831+0000 7fdb71e00700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb580a2ca0 0x7fdb580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.832+0000 7fdb71e00700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb580a2ca0 0x7fdb580a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009680 tx=0x7fdb64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:28.833+0000 7fdb6aa00700 1 -- 172.21.15.26:0/423685703 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdb6c02c020 con 0x7fdb741084e0 2024-06-13T06:55:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.064+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb7402dca0 con 0x7fdb741084e0 2024-06-13T06:55:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.066+0000 7fdb6aa00700 1 -- 172.21.15.26:0/423685703 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24656 (secure 0 0 0) 0x7fdb6c09cc90 con 0x7fdb741084e0 2024-06-13T06:55:29.066 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:55:29.066 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:55:28.335166+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:55:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb580a2ca0 msgr2=0x7fdb580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb580a2ca0 0x7fdb580a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009680 tx=0x7fdb64009260 comp rx=0 tx=0).stop 2024-06-13T06:55:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb741084e0 msgr2=0x7fdb7419f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb741084e0 0x7fdb7419f680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00d010 tx=0x7fdb6c009810 comp rx=0 tx=0).stop 2024-06-13T06:55:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 shutdown_connections 2024-06-13T06:55:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb580a2ca0 0x7fdb580a5160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb7406a170 0x7fdb7419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb741084e0 0x7fdb7419f680 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 --2- 172.21.15.26:0/423685703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb7406ab10 0x7fdb7419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.069+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 >> 172.21.15.26:0/423685703 conn(0x7fdb740782c0 msgr2=0x7fdb7410d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.070+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 shutdown_connections 2024-06-13T06:55:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:29.070+0000 7fdb79000700 1 -- 172.21.15.26:0/423685703 wait complete. 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-13T06:55:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-13T06:55:29.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]': finished 2024-06-13T06:55:29.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: osdmap e96: 12 total, 12 up, 12 in 2024-06-13T06:55:29.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:29 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/423685703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:29.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]': finished 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: osdmap e96: 12 total, 12 up, 12 in 2024-06-13T06:55:29.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:29 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/423685703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-06-13T06:55:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-13T06:55:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-13T06:55:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-06-13T06:55:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-06-13T06:55:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-06-13T06:55:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-13T06:55:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-13T06:55:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-13T06:55:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]': finished 2024-06-13T06:55:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: osdmap e96: 12 total, 12 up, 12 in 2024-06-13T06:55:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:29 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/423685703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:30.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:55:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:30 smithi136 ceph-mon[37229]: osdmap e97: 12 total, 12 up, 12 in 2024-06-13T06:55:30.684 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:30 smithi135 ceph-mon[36276]: osdmap e97: 12 total, 12 up, 12 in 2024-06-13T06:55:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:30 smithi026 ceph-mon[32041]: osdmap e97: 12 total, 12 up, 12 in 2024-06-13T06:55:31.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:31 smithi136 ceph-mon[37229]: pgmap v358: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-06-13T06:55:31.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:31 smithi026 ceph-mon[32041]: pgmap v358: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-06-13T06:55:31.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:31 smithi135 ceph-mon[36276]: pgmap v358: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-06-13T06:55:32.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 -- 172.21.15.26:0/3878253321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0360100ce0 msgr2=0x7f03601010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:32.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 --2- 172.21.15.26:0/3878253321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0360100ce0 0x7f03601010c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0350009990 tx=0x7f0350009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:32.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 -- 172.21.15.26:0/3878253321 shutdown_connections 2024-06-13T06:55:32.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 --2- 172.21.15.26:0/3878253321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0360113090 0x7f0360115530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:32.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 --2- 172.21.15.26:0/3878253321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0360100ce0 0x7f03601010c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:32.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 --2- 172.21.15.26:0/3878253321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0360101690 0x7f03601129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:32.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 -- 172.21.15.26:0/3878253321 >> 172.21.15.26:0/3878253321 conn(0x7f0360077a00 msgr2=0x7f03600fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:32.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 -- 172.21.15.26:0/3878253321 shutdown_connections 2024-06-13T06:55:32.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.817+0000 7f0365200700 1 -- 172.21.15.26:0/3878253321 wait complete. 2024-06-13T06:55:32.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.818+0000 7f0365200700 1 Processor -- start 2024-06-13T06:55:32.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.818+0000 7f0365200700 1 -- start start 2024-06-13T06:55:32.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f0365200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0360100ce0 0x7f036019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f0365200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0360101690 0x7f036019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f0365200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0360113090 0x7f03601a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f0365200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0360118da0 con 0x7f0360101690 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f0365200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0360118c20 con 0x7f0360113090 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f0365200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0360118f20 con 0x7f0360100ce0 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f035de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0360100ce0 0x7f036019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f035d400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0360101690 0x7f036019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f035de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0360100ce0 0x7f036019f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35744/0 (socket says 172.21.15.26:35744) 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f035de00700 1 -- 172.21.15.26:0/2328666981 learned_addr learned my addr 172.21.15.26:0/2328666981 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.819+0000 7f035e800700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0360113090 0x7f03601a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f035de00700 1 -- 172.21.15.26:0/2328666981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0360113090 msgr2=0x7f03601a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:32.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f035de00700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0360113090 0x7f03601a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f035de00700 1 -- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0360101690 msgr2=0x7f036019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f035de00700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0360101690 0x7f036019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f035de00700 1 -- 172.21.15.26:0/2328666981 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03601a42a0 con 0x7f0360100ce0 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f035e800700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0360113090 0x7f03601a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f035de00700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0360100ce0 0x7f036019f290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f035000bae0 tx=0x7f035000bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f0356a00700 1 -- 172.21.15.26:0/2328666981 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350026070 con 0x7f0360100ce0 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f0356a00700 1 -- 172.21.15.26:0/2328666981 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0350003d70 con 0x7f0360100ce0 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.820+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03601a4530 con 0x7f0360100ce0 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.821+0000 7f0356a00700 1 -- 172.21.15.26:0/2328666981 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350021410 con 0x7f0360100ce0 2024-06-13T06:55:32.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.821+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03601abe10 con 0x7f0360100ce0 2024-06-13T06:55:32.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.822+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0360069180 con 0x7f0360100ce0 2024-06-13T06:55:32.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.824+0000 7f0356a00700 1 -- 172.21.15.26:0/2328666981 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0350029070 con 0x7f0360100ce0 2024-06-13T06:55:32.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.825+0000 7f0356a00700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f03440a2c20 0x7f03440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:32.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.825+0000 7f0356a00700 1 -- 172.21.15.26:0/2328666981 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f03500d00c0 con 0x7f0360100ce0 2024-06-13T06:55:32.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.825+0000 7f035d400700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f03440a2c20 0x7f03440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:32.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.826+0000 7f035d400700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f03440a2c20 0x7f03440a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0360112990 tx=0x7f034c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:32.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:32.829+0000 7f0356a00700 1 -- 172.21.15.26:0/2328666981 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f035005d850 con 0x7f0360100ce0 2024-06-13T06:55:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.060+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f036002dca0 con 0x7f0360100ce0 2024-06-13T06:55:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.062+0000 7f0356a00700 1 -- 172.21.15.26:0/2328666981 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24656 (secure 0 0 0) 0x7f0350013070 con 0x7f0360100ce0 2024-06-13T06:55:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:55:33.063 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":97,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:55:29.352302+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:55:33.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.065+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f03440a2c20 msgr2=0x7f03440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:33.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.065+0000 7f0365200700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f03440a2c20 0x7f03440a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0360112990 tx=0x7f034c009260 comp rx=0 tx=0).stop 2024-06-13T06:55:33.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0360100ce0 msgr2=0x7f036019f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0360100ce0 0x7f036019f290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f035000bae0 tx=0x7f035000bb10 comp rx=0 tx=0).stop 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 shutdown_connections 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f03440a2c20 0x7f03440a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0360113090 0x7f03601a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0360100ce0 0x7f036019f290 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 --2- 172.21.15.26:0/2328666981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0360101690 0x7f036019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 >> 172.21.15.26:0/2328666981 conn(0x7f0360077a00 msgr2=0x7f0360111940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 shutdown_connections 2024-06-13T06:55:33.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:33.066+0000 7f0365200700 1 -- 172.21.15.26:0/2328666981 wait complete. 2024-06-13T06:55:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:33 smithi026 ceph-mon[32041]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-06-13T06:55:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:33 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2328666981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:33 smithi135 ceph-mon[36276]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-06-13T06:55:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:33 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2328666981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:33 smithi136 ceph-mon[37229]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-06-13T06:55:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:33 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2328666981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:34.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:55:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:35 smithi026 ceph-mon[32041]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-06-13T06:55:35.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:35 smithi135 ceph-mon[36276]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-06-13T06:55:35.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:35 smithi136 ceph-mon[37229]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-06-13T06:55:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 -- 172.21.15.26:0/1241867360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf3406ab60 msgr2=0x7faf3406afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 --2- 172.21.15.26:0/1241867360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf3406ab60 0x7faf3406afe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faf200098f0 tx=0x7faf20009c00 comp rx=0 tx=0).stop 2024-06-13T06:55:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 -- 172.21.15.26:0/1241867360 shutdown_connections 2024-06-13T06:55:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 --2- 172.21.15.26:0/1241867360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf341084f0 0x7faf341088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 --2- 172.21.15.26:0/1241867360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf3406b520 0x7faf3410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 --2- 172.21.15.26:0/1241867360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf3406ab60 0x7faf3406afe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 -- 172.21.15.26:0/1241867360 >> 172.21.15.26:0/1241867360 conn(0x7faf34078340 msgr2=0x7faf34078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 -- 172.21.15.26:0/1241867360 shutdown_connections 2024-06-13T06:55:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.850+0000 7faf38a00700 1 -- 172.21.15.26:0/1241867360 wait complete. 2024-06-13T06:55:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.852+0000 7faf38a00700 1 Processor -- start 2024-06-13T06:55:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.852+0000 7faf38a00700 1 -- start start 2024-06-13T06:55:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.852+0000 7faf38a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf3406ab60 0x7faf3419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.852+0000 7faf38a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf3406b520 0x7faf3419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.852+0000 7faf38a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf341084f0 0x7faf3419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.852+0000 7faf38a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf34112780 con 0x7faf341084f0 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.852+0000 7faf38a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf34112600 con 0x7faf3406ab60 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.852+0000 7faf38a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf34112900 con 0x7faf3406b520 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf32800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf341084f0 0x7faf3419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf3406b520 0x7faf3419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf3406ab60 0x7faf3419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf3406b520 0x7faf3419b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35766/0 (socket says 172.21.15.26:35766) 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 -- 172.21.15.26:0/258217939 learned_addr learned my addr 172.21.15.26:0/258217939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 -- 172.21.15.26:0/258217939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf3406ab60 msgr2=0x7faf3419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf3406ab60 0x7faf3419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 -- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf341084f0 msgr2=0x7faf3419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf341084f0 0x7faf3419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 -- 172.21.15.26:0/258217939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3419fec0 con 0x7faf3406b520 2024-06-13T06:55:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.853+0000 7faf31400700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf3406b520 0x7faf3419b3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faf20009230 tx=0x7faf2000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.854+0000 7faf2aa00700 1 -- 172.21.15.26:0/258217939 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20005180 con 0x7faf3406b520 2024-06-13T06:55:36.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.854+0000 7faf2aa00700 1 -- 172.21.15.26:0/258217939 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf2000b560 con 0x7faf3406b520 2024-06-13T06:55:36.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.854+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf341a0150 con 0x7faf3406b520 2024-06-13T06:55:36.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.854+0000 7faf2aa00700 1 -- 172.21.15.26:0/258217939 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20003a70 con 0x7faf3406b520 2024-06-13T06:55:36.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.854+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a7a30 con 0x7faf3406b520 2024-06-13T06:55:36.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.855+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf34103bd0 con 0x7faf3406b520 2024-06-13T06:55:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.858+0000 7faf2aa00700 1 -- 172.21.15.26:0/258217939 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faf20029060 con 0x7faf3406b520 2024-06-13T06:55:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.859+0000 7faf2aa00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf180a2c20 0x7faf180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.859+0000 7faf2aa00700 1 -- 172.21.15.26:0/258217939 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7faf200d02b0 con 0x7faf3406b520 2024-06-13T06:55:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.860+0000 7faf31e00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf180a2c20 0x7faf180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.860+0000 7faf31e00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf180a2c20 0x7faf180a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf24009be0 tx=0x7faf24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:36.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:36.861+0000 7faf2aa00700 1 -- 172.21.15.26:0/258217939 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faf2005d850 con 0x7faf3406b520 2024-06-13T06:55:37.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.139+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf34069180 con 0x7faf3406b520 2024-06-13T06:55:37.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.140+0000 7faf2aa00700 1 -- 172.21.15.26:0/258217939 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11798 (secure 0 0 0) 0x7faf20013030 con 0x7faf3406b520 2024-06-13T06:55:37.140 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:55:37.140 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T06:55:20.037369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24517,"qdb_cluster":[24517]},"id":1}]} 2024-06-13T06:55:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf180a2c20 msgr2=0x7faf180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf180a2c20 0x7faf180a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf24009be0 tx=0x7faf24009260 comp rx=0 tx=0).stop 2024-06-13T06:55:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf3406b520 msgr2=0x7faf3419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf3406b520 0x7faf3419b3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faf20009230 tx=0x7faf2000fa70 comp rx=0 tx=0).stop 2024-06-13T06:55:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 shutdown_connections 2024-06-13T06:55:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf180a2c20 0x7faf180a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf3406ab60 0x7faf3419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf341084f0 0x7faf3419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 --2- 172.21.15.26:0/258217939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf3406b520 0x7faf3419b3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 >> 172.21.15.26:0/258217939 conn(0x7faf34078340 msgr2=0x7faf3410d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.144+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 shutdown_connections 2024-06-13T06:55:37.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:37.145+0000 7faf38a00700 1 -- 172.21.15.26:0/258217939 wait complete. 2024-06-13T06:55:37.145 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-06-13T06:55:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:37 smithi026 ceph-mon[32041]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-06-13T06:55:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:37 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/258217939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:37 smithi135 ceph-mon[36276]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-06-13T06:55:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:37 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/258217939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:37 smithi136 ceph-mon[37229]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-06-13T06:55:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:37 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/258217939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:38.032 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:55:38.494 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:39.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:39 smithi026 ceph-mon[32041]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 164 B/s, 0 objects/s recovering 2024-06-13T06:55:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:39 smithi135 ceph-mon[36276]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 164 B/s, 0 objects/s recovering 2024-06-13T06:55:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:39 smithi136 ceph-mon[37229]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 164 B/s, 0 objects/s recovering 2024-06-13T06:55:40.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.825+0000 7f1655e00700 1 -- 172.21.15.26:0/3020079355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1650108500 msgr2=0x7f16501088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:40.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.825+0000 7f1655e00700 1 --2- 172.21.15.26:0/3020079355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1650108500 0x7f16501088e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16400098f0 tx=0x7f1640009c00 comp rx=0 tx=0).stop 2024-06-13T06:55:40.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.825+0000 7f1655e00700 1 -- 172.21.15.26:0/3020079355 shutdown_connections 2024-06-13T06:55:40.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.825+0000 7f1655e00700 1 --2- 172.21.15.26:0/3020079355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f165006ab30 0x7f165010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:40.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.825+0000 7f1655e00700 1 --2- 172.21.15.26:0/3020079355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1650108500 0x7f16501088e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:40.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.825+0000 7f1655e00700 1 --2- 172.21.15.26:0/3020079355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165006a170 0x7f165006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:40.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.825+0000 7f1655e00700 1 -- 172.21.15.26:0/3020079355 >> 172.21.15.26:0/3020079355 conn(0x7f1650078340 msgr2=0x7f1650078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:40.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.825+0000 7f1655e00700 1 -- 172.21.15.26:0/3020079355 shutdown_connections 2024-06-13T06:55:40.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.826+0000 7f1655e00700 1 -- 172.21.15.26:0/3020079355 wait complete. 2024-06-13T06:55:40.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.827+0000 7f1655e00700 1 Processor -- start 2024-06-13T06:55:40.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.827+0000 7f1655e00700 1 -- start start 2024-06-13T06:55:40.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.827+0000 7f1655e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f165006a170 0x7f165019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:40.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.827+0000 7f1655e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f165006ab30 0x7f165019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:40.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.827+0000 7f1655e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1650108500 0x7f165019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:40.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.827+0000 7f1655e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f16501127d0 con 0x7f1650108500 2024-06-13T06:55:40.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.827+0000 7f1655e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1650112650 con 0x7f165006ab30 2024-06-13T06:55:40.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.827+0000 7f1655e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1650112950 con 0x7f165006a170 2024-06-13T06:55:40.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f165006a170 0x7f165019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:40.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f165006a170 0x7f165019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35780/0 (socket says 172.21.15.26:35780) 2024-06-13T06:55:40.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164ea00700 1 -- 172.21.15.26:0/1718214291 learned_addr learned my addr 172.21.15.26:0/1718214291 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:40.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164f400700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1650108500 0x7f165019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:40.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164e000700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f165006ab30 0x7f165019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164e000700 1 -- 172.21.15.26:0/1718214291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f165006a170 msgr2=0x7f165019af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164e000700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f165006a170 0x7f165019af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164e000700 1 -- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1650108500 msgr2=0x7f165019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164e000700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1650108500 0x7f165019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.828+0000 7f164e000700 1 -- 172.21.15.26:0/1718214291 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165019ff30 con 0x7f165006ab30 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.829+0000 7f164e000700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f165006ab30 0x7f165019b460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f163c00e9c0 tx=0x7f163c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.829+0000 7f1654a00700 1 -- 172.21.15.26:0/1718214291 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c00c9c0 con 0x7f165006ab30 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.829+0000 7f1654a00700 1 -- 172.21.15.26:0/1718214291 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163c00cb60 con 0x7f165006ab30 2024-06-13T06:55:40.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.829+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16501a0220 con 0x7f165006ab30 2024-06-13T06:55:40.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.829+0000 7f1654a00700 1 -- 172.21.15.26:0/1718214291 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c018680 con 0x7f165006ab30 2024-06-13T06:55:40.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.829+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16501a7aa0 con 0x7f165006ab30 2024-06-13T06:55:40.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.830+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1650069180 con 0x7f165006ab30 2024-06-13T06:55:40.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.832+0000 7f1654a00700 1 -- 172.21.15.26:0/1718214291 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f163c00cd10 con 0x7f165006ab30 2024-06-13T06:55:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.833+0000 7f1654a00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f16340a2ca0 0x7f16340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.834+0000 7f1654a00700 1 -- 172.21.15.26:0/1718214291 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f163c0cf7f0 con 0x7f165006ab30 2024-06-13T06:55:40.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.834+0000 7f164ea00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f16340a2ca0 0x7f16340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:40.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.834+0000 7f164ea00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f16340a2ca0 0x7f16340a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f164000c010 tx=0x7f1640019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:40.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:40.836+0000 7f1654a00700 1 -- 172.21.15.26:0/1718214291 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f163c09c390 con 0x7f165006ab30 2024-06-13T06:55:41.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.069+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16501a7cd0 con 0x7f165006ab30 2024-06-13T06:55:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.071+0000 7f1654a00700 1 -- 172.21.15.26:0/1718214291 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24656 (secure 0 0 0) 0x7f163c09bc90 con 0x7f165006ab30 2024-06-13T06:55:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:55:41.071 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":97,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:55:29.352302+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:55:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f16340a2ca0 msgr2=0x7f16340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f16340a2ca0 0x7f16340a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f164000c010 tx=0x7f1640019040 comp rx=0 tx=0).stop 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f165006ab30 msgr2=0x7f165019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f165006ab30 0x7f165019b460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f163c00e9c0 tx=0x7f163c00ecd0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 shutdown_connections 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f16340a2ca0 0x7f16340a5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1650108500 0x7f165019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f165006a170 0x7f165019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 --2- 172.21.15.26:0/1718214291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f165006ab30 0x7f165019b460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 >> 172.21.15.26:0/1718214291 conn(0x7f1650078340 msgr2=0x7f165010d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.074+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 shutdown_connections 2024-06-13T06:55:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:41.075+0000 7f1655e00700 1 -- 172.21.15.26:0/1718214291 wait complete. 2024-06-13T06:55:41.679 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:41 smithi136 ceph-mon[37229]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 149 B/s, 0 objects/s recovering 2024-06-13T06:55:41.679 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:41 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1718214291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:41.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:41 smithi026 ceph-mon[32041]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 149 B/s, 0 objects/s recovering 2024-06-13T06:55:41.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:41 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1718214291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:41 smithi135 ceph-mon[36276]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 149 B/s, 0 objects/s recovering 2024-06-13T06:55:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:41 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1718214291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:42.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-13T06:55:01.881338+0000', 'modified': '2024-06-13T06:55:20.037369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24517': {'gid': 24517, 'name': 'g', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.26:6839/173910348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 173910348}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 173910348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24517, 'qdb_cluster': [24517]} 2024-06-13T06:55:42.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-13T06:55:42.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.g status 2024-06-13T06:55:42.642 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:43.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:43 smithi026 ceph-mon[32041]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-06-13T06:55:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:43 smithi135 ceph-mon[36276]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-06-13T06:55:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:43 smithi136 ceph-mon[37229]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-06-13T06:55:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1918351010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 msgr2=0x7f7cd01112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1918351010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 0x7f7cd01112e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009990 tx=0x7f7cc8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1918351010 shutdown_connections 2024-06-13T06:55:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1918351010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 0x7f7cd01112e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1918351010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cd00746c0 0x7f7cd0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1918351010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd0074fe0 0x7f7cd010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1918351010 >> 172.21.15.26:0/1918351010 conn(0x7f7cd00795f0 msgr2=0x7f7cd0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1918351010 shutdown_connections 2024-06-13T06:55:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.148+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1918351010 wait complete. 2024-06-13T06:55:45.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.149+0000 7f7cd7e00700 1 Processor -- start 2024-06-13T06:55:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.149+0000 7f7cd7e00700 1 -- start start 2024-06-13T06:55:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00746c0 0x7f7cd019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cd0074fe0 0x7f7cd019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd7e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 0x7f7cd019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd7e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0114a00 con 0x7f7cd010eef0 2024-06-13T06:55:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd7e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0114880 con 0x7f7cd00746c0 2024-06-13T06:55:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd7e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0114b80 con 0x7f7cd0074fe0 2024-06-13T06:55:45.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd5e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 0x7f7cd019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00746c0 0x7f7cd019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd4a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cd0074fe0 0x7f7cd019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00746c0 0x7f7cd019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39714/0 (socket says 172.21.15.26:39714) 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd4a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cd0074fe0 0x7f7cd019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53280/0 (socket says 172.21.15.26:53280) 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.150+0000 7f7cd5400700 1 -- 172.21.15.26:0/1513933361 learned_addr learned my addr 172.21.15.26:0/1513933361 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd5e00700 1 -- 172.21.15.26:0/1513933361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cd0074fe0 msgr2=0x7f7cd019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cd0074fe0 0x7f7cd019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd5e00700 1 -- 172.21.15.26:0/1513933361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00746c0 msgr2=0x7f7cd019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00746c0 0x7f7cd019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd5e00700 1 -- 172.21.15.26:0/1513933361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd019fe00 con 0x7f7cd010eef0 2024-06-13T06:55:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd4a00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cd0074fe0 0x7f7cd019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:45.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 0x7f7cd019f6c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cd006acc0 tx=0x7f7cc80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:45.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cce000700 1 -- 172.21.15.26:0/1513933361 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc800f880 con 0x7f7cd010eef0 2024-06-13T06:55:45.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1513933361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd01a0090 con 0x7f7cd010eef0 2024-06-13T06:55:45.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1513933361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd01a7970 con 0x7f7cd010eef0 2024-06-13T06:55:45.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cce000700 1 -- 172.21.15.26:0/1513933361 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc8004830 con 0x7f7cd010eef0 2024-06-13T06:55:45.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.151+0000 7f7cce000700 1 -- 172.21.15.26:0/1513933361 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8005420 con 0x7f7cd010eef0 2024-06-13T06:55:45.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.155+0000 7f7cce000700 1 -- 172.21.15.26:0/1513933361 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7cc8009d10 con 0x7f7cd010eef0 2024-06-13T06:55:45.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.156+0000 7f7cce000700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7cb80a2ca0 0x7f7cb80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.156+0000 7f7ce287ab80 1 Processor -- start 2024-06-13T06:55:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.156+0000 7f7ce287ab80 1 -- start start 2024-06-13T06:55:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.156+0000 7f7cce000700 1 -- 172.21.15.26:0/1513933361 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f7cc80cfc10 con 0x7f7cd010eef0 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.156+0000 7f7cd5400700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7cb80a2ca0 0x7f7cb80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e6a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e6a560 0x55a5c4e6a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4d799c0 con 0x55a5c4e6a560 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4dbcab0 con 0x55a5c4de6df0 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4d778a0 con 0x55a5c4e752b0 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7cd4a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e6a560 0x55a5c4e6a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7cd5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7cd5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e776a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53296/0 (socket says 172.21.15.26:53296) 2024-06-13T06:55:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7cd5e00700 1 -- 172.21.15.26:0/2089415991 learned_addr learned my addr 172.21.15.26:0/2089415991 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7cd5400700 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e6a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7cd5e00700 1 -- 172.21.15.26:0/2089415991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 msgr2=0x55a5c4e6a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.157+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e6a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cd5e00700 1 -- 172.21.15.26:0/2089415991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e6a560 msgr2=0x55a5c4e6a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e6a560 0x55a5c4e6a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cd5e00700 1 -- 172.21.15.26:0/2089415991 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c4d64f40 con 0x55a5c4e752b0 2024-06-13T06:55:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cd5400700 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e6a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:55:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cd5400700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7cb80a2ca0 0x7f7cb80a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009c10 tx=0x7f7cc400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e776a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80985a0 tx=0x7f7cc802e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74a8fb6d31837610 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cc2a00700 1 -- 172.21.15.26:0/2089415991 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc809c140 con 0x55a5c4e752b0 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cc2a00700 1 -- 172.21.15.26:0/2089415991 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc80a04f0 con 0x55a5c4e752b0 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.158+0000 7f7cc2a00700 1 -- 172.21.15.26:0/2089415991 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc809c3a0 con 0x55a5c4e752b0 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2089415991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 msgr2=0x55a5c4e776a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e776a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80985a0 tx=0x7f7cc802e000 comp rx=0 tx=0).stop 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2089415991 shutdown_connections 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e6a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e776a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2089415991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e6a560 0x55a5c4e6a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2089415991 >> 172.21.15.26:0/2089415991 conn(0x55a5c4d97120 msgr2=0x55a5c4e66460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2089415991 shutdown_connections 2024-06-13T06:55:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.159+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2089415991 wait complete. 2024-06-13T06:55:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.160+0000 7f7ce287ab80 1 -- start start 2024-06-13T06:55:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.160+0000 7f7ce287ab80 1 Processor -- start 2024-06-13T06:55:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e6a560 0x55a5c4e71c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e72190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4d58f10 con 0x55a5c4e752b0 2024-06-13T06:55:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4db8a70 con 0x55a5c4de6df0 2024-06-13T06:55:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4e77d20 con 0x55a5c4e6a560 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7ce287ab80 10 client.0 _ll_register_callbacks cb 0x55a5c4d566d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e72190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd4a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e6a560 0x55a5c4e71c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e72190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53586/0 (socket says 172.21.15.26:53586) 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5e00700 1 -- 172.21.15.26:0/3891656307 learned_addr learned my addr 172.21.15.26:0/3891656307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5e00700 1 -- 172.21.15.26:0/3891656307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e6a560 msgr2=0x55a5c4e71c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e6a560 0x55a5c4e71c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5e00700 1 -- 172.21.15.26:0/3891656307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 msgr2=0x55a5c4e74950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e74950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5e00700 1 -- 172.21.15.26:0/3891656307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c4d0b880 con 0x55a5c4e752b0 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd5400700 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e74950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.161+0000 7f7cd4a00700 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e6a560 0x55a5c4e71c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.162+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e72190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80098e0 tx=0x7f7cc8098510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84d05cc503e6eea server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.162+0000 7f7cb7400700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.162+0000 7f7cb7400700 1 -- 172.21.15.26:0/3891656307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc809c140 con 0x55a5c4e752b0 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.162+0000 7f7cb7400700 1 -- 172.21.15.26:0/3891656307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f7cc80a01b0 con 0x55a5c4e752b0 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.162+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3891656307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc80a04f0 con 0x55a5c4e752b0 2024-06-13T06:55:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.162+0000 7f7cb7400700 1 -- 172.21.15.26:0/3891656307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7cc809c3a0 con 0x55a5c4e752b0 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.162+0000 7f7ce287ab80 10 client.14511 fetch_fsmap learned FSMap version 17 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.162+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3891656307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a5c4e82380 con 0x55a5c4e752b0 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7cb7400700 1 -- 172.21.15.26:0/3891656307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc80a04f0 con 0x55a5c4e752b0 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7cb7400700 1 -- 172.21.15.26:0/3891656307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f7cc809b590 con 0x55a5c4e752b0 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7ce287ab80 10 client.14511 fetch_fsmap finished waiting for FSMap version 17 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7ce287ab80 10 client.14511 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7ce287ab80 20 client.14511 populate_metadata read hostname 'smithi026' 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a5c4e84790 0x55a5c4e85c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7ce287ab80 4 client.14511 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:55:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3891656307 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5c4dc9fb0 con 0x55a5c4e84790 2024-06-13T06:55:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.163+0000 7f7cd5400700 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a5c4e84790 0x55a5c4e85c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.164+0000 7f7cd5400700 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a5c4e84790 0x55a5c4e85c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e00c50d71103503 server_cookie=e9db782ce7f38606 in_seq=0 out_seq=0 2024-06-13T06:55:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.164+0000 7f7cb7400700 10 client.14511 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T06:55:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.165+0000 7f7cb7400700 1 -- 172.21.15.26:0/3891656307 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a5c4dc9fb0 con 0x55a5c4e84790 2024-06-13T06:55:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.165+0000 7f7cb7400700 10 client.14511 handle_command_reply: tid=0 2024-06-13T06:55:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.165+0000 7f7ce287ab80 1 client.14511 shutdown 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.165+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3891656307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 msgr2=0x55a5c4e72190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e72190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80098e0 tx=0x7f7cc8098510 comp rx=0 tx=0).stop 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3891656307 shutdown_connections 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a5c4e84790 0x55a5c4e85c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4de6df0 0x55a5c4e74950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e72190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3891656307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e6a560 0x55a5c4e71c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3891656307 >> 172.21.15.26:0/3891656307 conn(0x55a5c4d97120 msgr2=0x55a5c4e656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7cb7400700 0 client.14511 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3891656307 shutdown_connections 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3891656307 wait complete. 2024-06-13T06:55:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.166+0000 7f7ce287ab80 20 client.14511 trim_cache size 0 max 16384 2024-06-13T06:55:45.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7ce287ab80 1 Processor -- start 2024-06-13T06:55:45.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7ce287ab80 1 -- start start 2024-06-13T06:55:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4de6df0 0x55a5c4e85c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4e83f40 con 0x55a5c4de6df0 2024-06-13T06:55:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4e84610 con 0x55a5c4e6a560 2024-06-13T06:55:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4e84ce0 con 0x55a5c4e752b0 2024-06-13T06:55:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7cd5400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4de6df0 0x55a5c4e85c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7cd5400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4de6df0 0x55a5c4e85c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53602/0 (socket says 172.21.15.26:53602) 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7cd5400700 1 -- 172.21.15.26:0/3507663257 learned_addr learned my addr 172.21.15.26:0/3507663257 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.190+0000 7f7cd4a00700 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cd5400700 1 -- 172.21.15.26:0/3507663257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 msgr2=0x55a5c4e836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cd5400700 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cd5400700 1 -- 172.21.15.26:0/3507663257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 msgr2=0x55a5c4e840c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cd5400700 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e840c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cd5400700 1 -- 172.21.15.26:0/3507663257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c4e6bb80 con 0x55a5c4de6df0 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cd5400700 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4de6df0 0x55a5c4e85c00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009fd0 tx=0x7f7cc401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9da7f8c88719689 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:45.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cc2000700 1 -- 172.21.15.26:0/3507663257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4019110 con 0x55a5c4de6df0 2024-06-13T06:55:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cc2000700 1 -- 172.21.15.26:0/3507663257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc40192b0 con 0x55a5c4de6df0 2024-06-13T06:55:45.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.191+0000 7f7cc2000700 1 -- 172.21.15.26:0/3507663257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4008800 con 0x55a5c4de6df0 2024-06-13T06:55:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3507663257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4de6df0 msgr2=0x55a5c4e85c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4de6df0 0x55a5c4e85c00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009fd0 tx=0x7f7cc401c000 comp rx=0 tx=0).stop 2024-06-13T06:55:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3507663257 shutdown_connections 2024-06-13T06:55:45.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4e752b0 0x55a5c4e836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4de6df0 0x55a5c4e85c00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/3507663257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e840c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3507663257 >> 172.21.15.26:0/3507663257 conn(0x55a5c4d97120 msgr2=0x55a5c4e6d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3507663257 shutdown_connections 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.192+0000 7f7ce287ab80 1 -- 172.21.15.26:0/3507663257 wait complete. 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.193+0000 7f7ce287ab80 1 -- start start 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.193+0000 7f7ce287ab80 1 Processor -- start 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.193+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4de6df0 0x55a5c4e75ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.193+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e76430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7ce287ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e76970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4e85a80 con 0x55a5c4e752b0 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4e853b0 con 0x55a5c4e6a560 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7ce287ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4e86150 con 0x55a5c4de6df0 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7ce287ab80 10 client.0 _ll_register_callbacks cb 0x55a5c4d566d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd4a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e76430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd5e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e76970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd4a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e76430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39748/0 (socket says 172.21.15.26:39748) 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd4a00700 1 -- 172.21.15.26:0/2031420744 learned_addr learned my addr 172.21.15.26:0/2031420744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:45.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd5400700 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4de6df0 0x55a5c4e75ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd5e00700 1 -- 172.21.15.26:0/2031420744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4de6df0 msgr2=0x55a5c4e75ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4de6df0 0x55a5c4e75ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd5e00700 1 -- 172.21.15.26:0/2031420744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 msgr2=0x55a5c4e76430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e76430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd5e00700 1 -- 172.21.15.26:0/2031420744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c4e8e170 con 0x55a5c4e752b0 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.194+0000 7f7cd4a00700 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e76430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.195+0000 7f7cd5e00700 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e76970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7cc809a3f0 tx=0x7f7cc809a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ad30c5d007ce42 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.195+0000 7f7cb7400700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T06:55:45.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.195+0000 7f7cb7400700 1 -- 172.21.15.26:0/2031420744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc809c140 con 0x55a5c4e752b0 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.195+0000 7f7cb7400700 1 -- 172.21.15.26:0/2031420744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f7cc80b0250 con 0x55a5c4e752b0 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.195+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2031420744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc40192b0 con 0x55a5c4e752b0 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.195+0000 7f7cb7400700 1 -- 172.21.15.26:0/2031420744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7cc80a04f0 con 0x55a5c4e752b0 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.195+0000 7f7ce287ab80 10 client.14517 fetch_fsmap learned FSMap version 17 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.195+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2031420744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a5c4e8e170 con 0x55a5c4e752b0 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7cb7400700 1 -- 172.21.15.26:0/2031420744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc40192b0 con 0x55a5c4e752b0 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7cb7400700 1 -- 172.21.15.26:0/2031420744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f7cc809b000 con 0x55a5c4e752b0 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7ce287ab80 10 client.14517 fetch_fsmap finished waiting for FSMap version 17 2024-06-13T06:55:45.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7ce287ab80 10 client.14517 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7ce287ab80 20 client.14517 populate_metadata read hostname 'smithi026' 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a5c4ef3de0 0x55a5c4ef2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7ce287ab80 4 client.14517 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2031420744 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "status"}) -- 0x55a5c4dc4530 con 0x55a5c4ef3de0 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.196+0000 7f7cd5400700 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a5c4ef3de0 0x55a5c4ef2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7cd5400700 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a5c4ef3de0 0x55a5c4ef2af0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f772b43b396bfff server_cookie=bc987e818cd3a493 in_seq=0 out_seq=0 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7cb7400700 10 client.14517 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7cb7400700 1 -- 172.21.15.26:0/2031420744 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a5c4dc4530 con 0x55a5c4ef3de0 2024-06-13T06:55:45.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7cb7400700 10 client.14517 handle_command_reply: tid=0 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 client.14517 shutdown 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2031420744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 msgr2=0x55a5c4e76970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e76970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7cc809a3f0 tx=0x7f7cc809a650 comp rx=0 tx=0).stop 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2031420744 shutdown_connections 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a5c4ef3de0 0x55a5c4ef2af0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a5c4e752b0 0x55a5c4e76970 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5c4de6df0 0x55a5c4e75ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 --2- 172.21.15.26:0/2031420744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5c4e6a560 0x55a5c4e76430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7cb7400700 0 client.14517 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.197+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2031420744 >> 172.21.15.26:0/2031420744 conn(0x55a5c4d97120 msgr2=0x55a5c4e6a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:45.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.198+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2031420744 shutdown_connections 2024-06-13T06:55:45.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.198+0000 7f7ce287ab80 1 -- 172.21.15.26:0/2031420744 wait complete. 2024-06-13T06:55:45.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.198+0000 7f7ce287ab80 20 client.14517 trim_cache size 0 max 16384 2024-06-13T06:55:45.204 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T06:55:45.204 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24517, 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 43.290637021000002, 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 17, 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 97, 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 95, 2024-06-13T06:55:45.205 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 75.949536671999994 2024-06-13T06:55:45.206 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T06:55:45.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7cb80a2ca0 msgr2=0x7f7cb80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7cb80a2ca0 0x7f7cb80a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009c10 tx=0x7f7cc400b040 comp rx=0 tx=0).stop 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 msgr2=0x7f7cd019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 0x7f7cd019f6c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cd006acc0 tx=0x7f7cc80047b0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1513933361 shutdown_connections 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7cb80a2ca0 0x7f7cb80a5160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00746c0 0x7f7cd019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd010eef0 0x7f7cd019f6c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 --2- 172.21.15.26:0/1513933361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cd0074fe0 0x7f7cd019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1513933361 >> 172.21.15.26:0/1513933361 conn(0x7f7cd00795f0 msgr2=0x7f7cd00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1513933361 shutdown_connections 2024-06-13T06:55:45.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:45.199+0000 7f7cd7e00700 1 -- 172.21.15.26:0/1513933361 wait complete. 2024-06-13T06:55:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:45 smithi026 ceph-mon[32041]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-06-13T06:55:45.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:45 smithi135 ceph-mon[36276]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-06-13T06:55:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:45 smithi136 ceph-mon[37229]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 134 B/s, 0 objects/s recovering 2024-06-13T06:55:46.160 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-13T06:55:46.161 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:55:46.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs ls 2024-06-13T06:55:46.622 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:47.690 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:47 smithi135 ceph-mon[36276]: pgmap v366: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:47.691 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:55:47.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:47 smithi136 ceph-mon[37229]: pgmap v366: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:47.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:55:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:47 smithi026 ceph-mon[32041]: pgmap v366: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:55:49.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 -- 172.21.15.26:0/1764242624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0109480 msgr2=0x7f51a0115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:49.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 --2- 172.21.15.26:0/1764242624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0109480 0x7f51a0115990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f519800b0a0 tx=0x7f519800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 -- 172.21.15.26:0/1764242624 shutdown_connections 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 --2- 172.21.15.26:0/1764242624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0109480 0x7f51a0115990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 --2- 172.21.15.26:0/1764242624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51a010eac0 0x7f51a010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 --2- 172.21.15.26:0/1764242624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51a0108ac0 0x7f51a0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 -- 172.21.15.26:0/1764242624 >> 172.21.15.26:0/1764242624 conn(0x7f51a0075c60 msgr2=0x7f51a01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 -- 172.21.15.26:0/1764242624 shutdown_connections 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.140+0000 7f51a5000700 1 -- 172.21.15.26:0/1764242624 wait complete. 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.141+0000 7f51a5000700 1 Processor -- start 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.141+0000 7f51a5000700 1 -- start start 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f51a5000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0108ac0 0x7f51a019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f51a5000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51a0109480 0x7f51a019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f51a5000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51a010eac0 0x7f51a01a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:49.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f51a5000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0118e50 con 0x7f51a0108ac0 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f51a5000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0118cd0 con 0x7f51a010eac0 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f51a5000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0118fd0 con 0x7f51a0109480 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f519de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0108ac0 0x7f51a019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f519d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51a0109480 0x7f51a019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f519de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0108ac0 0x7f51a019f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53616/0 (socket says 172.21.15.26:53616) 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f519de00700 1 -- 172.21.15.26:0/1774938663 learned_addr learned my addr 172.21.15.26:0/1774938663 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.142+0000 7f519e800700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51a010eac0 0x7f51a01a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f519e800700 1 -- 172.21.15.26:0/1774938663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51a0109480 msgr2=0x7f51a019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f519e800700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51a0109480 0x7f51a019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f519e800700 1 -- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0108ac0 msgr2=0x7f51a019f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f519e800700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0108ac0 0x7f51a019f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f519e800700 1 -- 172.21.15.26:0/1774938663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a01a41d0 con 0x7f51a010eac0 2024-06-13T06:55:49.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f519d400700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51a0109480 0x7f51a019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f519e800700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51a010eac0 0x7f51a01a3a90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f519800b0a0 tx=0x7f5198004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f5196a00700 1 -- 172.21.15.26:0/1774938663 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198021070 con 0x7f51a010eac0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f5196a00700 1 -- 172.21.15.26:0/1774938663 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5198007b80 con 0x7f51a010eac0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a01a4460 con 0x7f51a010eac0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.143+0000 7f5196a00700 1 -- 172.21.15.26:0/1774938663 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51980052a0 con 0x7f51a010eac0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.144+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a01abd40 con 0x7f51a010eac0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.145+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a0069180 con 0x7f51a010eac0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.147+0000 7f5196a00700 1 -- 172.21.15.26:0/1774938663 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5198024070 con 0x7f51a010eac0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.148+0000 7f5196a00700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f51840a2c20 0x7f51840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.148+0000 7f5196a00700 1 -- 172.21.15.26:0/1774938663 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f519800f020 con 0x7f51a010eac0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.148+0000 7f519de00700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f51840a2c20 0x7f51840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.149+0000 7f519de00700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f51840a2c20 0x7f51840a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5190007c10 tx=0x7f5190007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.152+0000 7f5196a00700 1 -- 172.21.15.26:0/1774938663 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f519802b020 con 0x7f51a010eac0 2024-06-13T06:55:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.435+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f51a01abf10 con 0x7f51a010eac0 2024-06-13T06:55:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:49 smithi026 ceph-mon[32041]: pgmap v367: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.435+0000 7f5196a00700 1 -- 172.21.15.26:0/1774938663 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f519809d410 con 0x7f51a010eac0 2024-06-13T06:55:49.436 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-13T06:55:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.438+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f51840a2c20 msgr2=0x7f51840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f51840a2c20 0x7f51840a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5190007c10 tx=0x7f5190007480 comp rx=0 tx=0).stop 2024-06-13T06:55:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51a010eac0 msgr2=0x7f51a01a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51a010eac0 0x7f51a01a3a90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f519800b0a0 tx=0x7f5198004d00 comp rx=0 tx=0).stop 2024-06-13T06:55:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 shutdown_connections 2024-06-13T06:55:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f51840a2c20 0x7f51840a50e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51a010eac0 0x7f51a01a3a90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a0108ac0 0x7f51a019f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 --2- 172.21.15.26:0/1774938663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51a0109480 0x7f51a019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 >> 172.21.15.26:0/1774938663 conn(0x7f51a0075c60 msgr2=0x7f51a0076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 shutdown_connections 2024-06-13T06:55:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:49.439+0000 7f51a5000700 1 -- 172.21.15.26:0/1774938663 wait complete. 2024-06-13T06:55:49.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:49 smithi136 ceph-mon[37229]: pgmap v367: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:49.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:49 smithi135 ceph-mon[36276]: pgmap v367: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:49.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:49.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-13T06:55:50.508 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-13T06:55:50.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:50.509 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-06-13T06:55:50.562 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-13T06:55:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-13T06:55:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-06-13T06:55:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:55:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-06-13T06:55:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:55:50.562 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:55:50.562 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41982sec preferred_lft 41982sec 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:55:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-06-13T06:55:50.564 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-06-13T06:55:50.565 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-13T06:55:50.565 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:50.565 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T06:55:50.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-06-13T06:55:50.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-06-13T06:55:50.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-06-13T06:55:50.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-13T06:55:50.566 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T06:55:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:50 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1774938663' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:55:50.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:50.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:50 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1774938663' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:55:50.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:50 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1774938663' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:55:51.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:51.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:51.071 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-13T06:55:51.135 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-06-13T06:55:51.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:51.138 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-06-13T06:55:51.188 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-06-13T06:55:51.189 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-13T06:55:51.189 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-13T06:55:51.189 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-13T06:55:51.189 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-13T06:55:51.190 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:51.190 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T06:55:51.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-13T06:55:51.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-13T06:55:51.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-13T06:55:51.190 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T06:55:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:51.569 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:51 smithi026 ceph-mon[32041]: pgmap v368: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:51.569 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:51.569 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:51 smithi136 ceph-mon[37229]: pgmap v368: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:51.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:51 smithi135 ceph-mon[36276]: pgmap v368: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:51.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:51.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:51.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:51.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:51.925 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-06-13T06:55:51.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:51.967 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-06-13T06:55:52.063 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:52.063 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T06:55:52.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:55:52.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-13T06:55:52.063 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T06:55:52.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:52.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:52.243 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-13T06:55:52.244 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:52.244 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T06:55:52.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-13T06:55:52.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-13T06:55:52.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-13T06:55:52.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-13T06:55:52.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-13T06:55:52.244 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T06:55:52.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:52.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:52.596 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:52.596 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T06:55:52.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-06-13T06:55:52.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-06-13T06:55:52.596 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T06:55:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:52.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:52.830 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-13T06:55:52.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-13T06:55:52.830 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:55:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-13T06:55:52.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-13T06:55:52.886 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:55:52.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-06-13T06:55:53.039 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-06-13T06:55:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-06-13T06:55:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-06-13T06:55:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-06-13T06:55:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1337e8c5d9094e3deb947bac572c8db6588a1b86008e9de02c485f40cf519395/merged 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/774019c5ed061b6316b32373d60de9652338cd7a4e1272ca7c049e249476e04c/merged 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/0 2024-06-13T06:55:53.101 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/66a0dd08edb1df6c5a62e79ce198f5ac402bf54a867b06c77de35438a9adb0cf/merged 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1519aa908434d0d5d394039a605cc4c776d1f81de848ae3527078b2ae64d14d6/merged 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/0fa84763735caf5588d2e9049a4973b968242480c019d0bfe70fc685f5e9c5fe/merged 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/39c8d35054f10f6166506ffa72bb99e88d663bd42485b9935572a008c03c6f2f/merged 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/043be13507077f8baedbe915fbaee702912cb0f9c1f52f7ccafac6363bc75961/merged 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/77d7dc88a6d480337b64f46566f406a6f42b61bd200cf215828aa883cb360763/merged 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/ec3d2a8f67db94d858a444f359b7a03cfe35091ec57bbb597906f03a7f22f638/merged 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/2a3ca69562047921d82a707bc1ff2a42690f8b4fa6e07ae56376dfad922b34d5/merged 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-06-13T06:55:53.102 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:55:53.103 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:55:53.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:53.104 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-06-13T06:55:53.161 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-13T06:55:53.161 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-13T06:55:53.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-06-13T06:55:53.240 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-06-13T06:55:53.284 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-13T06:55:53.285 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1337e8c5d9094e3deb947bac572c8db6588a1b86008e9de02c485f40cf519395/merged 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/774019c5ed061b6316b32373d60de9652338cd7a4e1272ca7c049e249476e04c/merged 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/0 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/66a0dd08edb1df6c5a62e79ce198f5ac402bf54a867b06c77de35438a9adb0cf/merged 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1519aa908434d0d5d394039a605cc4c776d1f81de848ae3527078b2ae64d14d6/merged 2024-06-13T06:55:53.286 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/0fa84763735caf5588d2e9049a4973b968242480c019d0bfe70fc685f5e9c5fe/merged 2024-06-13T06:55:53.287 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/39c8d35054f10f6166506ffa72bb99e88d663bd42485b9935572a008c03c6f2f/merged 2024-06-13T06:55:53.287 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/043be13507077f8baedbe915fbaee702912cb0f9c1f52f7ccafac6363bc75961/merged 2024-06-13T06:55:53.287 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/77d7dc88a6d480337b64f46566f406a6f42b61bd200cf215828aa883cb360763/merged 2024-06-13T06:55:53.287 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/ec3d2a8f67db94d858a444f359b7a03cfe35091ec57bbb597906f03a7f22f638/merged 2024-06-13T06:55:53.287 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/2a3ca69562047921d82a707bc1ff2a42690f8b4fa6e07ae56376dfad922b34d5/merged 2024-06-13T06:55:53.287 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-06-13T06:55:53.287 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:55:53.287 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:55:53.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:53.288 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-06-13T06:55:53.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi026.stderr:2024-06-13T06:55:53.409+0000 7f0822835480 -1 init, newargv = 0x56368cc38c30 newargc=15 2024-06-13T06:55:53.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi026.stderr:ceph-fuse[78734]: starting ceph client 2024-06-13T06:55:53.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi026.stderr:ceph-fuse[78734]: starting fuse 2024-06-13T06:55:53.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:53 smithi136 ceph-mon[37229]: pgmap v369: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:53.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:55:53.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:55:53.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:53.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:53 smithi026 ceph-mon[32041]: pgmap v369: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:53.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:55:53.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:55:53.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:53.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:53 smithi135 ceph-mon[36276]: pgmap v369: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:55:53.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:55:53.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:55:53.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:55:54.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-06-13T06:55:54.584 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-06-13T06:55:54.644 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-06-13T06:55:54.645 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-06-13T06:55:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1337e8c5d9094e3deb947bac572c8db6588a1b86008e9de02c485f40cf519395/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/774019c5ed061b6316b32373d60de9652338cd7a4e1272ca7c049e249476e04c/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/0 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/66a0dd08edb1df6c5a62e79ce198f5ac402bf54a867b06c77de35438a9adb0cf/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1519aa908434d0d5d394039a605cc4c776d1f81de848ae3527078b2ae64d14d6/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/0fa84763735caf5588d2e9049a4973b968242480c019d0bfe70fc685f5e9c5fe/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/39c8d35054f10f6166506ffa72bb99e88d663bd42485b9935572a008c03c6f2f/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/043be13507077f8baedbe915fbaee702912cb0f9c1f52f7ccafac6363bc75961/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/77d7dc88a6d480337b64f46566f406a6f42b61bd200cf215828aa883cb360763/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/ec3d2a8f67db94d858a444f359b7a03cfe35091ec57bbb597906f03a7f22f638/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/2a3ca69562047921d82a707bc1ff2a42690f8b4fa6e07ae56376dfad922b34d5/merged 2024-06-13T06:55:54.647 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-06-13T06:55:54.648 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:55:54.648 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:55:54.648 INFO:teuthology.orchestra.run.smithi026.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-13T06:55:54.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:54.649 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-06-13T06:55:54.705 INFO:teuthology.orchestra.run.smithi026.stdout:119 2024-06-13T06:55:54.705 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-13T06:55:54.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-06-13T06:55:54.705 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-06-13T06:55:54.705 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-06-13T06:55:54.705 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-06-13T06:55:54.706 DEBUG:teuthology.orchestra.run.smithi026:> 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.admin")) 2024-06-13T06:55:54.707 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-06-13T06:55:54.878 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.admin.78734.asok 2024-06-13T06:55:54.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:54.900 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78734.asok 2024-06-13T06:55:54.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:54.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78734.asok status 2024-06-13T06:55:55.138 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "admin", 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "78734", 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24652, 2024-06-13T06:55:55.139 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 24652 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2272830055 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-06-13T06:55:55.140 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2272830055 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.24652 172.21.15.26:0/2272830055", 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "172.21.15.26:0/2272830055", 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 17, 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 97, 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-06-13T06:55:55.141 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T06:55:55.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:55.155 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-06-13T06:55:55.155 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:55:55.211 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-06-13T06:55:55.211 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:55:55.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:55.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:55:55.307 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-13T06:55:55.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-13T06:55:55.461 INFO:teuthology.orchestra.run.smithi026.stdout:# file: . 2024-06-13T06:55:55.462 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-13T06:55:55.462 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:55:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:55.473 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:55:55.473 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T06:55:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:55 smithi026 ceph-mon[32041]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:55.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd blocklist ls 2024-06-13T06:55:55.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:55 smithi136 ceph-mon[37229]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:55.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:55 smithi135 ceph-mon[36276]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:56.036 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:57.623 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:57 smithi026 ceph-mon[32041]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:57.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:57 smithi136 ceph-mon[37229]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:57.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:57 smithi135 ceph-mon[36276]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 -- 172.21.15.26:0/353240309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5601030f0 msgr2=0x7fa560107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 --2- 172.21.15.26:0/353240309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5601030f0 0x7fa560107110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0099c0 tx=0x7fa54c009cd0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 -- 172.21.15.26:0/353240309 shutdown_connections 2024-06-13T06:55:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 --2- 172.21.15.26:0/353240309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa560102740 0x7fa560102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 --2- 172.21.15.26:0/353240309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560107840 0x7fa56010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 --2- 172.21.15.26:0/353240309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5601030f0 0x7fa560107110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 -- 172.21.15.26:0/353240309 >> 172.21.15.26:0/353240309 conn(0x7fa5600795f0 msgr2=0x7fa560079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 -- 172.21.15.26:0/353240309 shutdown_connections 2024-06-13T06:55:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.489+0000 7fa568400700 1 -- 172.21.15.26:0/353240309 wait complete. 2024-06-13T06:55:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.490+0000 7fa568400700 1 Processor -- start 2024-06-13T06:55:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa568400700 1 -- start start 2024-06-13T06:55:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa568400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560102740 0x7fa56019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa568400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5601030f0 0x7fa56019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa568400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa560107840 0x7fa56019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa568400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa560112770 con 0x7fa5601030f0 2024-06-13T06:55:58.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa568400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601125f0 con 0x7fa560107840 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa568400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601128f0 con 0x7fa560102740 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa565a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560102740 0x7fa56019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa565000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5601030f0 0x7fa56019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa565a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560102740 0x7fa56019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50804/0 (socket says 172.21.15.26:50804) 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa565a00700 1 -- 172.21.15.26:0/3808875532 learned_addr learned my addr 172.21.15.26:0/3808875532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.491+0000 7fa566400700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa560107840 0x7fa56019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.492+0000 7fa565a00700 1 -- 172.21.15.26:0/3808875532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa560107840 msgr2=0x7fa56019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.492+0000 7fa565a00700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa560107840 0x7fa56019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.492+0000 7fa565a00700 1 -- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5601030f0 msgr2=0x7fa56019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.492+0000 7fa565a00700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5601030f0 0x7fa56019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.492+0000 7fa565a00700 1 -- 172.21.15.26:0/3808875532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56019fdb0 con 0x7fa560102740 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.492+0000 7fa565a00700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560102740 0x7fa56019ada0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa55400e9a0 tx=0x7fa55400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.493+0000 7fa55e000700 1 -- 172.21.15.26:0/3808875532 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400ca10 con 0x7fa560102740 2024-06-13T06:55:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.493+0000 7fa55e000700 1 -- 172.21.15.26:0/3808875532 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55400cbb0 con 0x7fa560102740 2024-06-13T06:55:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.493+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5601a00a0 con 0x7fa560102740 2024-06-13T06:55:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.493+0000 7fa55e000700 1 -- 172.21.15.26:0/3808875532 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554018680 con 0x7fa560102740 2024-06-13T06:55:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.493+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5601a7920 con 0x7fa560102740 2024-06-13T06:55:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.495+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa560069180 con 0x7fa560102740 2024-06-13T06:55:58.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.496+0000 7fa55e000700 1 -- 172.21.15.26:0/3808875532 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa554010430 con 0x7fa560102740 2024-06-13T06:55:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.499+0000 7fa55e000700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5480a2ca0 0x7fa5480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.499+0000 7fa55e000700 1 -- 172.21.15.26:0/3808875532 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fa5540cf580 con 0x7fa560102740 2024-06-13T06:55:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.499+0000 7fa565000700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5480a2ca0 0x7fa5480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.500+0000 7fa565000700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5480a2ca0 0x7fa5480a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa54c006010 tx=0x7fa54c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.502+0000 7fa55e000700 1 -- 172.21.15.26:0/3808875532 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa55409c120 con 0x7fa560102740 2024-06-13T06:55:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.752+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa5601a7b50 con 0x7fa560102740 2024-06-13T06:55:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.752+0000 7fa55e000700 1 -- 172.21.15.26:0/3808875532 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+591 (secure 0 0 0) 0x7fa55409ba20 con 0x7fa560102740 2024-06-13T06:55:58.752 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3363905776 2024-06-14T06:46:28.109797+0000 2024-06-13T06:55:58.752 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/3558618334 2024-06-14T06:46:28.109797+0000 2024-06-13T06:55:58.752 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2900186702 2024-06-14T06:46:28.109797+0000 2024-06-13T06:55:58.753 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/2776242437 2024-06-14T06:45:51.299898+0000 2024-06-13T06:55:58.753 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/2776242437 2024-06-14T06:45:51.299898+0000 2024-06-13T06:55:58.753 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/3558618334 2024-06-14T06:46:28.109797+0000 2024-06-13T06:55:58.753 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/106059480 2024-06-14T06:45:51.299898+0000 2024-06-13T06:55:58.753 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2539477633 2024-06-14T06:45:51.299898+0000 2024-06-13T06:55:58.753 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3002519462 2024-06-14T06:45:51.299898+0000 2024-06-13T06:55:58.753 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2082657826 2024-06-14T06:46:28.109797+0000 2024-06-13T06:55:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.756+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5480a2ca0 msgr2=0x7fa5480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.756+0000 7fa568400700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5480a2ca0 0x7fa5480a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa54c006010 tx=0x7fa54c005960 comp rx=0 tx=0).stop 2024-06-13T06:55:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.756+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560102740 msgr2=0x7fa56019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.756+0000 7fa568400700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560102740 0x7fa56019ada0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa55400e9a0 tx=0x7fa55400ecb0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.757+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 shutdown_connections 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.757+0000 7fa568400700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5480a2ca0 0x7fa5480a5160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.757+0000 7fa568400700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa560107840 0x7fa56019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.757+0000 7fa568400700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560102740 0x7fa56019ada0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.757+0000 7fa568400700 1 --2- 172.21.15.26:0/3808875532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5601030f0 0x7fa56019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.757+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 >> 172.21.15.26:0/3808875532 conn(0x7fa5600795f0 msgr2=0x7fa560108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.757+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 shutdown_connections 2024-06-13T06:55:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:55:58.757+0000 7fa568400700 1 -- 172.21.15.26:0/3808875532 wait complete. 2024-06-13T06:55:58.758 INFO:teuthology.orchestra.run.smithi026.stderr:listed 10 entries 2024-06-13T06:55:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:59 smithi026 ceph-mon[32041]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:55:59 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3808875532' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:55:59.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:59 smithi136 ceph-mon[37229]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:59.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:55:59 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3808875532' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:55:59.749 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:55:59.749 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T06:55:59.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:59 smithi135 ceph-mon[36276]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:55:59.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:55:59 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3808875532' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:55:59.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd blocklist ls 2024-06-13T06:56:00.310 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:01.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:01 smithi026 ceph-mon[32041]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:01.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:01 smithi136 ceph-mon[37229]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:01.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:01 smithi135 ceph-mon[36276]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.818+0000 7f2d95600700 1 -- 172.21.15.26:0/3091339079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 msgr2=0x7f2d9010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.818+0000 7f2d95600700 1 --2- 172.21.15.26:0/3091339079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 0x7f2d9010f320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800b0a0 tx=0x7f2d8800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:56:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.819+0000 7f2d95600700 1 -- 172.21.15.26:0/3091339079 shutdown_connections 2024-06-13T06:56:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.819+0000 7f2d95600700 1 --2- 172.21.15.26:0/3091339079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 0x7f2d9010f320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.819+0000 7f2d95600700 1 --2- 172.21.15.26:0/3091339079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d90108500 0x7f2d901088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.819+0000 7f2d95600700 1 --2- 172.21.15.26:0/3091339079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d9006ab40 0x7f2d9006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.819+0000 7f2d95600700 1 -- 172.21.15.26:0/3091339079 >> 172.21.15.26:0/3091339079 conn(0x7f2d90078300 msgr2=0x7f2d90078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:02.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.819+0000 7f2d95600700 1 -- 172.21.15.26:0/3091339079 shutdown_connections 2024-06-13T06:56:02.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.819+0000 7f2d95600700 1 -- 172.21.15.26:0/3091339079 wait complete. 2024-06-13T06:56:02.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.820+0000 7f2d95600700 1 Processor -- start 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.820+0000 7f2d95600700 1 -- start start 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d95600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d9006ab40 0x7f2d90101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d95600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 0x7f2d901019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d95600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d90108500 0x7f2d90101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d95600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901127b0 con 0x7f2d9006ab40 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d95600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90112630 con 0x7f2d9006b500 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d95600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90112930 con 0x7f2d90108500 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d8ea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d9006ab40 0x7f2d90101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d8e000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 0x7f2d901019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d8e000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 0x7f2d901019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51472/0 (socket says 172.21.15.26:51472) 2024-06-13T06:56:02.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d8e000700 1 -- 172.21.15.26:0/496252751 learned_addr learned my addr 172.21.15.26:0/496252751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:02.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d8f400700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d90108500 0x7f2d90101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:02.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d8e000700 1 -- 172.21.15.26:0/496252751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d90108500 msgr2=0x7f2d90101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:02.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d8e000700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d90108500 0x7f2d90101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.821+0000 7f2d8e000700 1 -- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d9006ab40 msgr2=0x7f2d90101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.822+0000 7f2d8e000700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d9006ab40 0x7f2d90101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.822+0000 7f2d8e000700 1 -- 172.21.15.26:0/496252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d901a7130 con 0x7f2d9006b500 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.822+0000 7f2d8e000700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 0x7f2d901019a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00e9a0 tx=0x7f2d7c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.822+0000 7f2d87400700 1 -- 172.21.15.26:0/496252751 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00ca10 con 0x7f2d9006b500 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.822+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d901a7420 con 0x7f2d9006b500 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.822+0000 7f2d87400700 1 -- 172.21.15.26:0/496252751 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d7c00cbb0 con 0x7f2d9006b500 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.822+0000 7f2d87400700 1 -- 172.21.15.26:0/496252751 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c018680 con 0x7f2d9006b500 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.822+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d901a79b0 con 0x7f2d9006b500 2024-06-13T06:56:02.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.824+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d90103bc0 con 0x7f2d9006b500 2024-06-13T06:56:02.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.826+0000 7f2d87400700 1 -- 172.21.15.26:0/496252751 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2d7c010430 con 0x7f2d9006b500 2024-06-13T06:56:02.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.828+0000 7f2d87400700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2d740a2ca0 0x7f2d740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:02.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.828+0000 7f2d87400700 1 -- 172.21.15.26:0/496252751 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f2d7c0cf580 con 0x7f2d9006b500 2024-06-13T06:56:02.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.828+0000 7f2d8ea00700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2d740a2ca0 0x7f2d740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:02.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.829+0000 7f2d8ea00700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2d740a2ca0 0x7f2d740a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005950 tx=0x7f2d8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:02.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:02.832+0000 7f2d87400700 1 -- 172.21.15.26:0/496252751 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2d7c09c120 con 0x7f2d9006b500 2024-06-13T06:56:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.085+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2d9002dca0 con 0x7f2d9006b500 2024-06-13T06:56:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.086+0000 7f2d87400700 1 -- 172.21.15.26:0/496252751 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+591 (secure 0 0 0) 0x7f2d7c09ba20 con 0x7f2d9006b500 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3363905776 2024-06-14T06:46:28.109797+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/3558618334 2024-06-14T06:46:28.109797+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2900186702 2024-06-14T06:46:28.109797+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/2776242437 2024-06-14T06:45:51.299898+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/2776242437 2024-06-14T06:45:51.299898+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/3558618334 2024-06-14T06:46:28.109797+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/106059480 2024-06-14T06:45:51.299898+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2539477633 2024-06-14T06:45:51.299898+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3002519462 2024-06-14T06:45:51.299898+0000 2024-06-13T06:56:03.087 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2082657826 2024-06-14T06:46:28.109797+0000 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2d740a2ca0 msgr2=0x7f2d740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2d740a2ca0 0x7f2d740a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005950 tx=0x7f2d8000b410 comp rx=0 tx=0).stop 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 msgr2=0x7f2d901019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 0x7f2d901019a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00e9a0 tx=0x7f2d7c00ecb0 comp rx=0 tx=0).stop 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 shutdown_connections 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2d740a2ca0 0x7f2d740a5160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d90108500 0x7f2d90101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d9006ab40 0x7f2d90101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 --2- 172.21.15.26:0/496252751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d9006b500 0x7f2d901019a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.089+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 >> 172.21.15.26:0/496252751 conn(0x7f2d90078300 msgr2=0x7f2d900fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.090+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 shutdown_connections 2024-06-13T06:56:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:03.090+0000 7f2d95600700 1 -- 172.21.15.26:0/496252751 wait complete. 2024-06-13T06:56:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:listed 10 entries 2024-06-13T06:56:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:03 smithi026 ceph-mon[32041]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:03 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/496252751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:56:03.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:03 smithi135 ceph-mon[36276]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:03.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:03 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/496252751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:56:03.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:03 smithi136 ceph-mon[37229]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:03.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:03 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/496252751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:56:04.109 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-06-13T06:56:04.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:56:04.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:56:04.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi026.stderr:ceph-fuse[78734]: fuse finished with error 0 and tester_r 0 2024-06-13T06:56:04.211 INFO:teuthology.orchestra.run:waiting for 300 2024-06-13T06:56:05.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:05 smithi026 ceph-mon[32041]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:05.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:05 smithi135 ceph-mon[36276]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:05.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:05 smithi136 ceph-mon[37229]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:56:07.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:07 smithi026 ceph-mon[32041]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:56:07.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:07 smithi135 ceph-mon[36276]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:56:07.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:07 smithi136 ceph-mon[37229]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:56:09.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:09 smithi026 ceph-mon[32041]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:56:09.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:09 smithi135 ceph-mon[36276]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:56:09.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:09 smithi136 ceph-mon[37229]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-13T06:56:10.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T06:56:10.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:56:10.313 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-13T06:56:10.390 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:56:10.858 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:11.679 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:11 smithi136 ceph-mon[37229]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-13T06:56:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:11 smithi026 ceph-mon[32041]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-13T06:56:11.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:11 smithi135 ceph-mon[36276]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-13T06:56:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.324+0000 7f6a29e00700 1 -- 172.21.15.26:0/1087364821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24108ac0 msgr2=0x7f6a24108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.324+0000 7f6a29e00700 1 --2- 172.21.15.26:0/1087364821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24108ac0 0x7f6a24108f40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a100098f0 tx=0x7f6a10009c00 comp rx=0 tx=0).stop 2024-06-13T06:56:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.324+0000 7f6a29e00700 1 -- 172.21.15.26:0/1087364821 shutdown_connections 2024-06-13T06:56:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.324+0000 7f6a29e00700 1 --2- 172.21.15.26:0/1087364821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24109480 0x7f6a24115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.324+0000 7f6a29e00700 1 --2- 172.21.15.26:0/1087364821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2410eac0 0x7f6a2410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.324+0000 7f6a29e00700 1 --2- 172.21.15.26:0/1087364821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24108ac0 0x7f6a24108f40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.324+0000 7f6a29e00700 1 -- 172.21.15.26:0/1087364821 >> 172.21.15.26:0/1087364821 conn(0x7f6a24075c60 msgr2=0x7f6a241072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.325+0000 7f6a29e00700 1 -- 172.21.15.26:0/1087364821 shutdown_connections 2024-06-13T06:56:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.325+0000 7f6a29e00700 1 -- 172.21.15.26:0/1087364821 wait complete. 2024-06-13T06:56:13.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.326+0000 7f6a29e00700 1 Processor -- start 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.326+0000 7f6a29e00700 1 -- start start 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.326+0000 7f6a29e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24108ac0 0x7f6a2419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.326+0000 7f6a29e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24109480 0x7f6a2419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.326+0000 7f6a29e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a2410eac0 0x7f6a241a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.326+0000 7f6a29e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24118e10 con 0x7f6a24108ac0 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.326+0000 7f6a29e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24118c90 con 0x7f6a24109480 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.326+0000 7f6a29e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24118f90 con 0x7f6a2410eac0 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24108ac0 0x7f6a2419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24109480 0x7f6a2419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24108ac0 0x7f6a2419f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48188/0 (socket says 172.21.15.26:48188) 2024-06-13T06:56:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22a00700 1 -- 172.21.15.26:0/2918852071 learned_addr learned my addr 172.21.15.26:0/2918852071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a23400700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a2410eac0 0x7f6a241a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22000700 1 -- 172.21.15.26:0/2918852071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a2410eac0 msgr2=0x7f6a241a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22000700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a2410eac0 0x7f6a241a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22000700 1 -- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24108ac0 msgr2=0x7f6a2419f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22000700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24108ac0 0x7f6a2419f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.327+0000 7f6a22000700 1 -- 172.21.15.26:0/2918852071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a241a4290 con 0x7f6a24109480 2024-06-13T06:56:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.328+0000 7f6a22000700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24109480 0x7f6a2419f7c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000c010 tx=0x7f6a1000fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.328+0000 7f6a28a00700 1 -- 172.21.15.26:0/2918852071 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10005280 con 0x7f6a24109480 2024-06-13T06:56:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.328+0000 7f6a28a00700 1 -- 172.21.15.26:0/2918852071 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1000ba40 con 0x7f6a24109480 2024-06-13T06:56:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.328+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241a4520 con 0x7f6a24109480 2024-06-13T06:56:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.328+0000 7f6a28a00700 1 -- 172.21.15.26:0/2918852071 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10003b20 con 0x7f6a24109480 2024-06-13T06:56:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.328+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241abe00 con 0x7f6a24109480 2024-06-13T06:56:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.329+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2410a200 con 0x7f6a24109480 2024-06-13T06:56:13.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.331+0000 7f6a28a00700 1 -- 172.21.15.26:0/2918852071 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6a1000bbf0 con 0x7f6a24109480 2024-06-13T06:56:13.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.333+0000 7f6a28a00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a080a2c20 0x7f6a080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:13.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.333+0000 7f6a28a00700 1 -- 172.21.15.26:0/2918852071 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f6a100d0600 con 0x7f6a24109480 2024-06-13T06:56:13.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.333+0000 7f6a22a00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a080a2c20 0x7f6a080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:13.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.334+0000 7f6a22a00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a080a2c20 0x7f6a080a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a14005950 tx=0x7f6a1400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:13.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.335+0000 7f6a28a00700 1 -- 172.21.15.26:0/2918852071 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a10013030 con 0x7f6a24109480 2024-06-13T06:56:13.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.567+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a2402dca0 con 0x7f6a24109480 2024-06-13T06:56:13.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.569+0000 7f6a28a00700 1 -- 172.21.15.26:0/2918852071 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24656 (secure 0 0 0) 0x7f6a1009d230 con 0x7f6a24109480 2024-06-13T06:56:13.569 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:56:13.570 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":97,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:55:29.352302+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:56:13.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.572+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a080a2c20 msgr2=0x7f6a080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:13.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.572+0000 7f6a29e00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a080a2c20 0x7f6a080a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a14005950 tx=0x7f6a1400b500 comp rx=0 tx=0).stop 2024-06-13T06:56:13.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.572+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24109480 msgr2=0x7f6a2419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:13.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.572+0000 7f6a29e00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24109480 0x7f6a2419f7c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000c010 tx=0x7f6a1000fb40 comp rx=0 tx=0).stop 2024-06-13T06:56:13.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.572+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 shutdown_connections 2024-06-13T06:56:13.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.572+0000 7f6a29e00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a080a2c20 0x7f6a080a50e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.573+0000 7f6a29e00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a2410eac0 0x7f6a241a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.573+0000 7f6a29e00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a24108ac0 0x7f6a2419f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.573+0000 7f6a29e00700 1 --2- 172.21.15.26:0/2918852071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24109480 0x7f6a2419f7c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:13.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.573+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 >> 172.21.15.26:0/2918852071 conn(0x7f6a24075c60 msgr2=0x7f6a24076240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:13.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.573+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 shutdown_connections 2024-06-13T06:56:13.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:13.573+0000 7f6a29e00700 1 -- 172.21.15.26:0/2918852071 wait complete. 2024-06-13T06:56:13.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:13 smithi026 ceph-mon[32041]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:13.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:13 smithi135 ceph-mon[36276]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:13.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:13 smithi136 ceph-mon[37229]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:14.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:56:14.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:14 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2918852071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:56:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:14 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2918852071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:56:14.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:14 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2918852071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:56:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:15 smithi026 ceph-mon[32041]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:15 smithi135 ceph-mon[36276]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:15.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:15 smithi136 ceph-mon[37229]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 -- 172.21.15.26:0/960626128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb874104fe0 msgr2=0x7fb8741114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:17.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 --2- 172.21.15.26:0/960626128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb874104fe0 0x7fb8741114f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b0a0 tx=0x7fb86c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:17 smithi026 ceph-mon[32041]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 -- 172.21.15.26:0/960626128 shutdown_connections 2024-06-13T06:56:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 --2- 172.21.15.26:0/960626128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb874104fe0 0x7fb8741114f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 --2- 172.21.15.26:0/960626128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb87410a620 0x7fb87410aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 --2- 172.21.15.26:0/960626128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb874104620 0x7fb874104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 -- 172.21.15.26:0/960626128 >> 172.21.15.26:0/960626128 conn(0x7fb874100110 msgr2=0x7fb874102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 -- 172.21.15.26:0/960626128 shutdown_connections 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.586+0000 7fb87a200700 1 -- 172.21.15.26:0/960626128 wait complete. 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.587+0000 7fb87a200700 1 Processor -- start 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb87a200700 1 -- start start 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb87a200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb874104620 0x7fb87419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb87a200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb874104fe0 0x7fb87419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb87a200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87410a620 0x7fb87419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb87a200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741149c0 con 0x7fb87410a620 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb87a200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb874114840 con 0x7fb874104620 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb87a200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb874114b40 con 0x7fb874104fe0 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb879800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87410a620 0x7fb87419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:17.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb873e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb874104fe0 0x7fb87419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb879800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87410a620 0x7fb87419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48204/0 (socket says 172.21.15.26:48204) 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.588+0000 7fb879800700 1 -- 172.21.15.26:0/859753686 learned_addr learned my addr 172.21.15.26:0/859753686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.589+0000 7fb878e00700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb874104620 0x7fb87419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.589+0000 7fb879800700 1 -- 172.21.15.26:0/859753686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb874104fe0 msgr2=0x7fb87419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.589+0000 7fb879800700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb874104fe0 0x7fb87419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.589+0000 7fb879800700 1 -- 172.21.15.26:0/859753686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb874104620 msgr2=0x7fb87419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.589+0000 7fb879800700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb874104620 0x7fb87419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.589+0000 7fb879800700 1 -- 172.21.15.26:0/859753686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87419fe00 con 0x7fb87410a620 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.589+0000 7fb873e00700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb874104fe0 0x7fb87419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.589+0000 7fb879800700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87410a620 0x7fb87419f6c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b070 tx=0x7fb86c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.590+0000 7fb871600700 1 -- 172.21.15.26:0/859753686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c021070 con 0x7fb87410a620 2024-06-13T06:56:17.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.590+0000 7fb871600700 1 -- 172.21.15.26:0/859753686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c003c00 con 0x7fb87410a620 2024-06-13T06:56:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.590+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8741a0090 con 0x7fb87410a620 2024-06-13T06:56:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.590+0000 7fb871600700 1 -- 172.21.15.26:0/859753686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c004740 con 0x7fb87410a620 2024-06-13T06:56:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.590+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8741a7970 con 0x7fb87410a620 2024-06-13T06:56:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.591+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb874069180 con 0x7fb87410a620 2024-06-13T06:56:17.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.593+0000 7fb871600700 1 -- 172.21.15.26:0/859753686 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb86c007980 con 0x7fb87410a620 2024-06-13T06:56:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.594+0000 7fb871600700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb85c0a2ca0 0x7fb85c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.594+0000 7fb871600700 1 -- 172.21.15.26:0/859753686 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fb86c00f020 con 0x7fb87410a620 2024-06-13T06:56:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.595+0000 7fb878e00700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb85c0a2ca0 0x7fb85c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.595+0000 7fb878e00700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb85c0a2ca0 0x7fb85c0a5160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb868009be0 tx=0x7fb868009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:17.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.599+0000 7fb871600700 1 -- 172.21.15.26:0/859753686 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb86c02a020 con 0x7fb87410a620 2024-06-13T06:56:17.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:17 smithi135 ceph-mon[36276]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:17.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.818+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb87402dca0 con 0x7fb87410a620 2024-06-13T06:56:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.821+0000 7fb871600700 1 -- 172.21.15.26:0/859753686 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24656 (secure 0 0 0) 0x7fb86c09d2b0 con 0x7fb87410a620 2024-06-13T06:56:17.821 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:56:17.821 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":97,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:55:29.352302+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:56:17.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.823+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb85c0a2ca0 msgr2=0x7fb85c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:17.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.823+0000 7fb87a200700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb85c0a2ca0 0x7fb85c0a5160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb868009be0 tx=0x7fb868009260 comp rx=0 tx=0).stop 2024-06-13T06:56:17.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.823+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87410a620 msgr2=0x7fb87419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:17.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.823+0000 7fb87a200700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87410a620 0x7fb87419f6c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b070 tx=0x7fb86c004c60 comp rx=0 tx=0).stop 2024-06-13T06:56:17.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.824+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 shutdown_connections 2024-06-13T06:56:17.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.824+0000 7fb87a200700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb85c0a2ca0 0x7fb85c0a5160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.824+0000 7fb87a200700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb874104620 0x7fb87419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.824+0000 7fb87a200700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb87410a620 0x7fb87419f6c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.824+0000 7fb87a200700 1 --2- 172.21.15.26:0/859753686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb874104fe0 0x7fb87419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.824+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 >> 172.21.15.26:0/859753686 conn(0x7fb874100110 msgr2=0x7fb874100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:17.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.824+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 shutdown_connections 2024-06-13T06:56:17.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:17.824+0000 7fb87a200700 1 -- 172.21.15.26:0/859753686 wait complete. 2024-06-13T06:56:17.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:17 smithi136 ceph-mon[37229]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:18.744 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:18 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/859753686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:56:18.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:18 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/859753686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:56:18.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-13T06:56:18.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:18 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/859753686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:56:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:19.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:19 smithi026 ceph-mon[32041]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:19.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:19 smithi135 ceph-mon[36276]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:19.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:19 smithi136 ceph-mon[37229]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:21.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:21 smithi026 ceph-mon[32041]: pgmap v383: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:21.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:21 smithi135 ceph-mon[36276]: pgmap v383: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.836+0000 7f79a2800700 1 -- 172.21.15.26:0/938246054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 msgr2=0x7f799c104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.836+0000 7f79a2800700 1 --2- 172.21.15.26:0/938246054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 0x7f799c104b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7988009990 tx=0x7f7988009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.836+0000 7f79a2800700 1 -- 172.21.15.26:0/938246054 shutdown_connections 2024-06-13T06:56:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.836+0000 7f79a2800700 1 --2- 172.21.15.26:0/938246054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f799c10a6b0 0x7f799c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.836+0000 7f79a2800700 1 --2- 172.21.15.26:0/938246054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f799c105070 0x7f799c111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.836+0000 7f79a2800700 1 --2- 172.21.15.26:0/938246054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 0x7f799c104b30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.836+0000 7f79a2800700 1 -- 172.21.15.26:0/938246054 >> 172.21.15.26:0/938246054 conn(0x7f799c1001a0 msgr2=0x7f799c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.837+0000 7f79a2800700 1 -- 172.21.15.26:0/938246054 shutdown_connections 2024-06-13T06:56:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.837+0000 7f79a2800700 1 -- 172.21.15.26:0/938246054 wait complete. 2024-06-13T06:56:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.838+0000 7f79a2800700 1 Processor -- start 2024-06-13T06:56:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.838+0000 7f79a2800700 1 -- start start 2024-06-13T06:56:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f79a2800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 0x7f799c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f79a2800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f799c105070 0x7f799c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f79a2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f799c10a6b0 0x7f799c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f79a2800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f799c114a90 con 0x7f799c1046b0 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f79a2800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f799c114910 con 0x7f799c10a6b0 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f79a2800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f799c114c10 con 0x7f799c105070 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f799b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 0x7f799c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f799aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f799c105070 0x7f799c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f799b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 0x7f799c19aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48220/0 (socket says 172.21.15.26:48220) 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f799aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f799c105070 0x7f799c19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57926/0 (socket says 172.21.15.26:57926) 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f799b400700 1 -- 172.21.15.26:0/2952436387 learned_addr learned my addr 172.21.15.26:0/2952436387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.839+0000 7f799be00700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f799c10a6b0 0x7f799c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.840+0000 7f799b400700 1 -- 172.21.15.26:0/2952436387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f799c105070 msgr2=0x7f799c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.840+0000 7f799b400700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f799c105070 0x7f799c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.840+0000 7f799b400700 1 -- 172.21.15.26:0/2952436387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f799c10a6b0 msgr2=0x7f799c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.840+0000 7f799b400700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f799c10a6b0 0x7f799c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.840+0000 7f799b400700 1 -- 172.21.15.26:0/2952436387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c19fe80 con 0x7f799c1046b0 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.840+0000 7f799be00700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f799c10a6b0 0x7f799c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.840+0000 7f799b400700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 0x7f799c19aec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f798c00e9a0 tx=0x7f798c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.840+0000 7f7993e00700 1 -- 172.21.15.26:0/2952436387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c00c9c0 con 0x7f799c1046b0 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.841+0000 7f7993e00700 1 -- 172.21.15.26:0/2952436387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798c00cb60 con 0x7f799c1046b0 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.841+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c1a0170 con 0x7f799c1046b0 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.841+0000 7f7993e00700 1 -- 172.21.15.26:0/2952436387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c018660 con 0x7f799c1046b0 2024-06-13T06:56:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.841+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1a79f0 con 0x7f799c1046b0 2024-06-13T06:56:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.842+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c069180 con 0x7f799c1046b0 2024-06-13T06:56:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.844+0000 7f7993e00700 1 -- 172.21.15.26:0/2952436387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f798c00cd10 con 0x7f799c1046b0 2024-06-13T06:56:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.845+0000 7f7993e00700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79800a2ca0 0x7f79800a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.845+0000 7f7993e00700 1 -- 172.21.15.26:0/2952436387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f798c0cf480 con 0x7f799c1046b0 2024-06-13T06:56:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.846+0000 7f799aa00700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79800a2ca0 0x7f79800a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.846+0000 7f799aa00700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79800a2ca0 0x7f79800a5160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f799c102750 tx=0x7f798800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:21.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:21.850+0000 7f7993e00700 1 -- 172.21.15.26:0/2952436387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f798c09c020 con 0x7f799c1046b0 2024-06-13T06:56:21.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:21 smithi136 ceph-mon[37229]: pgmap v383: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:56:22.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.134+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f799c19c1d0 con 0x7f799c1046b0 2024-06-13T06:56:22.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.457+0000 7f7993e00700 1 -- 172.21.15.26:0/2952436387 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f798c09b920 con 0x7f799c1046b0 2024-06-13T06:56:22.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79800a2ca0 msgr2=0x7f79800a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:22.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79800a2ca0 0x7f79800a5160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f799c102750 tx=0x7f798800b260 comp rx=0 tx=0).stop 2024-06-13T06:56:22.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 msgr2=0x7f799c19aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:22.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 0x7f799c19aec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f798c00e9a0 tx=0x7f798c00ecb0 comp rx=0 tx=0).stop 2024-06-13T06:56:22.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 shutdown_connections 2024-06-13T06:56:22.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79800a2ca0 0x7f79800a5160 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:22.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f799c10a6b0 0x7f799c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:22.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f799c1046b0 0x7f799c19aec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:22.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 --2- 172.21.15.26:0/2952436387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f799c105070 0x7f799c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:22.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.463+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 >> 172.21.15.26:0/2952436387 conn(0x7f799c1001a0 msgr2=0x7f799c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:22.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.464+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 shutdown_connections 2024-06-13T06:56:22.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:22.464+0000 7f79a2800700 1 -- 172.21.15.26:0/2952436387 wait complete. 2024-06-13T06:56:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:22 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2952436387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:56:22.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:22 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2952436387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:56:22.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:22 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2952436387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:56:23.520 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-06-13T06:56:23.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-06-13T06:56:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:23 smithi026 ceph-mon[32041]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:23.822 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:23 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2952436387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-13T06:56:23.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:23 smithi026 ceph-mon[32041]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:56:23.823 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:23 smithi135 ceph-mon[36276]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:23.823 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:23 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2952436387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-13T06:56:23.823 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:23 smithi135 ceph-mon[36276]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:56:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:23 smithi136 ceph-mon[37229]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:23 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2952436387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-13T06:56:23.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:23 smithi136 ceph-mon[37229]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:25 smithi026 ceph-mon[32041]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:25 smithi135 ceph-mon[36276]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:25 smithi136 ceph-mon[37229]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 -- 172.21.15.26:0/1229375724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 msgr2=0x7f2cbc06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 --2- 172.21.15.26:0/1229375724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 0x7f2cbc06b040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80099c0 tx=0x7f2ca8009cd0 comp rx=0 tx=0).stop 2024-06-13T06:56:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 -- 172.21.15.26:0/1229375724 shutdown_connections 2024-06-13T06:56:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 --2- 172.21.15.26:0/1229375724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cbc06b580 0x7f2cbc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 --2- 172.21.15.26:0/1229375724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cbc108520 0x7f2cbc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 --2- 172.21.15.26:0/1229375724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 0x7f2cbc06b040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 -- 172.21.15.26:0/1229375724 >> 172.21.15.26:0/1229375724 conn(0x7f2cbc078330 msgr2=0x7f2cbc078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 -- 172.21.15.26:0/1229375724 shutdown_connections 2024-06-13T06:56:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.397+0000 7f2cc4000700 1 -- 172.21.15.26:0/1229375724 wait complete. 2024-06-13T06:56:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.399+0000 7f2cc4000700 1 Processor -- start 2024-06-13T06:56:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.399+0000 7f2cc4000700 1 -- start start 2024-06-13T06:56:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc4000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 0x7f2cbc101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc4000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cbc06b580 0x7f2cbc1019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc4000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cbc108520 0x7f2cbc101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc4000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc1127c0 con 0x7f2cbc108520 2024-06-13T06:56:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc4000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc112640 con 0x7f2cbc06abc0 2024-06-13T06:56:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc4000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc112940 con 0x7f2cbc06b580 2024-06-13T06:56:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc1600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 0x7f2cbc101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc2000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cbc108520 0x7f2cbc101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc1600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 0x7f2cbc101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38590/0 (socket says 172.21.15.26:38590) 2024-06-13T06:56:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc1600700 1 -- 172.21.15.26:0/3645766702 learned_addr learned my addr 172.21.15.26:0/3645766702 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.400+0000 7f2cc0c00700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cbc06b580 0x7f2cbc1019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cc1600700 1 -- 172.21.15.26:0/3645766702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cbc06b580 msgr2=0x7f2cbc1019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cc1600700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cbc06b580 0x7f2cbc1019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cc1600700 1 -- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cbc108520 msgr2=0x7f2cbc101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cc1600700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cbc108520 0x7f2cbc101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cc1600700 1 -- 172.21.15.26:0/3645766702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1a71e0 con 0x7f2cbc06abc0 2024-06-13T06:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cc1600700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 0x7f2cbc101490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000e9a0 tx=0x7f2cb000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cba000700 1 -- 172.21.15.26:0/3645766702 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000c9c0 con 0x7f2cbc06abc0 2024-06-13T06:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cba000700 1 -- 172.21.15.26:0/3645766702 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb000cb60 con 0x7f2cbc06abc0 2024-06-13T06:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.401+0000 7f2cba000700 1 -- 172.21.15.26:0/3645766702 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0018660 con 0x7f2cbc06abc0 2024-06-13T06:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.402+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1a74d0 con 0x7f2cbc06abc0 2024-06-13T06:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.402+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1a7990 con 0x7f2cbc06abc0 2024-06-13T06:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.403+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc103bc0 con 0x7f2cbc06abc0 2024-06-13T06:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.405+0000 7f2cba000700 1 -- 172.21.15.26:0/3645766702 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2cb000cd10 con 0x7f2cbc06abc0 2024-06-13T06:56:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.406+0000 7f2cba000700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ca40a2ca0 0x7f2ca40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.407+0000 7f2cba000700 1 -- 172.21.15.26:0/3645766702 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f2cb00cf6a0 con 0x7f2cbc06abc0 2024-06-13T06:56:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.407+0000 7f2cc0c00700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ca40a2ca0 0x7f2ca40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.407+0000 7f2cc0c00700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ca40a2ca0 0x7f2ca40a5160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc102f20 tx=0x7f2ca8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.409+0000 7f2cba000700 1 -- 172.21.15.26:0/3645766702 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2cb009c240 con 0x7f2cbc06abc0 2024-06-13T06:56:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:26.692+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f2cbc02de60 con 0x7f2cbc06abc0 2024-06-13T06:56:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.484+0000 7f2cba000700 1 -- 172.21.15.26:0/3645766702 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f2cb009bb40 con 0x7f2cbc06abc0 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ca40a2ca0 msgr2=0x7f2ca40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ca40a2ca0 0x7f2ca40a5160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc102f20 tx=0x7f2ca8005fb0 comp rx=0 tx=0).stop 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 msgr2=0x7f2cbc101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 0x7f2cbc101490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000e9a0 tx=0x7f2cb000ecb0 comp rx=0 tx=0).stop 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 shutdown_connections 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ca40a2ca0 0x7f2ca40a5160 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cbc06abc0 0x7f2cbc101490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cbc108520 0x7f2cbc101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 --2- 172.21.15.26:0/3645766702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cbc06b580 0x7f2cbc1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 >> 172.21.15.26:0/3645766702 conn(0x7f2cbc078330 msgr2=0x7f2cbc100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 shutdown_connections 2024-06-13T06:56:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:27.488+0000 7f2cc4000700 1 -- 172.21.15.26:0/3645766702 wait complete. 2024-06-13T06:56:27.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:27 smithi026 ceph-mon[32041]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:27.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:27 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3645766702' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:56:27.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:27 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:56:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:27 smithi135 ceph-mon[36276]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:27 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3645766702' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:56:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:27 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:56:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:27 smithi136 ceph-mon[37229]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:56:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:27 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3645766702' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:56:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:27 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:56:28.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-13T06:56:28.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-13T06:56:28.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-13T06:56:28.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-13T06:56:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:56:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: fsmap cephfs:2 {0=g=up:active,1=i=up:creating} 10 up:standby 2024-06-13T06:56:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-13T06:56:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-13T06:56:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-06-13T06:56:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-13T06:56:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-06-13T06:56:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-13T06:56:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-13T06:56:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-13T06:56:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-13T06:56:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:56:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: fsmap cephfs:2 {0=g=up:active,1=i=up:creating} 10 up:standby 2024-06-13T06:56:28.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-13T06:56:28.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-13T06:56:28.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-06-13T06:56:28.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-13T06:56:28.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-06-13T06:56:28.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-13T06:56:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:28.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: fsmap cephfs:2 {0=g=up:active,1=i=up:creating} 10 up:standby 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-06-13T06:56:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-13T06:56:29.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:active 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: Cluster is now healthy 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: fsmap cephfs:2 {0=g=up:active,1=i=up:active} 10 up:standby 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: osdmap e98: 12 total, 12 up, 12 in 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:creating} 9 up:standby 2024-06-13T06:56:29.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-mon[32041]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:active 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: Cluster is now healthy 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: fsmap cephfs:2 {0=g=up:active,1=i=up:active} 10 up:standby 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-13T06:56:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-06-13T06:56:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-13T06:56:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: osdmap e98: 12 total, 12 up, 12 in 2024-06-13T06:56:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:creating} 9 up:standby 2024-06-13T06:56:29.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:29 smithi135 ceph-mon[36276]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-13T06:56:29.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:active 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: Cluster is now healthy 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: fsmap cephfs:2 {0=g=up:active,1=i=up:active} 10 up:standby 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: osdmap e98: 12 total, 12 up, 12 in 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:creating} 9 up:standby 2024-06-13T06:56:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:29 smithi136 ceph-mon[37229]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-13T06:56:29.952 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: 2024-06-13T06:56:29.674+0000 7f4b66e00700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-06-13T06:56:29.952 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 06:56:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: 2024-06-13T06:56:29.674+0000 7f4b66e00700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-06-13T06:56:30.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:30 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:active 2024-06-13T06:56:30.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:30 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2024-06-13T06:56:30.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:30 smithi026 ceph-mon[32041]: osdmap e99: 12 total, 12 up, 12 in 2024-06-13T06:56:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:30 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:active 2024-06-13T06:56:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:30 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2024-06-13T06:56:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:30 smithi135 ceph-mon[36276]: osdmap e99: 12 total, 12 up, 12 in 2024-06-13T06:56:30.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:30 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:active 2024-06-13T06:56:30.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:30 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2024-06-13T06:56:30.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:30 smithi136 ceph-mon[37229]: osdmap e99: 12 total, 12 up, 12 in 2024-06-13T06:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.281+0000 7fba93e00700 1 -- 172.21.15.26:0/1220401493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba94075460 msgr2=0x7fba9410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.281+0000 7fba93e00700 1 --2- 172.21.15.26:0/1220401493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba94075460 0x7fba9410e690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009990 tx=0x7fba7c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.281+0000 7fba93e00700 1 -- 172.21.15.26:0/1220401493 shutdown_connections 2024-06-13T06:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.281+0000 7fba93e00700 1 --2- 172.21.15.26:0/1220401493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba9410ed00 0x7fba94111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.281+0000 7fba93e00700 1 --2- 172.21.15.26:0/1220401493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba94074ab0 0x7fba94074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.281+0000 7fba93e00700 1 --2- 172.21.15.26:0/1220401493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba94075460 0x7fba9410e690 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.281+0000 7fba93e00700 1 -- 172.21.15.26:0/1220401493 >> 172.21.15.26:0/1220401493 conn(0x7fba940fde80 msgr2=0x7fba941002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.282+0000 7fba93e00700 1 -- 172.21.15.26:0/1220401493 shutdown_connections 2024-06-13T06:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.282+0000 7fba93e00700 1 -- 172.21.15.26:0/1220401493 wait complete. 2024-06-13T06:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.283+0000 7fba93e00700 1 Processor -- start 2024-06-13T06:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.283+0000 7fba93e00700 1 -- start start 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.283+0000 7fba93e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba94074ab0 0x7fba9419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.283+0000 7fba93e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba94075460 0x7fba9419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.283+0000 7fba93e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba9410ed00 0x7fba9419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.283+0000 7fba93e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba94114a30 con 0x7fba94075460 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.283+0000 7fba93e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba941148b0 con 0x7fba94074ab0 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.283+0000 7fba93e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba94114bb0 con 0x7fba9410ed00 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba94074ab0 0x7fba9419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba94074ab0 0x7fba9419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38620/0 (socket says 172.21.15.26:38620) 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 -- 172.21.15.26:0/3905684914 learned_addr learned my addr 172.21.15.26:0/3905684914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba8a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba94075460 0x7fba9419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba93400700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba9410ed00 0x7fba9419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 -- 172.21.15.26:0/3905684914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba9410ed00 msgr2=0x7fba9419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba9410ed00 0x7fba9419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 -- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba94075460 msgr2=0x7fba9419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba94075460 0x7fba9419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 -- 172.21.15.26:0/3905684914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9419fee0 con 0x7fba94074ab0 2024-06-13T06:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.284+0000 7fba92a00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba94074ab0 0x7fba9419af20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba8400e9a0 tx=0x7fba8400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.285+0000 7fba8be00700 1 -- 172.21.15.26:0/3905684914 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8400ca10 con 0x7fba94074ab0 2024-06-13T06:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.285+0000 7fba8be00700 1 -- 172.21.15.26:0/3905684914 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8400cbb0 con 0x7fba94074ab0 2024-06-13T06:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.285+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba941a01d0 con 0x7fba94074ab0 2024-06-13T06:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.285+0000 7fba8be00700 1 -- 172.21.15.26:0/3905684914 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84018680 con 0x7fba94074ab0 2024-06-13T06:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.285+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba941a7a50 con 0x7fba94074ab0 2024-06-13T06:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.286+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba94069180 con 0x7fba94074ab0 2024-06-13T06:56:31.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.288+0000 7fba8be00700 1 -- 172.21.15.26:0/3905684914 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fba84010430 con 0x7fba94074ab0 2024-06-13T06:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.290+0000 7fba8be00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba780a2c20 0x7fba780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.290+0000 7fba8be00700 1 -- 172.21.15.26:0/3905684914 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fba840cf6d0 con 0x7fba94074ab0 2024-06-13T06:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.290+0000 7fba8a000700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba780a2c20 0x7fba780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:31.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.291+0000 7fba8a000700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba780a2c20 0x7fba780a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba9410fff0 tx=0x7fba7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:31.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.294+0000 7fba8be00700 1 -- 172.21.15.26:0/3905684914 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba8409c1e0 con 0x7fba94074ab0 2024-06-13T06:56:31.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:31.582+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fba9402dca0 con 0x7fba94074ab0 2024-06-13T06:56:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:31 smithi135 ceph-mon[36276]: pgmap v390: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:31.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:31 smithi136 ceph-mon[37229]: pgmap v390: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:31 smithi026 ceph-mon[32041]: pgmap v390: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:32.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.542+0000 7fba8be00700 1 -- 172.21.15.26:0/3905684914 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fba8409bae0 con 0x7fba94074ab0 2024-06-13T06:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba780a2c20 msgr2=0x7fba780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba780a2c20 0x7fba780a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba9410fff0 tx=0x7fba7c005fd0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba94074ab0 msgr2=0x7fba9419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba94074ab0 0x7fba9419af20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba8400e9a0 tx=0x7fba8400ecb0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 shutdown_connections 2024-06-13T06:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba780a2c20 0x7fba780a50e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba94074ab0 0x7fba9419af20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba9410ed00 0x7fba9419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 --2- 172.21.15.26:0/3905684914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba94075460 0x7fba9419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 >> 172.21.15.26:0/3905684914 conn(0x7fba940fde80 msgr2=0x7fba940feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.547+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 shutdown_connections 2024-06-13T06:56:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:32.548+0000 7fba93e00700 1 -- 172.21.15.26:0/3905684914 wait complete. 2024-06-13T06:56:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:32 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3905684914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:56:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:32 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:56:32.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:32 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3905684914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:56:32.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:32 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:56:32.985 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:32 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3905684914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:56:32.985 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:32 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:56:33.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-13T06:56:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:33 smithi026 ceph-mon[32041]: pgmap v391: 129 pgs: 129 active+clean; 583 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:33 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-13T06:56:33.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:33 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2024-06-13T06:56:33.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:33 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:33 smithi136 ceph-mon[37229]: pgmap v391: 129 pgs: 129 active+clean; 583 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:33 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-13T06:56:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:33 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2024-06-13T06:56:33.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:33 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:33 smithi135 ceph-mon[36276]: pgmap v391: 129 pgs: 129 active+clean; 583 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:33 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-13T06:56:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:33 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2024-06-13T06:56:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:33 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:35.727 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:35 smithi026 ceph-mon[32041]: pgmap v392: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:35.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:35 smithi136 ceph-mon[37229]: pgmap v392: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:35 smithi135 ceph-mon[36276]: pgmap v392: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 -- 172.21.15.26:0/1388875242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 msgr2=0x7f82b010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 --2- 172.21.15.26:0/1388875242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 0x7f82b010eea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82a00098f0 tx=0x7f82a0009c00 comp rx=0 tx=0).stop 2024-06-13T06:56:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 -- 172.21.15.26:0/1388875242 shutdown_connections 2024-06-13T06:56:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 --2- 172.21.15.26:0/1388875242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 0x7f82b010eea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 --2- 172.21.15.26:0/1388875242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82b0109480 0x7f82b0115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 --2- 172.21.15.26:0/1388875242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0108ac0 0x7f82b0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 -- 172.21.15.26:0/1388875242 >> 172.21.15.26:0/1388875242 conn(0x7f82b0075c60 msgr2=0x7f82b01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 -- 172.21.15.26:0/1388875242 shutdown_connections 2024-06-13T06:56:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.312+0000 7f82b4a00700 1 -- 172.21.15.26:0/1388875242 wait complete. 2024-06-13T06:56:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.313+0000 7f82b4a00700 1 Processor -- start 2024-06-13T06:56:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.314+0000 7f82b4a00700 1 -- start start 2024-06-13T06:56:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.314+0000 7f82b4a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0108ac0 0x7f82b019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.314+0000 7f82b4a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82b0109480 0x7f82b019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.314+0000 7f82b4a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 0x7f82b01a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.314+0000 7f82b4a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0118e40 con 0x7f82b0109480 2024-06-13T06:56:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.314+0000 7f82b4a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0118cc0 con 0x7f82b010eac0 2024-06-13T06:56:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.314+0000 7f82b4a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0118fc0 con 0x7f82b0108ac0 2024-06-13T06:56:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ade00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0108ac0 0x7f82b019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ad400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82b0109480 0x7f82b019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ae800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 0x7f82b01a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ae800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 0x7f82b01a3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:45542/0 (socket says 172.21.15.26:45542) 2024-06-13T06:56:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ae800700 1 -- 172.21.15.26:0/448618140 learned_addr learned my addr 172.21.15.26:0/448618140 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ad400700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82b0109480 0x7f82b019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44822/0 (socket says 172.21.15.26:44822) 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ade00700 1 -- 172.21.15.26:0/448618140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 msgr2=0x7f82b01a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ade00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 0x7f82b01a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ade00700 1 -- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82b0109480 msgr2=0x7f82b019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ade00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82b0109480 0x7f82b019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ade00700 1 -- 172.21.15.26:0/448618140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b01a42e0 con 0x7f82b0108ac0 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.315+0000 7f82ae800700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 0x7f82b01a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.316+0000 7f82ade00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0108ac0 0x7f82b019f2d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000c00 tx=0x7f82a000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.316+0000 7f82a6a00700 1 -- 172.21.15.26:0/448618140 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0005040 con 0x7f82b0108ac0 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.316+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b01a4570 con 0x7f82b0108ac0 2024-06-13T06:56:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.316+0000 7f82a6a00700 1 -- 172.21.15.26:0/448618140 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a00056c0 con 0x7f82b0108ac0 2024-06-13T06:56:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.316+0000 7f82a6a00700 1 -- 172.21.15.26:0/448618140 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0003a00 con 0x7f82b0108ac0 2024-06-13T06:56:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.316+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b01abe50 con 0x7f82b0108ac0 2024-06-13T06:56:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.317+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b0069180 con 0x7f82b0108ac0 2024-06-13T06:56:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.320+0000 7f82a6a00700 1 -- 172.21.15.26:0/448618140 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f82a000b560 con 0x7f82b0108ac0 2024-06-13T06:56:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.321+0000 7f82a6a00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82940a2c20 0x7f82940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.321+0000 7f82a6a00700 1 -- 172.21.15.26:0/448618140 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f82a00d0610 con 0x7f82b0108ac0 2024-06-13T06:56:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.321+0000 7f82ad400700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82940a2c20 0x7f82940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.322+0000 7f82ad400700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82940a2c20 0x7f82940a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f829c009750 tx=0x7f829c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.323+0000 7f82a6a00700 1 -- 172.21.15.26:0/448618140 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f82a0013030 con 0x7f82b0108ac0 2024-06-13T06:56:36.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:36.602+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f82b01a0630 con 0x7f82b0108ac0 2024-06-13T06:56:37.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.555+0000 7f82a6a00700 1 -- 172.21.15.26:0/448618140 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f82a009d1b0 con 0x7f82b0108ac0 2024-06-13T06:56:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82940a2c20 msgr2=0x7f82940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82940a2c20 0x7f82940a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f829c009750 tx=0x7f829c009310 comp rx=0 tx=0).stop 2024-06-13T06:56:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0108ac0 msgr2=0x7f82b019f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0108ac0 0x7f82b019f2d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000c00 tx=0x7f82a000fa00 comp rx=0 tx=0).stop 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 shutdown_connections 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82940a2c20 0x7f82940a50e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b010eac0 0x7f82b01a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0108ac0 0x7f82b019f2d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 --2- 172.21.15.26:0/448618140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82b0109480 0x7f82b019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.560+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 >> 172.21.15.26:0/448618140 conn(0x7f82b0075c60 msgr2=0x7f82b0076280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.561+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 shutdown_connections 2024-06-13T06:56:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:37.561+0000 7f82b4a00700 1 -- 172.21.15.26:0/448618140 wait complete. 2024-06-13T06:56:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:37 smithi026 ceph-mon[32041]: pgmap v393: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:37 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/448618140' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:56:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:37 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:56:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:37 smithi136 ceph-mon[37229]: pgmap v393: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:37 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/448618140' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:56:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:37 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:56:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:37 smithi135 ceph-mon[36276]: pgmap v393: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:37 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/448618140' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:56:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:37 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:56:38.431 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-06-13T06:56:38.431 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-13T06:56:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:38 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-13T06:56:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:38 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:38.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:38 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-13T06:56:38.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:38 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:38.906 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:39.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:38 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-13T06:56:39.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:38 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:39.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:39 smithi026 ceph-mon[32041]: pgmap v394: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:39 smithi136 ceph-mon[37229]: pgmap v394: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:39 smithi135 ceph-mon[36276]: pgmap v394: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 -- 172.21.15.26:0/3542174053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 msgr2=0x7ff4c806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/3542174053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 0x7ff4c806a5f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009990 tx=0x7ff4b4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 -- 172.21.15.26:0/3542174053 shutdown_connections 2024-06-13T06:56:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/3542174053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c806ab30 0x7ff4c810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/3542174053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c8108500 0x7ff4c81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/3542174053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 0x7ff4c806a5f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 -- 172.21.15.26:0/3542174053 >> 172.21.15.26:0/3542174053 conn(0x7ff4c8078340 msgr2=0x7ff4c8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 -- 172.21.15.26:0/3542174053 shutdown_connections 2024-06-13T06:56:41.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.511+0000 7ff4cfe00700 1 -- 172.21.15.26:0/3542174053 wait complete. 2024-06-13T06:56:41.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.513+0000 7ff4cfe00700 1 Processor -- start 2024-06-13T06:56:41.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.513+0000 7ff4cfe00700 1 -- start start 2024-06-13T06:56:41.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.513+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 0x7ff4c8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.513+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c806ab30 0x7ff4c81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.513+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c8108500 0x7ff4c8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.513+0000 7ff4cfe00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c8112780 con 0x7ff4c806a170 2024-06-13T06:56:41.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.513+0000 7ff4cfe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c8112600 con 0x7ff4c8108500 2024-06-13T06:56:41.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.513+0000 7ff4cfe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c8112900 con 0x7ff4c806ab30 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cd400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 0x7ff4c8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cde00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c8108500 0x7ff4c8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cd400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 0x7ff4c8101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44842/0 (socket says 172.21.15.26:44842) 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cd400700 1 -- 172.21.15.26:0/110756409 learned_addr learned my addr 172.21.15.26:0/110756409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cca00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c806ab30 0x7ff4c81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cd400700 1 -- 172.21.15.26:0/110756409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c806ab30 msgr2=0x7ff4c81019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cd400700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c806ab30 0x7ff4c81019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cd400700 1 -- 172.21.15.26:0/110756409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c8108500 msgr2=0x7ff4c8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cd400700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c8108500 0x7ff4c8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cd400700 1 -- 172.21.15.26:0/110756409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c81a7140 con 0x7ff4c806a170 2024-06-13T06:56:41.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.514+0000 7ff4cde00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c8108500 0x7ff4c8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:41.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.515+0000 7ff4cd400700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 0x7ff4c8101460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00e9c0 tx=0x7ff4bc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:41.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.515+0000 7ff4c6000700 1 -- 172.21.15.26:0/110756409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00cbd0 con 0x7ff4c806a170 2024-06-13T06:56:41.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.515+0000 7ff4c6000700 1 -- 172.21.15.26:0/110756409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4bc00ee60 con 0x7ff4c806a170 2024-06-13T06:56:41.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.515+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c81a7430 con 0x7ff4c806a170 2024-06-13T06:56:41.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.515+0000 7ff4c6000700 1 -- 172.21.15.26:0/110756409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc005630 con 0x7ff4c806a170 2024-06-13T06:56:41.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.515+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c81a78f0 con 0x7ff4c806a170 2024-06-13T06:56:41.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.516+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c810a9e0 con 0x7ff4c806a170 2024-06-13T06:56:41.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.519+0000 7ff4c6000700 1 -- 172.21.15.26:0/110756409 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4bc004750 con 0x7ff4c806a170 2024-06-13T06:56:41.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.520+0000 7ff4c6000700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4b00a2c20 0x7ff4b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.520+0000 7ff4c6000700 1 -- 172.21.15.26:0/110756409 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff4bc0cf6b0 con 0x7ff4c806a170 2024-06-13T06:56:41.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.520+0000 7ff4cca00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4b00a2c20 0x7ff4b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.521+0000 7ff4cca00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4b00a2c20 0x7ff4b00a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098e0 tx=0x7ff4b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:41.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.525+0000 7ff4c6000700 1 -- 172.21.15.26:0/110756409 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff4bc09c1c0 con 0x7ff4c806a170 2024-06-13T06:56:41.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:41 smithi026 ceph-mon[32041]: pgmap v395: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.788+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff4c80628d0 con 0x7ff4b00a2c20 2024-06-13T06:56:41.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.827+0000 7ff4c6000700 1 -- 172.21.15.26:0/110756409 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff4c80628d0 con 0x7ff4b00a2c20 2024-06-13T06:56:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4b00a2c20 msgr2=0x7ff4b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4b00a2c20 0x7ff4b00a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098e0 tx=0x7ff4b4005960 comp rx=0 tx=0).stop 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 msgr2=0x7ff4c8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 0x7ff4c8101460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00e9c0 tx=0x7ff4bc00ecd0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 shutdown_connections 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4b00a2c20 0x7ff4b00a50e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c8108500 0x7ff4c8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4c806a170 0x7ff4c8101460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 --2- 172.21.15.26:0/110756409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c806ab30 0x7ff4c81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 >> 172.21.15.26:0/110756409 conn(0x7ff4c8078340 msgr2=0x7ff4c80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 shutdown_connections 2024-06-13T06:56:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:41.830+0000 7ff4cfe00700 1 -- 172.21.15.26:0/110756409 wait complete. 2024-06-13T06:56:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:41 smithi136 ceph-mon[37229]: pgmap v395: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:41 smithi135 ceph-mon[36276]: pgmap v395: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-06-13T06:56:42.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:56:41 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:56:41.800+0000 7f67d7800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:56:42.253 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:56:41 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:56:41.800+0000 7f67d7800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:56:42.253 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:56:41 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:56:41.800+0000 7f67d7800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:56:42.253 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:56:41 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:56:41.800+0000 7f67d7800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:56:42.253 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 06:56:41 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T06:56:41.800+0000 7f67d7800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:56:42.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-13T06:56:42.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:42 smithi136 ceph-mon[37229]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:43.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:42 smithi026 ceph-mon[32041]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:43.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:42 smithi135 ceph-mon[36276]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:43.234 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:43.679 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:43 smithi026 ceph-mon[32041]: pgmap v396: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 8 op/s 2024-06-13T06:56:43.679 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:43 smithi026 ceph-mon[32041]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-06-13T06:56:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:43 smithi136 ceph-mon[37229]: pgmap v396: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 8 op/s 2024-06-13T06:56:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:43 smithi136 ceph-mon[37229]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-06-13T06:56:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:43 smithi135 ceph-mon[36276]: pgmap v396: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 8 op/s 2024-06-13T06:56:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:43 smithi135 ceph-mon[36276]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-06-13T06:56:45.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:45 smithi026 ceph-mon[32041]: pgmap v397: 129 pgs: 129 active+clean; 590 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-13T06:56:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 -- 172.21.15.26:0/366728932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe95806a170 msgr2=0x7fe95806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 --2- 172.21.15.26:0/366728932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe95806a170 0x7fe95806a5f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe944009990 tx=0x7fe944009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 -- 172.21.15.26:0/366728932 shutdown_connections 2024-06-13T06:56:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 --2- 172.21.15.26:0/366728932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe95806ab30 0x7fe95810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 --2- 172.21.15.26:0/366728932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe958108500 0x7fe9581088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 --2- 172.21.15.26:0/366728932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe95806a170 0x7fe95806a5f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 -- 172.21.15.26:0/366728932 >> 172.21.15.26:0/366728932 conn(0x7fe958078340 msgr2=0x7fe958078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 -- 172.21.15.26:0/366728932 shutdown_connections 2024-06-13T06:56:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.672+0000 7fe95fc00700 1 -- 172.21.15.26:0/366728932 wait complete. 2024-06-13T06:56:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.673+0000 7fe95fc00700 1 Processor -- start 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.673+0000 7fe95fc00700 1 -- start start 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe95fc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe95806a170 0x7fe95819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe95fc00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95806ab30 0x7fe95819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe95fc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe958108500 0x7fe95819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe95fc00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe958112760 con 0x7fe95806ab30 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe95fc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581125e0 con 0x7fe958108500 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe95fc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581128e0 con 0x7fe95806a170 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe957e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95806ab30 0x7fe95819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe957e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95806ab30 0x7fe95819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47446/0 (socket says 172.21.15.26:47446) 2024-06-13T06:56:45.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe957e00700 1 -- 172.21.15.26:0/399110119 learned_addr learned my addr 172.21.15.26:0/399110119 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:45.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.674+0000 7fe95d200700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe95806a170 0x7fe95819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe95dc00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe958108500 0x7fe95819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe957e00700 1 -- 172.21.15.26:0/399110119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe95806a170 msgr2=0x7fe95819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe957e00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe95806a170 0x7fe95819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe957e00700 1 -- 172.21.15.26:0/399110119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe958108500 msgr2=0x7fe95819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe957e00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe958108500 0x7fe95819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe957e00700 1 -- 172.21.15.26:0/399110119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95819fe40 con 0x7fe95806ab30 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe95dc00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe958108500 0x7fe95819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe957e00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95806ab30 0x7fe95819b370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe944000c00 tx=0x7fe94400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.675+0000 7fe955600700 1 -- 172.21.15.26:0/399110119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944027070 con 0x7fe95806ab30 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.676+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9581a00d0 con 0x7fe95806ab30 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.676+0000 7fe955600700 1 -- 172.21.15.26:0/399110119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe944003d70 con 0x7fe95806ab30 2024-06-13T06:56:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.676+0000 7fe955600700 1 -- 172.21.15.26:0/399110119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94400f460 con 0x7fe95806ab30 2024-06-13T06:56:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.676+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9581a79b0 con 0x7fe95806ab30 2024-06-13T06:56:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.677+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe958069180 con 0x7fe95806ab30 2024-06-13T06:56:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.681+0000 7fe955600700 1 -- 172.21.15.26:0/399110119 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe94402a070 con 0x7fe95806ab30 2024-06-13T06:56:45.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.683+0000 7fe955600700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe9400a2cb0 0x7fe9400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.683+0000 7fe955600700 1 -- 172.21.15.26:0/399110119 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe9440d0320 con 0x7fe95806ab30 2024-06-13T06:56:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.683+0000 7fe95d200700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe9400a2cb0 0x7fe9400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.684+0000 7fe95d200700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe9400a2cb0 0x7fe9400a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009be0 tx=0x7fe94c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.688+0000 7fe955600700 1 -- 172.21.15.26:0/399110119 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe944013070 con 0x7fe95806ab30 2024-06-13T06:56:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:45 smithi136 ceph-mon[37229]: pgmap v397: 129 pgs: 129 active+clean; 590 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-13T06:56:45.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:45.954+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe95810dd60 con 0x7fe9400a2cb0 2024-06-13T06:56:46.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.004+0000 7fe955600700 1 -- 172.21.15.26:0/399110119 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe95810dd60 con 0x7fe9400a2cb0 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe9400a2cb0 msgr2=0x7fe9400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe9400a2cb0 0x7fe9400a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009be0 tx=0x7fe94c009260 comp rx=0 tx=0).stop 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95806ab30 msgr2=0x7fe95819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95806ab30 0x7fe95819b370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe944000c00 tx=0x7fe94400bab0 comp rx=0 tx=0).stop 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 shutdown_connections 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe9400a2cb0 0x7fe9400a5170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe958108500 0x7fe95819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe95806a170 0x7fe95819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 --2- 172.21.15.26:0/399110119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe95806ab30 0x7fe95819b370 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 >> 172.21.15.26:0/399110119 conn(0x7fe958078340 msgr2=0x7fe95810d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:46.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.008+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 shutdown_connections 2024-06-13T06:56:46.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:46.009+0000 7fe95fc00700 1 -- 172.21.15.26:0/399110119 wait complete. 2024-06-13T06:56:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:45 smithi135 ceph-mon[36276]: pgmap v397: 129 pgs: 129 active+clean; 590 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-13T06:56:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:46 smithi026 ceph-mon[32041]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:46.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:46 smithi136 ceph-mon[37229]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:46.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-13T06:56:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:46 smithi135 ceph-mon[36276]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:47.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:47 smithi026 ceph-mon[32041]: pgmap v398: 129 pgs: 129 active+clean; 590 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 6 op/s 2024-06-13T06:56:47.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:47 smithi136 ceph-mon[37229]: pgmap v398: 129 pgs: 129 active+clean; 590 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 6 op/s 2024-06-13T06:56:48.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:47 smithi135 ceph-mon[36276]: pgmap v398: 129 pgs: 129 active+clean; 590 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 6 op/s 2024-06-13T06:56:49.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.722+0000 7f679aa00700 1 -- 172.21.15.26:0/3458620666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f679406b610 msgr2=0x7f6794107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:49.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.722+0000 7f679aa00700 1 --2- 172.21.15.26:0/3458620666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f679406b610 0x7f6794107050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f67800098f0 tx=0x7f6780009c00 comp rx=0 tx=0).stop 2024-06-13T06:56:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.723+0000 7f679aa00700 1 -- 172.21.15.26:0/3458620666 shutdown_connections 2024-06-13T06:56:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.723+0000 7f679aa00700 1 --2- 172.21.15.26:0/3458620666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6794107780 0x7f679410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.723+0000 7f679aa00700 1 --2- 172.21.15.26:0/3458620666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f679406ac60 0x7f679406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.723+0000 7f679aa00700 1 --2- 172.21.15.26:0/3458620666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f679406b610 0x7f6794107050 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.723+0000 7f679aa00700 1 -- 172.21.15.26:0/3458620666 >> 172.21.15.26:0/3458620666 conn(0x7f67940791d0 msgr2=0x7f67940795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.723+0000 7f679aa00700 1 -- 172.21.15.26:0/3458620666 shutdown_connections 2024-06-13T06:56:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.723+0000 7f679aa00700 1 -- 172.21.15.26:0/3458620666 wait complete. 2024-06-13T06:56:49.725 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:49 smithi026 ceph-mon[32041]: pgmap v399: 129 pgs: 129 active+clean; 591 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:56:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.724+0000 7f679aa00700 1 Processor -- start 2024-06-13T06:56:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.724+0000 7f679aa00700 1 -- start start 2024-06-13T06:56:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f679aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f679406ac60 0x7f679419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f679aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6794107780 0x7f679419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f679aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f679419f850 0x7f679419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f679aa00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6794112710 con 0x7f679419f850 2024-06-13T06:56:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f679aa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6794112590 con 0x7f679406ac60 2024-06-13T06:56:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f679aa00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6794112890 con 0x7f6794107780 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f679a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f679419f850 0x7f679419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f6799600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f679406ac60 0x7f679419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f6798c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6794107780 0x7f679419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f6798c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6794107780 0x7f679419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39000/0 (socket says 172.21.15.26:39000) 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f679a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f679419f850 0x7f679419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47456/0 (socket says 172.21.15.26:47456) 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f6799600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f679406ac60 0x7f679419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51360/0 (socket says 172.21.15.26:51360) 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.725+0000 7f6798c00700 1 -- 172.21.15.26:0/3799695667 learned_addr learned my addr 172.21.15.26:0/3799695667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.726+0000 7f6798c00700 1 -- 172.21.15.26:0/3799695667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f679406ac60 msgr2=0x7f679419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.726+0000 7f6798c00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f679406ac60 0x7f679419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.726+0000 7f6798c00700 1 -- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f679419f850 msgr2=0x7f679419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.726+0000 7f6798c00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f679419f850 0x7f679419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.726+0000 7f6798c00700 1 -- 172.21.15.26:0/3799695667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67941a0370 con 0x7f6794107780 2024-06-13T06:56:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.726+0000 7f6798c00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6794107780 0x7f679419b4c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6780000c00 tx=0x7f678000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.727+0000 7f6792000700 1 -- 172.21.15.26:0/3799695667 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780005140 con 0x7f6794107780 2024-06-13T06:56:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.727+0000 7f6792000700 1 -- 172.21.15.26:0/3799695667 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f678000b560 con 0x7f6794107780 2024-06-13T06:56:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.727+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67941a0600 con 0x7f6794107780 2024-06-13T06:56:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.727+0000 7f6792000700 1 -- 172.21.15.26:0/3799695667 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780003a20 con 0x7f6794107780 2024-06-13T06:56:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.727+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67941a7cd0 con 0x7f6794107780 2024-06-13T06:56:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.728+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6794069180 con 0x7f6794107780 2024-06-13T06:56:49.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.730+0000 7f6792000700 1 -- 172.21.15.26:0/3799695667 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f678000ba10 con 0x7f6794107780 2024-06-13T06:56:49.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.731+0000 7f6792000700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f677c0a2cb0 0x7f677c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:49.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.731+0000 7f6792000700 1 -- 172.21.15.26:0/3799695667 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f67800d0740 con 0x7f6794107780 2024-06-13T06:56:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.732+0000 7f6799600700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f677c0a2cb0 0x7f677c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.732+0000 7f6799600700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f677c0a2cb0 0x7f677c0a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6788005fd0 tx=0x7f6788009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:49.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:49.734+0000 7f6792000700 1 -- 172.21.15.26:0/3799695667 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6780013030 con 0x7f6794107780 2024-06-13T06:56:49.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:49 smithi136 ceph-mon[37229]: pgmap v399: 129 pgs: 129 active+clean; 591 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:56:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.003+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f67940ffb10 con 0x7f677c0a2cb0 2024-06-13T06:56:50.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.010+0000 7f6792000700 1 -- 172.21.15.26:0/3799695667 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f67940ffb10 con 0x7f677c0a2cb0 2024-06-13T06:56:50.010 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/qa/sv_0/27fce7d9-8647-4b1c-9fec-b0ffdb79386e 2024-06-13T06:56:50.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f677c0a2cb0 msgr2=0x7f677c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f677c0a2cb0 0x7f677c0a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6788005fd0 tx=0x7f6788009310 comp rx=0 tx=0).stop 2024-06-13T06:56:50.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6794107780 msgr2=0x7f679419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6794107780 0x7f679419b4c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6780000c00 tx=0x7f678000fa00 comp rx=0 tx=0).stop 2024-06-13T06:56:50.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 shutdown_connections 2024-06-13T06:56:50.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f677c0a2cb0 0x7f677c0a5170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f679406ac60 0x7f679419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f679419f850 0x7f679419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 --2- 172.21.15.26:0/3799695667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6794107780 0x7f679419b4c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.013+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 >> 172.21.15.26:0/3799695667 conn(0x7f67940791d0 msgr2=0x7f67940ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:50.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.014+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 shutdown_connections 2024-06-13T06:56:50.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:50.014+0000 7f679aa00700 1 -- 172.21.15.26:0/3799695667 wait complete. 2024-06-13T06:56:50.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:49 smithi135 ceph-mon[36276]: pgmap v399: 129 pgs: 129 active+clean; 591 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:56:50.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:50 smithi026 ceph-mon[32041]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:50.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:50 smithi136 ceph-mon[37229]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:51.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-13T06:56:51.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:50 smithi135 ceph-mon[36276]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:51.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:51 smithi026 ceph-mon[32041]: pgmap v400: 129 pgs: 129 active+clean; 605 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-13T06:56:52.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:51 smithi135 ceph-mon[36276]: pgmap v400: 129 pgs: 129 active+clean; 605 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-13T06:56:52.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:51 smithi136 ceph-mon[37229]: pgmap v400: 129 pgs: 129 active+clean; 605 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-13T06:56:53.841 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:53 smithi136 ceph-mon[37229]: pgmap v401: 129 pgs: 129 active+clean; 605 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-06-13T06:56:53.841 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:53 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:56:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 -- 172.21.15.26:0/1227074182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12140746c0 msgr2=0x7f1214074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 --2- 172.21.15.26:0/1227074182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12140746c0 0x7f1214074aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f12040099c0 tx=0x7f1204009cd0 comp rx=0 tx=0).stop 2024-06-13T06:56:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 -- 172.21.15.26:0/1227074182 shutdown_connections 2024-06-13T06:56:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 --2- 172.21.15.26:0/1227074182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1214113660 0x7f1214115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 --2- 172.21.15.26:0/1227074182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12140746c0 0x7f1214074aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 --2- 172.21.15.26:0/1227074182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214074fe0 0x7f1214113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 -- 172.21.15.26:0/1227074182 >> 172.21.15.26:0/1227074182 conn(0x7f12140795f0 msgr2=0x7f1214079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:53.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 -- 172.21.15.26:0/1227074182 shutdown_connections 2024-06-13T06:56:53.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.846+0000 7f121aa00700 1 -- 172.21.15.26:0/1227074182 wait complete. 2024-06-13T06:56:53.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.848+0000 7f121aa00700 1 Processor -- start 2024-06-13T06:56:53.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.848+0000 7f121aa00700 1 -- start start 2024-06-13T06:56:53.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.848+0000 7f121aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140746c0 0x7f121419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:53.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.848+0000 7f121aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1214074fe0 0x7f121419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:53.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.848+0000 7f121aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1214113660 0x7f12141a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:53.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f121aa00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1214118ec0 con 0x7f12140746c0 2024-06-13T06:56:53.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f121aa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1214118d40 con 0x7f1214074fe0 2024-06-13T06:56:53.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f121aa00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1214119040 con 0x7f1214113660 2024-06-13T06:56:53.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140746c0 0x7f121419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:53.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140746c0 0x7f121419f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45938/0 (socket says 172.21.15.26:45938) 2024-06-13T06:56:53.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213e00700 1 -- 172.21.15.26:0/244136483 learned_addr learned my addr 172.21.15.26:0/244136483 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:53.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1214074fe0 0x7f121419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:53.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1218a00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1214113660 0x7f12141a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213400700 1 -- 172.21.15.26:0/244136483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1214113660 msgr2=0x7f12141a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213400700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1214113660 0x7f12141a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213400700 1 -- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140746c0 msgr2=0x7f121419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213400700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140746c0 0x7f121419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1213400700 1 -- 172.21.15.26:0/244136483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12141a42a0 con 0x7f1214074fe0 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.849+0000 7f1218a00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1214113660 0x7f12141a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.850+0000 7f1213400700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1214074fe0 0x7f121419f7d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f120000e9c0 tx=0x7f120000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.850+0000 7f1210c00700 1 -- 172.21.15.26:0/244136483 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200004220 con 0x7f1214074fe0 2024-06-13T06:56:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.850+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141a4590 con 0x7f1214074fe0 2024-06-13T06:56:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.850+0000 7f1210c00700 1 -- 172.21.15.26:0/244136483 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12000043c0 con 0x7f1214074fe0 2024-06-13T06:56:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.850+0000 7f1210c00700 1 -- 172.21.15.26:0/244136483 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200003cc0 con 0x7f1214074fe0 2024-06-13T06:56:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.850+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141abe10 con 0x7f1214074fe0 2024-06-13T06:56:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.851+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1214069180 con 0x7f1214074fe0 2024-06-13T06:56:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.855+0000 7f1210c00700 1 -- 172.21.15.26:0/244136483 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1200043b10 con 0x7f1214074fe0 2024-06-13T06:56:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.856+0000 7f1210c00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11f80a2d30 0x7f11f80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.856+0000 7f1210c00700 1 -- 172.21.15.26:0/244136483 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f12000cf7b0 con 0x7f1214074fe0 2024-06-13T06:56:53.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.856+0000 7f1213e00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11f80a2d30 0x7f11f80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:53.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.857+0000 7f1210c00700 1 -- 172.21.15.26:0/244136483 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12000203e0 con 0x7f1214074fe0 2024-06-13T06:56:53.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:53.857+0000 7f1213e00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11f80a2d30 0x7f11f80a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f1204005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:54.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:53 smithi026 ceph-mon[32041]: pgmap v401: 129 pgs: 129 active+clean; 605 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-06-13T06:56:54.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:53 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:56:54.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:53 smithi135 ceph-mon[36276]: pgmap v401: 129 pgs: 129 active+clean; 605 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-06-13T06:56:54.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:53 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:56:54.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.129+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1214112ac0 con 0x7f11f80a2d30 2024-06-13T06:56:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.179+0000 7f1210c00700 1 -- 172.21.15.26:0/244136483 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1214112ac0 con 0x7f11f80a2d30 2024-06-13T06:56:54.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.182+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11f80a2d30 msgr2=0x7f11f80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.182+0000 7f121aa00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11f80a2d30 0x7f11f80a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f1204005fb0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.182+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1214074fe0 msgr2=0x7f121419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.182+0000 7f121aa00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1214074fe0 0x7f121419f7d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f120000e9c0 tx=0x7f120000ecd0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.183+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 shutdown_connections 2024-06-13T06:56:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.183+0000 7f121aa00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11f80a2d30 0x7f11f80a51f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.183+0000 7f121aa00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1214113660 0x7f12141a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.183+0000 7f121aa00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12140746c0 0x7f121419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.183+0000 7f121aa00700 1 --2- 172.21.15.26:0/244136483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1214074fe0 0x7f121419f7d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.183+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 >> 172.21.15.26:0/244136483 conn(0x7f12140795f0 msgr2=0x7f12141123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.183+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 shutdown_connections 2024-06-13T06:56:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:54.183+0000 7f121aa00700 1 -- 172.21.15.26:0/244136483 wait complete. 2024-06-13T06:56:55.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.104 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-13T06:56:55.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:55.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:56:56.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:55 smithi026 ceph-mon[32041]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:56.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:55 smithi026 ceph-mon[32041]: pgmap v402: 129 pgs: 129 active+clean; 610 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-13T06:56:56.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:55 smithi135 ceph-mon[36276]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:56.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:55 smithi135 ceph-mon[36276]: pgmap v402: 129 pgs: 129 active+clean; 610 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-13T06:56:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:55 smithi136 ceph-mon[37229]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:56:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:55 smithi136 ceph-mon[37229]: pgmap v402: 129 pgs: 129 active+clean; 610 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-13T06:56:56.142 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:57.992 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:57 smithi026 ceph-mon[32041]: pgmap v403: 129 pgs: 129 active+clean; 610 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:56:58.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:57 smithi135 ceph-mon[36276]: pgmap v403: 129 pgs: 129 active+clean; 610 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:56:58.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:57 smithi136 ceph-mon[37229]: pgmap v403: 129 pgs: 129 active+clean; 610 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:56:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 -- 172.21.15.26:0/867380273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5888074fe0 msgr2=0x7f5888113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 --2- 172.21.15.26:0/867380273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5888074fe0 0x7f5888113120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f58740098f0 tx=0x7f5874009c00 comp rx=0 tx=0).stop 2024-06-13T06:56:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 -- 172.21.15.26:0/867380273 shutdown_connections 2024-06-13T06:56:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 --2- 172.21.15.26:0/867380273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5888113660 0x7f5888115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 --2- 172.21.15.26:0/867380273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58880746c0 0x7f5888074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 --2- 172.21.15.26:0/867380273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5888074fe0 0x7f5888113120 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 -- 172.21.15.26:0/867380273 >> 172.21.15.26:0/867380273 conn(0x7f58880795f0 msgr2=0x7f5888079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:58.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 -- 172.21.15.26:0/867380273 shutdown_connections 2024-06-13T06:56:58.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.537+0000 7f588fa00700 1 -- 172.21.15.26:0/867380273 wait complete. 2024-06-13T06:56:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.538+0000 7f588fa00700 1 Processor -- start 2024-06-13T06:56:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.538+0000 7f588fa00700 1 -- start start 2024-06-13T06:56:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588fa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58880746c0 0x7f588819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588fa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5888074fe0 0x7f588819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588fa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5888113660 0x7f58881a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588fa00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5888118df0 con 0x7f5888113660 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588fa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5888118c70 con 0x7f58880746c0 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588fa00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5888118f70 con 0x7f5888074fe0 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588da00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5888113660 0x7f58881a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588da00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5888113660 0x7f58881a3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45958/0 (socket says 172.21.15.26:45958) 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588da00700 1 -- 172.21.15.26:0/1767189414 learned_addr learned my addr 172.21.15.26:0/1767189414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f5887e00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5888074fe0 0x7f588819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.539+0000 7f588d000700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58880746c0 0x7f588819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f588da00700 1 -- 172.21.15.26:0/1767189414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5888074fe0 msgr2=0x7f588819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f588da00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5888074fe0 0x7f588819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f588da00700 1 -- 172.21.15.26:0/1767189414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58880746c0 msgr2=0x7f588819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f588da00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58880746c0 0x7f588819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f588da00700 1 -- 172.21.15.26:0/1767189414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58881a4390 con 0x7f5888113660 2024-06-13T06:56:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f5887e00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5888074fe0 0x7f588819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f588da00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5888113660 0x7f58881a3c50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f588000ef90 tx=0x7f588000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f5885600700 1 -- 172.21.15.26:0/1767189414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58800092e0 con 0x7f5888113660 2024-06-13T06:56:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f5885600700 1 -- 172.21.15.26:0/1767189414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5880004bb0 con 0x7f5888113660 2024-06-13T06:56:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f5885600700 1 -- 172.21.15.26:0/1767189414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880004280 con 0x7f5888113660 2024-06-13T06:56:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.540+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58881a4620 con 0x7f5888113660 2024-06-13T06:56:58.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.541+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58881abf00 con 0x7f5888113660 2024-06-13T06:56:58.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.542+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5888075db0 con 0x7f5888113660 2024-06-13T06:56:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.544+0000 7f5885600700 1 -- 172.21.15.26:0/1767189414 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f588001b030 con 0x7f5888113660 2024-06-13T06:56:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.546+0000 7f5885600700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58700a2cb0 0x7f58700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.546+0000 7f5885600700 1 -- 172.21.15.26:0/1767189414 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f58800cf140 con 0x7f5888113660 2024-06-13T06:56:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.547+0000 7f588d000700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58700a2cb0 0x7f58700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.547+0000 7f588d000700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58700a2cb0 0x7f58700a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f587c005fd0 tx=0x7f587c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:58.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.552+0000 7f5885600700 1 -- 172.21.15.26:0/1767189414 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5880022020 con 0x7f5888113660 2024-06-13T06:56:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.833+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5888112bb0 con 0x7f58700a2cb0 2024-06-13T06:56:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.841+0000 7f5885600700 1 -- 172.21.15.26:0/1767189414 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5888112bb0 con 0x7f58700a2cb0 2024-06-13T06:56:58.841 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25 2024-06-13T06:56:58.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58700a2cb0 msgr2=0x7f58700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:58.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58700a2cb0 0x7f58700a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f587c005fd0 tx=0x7f587c009310 comp rx=0 tx=0).stop 2024-06-13T06:56:58.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5888113660 msgr2=0x7f58881a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:58.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5888113660 0x7f58881a3c50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f588000ef90 tx=0x7f588000c3a0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 shutdown_connections 2024-06-13T06:56:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58700a2cb0 0x7f58700a5170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58880746c0 0x7f588819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5888113660 0x7f58881a3c50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 --2- 172.21.15.26:0/1767189414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5888074fe0 0x7f588819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.845+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 >> 172.21.15.26:0/1767189414 conn(0x7f58880795f0 msgr2=0x7f58881124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.846+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 shutdown_connections 2024-06-13T06:56:58.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:56:58.846+0000 7f588fa00700 1 -- 172.21.15.26:0/1767189414 wait complete. 2024-06-13T06:56:59.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:57:00.148 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:59 smithi026 ceph-mon[32041]: pgmap v404: 129 pgs: 129 active+clean; 610 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-06-13T06:57:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:59 smithi026 ceph-mon[32041]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:57:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:56:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:59 smithi135 ceph-mon[36276]: pgmap v404: 129 pgs: 129 active+clean; 610 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-06-13T06:57:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:59 smithi135 ceph-mon[36276]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:57:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:56:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:59 smithi136 ceph-mon[37229]: pgmap v404: 129 pgs: 129 active+clean; 610 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-06-13T06:57:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:59 smithi136 ceph-mon[37229]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:57:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:56:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:01.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:00 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:57:01.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:00 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:57:01.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:00 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:01.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:00 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:57:01.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:00 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:57:01.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:00 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:01.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:00 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:57:01.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:00 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:57:01.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:00 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:57:02.090 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:01 smithi026 ceph-mon[32041]: pgmap v405: 129 pgs: 129 active+clean; 632 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-13T06:57:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:01 smithi135 ceph-mon[36276]: pgmap v405: 129 pgs: 129 active+clean; 632 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-13T06:57:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:01 smithi136 ceph-mon[37229]: pgmap v405: 129 pgs: 129 active+clean; 632 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.675+0000 7f596ec00700 1 -- 172.21.15.26:0/4193963734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968109480 msgr2=0x7f5968115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.675+0000 7f596ec00700 1 --2- 172.21.15.26:0/4193963734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968109480 0x7f5968115990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f596000b0a0 tx=0x7f596000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.676+0000 7f596ec00700 1 -- 172.21.15.26:0/4193963734 shutdown_connections 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.676+0000 7f596ec00700 1 --2- 172.21.15.26:0/4193963734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968109480 0x7f5968115990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.676+0000 7f596ec00700 1 --2- 172.21.15.26:0/4193963734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f596810eac0 0x7f596810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.676+0000 7f596ec00700 1 --2- 172.21.15.26:0/4193963734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5968108ac0 0x7f5968108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.676+0000 7f596ec00700 1 -- 172.21.15.26:0/4193963734 >> 172.21.15.26:0/4193963734 conn(0x7f5968075c60 msgr2=0x7f59681072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.676+0000 7f596ec00700 1 -- 172.21.15.26:0/4193963734 shutdown_connections 2024-06-13T06:57:02.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.676+0000 7f596ec00700 1 -- 172.21.15.26:0/4193963734 wait complete. 2024-06-13T06:57:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.677+0000 7f596ec00700 1 Processor -- start 2024-06-13T06:57:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.677+0000 7f596ec00700 1 -- start start 2024-06-13T06:57:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.677+0000 7f596ec00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968108ac0 0x7f596819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.677+0000 7f596ec00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5968109480 0x7f596819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.677+0000 7f596ec00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f596810eac0 0x7f59681a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f596ec00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5968118e40 con 0x7f5968108ac0 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f596ec00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5968118cc0 con 0x7f596810eac0 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f596ec00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5968118fc0 con 0x7f5968109480 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968108ac0 0x7f596819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5968109480 0x7f596819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968108ac0 0x7f596819f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35838/0 (socket says 172.21.15.26:35838) 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5968109480 0x7f596819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58802/0 (socket says 172.21.15.26:58802) 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967e00700 1 -- 172.21.15.26:0/4095565384 learned_addr learned my addr 172.21.15.26:0/4095565384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f596cc00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f596810eac0 0x7f59681a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967e00700 1 -- 172.21.15.26:0/4095565384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5968109480 msgr2=0x7f596819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967e00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5968109480 0x7f596819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967e00700 1 -- 172.21.15.26:0/4095565384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f596810eac0 msgr2=0x7f59681a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:02.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967e00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f596810eac0 0x7f59681a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967e00700 1 -- 172.21.15.26:0/4095565384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59681a4220 con 0x7f5968108ac0 2024-06-13T06:57:02.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f5967400700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5968109480 0x7f596819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:02.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.678+0000 7f596cc00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f596810eac0 0x7f59681a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:02.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.679+0000 7f5967e00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968108ac0 0x7f596819f210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f595c00eb30 tx=0x7f595c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:02.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.679+0000 7f5964c00700 1 -- 172.21.15.26:0/4095565384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c00cbd0 con 0x7f5968108ac0 2024-06-13T06:57:02.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.679+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59681a4510 con 0x7f5968108ac0 2024-06-13T06:57:02.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.679+0000 7f5964c00700 1 -- 172.21.15.26:0/4095565384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c004500 con 0x7f5968108ac0 2024-06-13T06:57:02.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.679+0000 7f5964c00700 1 -- 172.21.15.26:0/4095565384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c010430 con 0x7f5968108ac0 2024-06-13T06:57:02.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.679+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59681abd90 con 0x7f5968108ac0 2024-06-13T06:57:02.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.680+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5968069180 con 0x7f5968108ac0 2024-06-13T06:57:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.684+0000 7f5964c00700 1 -- 172.21.15.26:0/4095565384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f595c003f00 con 0x7f5968108ac0 2024-06-13T06:57:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.686+0000 7f5964c00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59500a2cb0 0x7f59500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.686+0000 7f5964c00700 1 -- 172.21.15.26:0/4095565384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f595c0cf9d0 con 0x7f5968108ac0 2024-06-13T06:57:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.686+0000 7f5967400700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59500a2cb0 0x7f59500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.687+0000 7f5967400700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59500a2cb0 0x7f59500a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5954009680 tx=0x7f5954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.689+0000 7f5964c00700 1 -- 172.21.15.26:0/4095565384 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f595c09c410 con 0x7f5968108ac0 2024-06-13T06:57:02.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.975+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59681a0570 con 0x7f5968108ac0 2024-06-13T06:57:02.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.976+0000 7f5964c00700 1 -- 172.21.15.26:0/4095565384 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f595c09bd10 con 0x7f5968108ac0 2024-06-13T06:57:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:57:02.976 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T06:56:36.605518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24511,24517,24506]},"id":1}]} 2024-06-13T06:57:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.979+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59500a2cb0 msgr2=0x7f59500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.979+0000 7f596ec00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59500a2cb0 0x7f59500a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5954009680 tx=0x7f5954009260 comp rx=0 tx=0).stop 2024-06-13T06:57:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.979+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968108ac0 msgr2=0x7f596819f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.979+0000 7f596ec00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968108ac0 0x7f596819f210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f595c00eb30 tx=0x7f595c00ee40 comp rx=0 tx=0).stop 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.980+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 shutdown_connections 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.980+0000 7f596ec00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59500a2cb0 0x7f59500a5170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.980+0000 7f596ec00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f596810eac0 0x7f59681a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.980+0000 7f596ec00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5968108ac0 0x7f596819f210 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.980+0000 7f596ec00700 1 --2- 172.21.15.26:0/4095565384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5968109480 0x7f596819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.980+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 >> 172.21.15.26:0/4095565384 conn(0x7f5968075c60 msgr2=0x7f5968076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.980+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 shutdown_connections 2024-06-13T06:57:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:02.980+0000 7f596ec00700 1 -- 172.21.15.26:0/4095565384 wait complete. 2024-06-13T06:57:02.981 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-06-13T06:57:03.867 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:57:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:03 smithi026 ceph-mon[32041]: pgmap v406: 129 pgs: 129 active+clean; 632 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-06-13T06:57:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:03 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/4095565384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:57:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:03 smithi135 ceph-mon[36276]: pgmap v406: 129 pgs: 129 active+clean; 632 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-06-13T06:57:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:03 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/4095565384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:57:04.354 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:03 smithi136 ceph-mon[37229]: pgmap v406: 129 pgs: 129 active+clean; 632 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-06-13T06:57:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:03 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/4095565384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:57:06.140 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:05 smithi026 ceph-mon[32041]: pgmap v407: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-06-13T06:57:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:05 smithi135 ceph-mon[36276]: pgmap v407: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-06-13T06:57:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:05 smithi136 ceph-mon[37229]: pgmap v407: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-06-13T06:57:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 -- 172.21.15.26:0/2792907268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c113660 msgr2=0x7fdd1c115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 --2- 172.21.15.26:0/2792907268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c113660 0x7fdd1c115a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400b0a0 tx=0x7fdd1400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 -- 172.21.15.26:0/2792907268 shutdown_connections 2024-06-13T06:57:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 --2- 172.21.15.26:0/2792907268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1c0746c0 0x7fdd1c074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 --2- 172.21.15.26:0/2792907268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c113660 0x7fdd1c115a50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 --2- 172.21.15.26:0/2792907268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c074fe0 0x7fdd1c113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 -- 172.21.15.26:0/2792907268 >> 172.21.15.26:0/2792907268 conn(0x7fdd1c0795f0 msgr2=0x7fdd1c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 -- 172.21.15.26:0/2792907268 shutdown_connections 2024-06-13T06:57:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.751+0000 7fdd23800700 1 -- 172.21.15.26:0/2792907268 wait complete. 2024-06-13T06:57:06.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.752+0000 7fdd23800700 1 Processor -- start 2024-06-13T06:57:06.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.752+0000 7fdd23800700 1 -- start start 2024-06-13T06:57:06.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd23800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c0746c0 0x7fdd1c19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:06.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd23800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1c074fe0 0x7fdd1c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:06.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd23800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c113660 0x7fdd1c1a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:06.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd23800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c118d80 con 0x7fdd1c113660 2024-06-13T06:57:06.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd23800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c118c00 con 0x7fdd1c074fe0 2024-06-13T06:57:06.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd23800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c118f00 con 0x7fdd1c0746c0 2024-06-13T06:57:06.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd21800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c113660 0x7fdd1c1a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:06.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd20e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c0746c0 0x7fdd1c19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:06.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd20e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c0746c0 0x7fdd1c19f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58816/0 (socket says 172.21.15.26:58816) 2024-06-13T06:57:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd20e00700 1 -- 172.21.15.26:0/3125006306 learned_addr learned my addr 172.21.15.26:0/3125006306 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.753+0000 7fdd1be00700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1c074fe0 0x7fdd1c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.754+0000 7fdd20e00700 1 -- 172.21.15.26:0/3125006306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1c074fe0 msgr2=0x7fdd1c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.754+0000 7fdd20e00700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1c074fe0 0x7fdd1c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.754+0000 7fdd20e00700 1 -- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c113660 msgr2=0x7fdd1c1a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.754+0000 7fdd20e00700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c113660 0x7fdd1c1a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.754+0000 7fdd20e00700 1 -- 172.21.15.26:0/3125006306 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c1a41c0 con 0x7fdd1c0746c0 2024-06-13T06:57:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.754+0000 7fdd21800700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c113660 0x7fdd1c1a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:06.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.754+0000 7fdd20e00700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c0746c0 0x7fdd1c19f1b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000e9a0 tx=0x7fdd1000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:06.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.754+0000 7fdd19600700 1 -- 172.21.15.26:0/3125006306 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000ca10 con 0x7fdd1c0746c0 2024-06-13T06:57:06.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.755+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c1a44b0 con 0x7fdd1c0746c0 2024-06-13T06:57:06.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.755+0000 7fdd19600700 1 -- 172.21.15.26:0/3125006306 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd1000cbb0 con 0x7fdd1c0746c0 2024-06-13T06:57:06.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.755+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c1abd30 con 0x7fdd1c0746c0 2024-06-13T06:57:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.755+0000 7fdd19600700 1 -- 172.21.15.26:0/3125006306 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10018680 con 0x7fdd1c0746c0 2024-06-13T06:57:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.756+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1c069180 con 0x7fdd1c0746c0 2024-06-13T06:57:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.758+0000 7fdd19600700 1 -- 172.21.15.26:0/3125006306 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdd10018820 con 0x7fdd1c0746c0 2024-06-13T06:57:06.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.759+0000 7fdd19600700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdd040a2d30 0x7fdd040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:06.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.759+0000 7fdd19600700 1 -- 172.21.15.26:0/3125006306 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdd100cf7d0 con 0x7fdd1c0746c0 2024-06-13T06:57:06.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.760+0000 7fdd1be00700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdd040a2d30 0x7fdd040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:06.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.760+0000 7fdd1be00700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdd040a2d30 0x7fdd040a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009680 tx=0x7fdd08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:06.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:06.762+0000 7fdd19600700 1 -- 172.21.15.26:0/3125006306 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdd1009c180 con 0x7fdd1c0746c0 2024-06-13T06:57:07.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.002+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd1c1abf90 con 0x7fdd1c0746c0 2024-06-13T06:57:07.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.004+0000 7fdd19600700 1 -- 172.21.15.26:0/3125006306 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7fdd1009ba80 con 0x7fdd1c0746c0 2024-06-13T06:57:07.004 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:57:07.004 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:57:07.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdd040a2d30 msgr2=0x7fdd040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:07.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdd040a2d30 0x7fdd040a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009680 tx=0x7fdd08009260 comp rx=0 tx=0).stop 2024-06-13T06:57:07.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c0746c0 msgr2=0x7fdd1c19f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c0746c0 0x7fdd1c19f1b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000e9a0 tx=0x7fdd1000ecb0 comp rx=0 tx=0).stop 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 shutdown_connections 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdd040a2d30 0x7fdd040a51f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c113660 0x7fdd1c1a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c0746c0 0x7fdd1c19f1b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 --2- 172.21.15.26:0/3125006306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1c074fe0 0x7fdd1c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 >> 172.21.15.26:0/3125006306 conn(0x7fdd1c0795f0 msgr2=0x7fdd1c112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 shutdown_connections 2024-06-13T06:57:07.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:07.007+0000 7fdd23800700 1 -- 172.21.15.26:0/3125006306 wait complete. 2024-06-13T06:57:07.878 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-13T06:57:07.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph pg dump --format=json 2024-06-13T06:57:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:07 smithi026 ceph-mon[32041]: pgmap v408: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-06-13T06:57:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:07 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3125006306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:08.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:07 smithi135 ceph-mon[36276]: pgmap v408: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-06-13T06:57:08.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:07 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3125006306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:08.348 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:07 smithi136 ceph-mon[37229]: pgmap v408: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-06-13T06:57:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:07 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3125006306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:10 smithi026 ceph-mon[32041]: pgmap v409: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-13T06:57:10.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:10 smithi136 ceph-mon[37229]: pgmap v409: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-13T06:57:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:10 smithi135 ceph-mon[36276]: pgmap v409: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-13T06:57:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.811+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1730638104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc06b3c0 msgr2=0x7f0cdc10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.811+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1730638104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc06b3c0 0x7f0cdc10f310 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400b0a0 tx=0x7f0cd400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.812+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1730638104 shutdown_connections 2024-06-13T06:57:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.812+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1730638104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc1084f0 0x7f0cdc1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.812+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1730638104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc06b3c0 0x7f0cdc10f310 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.812+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1730638104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cdc06aa00 0x7f0cdc06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.812+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1730638104 >> 172.21.15.26:0/1730638104 conn(0x7f0cdc078300 msgr2=0x7f0cdc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.812+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1730638104 shutdown_connections 2024-06-13T06:57:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.812+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1730638104 wait complete. 2024-06-13T06:57:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.813+0000 7f0ce0a00700 1 Processor -- start 2024-06-13T06:57:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0ce0a00700 1 -- start start 2024-06-13T06:57:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0ce0a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cdc06aa00 0x7f0cdc101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0ce0a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc06b3c0 0x7f0cdc101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0ce0a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc1084f0 0x7f0cdc101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0ce0a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc112750 con 0x7f0cdc1084f0 2024-06-13T06:57:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0ce0a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc1125d0 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0ce0a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc1128d0 con 0x7f0cdc06aa00 2024-06-13T06:57:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0cd9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cdc06aa00 0x7f0cdc101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0cda800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc1084f0 0x7f0cdc101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.814+0000 7f0cd9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cdc06aa00 0x7f0cdc101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58844/0 (socket says 172.21.15.26:58844) 2024-06-13T06:57:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cd9e00700 1 -- 172.21.15.26:0/1952289114 learned_addr learned my addr 172.21.15.26:0/1952289114 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cd9400700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc06b3c0 0x7f0cdc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cd9400700 1 -- 172.21.15.26:0/1952289114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cdc06aa00 msgr2=0x7f0cdc101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cd9400700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cdc06aa00 0x7f0cdc101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cd9400700 1 -- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc1084f0 msgr2=0x7f0cdc101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cd9400700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc1084f0 0x7f0cdc101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cd9400700 1 -- 172.21.15.26:0/1952289114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc1a70d0 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cda800700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc1084f0 0x7f0cdc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.815+0000 7f0cd9400700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc06b3c0 0x7f0cdc101990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800e9f0 tx=0x7f0cc800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.816+0000 7f0cd2a00700 1 -- 172.21.15.26:0/1952289114 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800c9c0 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.816+0000 7f0cd2a00700 1 -- 172.21.15.26:0/1952289114 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc800cb60 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.816+0000 7f0cd2a00700 1 -- 172.21.15.26:0/1952289114 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8018680 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.816+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc1a73c0 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.816+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc1a78b0 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.817+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cdc069180 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.820+0000 7f0cd2a00700 1 -- 172.21.15.26:0/1952289114 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0cc8010430 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.821+0000 7f0cd2a00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0cc00a2cb0 0x7f0cc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.821+0000 7f0cd2a00700 1 -- 172.21.15.26:0/1952289114 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0cc80cf730 con 0x7f0cdc06b3c0 2024-06-13T06:57:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.821+0000 7f0cd9e00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0cc00a2cb0 0x7f0cc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.822+0000 7f0cd9e00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0cc00a2cb0 0x7f0cc00a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009ca0 tx=0x7f0ccc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:10.826+0000 7f0cd2a00700 1 -- 172.21.15.26:0/1952289114 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0cc809c0e0 con 0x7f0cdc06b3c0 2024-06-13T06:57:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.056+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cdc0628d0 con 0x7f0cc00a2cb0 2024-06-13T06:57:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.080+0000 7f0cd2a00700 1 -- 172.21.15.26:0/1952289114 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340382 (secure 0 0 0) 0x7f0cdc0628d0 con 0x7f0cc00a2cb0 2024-06-13T06:57:11.081 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.088+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0cc00a2cb0 msgr2=0x7f0cc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.088+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0cc00a2cb0 0x7f0cc00a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009ca0 tx=0x7f0ccc009260 comp rx=0 tx=0).stop 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.088+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc06b3c0 msgr2=0x7f0cdc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.088+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc06b3c0 0x7f0cdc101990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800e9f0 tx=0x7f0cc800ed00 comp rx=0 tx=0).stop 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.089+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 shutdown_connections 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.089+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0cc00a2cb0 0x7f0cc00a5170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.089+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc1084f0 0x7f0cdc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.089+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cdc06aa00 0x7f0cdc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.089+0000 7f0ce0a00700 1 --2- 172.21.15.26:0/1952289114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc06b3c0 0x7f0cdc101990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.089+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 >> 172.21.15.26:0/1952289114 conn(0x7f0cdc078300 msgr2=0x7f0cdc0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.089+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 shutdown_connections 2024-06-13T06:57:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:11.089+0000 7f0ce0a00700 1 -- 172.21.15.26:0/1952289114 wait complete. 2024-06-13T06:57:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-06-13T06:57:12.114 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-06-13T06:57:10.324877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652992,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":365,"num_read_kb":296,"num_write":349,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182392,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":397880,"kb_used_data":15428,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543944,"statfs":{"total":1151940427776,"available":1151532998656,"internally_reserved":0,"allocated":15798272,"data_stored":8775602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28047,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":69,"num_write":16,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005115"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525058+0000","last_change":"2024-06-13T06:54:58.919669+0000","last_active":"2024-06-13T06:56:29.525058+0000","last_peered":"2024-06-13T06:56:29.525058+0000","last_clean":"2024-06-13T06:56:29.525058+0000","last_became_active":"2024-06-13T06:54:58.919334+0000","last_became_peered":"2024-06-13T06:54:58.919334+0000","last_unstale":"2024-06-13T06:56:29.525058+0000","last_undegraded":"2024-06-13T06:56:29.525058+0000","last_fullsized":"2024-06-13T06:56:29.525058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:45:40.831640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532319+0000","last_change":"2024-06-13T06:54:58.909508+0000","last_active":"2024-06-13T06:56:29.532319+0000","last_peered":"2024-06-13T06:56:29.532319+0000","last_clean":"2024-06-13T06:56:29.532319+0000","last_became_active":"2024-06-13T06:54:58.909221+0000","last_became_peered":"2024-06-13T06:54:58.909221+0000","last_unstale":"2024-06-13T06:56:29.532319+0000","last_undegraded":"2024-06-13T06:56:29.532319+0000","last_fullsized":"2024-06-13T06:56:29.532319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:17:07.476858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.531829+0000","last_change":"2024-06-13T06:54:58.909283+0000","last_active":"2024-06-13T06:56:29.531829+0000","last_peered":"2024-06-13T06:56:29.531829+0000","last_clean":"2024-06-13T06:56:29.531829+0000","last_became_active":"2024-06-13T06:54:58.908912+0000","last_became_peered":"2024-06-13T06:54:58.908912+0000","last_unstale":"2024-06-13T06:56:29.531829+0000","last_undegraded":"2024-06-13T06:56:29.531829+0000","last_fullsized":"2024-06-13T06:56:29.531829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:48:26.937971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608067+0000","last_change":"2024-06-13T06:54:58.902821+0000","last_active":"2024-06-13T06:56:29.608067+0000","last_peered":"2024-06-13T06:56:29.608067+0000","last_clean":"2024-06-13T06:56:29.608067+0000","last_became_active":"2024-06-13T06:54:58.902215+0000","last_became_peered":"2024-06-13T06:54:58.902215+0000","last_unstale":"2024-06-13T06:56:29.608067+0000","last_undegraded":"2024-06-13T06:56:29.608067+0000","last_fullsized":"2024-06-13T06:56:29.608067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:11:57.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522348+0000","last_change":"2024-06-13T06:54:58.897038+0000","last_active":"2024-06-13T06:56:29.522348+0000","last_peered":"2024-06-13T06:56:29.522348+0000","last_clean":"2024-06-13T06:56:29.522348+0000","last_became_active":"2024-06-13T06:54:58.896665+0000","last_became_peered":"2024-06-13T06:54:58.896665+0000","last_unstale":"2024-06-13T06:56:29.522348+0000","last_undegraded":"2024-06-13T06:56:29.522348+0000","last_fullsized":"2024-06-13T06:56:29.522348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:24:19.216932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523216+0000","last_change":"2024-06-13T06:54:58.895501+0000","last_active":"2024-06-13T06:56:29.523216+0000","last_peered":"2024-06-13T06:56:29.523216+0000","last_clean":"2024-06-13T06:56:29.523216+0000","last_became_active":"2024-06-13T06:54:58.895144+0000","last_became_peered":"2024-06-13T06:54:58.895144+0000","last_unstale":"2024-06-13T06:56:29.523216+0000","last_undegraded":"2024-06-13T06:56:29.523216+0000","last_fullsized":"2024-06-13T06:56:29.523216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:50.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607844+0000","last_change":"2024-06-13T06:54:58.901097+0000","last_active":"2024-06-13T06:56:29.607844+0000","last_peered":"2024-06-13T06:56:29.607844+0000","last_clean":"2024-06-13T06:56:29.607844+0000","last_became_active":"2024-06-13T06:54:58.900727+0000","last_became_peered":"2024-06-13T06:54:58.900727+0000","last_unstale":"2024-06-13T06:56:29.607844+0000","last_undegraded":"2024-06-13T06:56:29.607844+0000","last_fullsized":"2024-06-13T06:56:29.607844+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:09:38.349517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523777+0000","last_change":"2024-06-13T06:54:58.910740+0000","last_active":"2024-06-13T06:56:29.523777+0000","last_peered":"2024-06-13T06:56:29.523777+0000","last_clean":"2024-06-13T06:56:29.523777+0000","last_became_active":"2024-06-13T06:54:58.910351+0000","last_became_peered":"2024-06-13T06:54:58.910351+0000","last_unstale":"2024-06-13T06:56:29.523777+0000","last_undegraded":"2024-06-13T06:56:29.523777+0000","last_fullsized":"2024-06-13T06:56:29.523777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:45:12.854996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608887+0000","last_change":"2024-06-13T06:55:29.364516+0000","last_active":"2024-06-13T06:56:29.608887+0000","last_peered":"2024-06-13T06:56:29.608887+0000","last_clean":"2024-06-13T06:56:29.608887+0000","last_became_active":"2024-06-13T06:55:29.364079+0000","last_became_peered":"2024-06-13T06:55:29.364079+0000","last_unstale":"2024-06-13T06:56:29.608887+0000","last_undegraded":"2024-06-13T06:56:29.608887+0000","last_fullsized":"2024-06-13T06:56:29.608887+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:51:28.624359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523056+0000","last_change":"2024-06-13T06:54:58.903882+0000","last_active":"2024-06-13T06:56:29.523056+0000","last_peered":"2024-06-13T06:56:29.523056+0000","last_clean":"2024-06-13T06:56:29.523056+0000","last_became_active":"2024-06-13T06:54:58.903495+0000","last_became_peered":"2024-06-13T06:54:58.903495+0000","last_unstale":"2024-06-13T06:56:29.523056+0000","last_undegraded":"2024-06-13T06:56:29.523056+0000","last_fullsized":"2024-06-13T06:56:29.523056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:23.682993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609302+0000","last_change":"2024-06-13T06:54:58.900080+0000","last_active":"2024-06-13T06:56:29.609302+0000","last_peered":"2024-06-13T06:56:29.609302+0000","last_clean":"2024-06-13T06:56:29.609302+0000","last_became_active":"2024-06-13T06:54:58.899736+0000","last_became_peered":"2024-06-13T06:54:58.899736+0000","last_unstale":"2024-06-13T06:56:29.609302+0000","last_undegraded":"2024-06-13T06:56:29.609302+0000","last_fullsized":"2024-06-13T06:56:29.609302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:51:45.808821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608127+0000","last_change":"2024-06-13T06:54:58.900238+0000","last_active":"2024-06-13T06:56:29.608127+0000","last_peered":"2024-06-13T06:56:29.608127+0000","last_clean":"2024-06-13T06:56:29.608127+0000","last_became_active":"2024-06-13T06:54:58.899872+0000","last_became_peered":"2024-06-13T06:54:58.899872+0000","last_unstale":"2024-06-13T06:56:29.608127+0000","last_undegraded":"2024-06-13T06:56:29.608127+0000","last_fullsized":"2024-06-13T06:56:29.608127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:12:40.780894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.599885+0000","last_change":"2024-06-13T06:54:58.901981+0000","last_active":"2024-06-13T06:56:29.599885+0000","last_peered":"2024-06-13T06:56:29.599885+0000","last_clean":"2024-06-13T06:56:29.599885+0000","last_became_active":"2024-06-13T06:54:58.898704+0000","last_became_peered":"2024-06-13T06:54:58.898704+0000","last_unstale":"2024-06-13T06:56:29.599885+0000","last_undegraded":"2024-06-13T06:56:29.599885+0000","last_fullsized":"2024-06-13T06:56:29.599885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:03:56.658091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533036+0000","last_change":"2024-06-13T06:54:58.909022+0000","last_active":"2024-06-13T06:56:29.533036+0000","last_peered":"2024-06-13T06:56:29.533036+0000","last_clean":"2024-06-13T06:56:29.533036+0000","last_became_active":"2024-06-13T06:54:58.907438+0000","last_became_peered":"2024-06-13T06:54:58.907438+0000","last_unstale":"2024-06-13T06:56:29.533036+0000","last_undegraded":"2024-06-13T06:56:29.533036+0000","last_fullsized":"2024-06-13T06:56:29.533036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:40:34.190510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.531758+0000","last_change":"2024-06-13T06:54:58.897691+0000","last_active":"2024-06-13T06:56:29.531758+0000","last_peered":"2024-06-13T06:56:29.531758+0000","last_clean":"2024-06-13T06:56:29.531758+0000","last_became_active":"2024-06-13T06:54:58.897381+0000","last_became_peered":"2024-06-13T06:54:58.897381+0000","last_unstale":"2024-06-13T06:56:29.531758+0000","last_undegraded":"2024-06-13T06:56:29.531758+0000","last_fullsized":"2024-06-13T06:56:29.531758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:10.052052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607757+0000","last_change":"2024-06-13T06:54:58.907676+0000","last_active":"2024-06-13T06:56:29.607757+0000","last_peered":"2024-06-13T06:56:29.607757+0000","last_clean":"2024-06-13T06:56:29.607757+0000","last_became_active":"2024-06-13T06:54:58.907304+0000","last_became_peered":"2024-06-13T06:54:58.907304+0000","last_unstale":"2024-06-13T06:56:29.607757+0000","last_undegraded":"2024-06-13T06:56:29.607757+0000","last_fullsized":"2024-06-13T06:56:29.607757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:08:20.292516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524782+0000","last_change":"2024-06-13T06:54:58.920012+0000","last_active":"2024-06-13T06:56:29.524782+0000","last_peered":"2024-06-13T06:56:29.524782+0000","last_clean":"2024-06-13T06:56:29.524782+0000","last_became_active":"2024-06-13T06:54:58.919473+0000","last_became_peered":"2024-06-13T06:54:58.919473+0000","last_unstale":"2024-06-13T06:56:29.524782+0000","last_undegraded":"2024-06-13T06:56:29.524782+0000","last_fullsized":"2024-06-13T06:56:29.524782+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:49:13.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533070+0000","last_change":"2024-06-13T06:54:58.902576+0000","last_active":"2024-06-13T06:56:29.533070+0000","last_peered":"2024-06-13T06:56:29.533070+0000","last_clean":"2024-06-13T06:56:29.533070+0000","last_became_active":"2024-06-13T06:54:58.902205+0000","last_became_peered":"2024-06-13T06:54:58.902205+0000","last_unstale":"2024-06-13T06:56:29.533070+0000","last_undegraded":"2024-06-13T06:56:29.533070+0000","last_fullsized":"2024-06-13T06:56:29.533070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:13:24.107081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607362+0000","last_change":"2024-06-13T06:54:58.905392+0000","last_active":"2024-06-13T06:56:29.607362+0000","last_peered":"2024-06-13T06:56:29.607362+0000","last_clean":"2024-06-13T06:56:29.607362+0000","last_became_active":"2024-06-13T06:54:58.905017+0000","last_became_peered":"2024-06-13T06:54:58.905017+0000","last_unstale":"2024-06-13T06:56:29.607362+0000","last_undegraded":"2024-06-13T06:56:29.607362+0000","last_fullsized":"2024-06-13T06:56:29.607362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:05.931653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606834+0000","last_change":"2024-06-13T06:55:12.021359+0000","last_active":"2024-06-13T06:56:29.606834+0000","last_peered":"2024-06-13T06:56:29.606834+0000","last_clean":"2024-06-13T06:56:29.606834+0000","last_became_active":"2024-06-13T06:55:12.020860+0000","last_became_peered":"2024-06-13T06:55:12.020860+0000","last_unstale":"2024-06-13T06:56:29.606834+0000","last_undegraded":"2024-06-13T06:56:29.606834+0000","last_fullsized":"2024-06-13T06:56:29.606834+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:05:00.833905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.561282+0000","last_change":"2024-06-13T06:54:54.339220+0000","last_active":"2024-06-13T06:56:32.561282+0000","last_peered":"2024-06-13T06:56:32.561282+0000","last_clean":"2024-06-13T06:56:32.561282+0000","last_became_active":"2024-06-13T06:54:54.338573+0000","last_became_peered":"2024-06-13T06:54:54.338573+0000","last_unstale":"2024-06-13T06:56:32.561282+0000","last_undegraded":"2024-06-13T06:56:32.561282+0000","last_fullsized":"2024-06-13T06:56:32.561282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:53:50.258097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525854+0000","last_change":"2024-06-13T06:54:58.903920+0000","last_active":"2024-06-13T06:56:29.525854+0000","last_peered":"2024-06-13T06:56:29.525854+0000","last_clean":"2024-06-13T06:56:29.525854+0000","last_became_active":"2024-06-13T06:54:58.895364+0000","last_became_peered":"2024-06-13T06:54:58.895364+0000","last_unstale":"2024-06-13T06:56:29.525854+0000","last_undegraded":"2024-06-13T06:56:29.525854+0000","last_fullsized":"2024-06-13T06:56:29.525854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:04:35.109799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606948+0000","last_change":"2024-06-13T06:55:12.017561+0000","last_active":"2024-06-13T06:56:29.606948+0000","last_peered":"2024-06-13T06:56:29.606948+0000","last_clean":"2024-06-13T06:56:29.606948+0000","last_became_active":"2024-06-13T06:55:12.017104+0000","last_became_peered":"2024-06-13T06:55:12.017104+0000","last_unstale":"2024-06-13T06:56:29.606948+0000","last_undegraded":"2024-06-13T06:56:29.606948+0000","last_fullsized":"2024-06-13T06:56:29.606948+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:08:22.570491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526871+0000","last_change":"2024-06-13T06:54:54.631599+0000","last_active":"2024-06-13T06:56:29.526871+0000","last_peered":"2024-06-13T06:56:29.526871+0000","last_clean":"2024-06-13T06:56:29.526871+0000","last_became_active":"2024-06-13T06:54:54.631007+0000","last_became_peered":"2024-06-13T06:54:54.631007+0000","last_unstale":"2024-06-13T06:56:29.526871+0000","last_undegraded":"2024-06-13T06:56:29.526871+0000","last_fullsized":"2024-06-13T06:56:29.526871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:49:56.602192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532111+0000","last_change":"2024-06-13T06:54:58.917255+0000","last_active":"2024-06-13T06:56:29.532111+0000","last_peered":"2024-06-13T06:56:29.532111+0000","last_clean":"2024-06-13T06:56:29.532111+0000","last_became_active":"2024-06-13T06:54:58.916875+0000","last_became_peered":"2024-06-13T06:54:58.916875+0000","last_unstale":"2024-06-13T06:56:29.532111+0000","last_undegraded":"2024-06-13T06:56:29.532111+0000","last_fullsized":"2024-06-13T06:56:29.532111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:09:47.638802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608461+0000","last_change":"2024-06-13T06:55:12.018209+0000","last_active":"2024-06-13T06:56:29.608461+0000","last_peered":"2024-06-13T06:56:29.608461+0000","last_clean":"2024-06-13T06:56:29.608461+0000","last_became_active":"2024-06-13T06:55:12.017038+0000","last_became_peered":"2024-06-13T06:55:12.017038+0000","last_unstale":"2024-06-13T06:56:29.608461+0000","last_undegraded":"2024-06-13T06:56:29.608461+0000","last_fullsized":"2024-06-13T06:56:29.608461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:36:26.476454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522745+0000","last_change":"2024-06-13T06:54:54.328849+0000","last_active":"2024-06-13T06:56:29.522745+0000","last_peered":"2024-06-13T06:56:29.522745+0000","last_clean":"2024-06-13T06:56:29.522745+0000","last_became_active":"2024-06-13T06:54:54.328388+0000","last_became_peered":"2024-06-13T06:54:54.328388+0000","last_unstale":"2024-06-13T06:56:29.522745+0000","last_undegraded":"2024-06-13T06:56:29.522745+0000","last_fullsized":"2024-06-13T06:56:29.522745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:41.264403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607188+0000","last_change":"2024-06-13T06:54:58.897588+0000","last_active":"2024-06-13T06:56:29.607188+0000","last_peered":"2024-06-13T06:56:29.607188+0000","last_clean":"2024-06-13T06:56:29.607188+0000","last_became_active":"2024-06-13T06:54:58.897236+0000","last_became_peered":"2024-06-13T06:54:58.897236+0000","last_unstale":"2024-06-13T06:56:29.607188+0000","last_undegraded":"2024-06-13T06:56:29.607188+0000","last_fullsized":"2024-06-13T06:56:29.607188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:20:38.460445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607238+0000","last_change":"2024-06-13T06:55:12.014080+0000","last_active":"2024-06-13T06:56:29.607238+0000","last_peered":"2024-06-13T06:56:29.607238+0000","last_clean":"2024-06-13T06:56:29.607238+0000","last_became_active":"2024-06-13T06:55:12.013627+0000","last_became_peered":"2024-06-13T06:55:12.013627+0000","last_unstale":"2024-06-13T06:56:29.607238+0000","last_undegraded":"2024-06-13T06:56:29.607238+0000","last_fullsized":"2024-06-13T06:56:29.607238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:29:54.081593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522192+0000","last_change":"2024-06-13T06:54:54.328249+0000","last_active":"2024-06-13T06:56:29.522192+0000","last_peered":"2024-06-13T06:56:29.522192+0000","last_clean":"2024-06-13T06:56:29.522192+0000","last_became_active":"2024-06-13T06:54:54.327832+0000","last_became_peered":"2024-06-13T06:54:54.327832+0000","last_unstale":"2024-06-13T06:56:29.522192+0000","last_undegraded":"2024-06-13T06:56:29.522192+0000","last_fullsized":"2024-06-13T06:56:29.522192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:21:05.624075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532096+0000","last_change":"2024-06-13T06:55:29.383305+0000","last_active":"2024-06-13T06:56:29.532096+0000","last_peered":"2024-06-13T06:56:29.532096+0000","last_clean":"2024-06-13T06:56:29.532096+0000","last_became_active":"2024-06-13T06:55:29.381491+0000","last_became_peered":"2024-06-13T06:55:29.381491+0000","last_unstale":"2024-06-13T06:56:29.532096+0000","last_undegraded":"2024-06-13T06:56:29.532096+0000","last_fullsized":"2024-06-13T06:56:29.532096+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:52:26.437237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607192+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:56:29.607192+0000","last_peered":"2024-06-13T06:56:29.607192+0000","last_clean":"2024-06-13T06:56:29.607192+0000","last_became_active":"2024-06-13T06:55:12.000759+0000","last_became_peered":"2024-06-13T06:55:12.000759+0000","last_unstale":"2024-06-13T06:56:29.607192+0000","last_undegraded":"2024-06-13T06:56:29.607192+0000","last_fullsized":"2024-06-13T06:56:29.607192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:11:35.805338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'6","reported_seq":93,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:04.620533+0000","last_change":"2024-06-13T06:55:29.367538+0000","last_active":"2024-06-13T06:57:04.620533+0000","last_peered":"2024-06-13T06:57:04.620533+0000","last_clean":"2024-06-13T06:57:04.620533+0000","last_became_active":"2024-06-13T06:55:29.367149+0000","last_became_peered":"2024-06-13T06:55:29.367149+0000","last_unstale":"2024-06-13T06:57:04.620533+0000","last_undegraded":"2024-06-13T06:57:04.620533+0000","last_fullsized":"2024-06-13T06:57:04.620533+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:30:55.632799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522180+0000","last_change":"2024-06-13T06:54:58.896258+0000","last_active":"2024-06-13T06:56:29.522180+0000","last_peered":"2024-06-13T06:56:29.522180+0000","last_clean":"2024-06-13T06:56:29.522180+0000","last_became_active":"2024-06-13T06:54:58.895904+0000","last_became_peered":"2024-06-13T06:54:58.895904+0000","last_unstale":"2024-06-13T06:56:29.522180+0000","last_undegraded":"2024-06-13T06:56:29.522180+0000","last_fullsized":"2024-06-13T06:56:29.522180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:28:33.615338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607190+0000","last_change":"2024-06-13T06:55:12.018499+0000","last_active":"2024-06-13T06:56:29.607190+0000","last_peered":"2024-06-13T06:56:29.607190+0000","last_clean":"2024-06-13T06:56:29.607190+0000","last_became_active":"2024-06-13T06:55:12.018073+0000","last_became_peered":"2024-06-13T06:55:12.018073+0000","last_unstale":"2024-06-13T06:56:29.607190+0000","last_undegraded":"2024-06-13T06:56:29.607190+0000","last_fullsized":"2024-06-13T06:56:29.607190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:59:53.206259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523078+0000","last_change":"2024-06-13T06:54:54.339482+0000","last_active":"2024-06-13T06:56:29.523078+0000","last_peered":"2024-06-13T06:56:29.523078+0000","last_clean":"2024-06-13T06:56:29.523078+0000","last_became_active":"2024-06-13T06:54:54.339066+0000","last_became_peered":"2024-06-13T06:54:54.339066+0000","last_unstale":"2024-06-13T06:56:29.523078+0000","last_undegraded":"2024-06-13T06:56:29.523078+0000","last_fullsized":"2024-06-13T06:56:29.523078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:31:04.136530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525738+0000","last_change":"2024-06-13T06:54:58.891154+0000","last_active":"2024-06-13T06:56:29.525738+0000","last_peered":"2024-06-13T06:56:29.525738+0000","last_clean":"2024-06-13T06:56:29.525738+0000","last_became_active":"2024-06-13T06:54:58.890810+0000","last_became_peered":"2024-06-13T06:54:58.890810+0000","last_unstale":"2024-06-13T06:56:29.525738+0000","last_undegraded":"2024-06-13T06:56:29.525738+0000","last_fullsized":"2024-06-13T06:56:29.525738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:14.618233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524690+0000","last_change":"2024-06-13T06:55:12.025619+0000","last_active":"2024-06-13T06:56:29.524690+0000","last_peered":"2024-06-13T06:56:29.524690+0000","last_clean":"2024-06-13T06:56:29.524690+0000","last_became_active":"2024-06-13T06:55:12.025153+0000","last_became_peered":"2024-06-13T06:55:12.025153+0000","last_unstale":"2024-06-13T06:56:29.524690+0000","last_undegraded":"2024-06-13T06:56:29.524690+0000","last_fullsized":"2024-06-13T06:56:29.524690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:12.124550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.596726+0000","last_change":"2024-06-13T06:54:54.341698+0000","last_active":"2024-06-13T06:56:29.596726+0000","last_peered":"2024-06-13T06:56:29.596726+0000","last_clean":"2024-06-13T06:56:29.596726+0000","last_became_active":"2024-06-13T06:54:54.341113+0000","last_became_peered":"2024-06-13T06:54:54.341113+0000","last_unstale":"2024-06-13T06:56:29.596726+0000","last_undegraded":"2024-06-13T06:56:29.596726+0000","last_fullsized":"2024-06-13T06:56:29.596726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:18:23.368705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524079+0000","last_change":"2024-06-13T06:54:58.913688+0000","last_active":"2024-06-13T06:56:29.524079+0000","last_peered":"2024-06-13T06:56:29.524079+0000","last_clean":"2024-06-13T06:56:29.524079+0000","last_became_active":"2024-06-13T06:54:58.910493+0000","last_became_peered":"2024-06-13T06:54:58.910493+0000","last_unstale":"2024-06-13T06:56:29.524079+0000","last_undegraded":"2024-06-13T06:56:29.524079+0000","last_fullsized":"2024-06-13T06:56:29.524079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:29:18.081743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.531815+0000","last_change":"2024-06-13T06:55:12.016257+0000","last_active":"2024-06-13T06:56:29.531815+0000","last_peered":"2024-06-13T06:56:29.531815+0000","last_clean":"2024-06-13T06:56:29.531815+0000","last_became_active":"2024-06-13T06:55:12.015594+0000","last_became_peered":"2024-06-13T06:55:12.015594+0000","last_unstale":"2024-06-13T06:56:29.531815+0000","last_undegraded":"2024-06-13T06:56:29.531815+0000","last_fullsized":"2024-06-13T06:56:29.531815+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:02:44.642075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525674+0000","last_change":"2024-06-13T06:54:54.330366+0000","last_active":"2024-06-13T06:56:29.525674+0000","last_peered":"2024-06-13T06:56:29.525674+0000","last_clean":"2024-06-13T06:56:29.525674+0000","last_became_active":"2024-06-13T06:54:54.330010+0000","last_became_peered":"2024-06-13T06:54:54.330010+0000","last_unstale":"2024-06-13T06:56:29.525674+0000","last_undegraded":"2024-06-13T06:56:29.525674+0000","last_fullsized":"2024-06-13T06:56:29.525674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:36:01.488766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.531617+0000","last_change":"2024-06-13T06:54:58.918076+0000","last_active":"2024-06-13T06:56:29.531617+0000","last_peered":"2024-06-13T06:56:29.531617+0000","last_clean":"2024-06-13T06:56:29.531617+0000","last_became_active":"2024-06-13T06:54:58.917631+0000","last_became_peered":"2024-06-13T06:54:58.917631+0000","last_unstale":"2024-06-13T06:56:29.531617+0000","last_undegraded":"2024-06-13T06:56:29.531617+0000","last_fullsized":"2024-06-13T06:56:29.531617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:31:00.056837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.611761+0000","last_change":"2024-06-13T06:56:29.611761+0000","last_active":"2024-06-13T06:56:29.611761+0000","last_peered":"2024-06-13T06:56:29.611761+0000","last_clean":"2024-06-13T06:56:29.611761+0000","last_became_active":"2024-06-13T06:56:29.610385+0000","last_became_peered":"2024-06-13T06:56:29.610385+0000","last_unstale":"2024-06-13T06:56:29.611761+0000","last_undegraded":"2024-06-13T06:56:29.611761+0000","last_fullsized":"2024-06-13T06:56:29.611761+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:26:10.678918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":83,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:06.623641+0000","last_change":"2024-06-13T06:54:54.335324+0000","last_active":"2024-06-13T06:57:06.623641+0000","last_peered":"2024-06-13T06:57:06.623641+0000","last_clean":"2024-06-13T06:57:06.623641+0000","last_became_active":"2024-06-13T06:54:54.334936+0000","last_became_peered":"2024-06-13T06:54:54.334936+0000","last_unstale":"2024-06-13T06:57:06.623641+0000","last_undegraded":"2024-06-13T06:57:06.623641+0000","last_fullsized":"2024-06-13T06:57:06.623641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:41:52.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608833+0000","last_change":"2024-06-13T06:54:58.902146+0000","last_active":"2024-06-13T06:56:29.608833+0000","last_peered":"2024-06-13T06:56:29.608833+0000","last_clean":"2024-06-13T06:56:29.608833+0000","last_became_active":"2024-06-13T06:54:58.901807+0000","last_became_peered":"2024-06-13T06:54:58.901807+0000","last_unstale":"2024-06-13T06:56:29.608833+0000","last_undegraded":"2024-06-13T06:56:29.608833+0000","last_fullsized":"2024-06-13T06:56:29.608833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:59:53.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533051+0000","last_change":"2024-06-13T06:55:12.025418+0000","last_active":"2024-06-13T06:56:29.533051+0000","last_peered":"2024-06-13T06:56:29.533051+0000","last_clean":"2024-06-13T06:56:29.533051+0000","last_became_active":"2024-06-13T06:55:12.024859+0000","last_became_peered":"2024-06-13T06:55:12.024859+0000","last_unstale":"2024-06-13T06:56:29.533051+0000","last_undegraded":"2024-06-13T06:56:29.533051+0000","last_fullsized":"2024-06-13T06:56:29.533051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:05:17.553169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609367+0000","last_change":"2024-06-13T06:54:54.326011+0000","last_active":"2024-06-13T06:56:29.609367+0000","last_peered":"2024-06-13T06:56:29.609367+0000","last_clean":"2024-06-13T06:56:29.609367+0000","last_became_active":"2024-06-13T06:54:54.325664+0000","last_became_peered":"2024-06-13T06:54:54.325664+0000","last_unstale":"2024-06-13T06:56:29.609367+0000","last_undegraded":"2024-06-13T06:56:29.609367+0000","last_fullsized":"2024-06-13T06:56:29.609367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:32.973094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532335+0000","last_change":"2024-06-13T06:55:29.383612+0000","last_active":"2024-06-13T06:56:29.532335+0000","last_peered":"2024-06-13T06:56:29.532335+0000","last_clean":"2024-06-13T06:56:29.532335+0000","last_became_active":"2024-06-13T06:55:29.382174+0000","last_became_peered":"2024-06-13T06:55:29.382174+0000","last_unstale":"2024-06-13T06:56:29.532335+0000","last_undegraded":"2024-06-13T06:56:29.532335+0000","last_fullsized":"2024-06-13T06:56:29.532335+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:09:35.178209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608655+0000","last_change":"2024-06-13T06:55:12.018654+0000","last_active":"2024-06-13T06:56:29.608655+0000","last_peered":"2024-06-13T06:56:29.608655+0000","last_clean":"2024-06-13T06:56:29.608655+0000","last_became_active":"2024-06-13T06:55:12.017219+0000","last_became_peered":"2024-06-13T06:55:12.017219+0000","last_unstale":"2024-06-13T06:56:29.608655+0000","last_undegraded":"2024-06-13T06:56:29.608655+0000","last_fullsized":"2024-06-13T06:56:29.608655+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:46:54.161912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526453+0000","last_change":"2024-06-13T06:54:54.632154+0000","last_active":"2024-06-13T06:56:29.526453+0000","last_peered":"2024-06-13T06:56:29.526453+0000","last_clean":"2024-06-13T06:56:29.526453+0000","last_became_active":"2024-06-13T06:54:54.631721+0000","last_became_peered":"2024-06-13T06:54:54.631721+0000","last_unstale":"2024-06-13T06:56:29.526453+0000","last_undegraded":"2024-06-13T06:56:29.526453+0000","last_fullsized":"2024-06-13T06:56:29.526453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:16:10.492144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607473+0000","last_change":"2024-06-13T06:54:58.898961+0000","last_active":"2024-06-13T06:56:29.607473+0000","last_peered":"2024-06-13T06:56:29.607473+0000","last_clean":"2024-06-13T06:56:29.607473+0000","last_became_active":"2024-06-13T06:54:58.898599+0000","last_became_peered":"2024-06-13T06:54:58.898599+0000","last_unstale":"2024-06-13T06:56:29.607473+0000","last_undegraded":"2024-06-13T06:56:29.607473+0000","last_fullsized":"2024-06-13T06:56:29.607473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:21:53.044205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.527672+0000","last_change":"2024-06-13T06:56:29.527242+0000","last_active":"2024-06-13T06:56:29.527672+0000","last_peered":"2024-06-13T06:56:29.527672+0000","last_clean":"2024-06-13T06:56:29.527672+0000","last_became_active":"2024-06-13T06:56:29.526816+0000","last_became_peered":"2024-06-13T06:56:29.526816+0000","last_unstale":"2024-06-13T06:56:29.527672+0000","last_undegraded":"2024-06-13T06:56:29.527672+0000","last_fullsized":"2024-06-13T06:56:29.527672+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:45:17.770029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'32","reported_seq":171,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:07.620942+0000","last_change":"2024-06-13T06:55:29.376808+0000","last_active":"2024-06-13T06:57:07.620942+0000","last_peered":"2024-06-13T06:57:07.620942+0000","last_clean":"2024-06-13T06:57:07.620942+0000","last_became_active":"2024-06-13T06:55:29.367134+0000","last_became_peered":"2024-06-13T06:55:29.367134+0000","last_unstale":"2024-06-13T06:57:07.620942+0000","last_undegraded":"2024-06-13T06:57:07.620942+0000","last_fullsized":"2024-06-13T06:57:07.620942+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:19:08.784438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58390,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":79,"num_read_kb":97,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522524+0000","last_change":"2024-06-13T06:54:58.897686+0000","last_active":"2024-06-13T06:56:29.522524+0000","last_peered":"2024-06-13T06:56:29.522524+0000","last_clean":"2024-06-13T06:56:29.522524+0000","last_became_active":"2024-06-13T06:54:58.897313+0000","last_became_peered":"2024-06-13T06:54:58.897313+0000","last_unstale":"2024-06-13T06:56:29.522524+0000","last_undegraded":"2024-06-13T06:56:29.522524+0000","last_fullsized":"2024-06-13T06:56:29.522524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:54:49.558542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526734+0000","last_change":"2024-06-13T06:55:12.013918+0000","last_active":"2024-06-13T06:56:29.526734+0000","last_peered":"2024-06-13T06:56:29.526734+0000","last_clean":"2024-06-13T06:56:29.526734+0000","last_became_active":"2024-06-13T06:55:12.013522+0000","last_became_peered":"2024-06-13T06:55:12.013522+0000","last_unstale":"2024-06-13T06:56:29.526734+0000","last_undegraded":"2024-06-13T06:56:29.526734+0000","last_fullsized":"2024-06-13T06:56:29.526734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:51:46.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607084+0000","last_change":"2024-06-13T06:54:54.328674+0000","last_active":"2024-06-13T06:56:29.607084+0000","last_peered":"2024-06-13T06:56:29.607084+0000","last_clean":"2024-06-13T06:56:29.607084+0000","last_became_active":"2024-06-13T06:54:54.326789+0000","last_became_peered":"2024-06-13T06:54:54.326789+0000","last_unstale":"2024-06-13T06:56:29.607084+0000","last_undegraded":"2024-06-13T06:56:29.607084+0000","last_fullsized":"2024-06-13T06:56:29.607084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:18.280514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.599635+0000","last_change":"2024-06-13T06:54:58.918598+0000","last_active":"2024-06-13T06:56:29.599635+0000","last_peered":"2024-06-13T06:56:29.599635+0000","last_clean":"2024-06-13T06:56:29.599635+0000","last_became_active":"2024-06-13T06:54:58.918247+0000","last_became_peered":"2024-06-13T06:54:58.918247+0000","last_unstale":"2024-06-13T06:56:29.599635+0000","last_undegraded":"2024-06-13T06:56:29.599635+0000","last_fullsized":"2024-06-13T06:56:29.599635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:08:56.738806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524251+0000","last_change":"2024-06-13T06:55:12.017074+0000","last_active":"2024-06-13T06:56:29.524251+0000","last_peered":"2024-06-13T06:56:29.524251+0000","last_clean":"2024-06-13T06:56:29.524251+0000","last_became_active":"2024-06-13T06:55:12.016675+0000","last_became_peered":"2024-06-13T06:55:12.016675+0000","last_unstale":"2024-06-13T06:56:29.524251+0000","last_undegraded":"2024-06-13T06:56:29.524251+0000","last_fullsized":"2024-06-13T06:56:29.524251+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:20:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525911+0000","last_change":"2024-06-13T06:54:54.334994+0000","last_active":"2024-06-13T06:56:29.525911+0000","last_peered":"2024-06-13T06:56:29.525911+0000","last_clean":"2024-06-13T06:56:29.525911+0000","last_became_active":"2024-06-13T06:54:54.334647+0000","last_became_peered":"2024-06-13T06:54:54.334647+0000","last_unstale":"2024-06-13T06:56:29.525911+0000","last_undegraded":"2024-06-13T06:56:29.525911+0000","last_fullsized":"2024-06-13T06:56:29.525911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:59:15.317976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608697+0000","last_change":"2024-06-13T06:54:58.899107+0000","last_active":"2024-06-13T06:56:29.608697+0000","last_peered":"2024-06-13T06:56:29.608697+0000","last_clean":"2024-06-13T06:56:29.608697+0000","last_became_active":"2024-06-13T06:54:58.896034+0000","last_became_peered":"2024-06-13T06:54:58.896034+0000","last_unstale":"2024-06-13T06:56:29.608697+0000","last_undegraded":"2024-06-13T06:56:29.608697+0000","last_fullsized":"2024-06-13T06:56:29.608697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:00:01.132565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533178+0000","last_change":"2024-06-13T06:55:12.025681+0000","last_active":"2024-06-13T06:56:29.533178+0000","last_peered":"2024-06-13T06:56:29.533178+0000","last_clean":"2024-06-13T06:56:29.533178+0000","last_became_active":"2024-06-13T06:55:12.025267+0000","last_became_peered":"2024-06-13T06:55:12.025267+0000","last_unstale":"2024-06-13T06:56:29.533178+0000","last_undegraded":"2024-06-13T06:56:29.533178+0000","last_fullsized":"2024-06-13T06:56:29.533178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:58:23.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:06.622689+0000","last_change":"2024-06-13T06:54:54.325404+0000","last_active":"2024-06-13T06:57:06.622689+0000","last_peered":"2024-06-13T06:57:06.622689+0000","last_clean":"2024-06-13T06:57:06.622689+0000","last_became_active":"2024-06-13T06:54:54.325024+0000","last_became_peered":"2024-06-13T06:54:54.325024+0000","last_unstale":"2024-06-13T06:57:06.622689+0000","last_undegraded":"2024-06-13T06:57:06.622689+0000","last_fullsized":"2024-06-13T06:57:06.622689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:23.726350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607302+0000","last_change":"2024-06-13T06:54:58.894737+0000","last_active":"2024-06-13T06:56:29.607302+0000","last_peered":"2024-06-13T06:56:29.607302+0000","last_clean":"2024-06-13T06:56:29.607302+0000","last_became_active":"2024-06-13T06:54:58.894224+0000","last_became_peered":"2024-06-13T06:54:58.894224+0000","last_unstale":"2024-06-13T06:56:29.607302+0000","last_undegraded":"2024-06-13T06:56:29.607302+0000","last_fullsized":"2024-06-13T06:56:29.607302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:03.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600126+0000","last_change":"2024-06-13T06:55:12.026587+0000","last_active":"2024-06-13T06:56:29.600126+0000","last_peered":"2024-06-13T06:56:29.600126+0000","last_clean":"2024-06-13T06:56:29.600126+0000","last_became_active":"2024-06-13T06:55:12.026114+0000","last_became_peered":"2024-06-13T06:55:12.026114+0000","last_unstale":"2024-06-13T06:56:29.600126+0000","last_undegraded":"2024-06-13T06:56:29.600126+0000","last_fullsized":"2024-06-13T06:56:29.600126+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:43:09.245641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522168+0000","last_change":"2024-06-13T06:54:54.632069+0000","last_active":"2024-06-13T06:56:29.522168+0000","last_peered":"2024-06-13T06:56:29.522168+0000","last_clean":"2024-06-13T06:56:29.522168+0000","last_became_active":"2024-06-13T06:54:54.631692+0000","last_became_peered":"2024-06-13T06:54:54.631692+0000","last_unstale":"2024-06-13T06:56:29.522168+0000","last_undegraded":"2024-06-13T06:56:29.522168+0000","last_fullsized":"2024-06-13T06:56:29.522168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:07.109507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526117+0000","last_change":"2024-06-13T06:55:29.366286+0000","last_active":"2024-06-13T06:56:29.526117+0000","last_peered":"2024-06-13T06:56:29.526117+0000","last_clean":"2024-06-13T06:56:29.526117+0000","last_became_active":"2024-06-13T06:55:29.365854+0000","last_became_peered":"2024-06-13T06:55:29.365854+0000","last_unstale":"2024-06-13T06:56:29.526117+0000","last_undegraded":"2024-06-13T06:56:29.526117+0000","last_fullsized":"2024-06-13T06:56:29.526117+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:04.209244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526156+0000","last_change":"2024-06-13T06:55:12.021621+0000","last_active":"2024-06-13T06:56:29.526156+0000","last_peered":"2024-06-13T06:56:29.526156+0000","last_clean":"2024-06-13T06:56:29.526156+0000","last_became_active":"2024-06-13T06:55:12.021150+0000","last_became_peered":"2024-06-13T06:55:12.021150+0000","last_unstale":"2024-06-13T06:56:29.526156+0000","last_undegraded":"2024-06-13T06:56:29.526156+0000","last_fullsized":"2024-06-13T06:56:29.526156+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:48.318918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:30.508342+0000","last_change":"2024-06-13T06:55:29.369972+0000","last_active":"2024-06-13T06:56:30.508342+0000","last_peered":"2024-06-13T06:56:30.508342+0000","last_clean":"2024-06-13T06:56:30.508342+0000","last_became_active":"2024-06-13T06:55:29.366390+0000","last_became_peered":"2024-06-13T06:55:29.366390+0000","last_unstale":"2024-06-13T06:56:30.508342+0000","last_undegraded":"2024-06-13T06:56:30.508342+0000","last_fullsized":"2024-06-13T06:56:30.508342+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:18.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524659+0000","last_change":"2024-06-13T06:54:58.891309+0000","last_active":"2024-06-13T06:56:29.524659+0000","last_peered":"2024-06-13T06:56:29.524659+0000","last_clean":"2024-06-13T06:56:29.524659+0000","last_became_active":"2024-06-13T06:54:58.890972+0000","last_became_peered":"2024-06-13T06:54:58.890972+0000","last_unstale":"2024-06-13T06:56:29.524659+0000","last_undegraded":"2024-06-13T06:56:29.524659+0000","last_fullsized":"2024-06-13T06:56:29.524659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:41.240061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.610713+0000","last_change":"2024-06-13T06:56:29.610713+0000","last_active":"2024-06-13T06:56:29.610713+0000","last_peered":"2024-06-13T06:56:29.610713+0000","last_clean":"2024-06-13T06:56:29.610713+0000","last_became_active":"2024-06-13T06:56:29.610139+0000","last_became_peered":"2024-06-13T06:56:29.610139+0000","last_unstale":"2024-06-13T06:56:29.610713+0000","last_undegraded":"2024-06-13T06:56:29.610713+0000","last_fullsized":"2024-06-13T06:56:29.610713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:14:36.601243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.534462+0000","last_change":"2024-06-13T06:54:54.633893+0000","last_active":"2024-06-13T06:56:29.534462+0000","last_peered":"2024-06-13T06:56:29.534462+0000","last_clean":"2024-06-13T06:56:29.534462+0000","last_became_active":"2024-06-13T06:54:54.632946+0000","last_became_peered":"2024-06-13T06:54:54.632946+0000","last_unstale":"2024-06-13T06:56:29.534462+0000","last_undegraded":"2024-06-13T06:56:29.534462+0000","last_fullsized":"2024-06-13T06:56:29.534462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:16:54.812477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522730+0000","last_change":"2024-06-13T06:54:58.907182+0000","last_active":"2024-06-13T06:56:29.522730+0000","last_peered":"2024-06-13T06:56:29.522730+0000","last_clean":"2024-06-13T06:56:29.522730+0000","last_became_active":"2024-06-13T06:54:58.906824+0000","last_became_peered":"2024-06-13T06:54:58.906824+0000","last_unstale":"2024-06-13T06:56:29.522730+0000","last_undegraded":"2024-06-13T06:56:29.522730+0000","last_fullsized":"2024-06-13T06:56:29.522730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:34:49.626991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.597070+0000","last_change":"2024-06-13T06:55:12.014803+0000","last_active":"2024-06-13T06:56:29.597070+0000","last_peered":"2024-06-13T06:56:29.597070+0000","last_clean":"2024-06-13T06:56:29.597070+0000","last_became_active":"2024-06-13T06:55:12.014398+0000","last_became_peered":"2024-06-13T06:55:12.014398+0000","last_unstale":"2024-06-13T06:56:29.597070+0000","last_undegraded":"2024-06-13T06:56:29.597070+0000","last_fullsized":"2024-06-13T06:56:29.597070+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:07:44.960446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607834+0000","last_change":"2024-06-13T06:54:54.339056+0000","last_active":"2024-06-13T06:56:29.607834+0000","last_peered":"2024-06-13T06:56:29.607834+0000","last_clean":"2024-06-13T06:56:29.607834+0000","last_became_active":"2024-06-13T06:54:54.338683+0000","last_became_peered":"2024-06-13T06:54:54.338683+0000","last_unstale":"2024-06-13T06:56:29.607834+0000","last_undegraded":"2024-06-13T06:56:29.607834+0000","last_fullsized":"2024-06-13T06:56:29.607834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:05:17.060471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522498+0000","last_change":"2024-06-13T06:54:58.919843+0000","last_active":"2024-06-13T06:56:29.522498+0000","last_peered":"2024-06-13T06:56:29.522498+0000","last_clean":"2024-06-13T06:56:29.522498+0000","last_became_active":"2024-06-13T06:54:58.919405+0000","last_became_peered":"2024-06-13T06:54:58.919405+0000","last_unstale":"2024-06-13T06:56:29.522498+0000","last_undegraded":"2024-06-13T06:56:29.522498+0000","last_fullsized":"2024-06-13T06:56:29.522498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:00:20.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600039+0000","last_change":"2024-06-13T06:55:12.097651+0000","last_active":"2024-06-13T06:56:29.600039+0000","last_peered":"2024-06-13T06:56:29.600039+0000","last_clean":"2024-06-13T06:56:29.600039+0000","last_became_active":"2024-06-13T06:55:12.095644+0000","last_became_peered":"2024-06-13T06:55:12.095644+0000","last_unstale":"2024-06-13T06:56:29.600039+0000","last_undegraded":"2024-06-13T06:56:29.600039+0000","last_fullsized":"2024-06-13T06:56:29.600039+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:09:54.328428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526081+0000","last_change":"2024-06-13T06:54:58.891187+0000","last_active":"2024-06-13T06:56:29.526081+0000","last_peered":"2024-06-13T06:56:29.526081+0000","last_clean":"2024-06-13T06:56:29.526081+0000","last_became_active":"2024-06-13T06:54:58.890848+0000","last_became_peered":"2024-06-13T06:54:58.890848+0000","last_unstale":"2024-06-13T06:56:29.526081+0000","last_undegraded":"2024-06-13T06:56:29.526081+0000","last_fullsized":"2024-06-13T06:56:29.526081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:13:31.709650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":84,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:05.619762+0000","last_change":"2024-06-13T06:54:54.631482+0000","last_active":"2024-06-13T06:57:05.619762+0000","last_peered":"2024-06-13T06:57:05.619762+0000","last_clean":"2024-06-13T06:57:05.619762+0000","last_became_active":"2024-06-13T06:54:54.630818+0000","last_became_peered":"2024-06-13T06:54:54.630818+0000","last_unstale":"2024-06-13T06:57:05.619762+0000","last_undegraded":"2024-06-13T06:57:05.619762+0000","last_fullsized":"2024-06-13T06:57:05.619762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:34:18.801727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607252+0000","last_change":"2024-06-13T06:55:12.018856+0000","last_active":"2024-06-13T06:56:29.607252+0000","last_peered":"2024-06-13T06:56:29.607252+0000","last_clean":"2024-06-13T06:56:29.607252+0000","last_became_active":"2024-06-13T06:55:12.018488+0000","last_became_peered":"2024-06-13T06:55:12.018488+0000","last_unstale":"2024-06-13T06:56:29.607252+0000","last_undegraded":"2024-06-13T06:56:29.607252+0000","last_fullsized":"2024-06-13T06:56:29.607252+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:14.973197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532812+0000","last_change":"2024-06-13T06:55:29.377203+0000","last_active":"2024-06-13T06:56:29.532812+0000","last_peered":"2024-06-13T06:56:29.532812+0000","last_clean":"2024-06-13T06:56:29.532812+0000","last_became_active":"2024-06-13T06:55:29.375263+0000","last_became_peered":"2024-06-13T06:55:29.375263+0000","last_unstale":"2024-06-13T06:56:29.532812+0000","last_undegraded":"2024-06-13T06:56:29.532812+0000","last_fullsized":"2024-06-13T06:56:29.532812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:06:07.935172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558452+0000","last_change":"2024-06-13T06:54:54.335319+0000","last_active":"2024-06-13T06:56:32.558452+0000","last_peered":"2024-06-13T06:56:32.558452+0000","last_clean":"2024-06-13T06:56:32.558452+0000","last_became_active":"2024-06-13T06:54:54.334896+0000","last_became_peered":"2024-06-13T06:54:54.334896+0000","last_unstale":"2024-06-13T06:56:32.558452+0000","last_undegraded":"2024-06-13T06:56:32.558452+0000","last_fullsized":"2024-06-13T06:56:32.558452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:40.331535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606983+0000","last_change":"2024-06-13T06:53:16.968830+0000","last_active":"2024-06-13T06:56:29.606983+0000","last_peered":"2024-06-13T06:56:29.606983+0000","last_clean":"2024-06-13T06:56:29.606983+0000","last_became_active":"2024-06-13T06:53:16.968361+0000","last_became_peered":"2024-06-13T06:53:16.968361+0000","last_unstale":"2024-06-13T06:56:29.606983+0000","last_undegraded":"2024-06-13T06:56:29.606983+0000","last_fullsized":"2024-06-13T06:56:29.606983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_clean_scrub_stamp":"2024-06-13T06:49:18.858540+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:32:26.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.602124+0000","last_change":"2024-06-13T06:56:29.602124+0000","last_active":"2024-06-13T06:56:29.602124+0000","last_peered":"2024-06-13T06:56:29.602124+0000","last_clean":"2024-06-13T06:56:29.602124+0000","last_became_active":"2024-06-13T06:56:29.601650+0000","last_became_peered":"2024-06-13T06:56:29.601650+0000","last_unstale":"2024-06-13T06:56:29.602124+0000","last_undegraded":"2024-06-13T06:56:29.602124+0000","last_fullsized":"2024-06-13T06:56:29.602124+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:37:23.903614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.596545+0000","last_change":"2024-06-13T06:54:58.898930+0000","last_active":"2024-06-13T06:56:29.596545+0000","last_peered":"2024-06-13T06:56:29.596545+0000","last_clean":"2024-06-13T06:56:29.596545+0000","last_became_active":"2024-06-13T06:54:58.898384+0000","last_became_peered":"2024-06-13T06:54:58.898384+0000","last_unstale":"2024-06-13T06:56:29.596545+0000","last_undegraded":"2024-06-13T06:56:29.596545+0000","last_fullsized":"2024-06-13T06:56:29.596545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:06:02.210336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607221+0000","last_change":"2024-06-13T06:54:54.328776+0000","last_active":"2024-06-13T06:56:29.607221+0000","last_peered":"2024-06-13T06:56:29.607221+0000","last_clean":"2024-06-13T06:56:29.607221+0000","last_became_active":"2024-06-13T06:54:54.327773+0000","last_became_peered":"2024-06-13T06:54:54.327773+0000","last_unstale":"2024-06-13T06:56:29.607221+0000","last_undegraded":"2024-06-13T06:56:29.607221+0000","last_fullsized":"2024-06-13T06:56:29.607221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:23:02.240774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524397+0000","last_change":"2024-06-13T06:55:12.013862+0000","last_active":"2024-06-13T06:56:29.524397+0000","last_peered":"2024-06-13T06:56:29.524397+0000","last_clean":"2024-06-13T06:56:29.524397+0000","last_became_active":"2024-06-13T06:55:12.013389+0000","last_became_peered":"2024-06-13T06:55:12.013389+0000","last_unstale":"2024-06-13T06:56:29.524397+0000","last_undegraded":"2024-06-13T06:56:29.524397+0000","last_fullsized":"2024-06-13T06:56:29.524397+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:49.701202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.831665+0000","last_change":"2024-06-13T06:56:29.539862+0000","last_active":"2024-06-13T06:56:29.831665+0000","last_peered":"2024-06-13T06:56:29.831665+0000","last_clean":"2024-06-13T06:56:29.831665+0000","last_became_active":"2024-06-13T06:56:29.536109+0000","last_became_peered":"2024-06-13T06:56:29.536109+0000","last_unstale":"2024-06-13T06:56:29.831665+0000","last_undegraded":"2024-06-13T06:56:29.831665+0000","last_fullsized":"2024-06-13T06:56:29.831665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:23.307850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.599679+0000","last_change":"2024-06-13T06:54:58.901773+0000","last_active":"2024-06-13T06:56:29.599679+0000","last_peered":"2024-06-13T06:56:29.599679+0000","last_clean":"2024-06-13T06:56:29.599679+0000","last_became_active":"2024-06-13T06:54:58.901168+0000","last_became_peered":"2024-06-13T06:54:58.901168+0000","last_unstale":"2024-06-13T06:56:29.599679+0000","last_undegraded":"2024-06-13T06:56:29.599679+0000","last_fullsized":"2024-06-13T06:56:29.599679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:57:15.848785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524077+0000","last_change":"2024-06-13T06:55:12.095890+0000","last_active":"2024-06-13T06:56:29.524077+0000","last_peered":"2024-06-13T06:56:29.524077+0000","last_clean":"2024-06-13T06:56:29.524077+0000","last_became_active":"2024-06-13T06:55:12.095397+0000","last_became_peered":"2024-06-13T06:55:12.095397+0000","last_unstale":"2024-06-13T06:56:29.524077+0000","last_undegraded":"2024-06-13T06:56:29.524077+0000","last_fullsized":"2024-06-13T06:56:29.524077+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:56:18.211036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'11","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:04.939835+0000","last_change":"2024-06-13T06:54:54.337946+0000","last_active":"2024-06-13T06:57:04.939835+0000","last_peered":"2024-06-13T06:57:04.939835+0000","last_clean":"2024-06-13T06:57:04.939835+0000","last_became_active":"2024-06-13T06:54:54.337028+0000","last_became_peered":"2024-06-13T06:54:54.337028+0000","last_unstale":"2024-06-13T06:57:04.939835+0000","last_undegraded":"2024-06-13T06:57:04.939835+0000","last_fullsized":"2024-06-13T06:57:04.939835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:08:52.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526176+0000","last_change":"2024-06-13T06:54:58.895110+0000","last_active":"2024-06-13T06:56:29.526176+0000","last_peered":"2024-06-13T06:56:29.526176+0000","last_clean":"2024-06-13T06:56:29.526176+0000","last_became_active":"2024-06-13T06:54:58.893481+0000","last_became_peered":"2024-06-13T06:54:58.893481+0000","last_unstale":"2024-06-13T06:56:29.526176+0000","last_undegraded":"2024-06-13T06:56:29.526176+0000","last_fullsized":"2024-06-13T06:56:29.526176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:54.932110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606822+0000","last_change":"2024-06-13T06:55:11.997712+0000","last_active":"2024-06-13T06:56:29.606822+0000","last_peered":"2024-06-13T06:56:29.606822+0000","last_clean":"2024-06-13T06:56:29.606822+0000","last_became_active":"2024-06-13T06:55:11.997193+0000","last_became_peered":"2024-06-13T06:55:11.997193+0000","last_unstale":"2024-06-13T06:56:29.606822+0000","last_undegraded":"2024-06-13T06:56:29.606822+0000","last_fullsized":"2024-06-13T06:56:29.606822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:15.168070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608714+0000","last_change":"2024-06-13T06:54:54.520121+0000","last_active":"2024-06-13T06:56:29.608714+0000","last_peered":"2024-06-13T06:56:29.608714+0000","last_clean":"2024-06-13T06:56:29.608714+0000","last_became_active":"2024-06-13T06:54:54.519723+0000","last_became_peered":"2024-06-13T06:54:54.519723+0000","last_unstale":"2024-06-13T06:56:29.608714+0000","last_undegraded":"2024-06-13T06:56:29.608714+0000","last_fullsized":"2024-06-13T06:56:29.608714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:19:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532474+0000","last_change":"2024-06-13T06:54:58.923626+0000","last_active":"2024-06-13T06:56:29.532474+0000","last_peered":"2024-06-13T06:56:29.532474+0000","last_clean":"2024-06-13T06:56:29.532474+0000","last_became_active":"2024-06-13T06:54:58.923192+0000","last_became_peered":"2024-06-13T06:54:58.923192+0000","last_unstale":"2024-06-13T06:56:29.532474+0000","last_undegraded":"2024-06-13T06:56:29.532474+0000","last_fullsized":"2024-06-13T06:56:29.532474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:26:13.231549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607658+0000","last_change":"2024-06-13T06:55:12.096244+0000","last_active":"2024-06-13T06:56:29.607658+0000","last_peered":"2024-06-13T06:56:29.607658+0000","last_clean":"2024-06-13T06:56:29.607658+0000","last_became_active":"2024-06-13T06:55:12.095758+0000","last_became_peered":"2024-06-13T06:55:12.095758+0000","last_unstale":"2024-06-13T06:56:29.607658+0000","last_undegraded":"2024-06-13T06:56:29.607658+0000","last_fullsized":"2024-06-13T06:56:29.607658+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:38.122282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607931+0000","last_change":"2024-06-13T06:54:54.630488+0000","last_active":"2024-06-13T06:56:29.607931+0000","last_peered":"2024-06-13T06:56:29.607931+0000","last_clean":"2024-06-13T06:56:29.607931+0000","last_became_active":"2024-06-13T06:54:54.630224+0000","last_became_peered":"2024-06-13T06:54:54.630224+0000","last_unstale":"2024-06-13T06:56:29.607931+0000","last_undegraded":"2024-06-13T06:56:29.607931+0000","last_fullsized":"2024-06-13T06:56:29.607931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:33:19.107026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522127+0000","last_change":"2024-06-13T06:54:58.894466+0000","last_active":"2024-06-13T06:56:29.522127+0000","last_peered":"2024-06-13T06:56:29.522127+0000","last_clean":"2024-06-13T06:56:29.522127+0000","last_became_active":"2024-06-13T06:54:58.894034+0000","last_became_peered":"2024-06-13T06:54:58.894034+0000","last_unstale":"2024-06-13T06:56:29.522127+0000","last_undegraded":"2024-06-13T06:56:29.522127+0000","last_fullsized":"2024-06-13T06:56:29.522127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:44.051565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533541+0000","last_change":"2024-06-13T06:55:12.022078+0000","last_active":"2024-06-13T06:56:29.533541+0000","last_peered":"2024-06-13T06:56:29.533541+0000","last_clean":"2024-06-13T06:56:29.533541+0000","last_became_active":"2024-06-13T06:55:12.021652+0000","last_became_peered":"2024-06-13T06:55:12.021652+0000","last_unstale":"2024-06-13T06:56:29.533541+0000","last_undegraded":"2024-06-13T06:56:29.533541+0000","last_fullsized":"2024-06-13T06:56:29.533541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:25:57.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524569+0000","last_change":"2024-06-13T06:54:54.336757+0000","last_active":"2024-06-13T06:56:29.524569+0000","last_peered":"2024-06-13T06:56:29.524569+0000","last_clean":"2024-06-13T06:56:29.524569+0000","last_became_active":"2024-06-13T06:54:54.336012+0000","last_became_peered":"2024-06-13T06:54:54.336012+0000","last_unstale":"2024-06-13T06:56:29.524569+0000","last_undegraded":"2024-06-13T06:56:29.524569+0000","last_fullsized":"2024-06-13T06:56:29.524569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:52:21.227464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524562+0000","last_change":"2024-06-13T06:54:58.919631+0000","last_active":"2024-06-13T06:56:29.524562+0000","last_peered":"2024-06-13T06:56:29.524562+0000","last_clean":"2024-06-13T06:56:29.524562+0000","last_became_active":"2024-06-13T06:54:58.919274+0000","last_became_peered":"2024-06-13T06:54:58.919274+0000","last_unstale":"2024-06-13T06:56:29.524562+0000","last_undegraded":"2024-06-13T06:56:29.524562+0000","last_fullsized":"2024-06-13T06:56:29.524562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:22.453797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608429+0000","last_change":"2024-06-13T06:55:12.017338+0000","last_active":"2024-06-13T06:56:29.608429+0000","last_peered":"2024-06-13T06:56:29.608429+0000","last_clean":"2024-06-13T06:56:29.608429+0000","last_became_active":"2024-06-13T06:55:12.016841+0000","last_became_peered":"2024-06-13T06:55:12.016841+0000","last_unstale":"2024-06-13T06:56:29.608429+0000","last_undegraded":"2024-06-13T06:56:29.608429+0000","last_fullsized":"2024-06-13T06:56:29.608429+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:40:13.525892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607062+0000","last_change":"2024-06-13T06:54:54.328808+0000","last_active":"2024-06-13T06:56:29.607062+0000","last_peered":"2024-06-13T06:56:29.607062+0000","last_clean":"2024-06-13T06:56:29.607062+0000","last_became_active":"2024-06-13T06:54:54.326982+0000","last_became_peered":"2024-06-13T06:54:54.326982+0000","last_unstale":"2024-06-13T06:56:29.607062+0000","last_undegraded":"2024-06-13T06:56:29.607062+0000","last_fullsized":"2024-06-13T06:56:29.607062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:22:53.854275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524893+0000","last_change":"2024-06-13T06:54:58.925728+0000","last_active":"2024-06-13T06:56:29.524893+0000","last_peered":"2024-06-13T06:56:29.524893+0000","last_clean":"2024-06-13T06:56:29.524893+0000","last_became_active":"2024-06-13T06:54:58.925407+0000","last_became_peered":"2024-06-13T06:54:58.925407+0000","last_unstale":"2024-06-13T06:56:29.524893+0000","last_undegraded":"2024-06-13T06:56:29.524893+0000","last_fullsized":"2024-06-13T06:56:29.524893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:38.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607449+0000","last_change":"2024-06-13T06:55:12.095733+0000","last_active":"2024-06-13T06:56:29.607449+0000","last_peered":"2024-06-13T06:56:29.607449+0000","last_clean":"2024-06-13T06:56:29.607449+0000","last_became_active":"2024-06-13T06:55:12.095064+0000","last_became_peered":"2024-06-13T06:55:12.095064+0000","last_unstale":"2024-06-13T06:56:29.607449+0000","last_undegraded":"2024-06-13T06:56:29.607449+0000","last_fullsized":"2024-06-13T06:56:29.607449+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:26:05.486701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556112+0000","last_change":"2024-06-13T06:54:54.331448+0000","last_active":"2024-06-13T06:56:32.556112+0000","last_peered":"2024-06-13T06:56:32.556112+0000","last_clean":"2024-06-13T06:56:32.556112+0000","last_became_active":"2024-06-13T06:54:54.330964+0000","last_became_peered":"2024-06-13T06:54:54.330964+0000","last_unstale":"2024-06-13T06:56:32.556112+0000","last_undegraded":"2024-06-13T06:56:32.556112+0000","last_fullsized":"2024-06-13T06:56:32.556112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:23:12.211412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607683+0000","last_change":"2024-06-13T06:54:58.899112+0000","last_active":"2024-06-13T06:56:29.607683+0000","last_peered":"2024-06-13T06:56:29.607683+0000","last_clean":"2024-06-13T06:56:29.607683+0000","last_became_active":"2024-06-13T06:54:58.898742+0000","last_became_peered":"2024-06-13T06:54:58.898742+0000","last_unstale":"2024-06-13T06:56:29.607683+0000","last_undegraded":"2024-06-13T06:56:29.607683+0000","last_fullsized":"2024-06-13T06:56:29.607683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:50:12.983287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608176+0000","last_change":"2024-06-13T06:55:12.014149+0000","last_active":"2024-06-13T06:56:29.608176+0000","last_peered":"2024-06-13T06:56:29.608176+0000","last_clean":"2024-06-13T06:56:29.608176+0000","last_became_active":"2024-06-13T06:55:12.013454+0000","last_became_peered":"2024-06-13T06:55:12.013454+0000","last_unstale":"2024-06-13T06:56:29.608176+0000","last_undegraded":"2024-06-13T06:56:29.608176+0000","last_fullsized":"2024-06-13T06:56:29.608176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:13:39.368540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607420+0000","last_change":"2024-06-13T06:54:54.633268+0000","last_active":"2024-06-13T06:56:29.607420+0000","last_peered":"2024-06-13T06:56:29.607420+0000","last_clean":"2024-06-13T06:56:29.607420+0000","last_became_active":"2024-06-13T06:54:54.632840+0000","last_became_peered":"2024-06-13T06:54:54.632840+0000","last_unstale":"2024-06-13T06:56:29.607420+0000","last_undegraded":"2024-06-13T06:56:29.607420+0000","last_fullsized":"2024-06-13T06:56:29.607420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:21.653727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609371+0000","last_change":"2024-06-13T06:54:58.893417+0000","last_active":"2024-06-13T06:56:29.609371+0000","last_peered":"2024-06-13T06:56:29.609371+0000","last_clean":"2024-06-13T06:56:29.609371+0000","last_became_active":"2024-06-13T06:54:58.892381+0000","last_became_peered":"2024-06-13T06:54:58.892381+0000","last_unstale":"2024-06-13T06:56:29.609371+0000","last_undegraded":"2024-06-13T06:56:29.609371+0000","last_fullsized":"2024-06-13T06:56:29.609371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:07.979654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533388+0000","last_change":"2024-06-13T06:55:12.018336+0000","last_active":"2024-06-13T06:56:29.533388+0000","last_peered":"2024-06-13T06:56:29.533388+0000","last_clean":"2024-06-13T06:56:29.533388+0000","last_became_active":"2024-06-13T06:55:12.017861+0000","last_became_peered":"2024-06-13T06:55:12.017861+0000","last_unstale":"2024-06-13T06:56:29.533388+0000","last_undegraded":"2024-06-13T06:56:29.533388+0000","last_fullsized":"2024-06-13T06:56:29.533388+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:41.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526569+0000","last_change":"2024-06-13T06:54:54.333036+0000","last_active":"2024-06-13T06:56:29.526569+0000","last_peered":"2024-06-13T06:56:29.526569+0000","last_clean":"2024-06-13T06:56:29.526569+0000","last_became_active":"2024-06-13T06:54:54.332642+0000","last_became_peered":"2024-06-13T06:54:54.332642+0000","last_unstale":"2024-06-13T06:56:29.526569+0000","last_undegraded":"2024-06-13T06:56:29.526569+0000","last_fullsized":"2024-06-13T06:56:29.526569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:06:06.809024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607529+0000","last_change":"2024-06-13T06:54:58.881894+0000","last_active":"2024-06-13T06:56:29.607529+0000","last_peered":"2024-06-13T06:56:29.607529+0000","last_clean":"2024-06-13T06:56:29.607529+0000","last_became_active":"2024-06-13T06:54:58.881591+0000","last_became_peered":"2024-06-13T06:54:58.881591+0000","last_unstale":"2024-06-13T06:56:29.607529+0000","last_undegraded":"2024-06-13T06:56:29.607529+0000","last_fullsized":"2024-06-13T06:56:29.607529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:49:00.944212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525378+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:56:29.525378+0000","last_peered":"2024-06-13T06:56:29.525378+0000","last_clean":"2024-06-13T06:56:29.525378+0000","last_became_active":"2024-06-13T06:55:12.000748+0000","last_became_peered":"2024-06-13T06:55:12.000748+0000","last_unstale":"2024-06-13T06:56:29.525378+0000","last_undegraded":"2024-06-13T06:56:29.525378+0000","last_fullsized":"2024-06-13T06:56:29.525378+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:22.035807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600084+0000","last_change":"2024-06-13T06:54:54.342724+0000","last_active":"2024-06-13T06:56:29.600084+0000","last_peered":"2024-06-13T06:56:29.600084+0000","last_clean":"2024-06-13T06:56:29.600084+0000","last_became_active":"2024-06-13T06:54:54.342267+0000","last_became_peered":"2024-06-13T06:54:54.342267+0000","last_unstale":"2024-06-13T06:56:29.600084+0000","last_undegraded":"2024-06-13T06:56:29.600084+0000","last_fullsized":"2024-06-13T06:56:29.600084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:27:53.039202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608347+0000","last_change":"2024-06-13T06:54:58.892365+0000","last_active":"2024-06-13T06:56:29.608347+0000","last_peered":"2024-06-13T06:56:29.608347+0000","last_clean":"2024-06-13T06:56:29.608347+0000","last_became_active":"2024-06-13T06:54:58.889395+0000","last_became_peered":"2024-06-13T06:54:58.889395+0000","last_unstale":"2024-06-13T06:56:29.608347+0000","last_undegraded":"2024-06-13T06:56:29.608347+0000","last_fullsized":"2024-06-13T06:56:29.608347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:55.798293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524334+0000","last_change":"2024-06-13T06:54:58.888570+0000","last_active":"2024-06-13T06:56:29.524334+0000","last_peered":"2024-06-13T06:56:29.524334+0000","last_clean":"2024-06-13T06:56:29.524334+0000","last_became_active":"2024-06-13T06:54:58.888180+0000","last_became_peered":"2024-06-13T06:54:58.888180+0000","last_unstale":"2024-06-13T06:56:29.524334+0000","last_undegraded":"2024-06-13T06:56:29.524334+0000","last_fullsized":"2024-06-13T06:56:29.524334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:46:42.537790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607162+0000","last_change":"2024-06-13T06:54:58.893476+0000","last_active":"2024-06-13T06:56:29.607162+0000","last_peered":"2024-06-13T06:56:29.607162+0000","last_clean":"2024-06-13T06:56:29.607162+0000","last_became_active":"2024-06-13T06:54:58.893021+0000","last_became_peered":"2024-06-13T06:54:58.893021+0000","last_unstale":"2024-06-13T06:56:29.607162+0000","last_undegraded":"2024-06-13T06:56:29.607162+0000","last_fullsized":"2024-06-13T06:56:29.607162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:38.739070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607046+0000","last_change":"2024-06-13T06:54:58.911042+0000","last_active":"2024-06-13T06:56:29.607046+0000","last_peered":"2024-06-13T06:56:29.607046+0000","last_clean":"2024-06-13T06:56:29.607046+0000","last_became_active":"2024-06-13T06:54:58.910641+0000","last_became_peered":"2024-06-13T06:54:58.910641+0000","last_unstale":"2024-06-13T06:56:29.607046+0000","last_undegraded":"2024-06-13T06:56:29.607046+0000","last_fullsized":"2024-06-13T06:56:29.607046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:21:39.467223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.534304+0000","last_change":"2024-06-13T06:55:29.382730+0000","last_active":"2024-06-13T06:56:29.534304+0000","last_peered":"2024-06-13T06:56:29.534304+0000","last_clean":"2024-06-13T06:56:29.534304+0000","last_became_active":"2024-06-13T06:55:29.382162+0000","last_became_peered":"2024-06-13T06:55:29.382162+0000","last_unstale":"2024-06-13T06:56:29.534304+0000","last_undegraded":"2024-06-13T06:56:29.534304+0000","last_fullsized":"2024-06-13T06:56:29.534304+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:05.486414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.599474+0000","last_change":"2024-06-13T06:55:29.385380+0000","last_active":"2024-06-13T06:56:29.599474+0000","last_peered":"2024-06-13T06:56:29.599474+0000","last_clean":"2024-06-13T06:56:29.599474+0000","last_became_active":"2024-06-13T06:55:29.384855+0000","last_became_peered":"2024-06-13T06:55:29.384855+0000","last_unstale":"2024-06-13T06:56:29.599474+0000","last_undegraded":"2024-06-13T06:56:29.599474+0000","last_fullsized":"2024-06-13T06:56:29.599474+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:31:01.945632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525533+0000","last_change":"2024-06-13T06:54:58.889128+0000","last_active":"2024-06-13T06:56:29.525533+0000","last_peered":"2024-06-13T06:56:29.525533+0000","last_clean":"2024-06-13T06:56:29.525533+0000","last_became_active":"2024-06-13T06:54:58.888606+0000","last_became_peered":"2024-06-13T06:54:58.888606+0000","last_unstale":"2024-06-13T06:56:29.525533+0000","last_undegraded":"2024-06-13T06:56:29.525533+0000","last_fullsized":"2024-06-13T06:56:29.525533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:53:43.762968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607435+0000","last_change":"2024-06-13T06:54:58.896098+0000","last_active":"2024-06-13T06:56:29.607435+0000","last_peered":"2024-06-13T06:56:29.607435+0000","last_clean":"2024-06-13T06:56:29.607435+0000","last_became_active":"2024-06-13T06:54:58.895722+0000","last_became_peered":"2024-06-13T06:54:58.895722+0000","last_unstale":"2024-06-13T06:56:29.607435+0000","last_undegraded":"2024-06-13T06:56:29.607435+0000","last_fullsized":"2024-06-13T06:56:29.607435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:18:14.421363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526635+0000","last_change":"2024-06-13T06:54:58.905590+0000","last_active":"2024-06-13T06:56:29.526635+0000","last_peered":"2024-06-13T06:56:29.526635+0000","last_clean":"2024-06-13T06:56:29.526635+0000","last_became_active":"2024-06-13T06:54:58.905146+0000","last_became_peered":"2024-06-13T06:54:58.905146+0000","last_unstale":"2024-06-13T06:56:29.526635+0000","last_undegraded":"2024-06-13T06:56:29.526635+0000","last_fullsized":"2024-06-13T06:56:29.526635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:42:08.010906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525789+0000","last_change":"2024-06-13T06:54:58.898037+0000","last_active":"2024-06-13T06:56:29.525789+0000","last_peered":"2024-06-13T06:56:29.525789+0000","last_clean":"2024-06-13T06:56:29.525789+0000","last_became_active":"2024-06-13T06:54:58.897692+0000","last_became_peered":"2024-06-13T06:54:58.897692+0000","last_unstale":"2024-06-13T06:56:29.525789+0000","last_undegraded":"2024-06-13T06:56:29.525789+0000","last_fullsized":"2024-06-13T06:56:29.525789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:44:30.018192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524735+0000","last_change":"2024-06-13T06:54:58.903441+0000","last_active":"2024-06-13T06:56:29.524735+0000","last_peered":"2024-06-13T06:56:29.524735+0000","last_clean":"2024-06-13T06:56:29.524735+0000","last_became_active":"2024-06-13T06:54:58.893598+0000","last_became_peered":"2024-06-13T06:54:58.893598+0000","last_unstale":"2024-06-13T06:56:29.524735+0000","last_undegraded":"2024-06-13T06:56:29.524735+0000","last_fullsized":"2024-06-13T06:56:29.524735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:38:29.968619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609194+0000","last_change":"2024-06-13T06:54:58.918594+0000","last_active":"2024-06-13T06:56:29.609194+0000","last_peered":"2024-06-13T06:56:29.609194+0000","last_clean":"2024-06-13T06:56:29.609194+0000","last_became_active":"2024-06-13T06:54:58.918255+0000","last_became_peered":"2024-06-13T06:54:58.918255+0000","last_unstale":"2024-06-13T06:56:29.609194+0000","last_undegraded":"2024-06-13T06:56:29.609194+0000","last_fullsized":"2024-06-13T06:56:29.609194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:39:48.527707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.597127+0000","last_change":"2024-06-13T06:54:58.926253+0000","last_active":"2024-06-13T06:56:29.597127+0000","last_peered":"2024-06-13T06:56:29.597127+0000","last_clean":"2024-06-13T06:56:29.597127+0000","last_became_active":"2024-06-13T06:54:58.924611+0000","last_became_peered":"2024-06-13T06:54:58.924611+0000","last_unstale":"2024-06-13T06:56:29.597127+0000","last_undegraded":"2024-06-13T06:56:29.597127+0000","last_fullsized":"2024-06-13T06:56:29.597127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:29:23.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524605+0000","last_change":"2024-06-13T06:54:58.890454+0000","last_active":"2024-06-13T06:56:29.524605+0000","last_peered":"2024-06-13T06:56:29.524605+0000","last_clean":"2024-06-13T06:56:29.524605+0000","last_became_active":"2024-06-13T06:54:58.890129+0000","last_became_peered":"2024-06-13T06:54:58.890129+0000","last_unstale":"2024-06-13T06:56:29.524605+0000","last_undegraded":"2024-06-13T06:56:29.524605+0000","last_fullsized":"2024-06-13T06:56:29.524605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:10.819536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62390,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":239,"num_read_kb":187,"num_write":166,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1656,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":570102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37120,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708032,"statfs":{"total":95995035648,"available":95957024768,"internally_reserved":0,"allocated":1826816,"data_stored":1215716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":574390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070525,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":570170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332163,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1166122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495687,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1134592,"data_stored":565470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36508,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708644,"statfs":{"total":95995035648,"available":95957651456,"internally_reserved":0,"allocated":1200128,"data_stored":623770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1159168,"data_stored":566858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":566108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182488,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":565664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32372,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712780,"statfs":{"total":95995035648,"available":95961886720,"internally_reserved":0,"allocated":1224704,"data_stored":629522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1161710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":59968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:57:12.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph pg dump --format=json 2024-06-13T06:57:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:12 smithi136 ceph-mon[37229]: pgmap v410: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-13T06:57:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:12 smithi136 ceph-mon[37229]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:57:12.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:12 smithi026 ceph-mon[32041]: pgmap v410: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-13T06:57:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:12 smithi026 ceph-mon[32041]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:57:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:12 smithi135 ceph-mon[36276]: pgmap v410: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-13T06:57:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:12 smithi135 ceph-mon[36276]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:57:12.614 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:14 smithi136 ceph-mon[37229]: pgmap v411: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-13T06:57:14.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:14 smithi026 ceph-mon[32041]: pgmap v411: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-13T06:57:14.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:14 smithi135 ceph-mon[36276]: pgmap v411: 129 pgs: 129 active+clean; 638 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-13T06:57:14.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 -- 172.21.15.26:0/1552982773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a0105090 msgr2=0x7f37a01115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1552982773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a0105090 0x7f37a01115a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f379800b0a0 tx=0x7f379800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 -- 172.21.15.26:0/1552982773 shutdown_connections 2024-06-13T06:57:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1552982773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a0105090 0x7f37a01115a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1552982773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37a010a6d0 0x7f37a010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1552982773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a01046d0 0x7f37a0104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 -- 172.21.15.26:0/1552982773 >> 172.21.15.26:0/1552982773 conn(0x7f37a01001c0 msgr2=0x7f37a01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 -- 172.21.15.26:0/1552982773 shutdown_connections 2024-06-13T06:57:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.992+0000 7f37a7a00700 1 -- 172.21.15.26:0/1552982773 wait complete. 2024-06-13T06:57:14.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.993+0000 7f37a7a00700 1 Processor -- start 2024-06-13T06:57:14.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.994+0000 7f37a7a00700 1 -- start start 2024-06-13T06:57:14.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.994+0000 7f37a7a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37a01046d0 0x7f37a019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.994+0000 7f37a7a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a0105090 0x7f37a019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.994+0000 7f37a7a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a010a6d0 0x7f37a019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.994+0000 7f37a7a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0114ab0 con 0x7f37a010a6d0 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.994+0000 7f37a7a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0114930 con 0x7f37a0105090 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.994+0000 7f37a7a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0114c30 con 0x7f37a01046d0 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37a01046d0 0x7f37a019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a010a6d0 0x7f37a019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37a01046d0 0x7f37a019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43758/0 (socket says 172.21.15.26:43758) 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5000700 1 -- 172.21.15.26:0/1574388897 learned_addr learned my addr 172.21.15.26:0/1574388897 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f379fe00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a0105090 0x7f37a019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5a00700 1 -- 172.21.15.26:0/1574388897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37a01046d0 msgr2=0x7f37a019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37a01046d0 0x7f37a019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5a00700 1 -- 172.21.15.26:0/1574388897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a0105090 msgr2=0x7f37a019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a0105090 0x7f37a019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5a00700 1 -- 172.21.15.26:0/1574388897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a019fe70 con 0x7f37a010a6d0 2024-06-13T06:57:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5000700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37a01046d0 0x7f37a019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.995+0000 7f37a5a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a010a6d0 0x7f37a019f730 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f379800b070 tx=0x7f3798004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.996+0000 7f379d600700 1 -- 172.21.15.26:0/1574388897 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798021070 con 0x7f37a010a6d0 2024-06-13T06:57:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.996+0000 7f379d600700 1 -- 172.21.15.26:0/1574388897 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3798007b80 con 0x7f37a010a6d0 2024-06-13T06:57:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.996+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a01a0100 con 0x7f37a010a6d0 2024-06-13T06:57:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.996+0000 7f379d600700 1 -- 172.21.15.26:0/1574388897 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798005040 con 0x7f37a010a6d0 2024-06-13T06:57:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.996+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a01a79e0 con 0x7f37a010a6d0 2024-06-13T06:57:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.997+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a0069180 con 0x7f37a010a6d0 2024-06-13T06:57:14.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:14.999+0000 7f379d600700 1 -- 172.21.15.26:0/1574388897 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3798024070 con 0x7f37a010a6d0 2024-06-13T06:57:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.000+0000 7f379d600700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37880a2d30 0x7f37880a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.001+0000 7f379d600700 1 -- 172.21.15.26:0/1574388897 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f379800f020 con 0x7f37a010a6d0 2024-06-13T06:57:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.001+0000 7f37a5000700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37880a2d30 0x7f37880a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:15.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.002+0000 7f37a5000700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37880a2d30 0x7f37880a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3794011550 tx=0x7f3794009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:15.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.005+0000 7f379d600700 1 -- 172.21.15.26:0/1574388897 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f379802b020 con 0x7f37a010a6d0 2024-06-13T06:57:15.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.233+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37a0100d50 con 0x7f37880a2d30 2024-06-13T06:57:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.244+0000 7f379d600700 1 -- 172.21.15.26:0/1574388897 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340381 (secure 0 0 0) 0x7f37a0100d50 con 0x7f37880a2d30 2024-06-13T06:57:15.246 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:57:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37880a2d30 msgr2=0x7f37880a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37880a2d30 0x7f37880a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3794011550 tx=0x7f3794009260 comp rx=0 tx=0).stop 2024-06-13T06:57:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a010a6d0 msgr2=0x7f37a019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a010a6d0 0x7f37a019f730 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f379800b070 tx=0x7f3798004c80 comp rx=0 tx=0).stop 2024-06-13T06:57:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 shutdown_connections 2024-06-13T06:57:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37880a2d30 0x7f37880a51f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a010a6d0 0x7f37a019f730 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37a01046d0 0x7f37a019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 --2- 172.21.15.26:0/1574388897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a0105090 0x7f37a019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.253+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 >> 172.21.15.26:0/1574388897 conn(0x7f37a01001c0 msgr2=0x7f37a0100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.254+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 shutdown_connections 2024-06-13T06:57:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:15.254+0000 7f37a7a00700 1 -- 172.21.15.26:0/1574388897 wait complete. 2024-06-13T06:57:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-06-13T06:57:16.040 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-06-13T06:57:14.327071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657257,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":396,"num_read_kb":316,"num_write":350,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182392,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":397892,"kb_used_data":15440,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543932,"statfs":{"total":1151940427776,"available":1151532986368,"internally_reserved":0,"allocated":15810560,"data_stored":8788397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9668,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":56,"num_write":6,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005281"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525058+0000","last_change":"2024-06-13T06:54:58.919669+0000","last_active":"2024-06-13T06:56:29.525058+0000","last_peered":"2024-06-13T06:56:29.525058+0000","last_clean":"2024-06-13T06:56:29.525058+0000","last_became_active":"2024-06-13T06:54:58.919334+0000","last_became_peered":"2024-06-13T06:54:58.919334+0000","last_unstale":"2024-06-13T06:56:29.525058+0000","last_undegraded":"2024-06-13T06:56:29.525058+0000","last_fullsized":"2024-06-13T06:56:29.525058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:45:40.831640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532319+0000","last_change":"2024-06-13T06:54:58.909508+0000","last_active":"2024-06-13T06:56:29.532319+0000","last_peered":"2024-06-13T06:56:29.532319+0000","last_clean":"2024-06-13T06:56:29.532319+0000","last_became_active":"2024-06-13T06:54:58.909221+0000","last_became_peered":"2024-06-13T06:54:58.909221+0000","last_unstale":"2024-06-13T06:56:29.532319+0000","last_undegraded":"2024-06-13T06:56:29.532319+0000","last_fullsized":"2024-06-13T06:56:29.532319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:17:07.476858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.531829+0000","last_change":"2024-06-13T06:54:58.909283+0000","last_active":"2024-06-13T06:56:29.531829+0000","last_peered":"2024-06-13T06:56:29.531829+0000","last_clean":"2024-06-13T06:56:29.531829+0000","last_became_active":"2024-06-13T06:54:58.908912+0000","last_became_peered":"2024-06-13T06:54:58.908912+0000","last_unstale":"2024-06-13T06:56:29.531829+0000","last_undegraded":"2024-06-13T06:56:29.531829+0000","last_fullsized":"2024-06-13T06:56:29.531829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:48:26.937971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608067+0000","last_change":"2024-06-13T06:54:58.902821+0000","last_active":"2024-06-13T06:56:29.608067+0000","last_peered":"2024-06-13T06:56:29.608067+0000","last_clean":"2024-06-13T06:56:29.608067+0000","last_became_active":"2024-06-13T06:54:58.902215+0000","last_became_peered":"2024-06-13T06:54:58.902215+0000","last_unstale":"2024-06-13T06:56:29.608067+0000","last_undegraded":"2024-06-13T06:56:29.608067+0000","last_fullsized":"2024-06-13T06:56:29.608067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:11:57.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522348+0000","last_change":"2024-06-13T06:54:58.897038+0000","last_active":"2024-06-13T06:56:29.522348+0000","last_peered":"2024-06-13T06:56:29.522348+0000","last_clean":"2024-06-13T06:56:29.522348+0000","last_became_active":"2024-06-13T06:54:58.896665+0000","last_became_peered":"2024-06-13T06:54:58.896665+0000","last_unstale":"2024-06-13T06:56:29.522348+0000","last_undegraded":"2024-06-13T06:56:29.522348+0000","last_fullsized":"2024-06-13T06:56:29.522348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:24:19.216932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523216+0000","last_change":"2024-06-13T06:54:58.895501+0000","last_active":"2024-06-13T06:56:29.523216+0000","last_peered":"2024-06-13T06:56:29.523216+0000","last_clean":"2024-06-13T06:56:29.523216+0000","last_became_active":"2024-06-13T06:54:58.895144+0000","last_became_peered":"2024-06-13T06:54:58.895144+0000","last_unstale":"2024-06-13T06:56:29.523216+0000","last_undegraded":"2024-06-13T06:56:29.523216+0000","last_fullsized":"2024-06-13T06:56:29.523216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:50.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607844+0000","last_change":"2024-06-13T06:54:58.901097+0000","last_active":"2024-06-13T06:56:29.607844+0000","last_peered":"2024-06-13T06:56:29.607844+0000","last_clean":"2024-06-13T06:56:29.607844+0000","last_became_active":"2024-06-13T06:54:58.900727+0000","last_became_peered":"2024-06-13T06:54:58.900727+0000","last_unstale":"2024-06-13T06:56:29.607844+0000","last_undegraded":"2024-06-13T06:56:29.607844+0000","last_fullsized":"2024-06-13T06:56:29.607844+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:09:38.349517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523777+0000","last_change":"2024-06-13T06:54:58.910740+0000","last_active":"2024-06-13T06:56:29.523777+0000","last_peered":"2024-06-13T06:56:29.523777+0000","last_clean":"2024-06-13T06:56:29.523777+0000","last_became_active":"2024-06-13T06:54:58.910351+0000","last_became_peered":"2024-06-13T06:54:58.910351+0000","last_unstale":"2024-06-13T06:56:29.523777+0000","last_undegraded":"2024-06-13T06:56:29.523777+0000","last_fullsized":"2024-06-13T06:56:29.523777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:45:12.854996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608887+0000","last_change":"2024-06-13T06:55:29.364516+0000","last_active":"2024-06-13T06:56:29.608887+0000","last_peered":"2024-06-13T06:56:29.608887+0000","last_clean":"2024-06-13T06:56:29.608887+0000","last_became_active":"2024-06-13T06:55:29.364079+0000","last_became_peered":"2024-06-13T06:55:29.364079+0000","last_unstale":"2024-06-13T06:56:29.608887+0000","last_undegraded":"2024-06-13T06:56:29.608887+0000","last_fullsized":"2024-06-13T06:56:29.608887+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:51:28.624359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523056+0000","last_change":"2024-06-13T06:54:58.903882+0000","last_active":"2024-06-13T06:56:29.523056+0000","last_peered":"2024-06-13T06:56:29.523056+0000","last_clean":"2024-06-13T06:56:29.523056+0000","last_became_active":"2024-06-13T06:54:58.903495+0000","last_became_peered":"2024-06-13T06:54:58.903495+0000","last_unstale":"2024-06-13T06:56:29.523056+0000","last_undegraded":"2024-06-13T06:56:29.523056+0000","last_fullsized":"2024-06-13T06:56:29.523056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:23.682993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609302+0000","last_change":"2024-06-13T06:54:58.900080+0000","last_active":"2024-06-13T06:56:29.609302+0000","last_peered":"2024-06-13T06:56:29.609302+0000","last_clean":"2024-06-13T06:56:29.609302+0000","last_became_active":"2024-06-13T06:54:58.899736+0000","last_became_peered":"2024-06-13T06:54:58.899736+0000","last_unstale":"2024-06-13T06:56:29.609302+0000","last_undegraded":"2024-06-13T06:56:29.609302+0000","last_fullsized":"2024-06-13T06:56:29.609302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:51:45.808821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608127+0000","last_change":"2024-06-13T06:54:58.900238+0000","last_active":"2024-06-13T06:56:29.608127+0000","last_peered":"2024-06-13T06:56:29.608127+0000","last_clean":"2024-06-13T06:56:29.608127+0000","last_became_active":"2024-06-13T06:54:58.899872+0000","last_became_peered":"2024-06-13T06:54:58.899872+0000","last_unstale":"2024-06-13T06:56:29.608127+0000","last_undegraded":"2024-06-13T06:56:29.608127+0000","last_fullsized":"2024-06-13T06:56:29.608127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:12:40.780894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.599885+0000","last_change":"2024-06-13T06:54:58.901981+0000","last_active":"2024-06-13T06:56:29.599885+0000","last_peered":"2024-06-13T06:56:29.599885+0000","last_clean":"2024-06-13T06:56:29.599885+0000","last_became_active":"2024-06-13T06:54:58.898704+0000","last_became_peered":"2024-06-13T06:54:58.898704+0000","last_unstale":"2024-06-13T06:56:29.599885+0000","last_undegraded":"2024-06-13T06:56:29.599885+0000","last_fullsized":"2024-06-13T06:56:29.599885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:03:56.658091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533036+0000","last_change":"2024-06-13T06:54:58.909022+0000","last_active":"2024-06-13T06:56:29.533036+0000","last_peered":"2024-06-13T06:56:29.533036+0000","last_clean":"2024-06-13T06:56:29.533036+0000","last_became_active":"2024-06-13T06:54:58.907438+0000","last_became_peered":"2024-06-13T06:54:58.907438+0000","last_unstale":"2024-06-13T06:56:29.533036+0000","last_undegraded":"2024-06-13T06:56:29.533036+0000","last_fullsized":"2024-06-13T06:56:29.533036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:40:34.190510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.531758+0000","last_change":"2024-06-13T06:54:58.897691+0000","last_active":"2024-06-13T06:56:29.531758+0000","last_peered":"2024-06-13T06:56:29.531758+0000","last_clean":"2024-06-13T06:56:29.531758+0000","last_became_active":"2024-06-13T06:54:58.897381+0000","last_became_peered":"2024-06-13T06:54:58.897381+0000","last_unstale":"2024-06-13T06:56:29.531758+0000","last_undegraded":"2024-06-13T06:56:29.531758+0000","last_fullsized":"2024-06-13T06:56:29.531758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:10.052052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607757+0000","last_change":"2024-06-13T06:54:58.907676+0000","last_active":"2024-06-13T06:56:29.607757+0000","last_peered":"2024-06-13T06:56:29.607757+0000","last_clean":"2024-06-13T06:56:29.607757+0000","last_became_active":"2024-06-13T06:54:58.907304+0000","last_became_peered":"2024-06-13T06:54:58.907304+0000","last_unstale":"2024-06-13T06:56:29.607757+0000","last_undegraded":"2024-06-13T06:56:29.607757+0000","last_fullsized":"2024-06-13T06:56:29.607757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:08:20.292516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524782+0000","last_change":"2024-06-13T06:54:58.920012+0000","last_active":"2024-06-13T06:56:29.524782+0000","last_peered":"2024-06-13T06:56:29.524782+0000","last_clean":"2024-06-13T06:56:29.524782+0000","last_became_active":"2024-06-13T06:54:58.919473+0000","last_became_peered":"2024-06-13T06:54:58.919473+0000","last_unstale":"2024-06-13T06:56:29.524782+0000","last_undegraded":"2024-06-13T06:56:29.524782+0000","last_fullsized":"2024-06-13T06:56:29.524782+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:49:13.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533070+0000","last_change":"2024-06-13T06:54:58.902576+0000","last_active":"2024-06-13T06:56:29.533070+0000","last_peered":"2024-06-13T06:56:29.533070+0000","last_clean":"2024-06-13T06:56:29.533070+0000","last_became_active":"2024-06-13T06:54:58.902205+0000","last_became_peered":"2024-06-13T06:54:58.902205+0000","last_unstale":"2024-06-13T06:56:29.533070+0000","last_undegraded":"2024-06-13T06:56:29.533070+0000","last_fullsized":"2024-06-13T06:56:29.533070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:13:24.107081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607362+0000","last_change":"2024-06-13T06:54:58.905392+0000","last_active":"2024-06-13T06:56:29.607362+0000","last_peered":"2024-06-13T06:56:29.607362+0000","last_clean":"2024-06-13T06:56:29.607362+0000","last_became_active":"2024-06-13T06:54:58.905017+0000","last_became_peered":"2024-06-13T06:54:58.905017+0000","last_unstale":"2024-06-13T06:56:29.607362+0000","last_undegraded":"2024-06-13T06:56:29.607362+0000","last_fullsized":"2024-06-13T06:56:29.607362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:05.931653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606834+0000","last_change":"2024-06-13T06:55:12.021359+0000","last_active":"2024-06-13T06:56:29.606834+0000","last_peered":"2024-06-13T06:56:29.606834+0000","last_clean":"2024-06-13T06:56:29.606834+0000","last_became_active":"2024-06-13T06:55:12.020860+0000","last_became_peered":"2024-06-13T06:55:12.020860+0000","last_unstale":"2024-06-13T06:56:29.606834+0000","last_undegraded":"2024-06-13T06:56:29.606834+0000","last_fullsized":"2024-06-13T06:56:29.606834+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:05:00.833905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.561282+0000","last_change":"2024-06-13T06:54:54.339220+0000","last_active":"2024-06-13T06:56:32.561282+0000","last_peered":"2024-06-13T06:56:32.561282+0000","last_clean":"2024-06-13T06:56:32.561282+0000","last_became_active":"2024-06-13T06:54:54.338573+0000","last_became_peered":"2024-06-13T06:54:54.338573+0000","last_unstale":"2024-06-13T06:56:32.561282+0000","last_undegraded":"2024-06-13T06:56:32.561282+0000","last_fullsized":"2024-06-13T06:56:32.561282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:53:50.258097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525854+0000","last_change":"2024-06-13T06:54:58.903920+0000","last_active":"2024-06-13T06:56:29.525854+0000","last_peered":"2024-06-13T06:56:29.525854+0000","last_clean":"2024-06-13T06:56:29.525854+0000","last_became_active":"2024-06-13T06:54:58.895364+0000","last_became_peered":"2024-06-13T06:54:58.895364+0000","last_unstale":"2024-06-13T06:56:29.525854+0000","last_undegraded":"2024-06-13T06:56:29.525854+0000","last_fullsized":"2024-06-13T06:56:29.525854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:04:35.109799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606948+0000","last_change":"2024-06-13T06:55:12.017561+0000","last_active":"2024-06-13T06:56:29.606948+0000","last_peered":"2024-06-13T06:56:29.606948+0000","last_clean":"2024-06-13T06:56:29.606948+0000","last_became_active":"2024-06-13T06:55:12.017104+0000","last_became_peered":"2024-06-13T06:55:12.017104+0000","last_unstale":"2024-06-13T06:56:29.606948+0000","last_undegraded":"2024-06-13T06:56:29.606948+0000","last_fullsized":"2024-06-13T06:56:29.606948+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:08:22.570491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526871+0000","last_change":"2024-06-13T06:54:54.631599+0000","last_active":"2024-06-13T06:56:29.526871+0000","last_peered":"2024-06-13T06:56:29.526871+0000","last_clean":"2024-06-13T06:56:29.526871+0000","last_became_active":"2024-06-13T06:54:54.631007+0000","last_became_peered":"2024-06-13T06:54:54.631007+0000","last_unstale":"2024-06-13T06:56:29.526871+0000","last_undegraded":"2024-06-13T06:56:29.526871+0000","last_fullsized":"2024-06-13T06:56:29.526871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:49:56.602192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532111+0000","last_change":"2024-06-13T06:54:58.917255+0000","last_active":"2024-06-13T06:56:29.532111+0000","last_peered":"2024-06-13T06:56:29.532111+0000","last_clean":"2024-06-13T06:56:29.532111+0000","last_became_active":"2024-06-13T06:54:58.916875+0000","last_became_peered":"2024-06-13T06:54:58.916875+0000","last_unstale":"2024-06-13T06:56:29.532111+0000","last_undegraded":"2024-06-13T06:56:29.532111+0000","last_fullsized":"2024-06-13T06:56:29.532111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:09:47.638802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608461+0000","last_change":"2024-06-13T06:55:12.018209+0000","last_active":"2024-06-13T06:56:29.608461+0000","last_peered":"2024-06-13T06:56:29.608461+0000","last_clean":"2024-06-13T06:56:29.608461+0000","last_became_active":"2024-06-13T06:55:12.017038+0000","last_became_peered":"2024-06-13T06:55:12.017038+0000","last_unstale":"2024-06-13T06:56:29.608461+0000","last_undegraded":"2024-06-13T06:56:29.608461+0000","last_fullsized":"2024-06-13T06:56:29.608461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:36:26.476454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522745+0000","last_change":"2024-06-13T06:54:54.328849+0000","last_active":"2024-06-13T06:56:29.522745+0000","last_peered":"2024-06-13T06:56:29.522745+0000","last_clean":"2024-06-13T06:56:29.522745+0000","last_became_active":"2024-06-13T06:54:54.328388+0000","last_became_peered":"2024-06-13T06:54:54.328388+0000","last_unstale":"2024-06-13T06:56:29.522745+0000","last_undegraded":"2024-06-13T06:56:29.522745+0000","last_fullsized":"2024-06-13T06:56:29.522745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:41.264403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607188+0000","last_change":"2024-06-13T06:54:58.897588+0000","last_active":"2024-06-13T06:56:29.607188+0000","last_peered":"2024-06-13T06:56:29.607188+0000","last_clean":"2024-06-13T06:56:29.607188+0000","last_became_active":"2024-06-13T06:54:58.897236+0000","last_became_peered":"2024-06-13T06:54:58.897236+0000","last_unstale":"2024-06-13T06:56:29.607188+0000","last_undegraded":"2024-06-13T06:56:29.607188+0000","last_fullsized":"2024-06-13T06:56:29.607188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:20:38.460445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607238+0000","last_change":"2024-06-13T06:55:12.014080+0000","last_active":"2024-06-13T06:56:29.607238+0000","last_peered":"2024-06-13T06:56:29.607238+0000","last_clean":"2024-06-13T06:56:29.607238+0000","last_became_active":"2024-06-13T06:55:12.013627+0000","last_became_peered":"2024-06-13T06:55:12.013627+0000","last_unstale":"2024-06-13T06:56:29.607238+0000","last_undegraded":"2024-06-13T06:56:29.607238+0000","last_fullsized":"2024-06-13T06:56:29.607238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:29:54.081593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522192+0000","last_change":"2024-06-13T06:54:54.328249+0000","last_active":"2024-06-13T06:56:29.522192+0000","last_peered":"2024-06-13T06:56:29.522192+0000","last_clean":"2024-06-13T06:56:29.522192+0000","last_became_active":"2024-06-13T06:54:54.327832+0000","last_became_peered":"2024-06-13T06:54:54.327832+0000","last_unstale":"2024-06-13T06:56:29.522192+0000","last_undegraded":"2024-06-13T06:56:29.522192+0000","last_fullsized":"2024-06-13T06:56:29.522192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:21:05.624075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532096+0000","last_change":"2024-06-13T06:55:29.383305+0000","last_active":"2024-06-13T06:56:29.532096+0000","last_peered":"2024-06-13T06:56:29.532096+0000","last_clean":"2024-06-13T06:56:29.532096+0000","last_became_active":"2024-06-13T06:55:29.381491+0000","last_became_peered":"2024-06-13T06:55:29.381491+0000","last_unstale":"2024-06-13T06:56:29.532096+0000","last_undegraded":"2024-06-13T06:56:29.532096+0000","last_fullsized":"2024-06-13T06:56:29.532096+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:52:26.437237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607192+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:56:29.607192+0000","last_peered":"2024-06-13T06:56:29.607192+0000","last_clean":"2024-06-13T06:56:29.607192+0000","last_became_active":"2024-06-13T06:55:12.000759+0000","last_became_peered":"2024-06-13T06:55:12.000759+0000","last_unstale":"2024-06-13T06:56:29.607192+0000","last_undegraded":"2024-06-13T06:56:29.607192+0000","last_fullsized":"2024-06-13T06:56:29.607192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:11:35.805338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'6","reported_seq":98,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:09.631303+0000","last_change":"2024-06-13T06:55:29.367538+0000","last_active":"2024-06-13T06:57:09.631303+0000","last_peered":"2024-06-13T06:57:09.631303+0000","last_clean":"2024-06-13T06:57:09.631303+0000","last_became_active":"2024-06-13T06:55:29.367149+0000","last_became_peered":"2024-06-13T06:55:29.367149+0000","last_unstale":"2024-06-13T06:57:09.631303+0000","last_undegraded":"2024-06-13T06:57:09.631303+0000","last_fullsized":"2024-06-13T06:57:09.631303+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:30:55.632799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522180+0000","last_change":"2024-06-13T06:54:58.896258+0000","last_active":"2024-06-13T06:56:29.522180+0000","last_peered":"2024-06-13T06:56:29.522180+0000","last_clean":"2024-06-13T06:56:29.522180+0000","last_became_active":"2024-06-13T06:54:58.895904+0000","last_became_peered":"2024-06-13T06:54:58.895904+0000","last_unstale":"2024-06-13T06:56:29.522180+0000","last_undegraded":"2024-06-13T06:56:29.522180+0000","last_fullsized":"2024-06-13T06:56:29.522180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:28:33.615338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607190+0000","last_change":"2024-06-13T06:55:12.018499+0000","last_active":"2024-06-13T06:56:29.607190+0000","last_peered":"2024-06-13T06:56:29.607190+0000","last_clean":"2024-06-13T06:56:29.607190+0000","last_became_active":"2024-06-13T06:55:12.018073+0000","last_became_peered":"2024-06-13T06:55:12.018073+0000","last_unstale":"2024-06-13T06:56:29.607190+0000","last_undegraded":"2024-06-13T06:56:29.607190+0000","last_fullsized":"2024-06-13T06:56:29.607190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:59:53.206259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523078+0000","last_change":"2024-06-13T06:54:54.339482+0000","last_active":"2024-06-13T06:56:29.523078+0000","last_peered":"2024-06-13T06:56:29.523078+0000","last_clean":"2024-06-13T06:56:29.523078+0000","last_became_active":"2024-06-13T06:54:54.339066+0000","last_became_peered":"2024-06-13T06:54:54.339066+0000","last_unstale":"2024-06-13T06:56:29.523078+0000","last_undegraded":"2024-06-13T06:56:29.523078+0000","last_fullsized":"2024-06-13T06:56:29.523078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:31:04.136530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525738+0000","last_change":"2024-06-13T06:54:58.891154+0000","last_active":"2024-06-13T06:56:29.525738+0000","last_peered":"2024-06-13T06:56:29.525738+0000","last_clean":"2024-06-13T06:56:29.525738+0000","last_became_active":"2024-06-13T06:54:58.890810+0000","last_became_peered":"2024-06-13T06:54:58.890810+0000","last_unstale":"2024-06-13T06:56:29.525738+0000","last_undegraded":"2024-06-13T06:56:29.525738+0000","last_fullsized":"2024-06-13T06:56:29.525738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:14.618233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524690+0000","last_change":"2024-06-13T06:55:12.025619+0000","last_active":"2024-06-13T06:56:29.524690+0000","last_peered":"2024-06-13T06:56:29.524690+0000","last_clean":"2024-06-13T06:56:29.524690+0000","last_became_active":"2024-06-13T06:55:12.025153+0000","last_became_peered":"2024-06-13T06:55:12.025153+0000","last_unstale":"2024-06-13T06:56:29.524690+0000","last_undegraded":"2024-06-13T06:56:29.524690+0000","last_fullsized":"2024-06-13T06:56:29.524690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:12.124550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.596726+0000","last_change":"2024-06-13T06:54:54.341698+0000","last_active":"2024-06-13T06:56:29.596726+0000","last_peered":"2024-06-13T06:56:29.596726+0000","last_clean":"2024-06-13T06:56:29.596726+0000","last_became_active":"2024-06-13T06:54:54.341113+0000","last_became_peered":"2024-06-13T06:54:54.341113+0000","last_unstale":"2024-06-13T06:56:29.596726+0000","last_undegraded":"2024-06-13T06:56:29.596726+0000","last_fullsized":"2024-06-13T06:56:29.596726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:18:23.368705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524079+0000","last_change":"2024-06-13T06:54:58.913688+0000","last_active":"2024-06-13T06:56:29.524079+0000","last_peered":"2024-06-13T06:56:29.524079+0000","last_clean":"2024-06-13T06:56:29.524079+0000","last_became_active":"2024-06-13T06:54:58.910493+0000","last_became_peered":"2024-06-13T06:54:58.910493+0000","last_unstale":"2024-06-13T06:56:29.524079+0000","last_undegraded":"2024-06-13T06:56:29.524079+0000","last_fullsized":"2024-06-13T06:56:29.524079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:29:18.081743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532034+0000","last_change":"2024-06-13T06:55:12.016257+0000","last_active":"2024-06-13T06:56:29.532034+0000","last_peered":"2024-06-13T06:56:29.532034+0000","last_clean":"2024-06-13T06:56:29.532034+0000","last_became_active":"2024-06-13T06:55:12.015594+0000","last_became_peered":"2024-06-13T06:55:12.015594+0000","last_unstale":"2024-06-13T06:56:29.532034+0000","last_undegraded":"2024-06-13T06:56:29.532034+0000","last_fullsized":"2024-06-13T06:56:29.532034+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:02:44.642075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525674+0000","last_change":"2024-06-13T06:54:54.330366+0000","last_active":"2024-06-13T06:56:29.525674+0000","last_peered":"2024-06-13T06:56:29.525674+0000","last_clean":"2024-06-13T06:56:29.525674+0000","last_became_active":"2024-06-13T06:54:54.330010+0000","last_became_peered":"2024-06-13T06:54:54.330010+0000","last_unstale":"2024-06-13T06:56:29.525674+0000","last_undegraded":"2024-06-13T06:56:29.525674+0000","last_fullsized":"2024-06-13T06:56:29.525674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:36:01.488766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.531617+0000","last_change":"2024-06-13T06:54:58.918076+0000","last_active":"2024-06-13T06:56:29.531617+0000","last_peered":"2024-06-13T06:56:29.531617+0000","last_clean":"2024-06-13T06:56:29.531617+0000","last_became_active":"2024-06-13T06:54:58.917631+0000","last_became_peered":"2024-06-13T06:54:58.917631+0000","last_unstale":"2024-06-13T06:56:29.531617+0000","last_undegraded":"2024-06-13T06:56:29.531617+0000","last_fullsized":"2024-06-13T06:56:29.531617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:31:00.056837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.611761+0000","last_change":"2024-06-13T06:56:29.611761+0000","last_active":"2024-06-13T06:56:29.611761+0000","last_peered":"2024-06-13T06:56:29.611761+0000","last_clean":"2024-06-13T06:56:29.611761+0000","last_became_active":"2024-06-13T06:56:29.610385+0000","last_became_peered":"2024-06-13T06:56:29.610385+0000","last_unstale":"2024-06-13T06:56:29.611761+0000","last_undegraded":"2024-06-13T06:56:29.611761+0000","last_fullsized":"2024-06-13T06:56:29.611761+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:26:10.678918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":88,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:11.634446+0000","last_change":"2024-06-13T06:54:54.335324+0000","last_active":"2024-06-13T06:57:11.634446+0000","last_peered":"2024-06-13T06:57:11.634446+0000","last_clean":"2024-06-13T06:57:11.634446+0000","last_became_active":"2024-06-13T06:54:54.334936+0000","last_became_peered":"2024-06-13T06:54:54.334936+0000","last_unstale":"2024-06-13T06:57:11.634446+0000","last_undegraded":"2024-06-13T06:57:11.634446+0000","last_fullsized":"2024-06-13T06:57:11.634446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:41:52.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608833+0000","last_change":"2024-06-13T06:54:58.902146+0000","last_active":"2024-06-13T06:56:29.608833+0000","last_peered":"2024-06-13T06:56:29.608833+0000","last_clean":"2024-06-13T06:56:29.608833+0000","last_became_active":"2024-06-13T06:54:58.901807+0000","last_became_peered":"2024-06-13T06:54:58.901807+0000","last_unstale":"2024-06-13T06:56:29.608833+0000","last_undegraded":"2024-06-13T06:56:29.608833+0000","last_fullsized":"2024-06-13T06:56:29.608833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:59:53.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533051+0000","last_change":"2024-06-13T06:55:12.025418+0000","last_active":"2024-06-13T06:56:29.533051+0000","last_peered":"2024-06-13T06:56:29.533051+0000","last_clean":"2024-06-13T06:56:29.533051+0000","last_became_active":"2024-06-13T06:55:12.024859+0000","last_became_peered":"2024-06-13T06:55:12.024859+0000","last_unstale":"2024-06-13T06:56:29.533051+0000","last_undegraded":"2024-06-13T06:56:29.533051+0000","last_fullsized":"2024-06-13T06:56:29.533051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:05:17.553169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609367+0000","last_change":"2024-06-13T06:54:54.326011+0000","last_active":"2024-06-13T06:56:29.609367+0000","last_peered":"2024-06-13T06:56:29.609367+0000","last_clean":"2024-06-13T06:56:29.609367+0000","last_became_active":"2024-06-13T06:54:54.325664+0000","last_became_peered":"2024-06-13T06:54:54.325664+0000","last_unstale":"2024-06-13T06:56:29.609367+0000","last_undegraded":"2024-06-13T06:56:29.609367+0000","last_fullsized":"2024-06-13T06:56:29.609367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:32.973094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532335+0000","last_change":"2024-06-13T06:55:29.383612+0000","last_active":"2024-06-13T06:56:29.532335+0000","last_peered":"2024-06-13T06:56:29.532335+0000","last_clean":"2024-06-13T06:56:29.532335+0000","last_became_active":"2024-06-13T06:55:29.382174+0000","last_became_peered":"2024-06-13T06:55:29.382174+0000","last_unstale":"2024-06-13T06:56:29.532335+0000","last_undegraded":"2024-06-13T06:56:29.532335+0000","last_fullsized":"2024-06-13T06:56:29.532335+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:09:35.178209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608655+0000","last_change":"2024-06-13T06:55:12.018654+0000","last_active":"2024-06-13T06:56:29.608655+0000","last_peered":"2024-06-13T06:56:29.608655+0000","last_clean":"2024-06-13T06:56:29.608655+0000","last_became_active":"2024-06-13T06:55:12.017219+0000","last_became_peered":"2024-06-13T06:55:12.017219+0000","last_unstale":"2024-06-13T06:56:29.608655+0000","last_undegraded":"2024-06-13T06:56:29.608655+0000","last_fullsized":"2024-06-13T06:56:29.608655+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:46:54.161912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526453+0000","last_change":"2024-06-13T06:54:54.632154+0000","last_active":"2024-06-13T06:56:29.526453+0000","last_peered":"2024-06-13T06:56:29.526453+0000","last_clean":"2024-06-13T06:56:29.526453+0000","last_became_active":"2024-06-13T06:54:54.631721+0000","last_became_peered":"2024-06-13T06:54:54.631721+0000","last_unstale":"2024-06-13T06:56:29.526453+0000","last_undegraded":"2024-06-13T06:56:29.526453+0000","last_fullsized":"2024-06-13T06:56:29.526453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:16:10.492144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607473+0000","last_change":"2024-06-13T06:54:58.898961+0000","last_active":"2024-06-13T06:56:29.607473+0000","last_peered":"2024-06-13T06:56:29.607473+0000","last_clean":"2024-06-13T06:56:29.607473+0000","last_became_active":"2024-06-13T06:54:58.898599+0000","last_became_peered":"2024-06-13T06:54:58.898599+0000","last_unstale":"2024-06-13T06:56:29.607473+0000","last_undegraded":"2024-06-13T06:56:29.607473+0000","last_fullsized":"2024-06-13T06:56:29.607473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:21:53.044205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.527672+0000","last_change":"2024-06-13T06:56:29.527242+0000","last_active":"2024-06-13T06:56:29.527672+0000","last_peered":"2024-06-13T06:56:29.527672+0000","last_clean":"2024-06-13T06:56:29.527672+0000","last_became_active":"2024-06-13T06:56:29.526816+0000","last_became_peered":"2024-06-13T06:56:29.526816+0000","last_unstale":"2024-06-13T06:56:29.527672+0000","last_undegraded":"2024-06-13T06:56:29.527672+0000","last_fullsized":"2024-06-13T06:56:29.527672+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:45:17.770029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'33","reported_seq":184,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.630031+0000","last_change":"2024-06-13T06:55:29.376808+0000","last_active":"2024-06-13T06:57:12.630031+0000","last_peered":"2024-06-13T06:57:12.630031+0000","last_clean":"2024-06-13T06:57:12.630031+0000","last_became_active":"2024-06-13T06:55:29.367134+0000","last_became_peered":"2024-06-13T06:55:29.367134+0000","last_unstale":"2024-06-13T06:57:12.630031+0000","last_undegraded":"2024-06-13T06:57:12.630031+0000","last_fullsized":"2024-06-13T06:57:12.630031+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:19:08.784438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62655,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":107,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522524+0000","last_change":"2024-06-13T06:54:58.897686+0000","last_active":"2024-06-13T06:56:29.522524+0000","last_peered":"2024-06-13T06:56:29.522524+0000","last_clean":"2024-06-13T06:56:29.522524+0000","last_became_active":"2024-06-13T06:54:58.897313+0000","last_became_peered":"2024-06-13T06:54:58.897313+0000","last_unstale":"2024-06-13T06:56:29.522524+0000","last_undegraded":"2024-06-13T06:56:29.522524+0000","last_fullsized":"2024-06-13T06:56:29.522524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:54:49.558542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526734+0000","last_change":"2024-06-13T06:55:12.013918+0000","last_active":"2024-06-13T06:56:29.526734+0000","last_peered":"2024-06-13T06:56:29.526734+0000","last_clean":"2024-06-13T06:56:29.526734+0000","last_became_active":"2024-06-13T06:55:12.013522+0000","last_became_peered":"2024-06-13T06:55:12.013522+0000","last_unstale":"2024-06-13T06:56:29.526734+0000","last_undegraded":"2024-06-13T06:56:29.526734+0000","last_fullsized":"2024-06-13T06:56:29.526734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:51:46.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607084+0000","last_change":"2024-06-13T06:54:54.328674+0000","last_active":"2024-06-13T06:56:29.607084+0000","last_peered":"2024-06-13T06:56:29.607084+0000","last_clean":"2024-06-13T06:56:29.607084+0000","last_became_active":"2024-06-13T06:54:54.326789+0000","last_became_peered":"2024-06-13T06:54:54.326789+0000","last_unstale":"2024-06-13T06:56:29.607084+0000","last_undegraded":"2024-06-13T06:56:29.607084+0000","last_fullsized":"2024-06-13T06:56:29.607084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:18.280514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.599635+0000","last_change":"2024-06-13T06:54:58.918598+0000","last_active":"2024-06-13T06:56:29.599635+0000","last_peered":"2024-06-13T06:56:29.599635+0000","last_clean":"2024-06-13T06:56:29.599635+0000","last_became_active":"2024-06-13T06:54:58.918247+0000","last_became_peered":"2024-06-13T06:54:58.918247+0000","last_unstale":"2024-06-13T06:56:29.599635+0000","last_undegraded":"2024-06-13T06:56:29.599635+0000","last_fullsized":"2024-06-13T06:56:29.599635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:08:56.738806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524251+0000","last_change":"2024-06-13T06:55:12.017074+0000","last_active":"2024-06-13T06:56:29.524251+0000","last_peered":"2024-06-13T06:56:29.524251+0000","last_clean":"2024-06-13T06:56:29.524251+0000","last_became_active":"2024-06-13T06:55:12.016675+0000","last_became_peered":"2024-06-13T06:55:12.016675+0000","last_unstale":"2024-06-13T06:56:29.524251+0000","last_undegraded":"2024-06-13T06:56:29.524251+0000","last_fullsized":"2024-06-13T06:56:29.524251+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:20:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525911+0000","last_change":"2024-06-13T06:54:54.334994+0000","last_active":"2024-06-13T06:56:29.525911+0000","last_peered":"2024-06-13T06:56:29.525911+0000","last_clean":"2024-06-13T06:56:29.525911+0000","last_became_active":"2024-06-13T06:54:54.334647+0000","last_became_peered":"2024-06-13T06:54:54.334647+0000","last_unstale":"2024-06-13T06:56:29.525911+0000","last_undegraded":"2024-06-13T06:56:29.525911+0000","last_fullsized":"2024-06-13T06:56:29.525911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:59:15.317976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608697+0000","last_change":"2024-06-13T06:54:58.899107+0000","last_active":"2024-06-13T06:56:29.608697+0000","last_peered":"2024-06-13T06:56:29.608697+0000","last_clean":"2024-06-13T06:56:29.608697+0000","last_became_active":"2024-06-13T06:54:58.896034+0000","last_became_peered":"2024-06-13T06:54:58.896034+0000","last_unstale":"2024-06-13T06:56:29.608697+0000","last_undegraded":"2024-06-13T06:56:29.608697+0000","last_fullsized":"2024-06-13T06:56:29.608697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:00:01.132565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533178+0000","last_change":"2024-06-13T06:55:12.025681+0000","last_active":"2024-06-13T06:56:29.533178+0000","last_peered":"2024-06-13T06:56:29.533178+0000","last_clean":"2024-06-13T06:56:29.533178+0000","last_became_active":"2024-06-13T06:55:12.025267+0000","last_became_peered":"2024-06-13T06:55:12.025267+0000","last_unstale":"2024-06-13T06:56:29.533178+0000","last_undegraded":"2024-06-13T06:56:29.533178+0000","last_fullsized":"2024-06-13T06:56:29.533178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:58:23.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":90,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:11.633844+0000","last_change":"2024-06-13T06:54:54.325404+0000","last_active":"2024-06-13T06:57:11.633844+0000","last_peered":"2024-06-13T06:57:11.633844+0000","last_clean":"2024-06-13T06:57:11.633844+0000","last_became_active":"2024-06-13T06:54:54.325024+0000","last_became_peered":"2024-06-13T06:54:54.325024+0000","last_unstale":"2024-06-13T06:57:11.633844+0000","last_undegraded":"2024-06-13T06:57:11.633844+0000","last_fullsized":"2024-06-13T06:57:11.633844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:23.726350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607302+0000","last_change":"2024-06-13T06:54:58.894737+0000","last_active":"2024-06-13T06:56:29.607302+0000","last_peered":"2024-06-13T06:56:29.607302+0000","last_clean":"2024-06-13T06:56:29.607302+0000","last_became_active":"2024-06-13T06:54:58.894224+0000","last_became_peered":"2024-06-13T06:54:58.894224+0000","last_unstale":"2024-06-13T06:56:29.607302+0000","last_undegraded":"2024-06-13T06:56:29.607302+0000","last_fullsized":"2024-06-13T06:56:29.607302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:03.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600126+0000","last_change":"2024-06-13T06:55:12.026587+0000","last_active":"2024-06-13T06:56:29.600126+0000","last_peered":"2024-06-13T06:56:29.600126+0000","last_clean":"2024-06-13T06:56:29.600126+0000","last_became_active":"2024-06-13T06:55:12.026114+0000","last_became_peered":"2024-06-13T06:55:12.026114+0000","last_unstale":"2024-06-13T06:56:29.600126+0000","last_undegraded":"2024-06-13T06:56:29.600126+0000","last_fullsized":"2024-06-13T06:56:29.600126+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:43:09.245641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522168+0000","last_change":"2024-06-13T06:54:54.632069+0000","last_active":"2024-06-13T06:56:29.522168+0000","last_peered":"2024-06-13T06:56:29.522168+0000","last_clean":"2024-06-13T06:56:29.522168+0000","last_became_active":"2024-06-13T06:54:54.631692+0000","last_became_peered":"2024-06-13T06:54:54.631692+0000","last_unstale":"2024-06-13T06:56:29.522168+0000","last_undegraded":"2024-06-13T06:56:29.522168+0000","last_fullsized":"2024-06-13T06:56:29.522168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:07.109507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526117+0000","last_change":"2024-06-13T06:55:29.366286+0000","last_active":"2024-06-13T06:56:29.526117+0000","last_peered":"2024-06-13T06:56:29.526117+0000","last_clean":"2024-06-13T06:56:29.526117+0000","last_became_active":"2024-06-13T06:55:29.365854+0000","last_became_peered":"2024-06-13T06:55:29.365854+0000","last_unstale":"2024-06-13T06:56:29.526117+0000","last_undegraded":"2024-06-13T06:56:29.526117+0000","last_fullsized":"2024-06-13T06:56:29.526117+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:04.209244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526156+0000","last_change":"2024-06-13T06:55:12.021621+0000","last_active":"2024-06-13T06:56:29.526156+0000","last_peered":"2024-06-13T06:56:29.526156+0000","last_clean":"2024-06-13T06:56:29.526156+0000","last_became_active":"2024-06-13T06:55:12.021150+0000","last_became_peered":"2024-06-13T06:55:12.021150+0000","last_unstale":"2024-06-13T06:56:29.526156+0000","last_undegraded":"2024-06-13T06:56:29.526156+0000","last_fullsized":"2024-06-13T06:56:29.526156+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:48.318918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:30.508342+0000","last_change":"2024-06-13T06:55:29.369972+0000","last_active":"2024-06-13T06:56:30.508342+0000","last_peered":"2024-06-13T06:56:30.508342+0000","last_clean":"2024-06-13T06:56:30.508342+0000","last_became_active":"2024-06-13T06:55:29.366390+0000","last_became_peered":"2024-06-13T06:55:29.366390+0000","last_unstale":"2024-06-13T06:56:30.508342+0000","last_undegraded":"2024-06-13T06:56:30.508342+0000","last_fullsized":"2024-06-13T06:56:30.508342+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:18.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524659+0000","last_change":"2024-06-13T06:54:58.891309+0000","last_active":"2024-06-13T06:56:29.524659+0000","last_peered":"2024-06-13T06:56:29.524659+0000","last_clean":"2024-06-13T06:56:29.524659+0000","last_became_active":"2024-06-13T06:54:58.890972+0000","last_became_peered":"2024-06-13T06:54:58.890972+0000","last_unstale":"2024-06-13T06:56:29.524659+0000","last_undegraded":"2024-06-13T06:56:29.524659+0000","last_fullsized":"2024-06-13T06:56:29.524659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:41.240061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.610713+0000","last_change":"2024-06-13T06:56:29.610713+0000","last_active":"2024-06-13T06:56:29.610713+0000","last_peered":"2024-06-13T06:56:29.610713+0000","last_clean":"2024-06-13T06:56:29.610713+0000","last_became_active":"2024-06-13T06:56:29.610139+0000","last_became_peered":"2024-06-13T06:56:29.610139+0000","last_unstale":"2024-06-13T06:56:29.610713+0000","last_undegraded":"2024-06-13T06:56:29.610713+0000","last_fullsized":"2024-06-13T06:56:29.610713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:14:36.601243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.534462+0000","last_change":"2024-06-13T06:54:54.633893+0000","last_active":"2024-06-13T06:56:29.534462+0000","last_peered":"2024-06-13T06:56:29.534462+0000","last_clean":"2024-06-13T06:56:29.534462+0000","last_became_active":"2024-06-13T06:54:54.632946+0000","last_became_peered":"2024-06-13T06:54:54.632946+0000","last_unstale":"2024-06-13T06:56:29.534462+0000","last_undegraded":"2024-06-13T06:56:29.534462+0000","last_fullsized":"2024-06-13T06:56:29.534462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:16:54.812477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522730+0000","last_change":"2024-06-13T06:54:58.907182+0000","last_active":"2024-06-13T06:56:29.522730+0000","last_peered":"2024-06-13T06:56:29.522730+0000","last_clean":"2024-06-13T06:56:29.522730+0000","last_became_active":"2024-06-13T06:54:58.906824+0000","last_became_peered":"2024-06-13T06:54:58.906824+0000","last_unstale":"2024-06-13T06:56:29.522730+0000","last_undegraded":"2024-06-13T06:56:29.522730+0000","last_fullsized":"2024-06-13T06:56:29.522730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:34:49.626991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.597070+0000","last_change":"2024-06-13T06:55:12.014803+0000","last_active":"2024-06-13T06:56:29.597070+0000","last_peered":"2024-06-13T06:56:29.597070+0000","last_clean":"2024-06-13T06:56:29.597070+0000","last_became_active":"2024-06-13T06:55:12.014398+0000","last_became_peered":"2024-06-13T06:55:12.014398+0000","last_unstale":"2024-06-13T06:56:29.597070+0000","last_undegraded":"2024-06-13T06:56:29.597070+0000","last_fullsized":"2024-06-13T06:56:29.597070+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:07:44.960446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607834+0000","last_change":"2024-06-13T06:54:54.339056+0000","last_active":"2024-06-13T06:56:29.607834+0000","last_peered":"2024-06-13T06:56:29.607834+0000","last_clean":"2024-06-13T06:56:29.607834+0000","last_became_active":"2024-06-13T06:54:54.338683+0000","last_became_peered":"2024-06-13T06:54:54.338683+0000","last_unstale":"2024-06-13T06:56:29.607834+0000","last_undegraded":"2024-06-13T06:56:29.607834+0000","last_fullsized":"2024-06-13T06:56:29.607834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:05:17.060471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522498+0000","last_change":"2024-06-13T06:54:58.919843+0000","last_active":"2024-06-13T06:56:29.522498+0000","last_peered":"2024-06-13T06:56:29.522498+0000","last_clean":"2024-06-13T06:56:29.522498+0000","last_became_active":"2024-06-13T06:54:58.919405+0000","last_became_peered":"2024-06-13T06:54:58.919405+0000","last_unstale":"2024-06-13T06:56:29.522498+0000","last_undegraded":"2024-06-13T06:56:29.522498+0000","last_fullsized":"2024-06-13T06:56:29.522498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:00:20.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600039+0000","last_change":"2024-06-13T06:55:12.097651+0000","last_active":"2024-06-13T06:56:29.600039+0000","last_peered":"2024-06-13T06:56:29.600039+0000","last_clean":"2024-06-13T06:56:29.600039+0000","last_became_active":"2024-06-13T06:55:12.095644+0000","last_became_peered":"2024-06-13T06:55:12.095644+0000","last_unstale":"2024-06-13T06:56:29.600039+0000","last_undegraded":"2024-06-13T06:56:29.600039+0000","last_fullsized":"2024-06-13T06:56:29.600039+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:09:54.328428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526081+0000","last_change":"2024-06-13T06:54:58.891187+0000","last_active":"2024-06-13T06:56:29.526081+0000","last_peered":"2024-06-13T06:56:29.526081+0000","last_clean":"2024-06-13T06:56:29.526081+0000","last_became_active":"2024-06-13T06:54:58.890848+0000","last_became_peered":"2024-06-13T06:54:58.890848+0000","last_unstale":"2024-06-13T06:56:29.526081+0000","last_undegraded":"2024-06-13T06:56:29.526081+0000","last_fullsized":"2024-06-13T06:56:29.526081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:13:31.709650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":89,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:10.630536+0000","last_change":"2024-06-13T06:54:54.631482+0000","last_active":"2024-06-13T06:57:10.630536+0000","last_peered":"2024-06-13T06:57:10.630536+0000","last_clean":"2024-06-13T06:57:10.630536+0000","last_became_active":"2024-06-13T06:54:54.630818+0000","last_became_peered":"2024-06-13T06:54:54.630818+0000","last_unstale":"2024-06-13T06:57:10.630536+0000","last_undegraded":"2024-06-13T06:57:10.630536+0000","last_fullsized":"2024-06-13T06:57:10.630536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:34:18.801727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607252+0000","last_change":"2024-06-13T06:55:12.018856+0000","last_active":"2024-06-13T06:56:29.607252+0000","last_peered":"2024-06-13T06:56:29.607252+0000","last_clean":"2024-06-13T06:56:29.607252+0000","last_became_active":"2024-06-13T06:55:12.018488+0000","last_became_peered":"2024-06-13T06:55:12.018488+0000","last_unstale":"2024-06-13T06:56:29.607252+0000","last_undegraded":"2024-06-13T06:56:29.607252+0000","last_fullsized":"2024-06-13T06:56:29.607252+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:14.973197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532812+0000","last_change":"2024-06-13T06:55:29.377203+0000","last_active":"2024-06-13T06:56:29.532812+0000","last_peered":"2024-06-13T06:56:29.532812+0000","last_clean":"2024-06-13T06:56:29.532812+0000","last_became_active":"2024-06-13T06:55:29.375263+0000","last_became_peered":"2024-06-13T06:55:29.375263+0000","last_unstale":"2024-06-13T06:56:29.532812+0000","last_undegraded":"2024-06-13T06:56:29.532812+0000","last_fullsized":"2024-06-13T06:56:29.532812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:06:07.935172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558452+0000","last_change":"2024-06-13T06:54:54.335319+0000","last_active":"2024-06-13T06:56:32.558452+0000","last_peered":"2024-06-13T06:56:32.558452+0000","last_clean":"2024-06-13T06:56:32.558452+0000","last_became_active":"2024-06-13T06:54:54.334896+0000","last_became_peered":"2024-06-13T06:54:54.334896+0000","last_unstale":"2024-06-13T06:56:32.558452+0000","last_undegraded":"2024-06-13T06:56:32.558452+0000","last_fullsized":"2024-06-13T06:56:32.558452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:40.331535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606983+0000","last_change":"2024-06-13T06:53:16.968830+0000","last_active":"2024-06-13T06:56:29.606983+0000","last_peered":"2024-06-13T06:56:29.606983+0000","last_clean":"2024-06-13T06:56:29.606983+0000","last_became_active":"2024-06-13T06:53:16.968361+0000","last_became_peered":"2024-06-13T06:53:16.968361+0000","last_unstale":"2024-06-13T06:56:29.606983+0000","last_undegraded":"2024-06-13T06:56:29.606983+0000","last_fullsized":"2024-06-13T06:56:29.606983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_clean_scrub_stamp":"2024-06-13T06:49:18.858540+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:32:26.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.602124+0000","last_change":"2024-06-13T06:56:29.602124+0000","last_active":"2024-06-13T06:56:29.602124+0000","last_peered":"2024-06-13T06:56:29.602124+0000","last_clean":"2024-06-13T06:56:29.602124+0000","last_became_active":"2024-06-13T06:56:29.601650+0000","last_became_peered":"2024-06-13T06:56:29.601650+0000","last_unstale":"2024-06-13T06:56:29.602124+0000","last_undegraded":"2024-06-13T06:56:29.602124+0000","last_fullsized":"2024-06-13T06:56:29.602124+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:37:23.903614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.596545+0000","last_change":"2024-06-13T06:54:58.898930+0000","last_active":"2024-06-13T06:56:29.596545+0000","last_peered":"2024-06-13T06:56:29.596545+0000","last_clean":"2024-06-13T06:56:29.596545+0000","last_became_active":"2024-06-13T06:54:58.898384+0000","last_became_peered":"2024-06-13T06:54:58.898384+0000","last_unstale":"2024-06-13T06:56:29.596545+0000","last_undegraded":"2024-06-13T06:56:29.596545+0000","last_fullsized":"2024-06-13T06:56:29.596545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:06:02.210336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607221+0000","last_change":"2024-06-13T06:54:54.328776+0000","last_active":"2024-06-13T06:56:29.607221+0000","last_peered":"2024-06-13T06:56:29.607221+0000","last_clean":"2024-06-13T06:56:29.607221+0000","last_became_active":"2024-06-13T06:54:54.327773+0000","last_became_peered":"2024-06-13T06:54:54.327773+0000","last_unstale":"2024-06-13T06:56:29.607221+0000","last_undegraded":"2024-06-13T06:56:29.607221+0000","last_fullsized":"2024-06-13T06:56:29.607221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:23:02.240774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524397+0000","last_change":"2024-06-13T06:55:12.013862+0000","last_active":"2024-06-13T06:56:29.524397+0000","last_peered":"2024-06-13T06:56:29.524397+0000","last_clean":"2024-06-13T06:56:29.524397+0000","last_became_active":"2024-06-13T06:55:12.013389+0000","last_became_peered":"2024-06-13T06:55:12.013389+0000","last_unstale":"2024-06-13T06:56:29.524397+0000","last_undegraded":"2024-06-13T06:56:29.524397+0000","last_fullsized":"2024-06-13T06:56:29.524397+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:49.701202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.831665+0000","last_change":"2024-06-13T06:56:29.539862+0000","last_active":"2024-06-13T06:56:29.831665+0000","last_peered":"2024-06-13T06:56:29.831665+0000","last_clean":"2024-06-13T06:56:29.831665+0000","last_became_active":"2024-06-13T06:56:29.536109+0000","last_became_peered":"2024-06-13T06:56:29.536109+0000","last_unstale":"2024-06-13T06:56:29.831665+0000","last_undegraded":"2024-06-13T06:56:29.831665+0000","last_fullsized":"2024-06-13T06:56:29.831665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:23.307850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.599679+0000","last_change":"2024-06-13T06:54:58.901773+0000","last_active":"2024-06-13T06:56:29.599679+0000","last_peered":"2024-06-13T06:56:29.599679+0000","last_clean":"2024-06-13T06:56:29.599679+0000","last_became_active":"2024-06-13T06:54:58.901168+0000","last_became_peered":"2024-06-13T06:54:58.901168+0000","last_unstale":"2024-06-13T06:56:29.599679+0000","last_undegraded":"2024-06-13T06:56:29.599679+0000","last_fullsized":"2024-06-13T06:56:29.599679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:57:15.848785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524234+0000","last_change":"2024-06-13T06:55:12.095890+0000","last_active":"2024-06-13T06:56:29.524234+0000","last_peered":"2024-06-13T06:56:29.524234+0000","last_clean":"2024-06-13T06:56:29.524234+0000","last_became_active":"2024-06-13T06:55:12.095397+0000","last_became_peered":"2024-06-13T06:55:12.095397+0000","last_unstale":"2024-06-13T06:56:29.524234+0000","last_undegraded":"2024-06-13T06:56:29.524234+0000","last_fullsized":"2024-06-13T06:56:29.524234+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:56:18.211036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'11","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:04.939835+0000","last_change":"2024-06-13T06:54:54.337946+0000","last_active":"2024-06-13T06:57:04.939835+0000","last_peered":"2024-06-13T06:57:04.939835+0000","last_clean":"2024-06-13T06:57:04.939835+0000","last_became_active":"2024-06-13T06:54:54.337028+0000","last_became_peered":"2024-06-13T06:54:54.337028+0000","last_unstale":"2024-06-13T06:57:04.939835+0000","last_undegraded":"2024-06-13T06:57:04.939835+0000","last_fullsized":"2024-06-13T06:57:04.939835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:08:52.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526297+0000","last_change":"2024-06-13T06:54:58.895110+0000","last_active":"2024-06-13T06:56:29.526297+0000","last_peered":"2024-06-13T06:56:29.526297+0000","last_clean":"2024-06-13T06:56:29.526297+0000","last_became_active":"2024-06-13T06:54:58.893481+0000","last_became_peered":"2024-06-13T06:54:58.893481+0000","last_unstale":"2024-06-13T06:56:29.526297+0000","last_undegraded":"2024-06-13T06:56:29.526297+0000","last_fullsized":"2024-06-13T06:56:29.526297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:54.932110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606822+0000","last_change":"2024-06-13T06:55:11.997712+0000","last_active":"2024-06-13T06:56:29.606822+0000","last_peered":"2024-06-13T06:56:29.606822+0000","last_clean":"2024-06-13T06:56:29.606822+0000","last_became_active":"2024-06-13T06:55:11.997193+0000","last_became_peered":"2024-06-13T06:55:11.997193+0000","last_unstale":"2024-06-13T06:56:29.606822+0000","last_undegraded":"2024-06-13T06:56:29.606822+0000","last_fullsized":"2024-06-13T06:56:29.606822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:15.168070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608714+0000","last_change":"2024-06-13T06:54:54.520121+0000","last_active":"2024-06-13T06:56:29.608714+0000","last_peered":"2024-06-13T06:56:29.608714+0000","last_clean":"2024-06-13T06:56:29.608714+0000","last_became_active":"2024-06-13T06:54:54.519723+0000","last_became_peered":"2024-06-13T06:54:54.519723+0000","last_unstale":"2024-06-13T06:56:29.608714+0000","last_undegraded":"2024-06-13T06:56:29.608714+0000","last_fullsized":"2024-06-13T06:56:29.608714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:19:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532607+0000","last_change":"2024-06-13T06:54:58.923626+0000","last_active":"2024-06-13T06:56:29.532607+0000","last_peered":"2024-06-13T06:56:29.532607+0000","last_clean":"2024-06-13T06:56:29.532607+0000","last_became_active":"2024-06-13T06:54:58.923192+0000","last_became_peered":"2024-06-13T06:54:58.923192+0000","last_unstale":"2024-06-13T06:56:29.532607+0000","last_undegraded":"2024-06-13T06:56:29.532607+0000","last_fullsized":"2024-06-13T06:56:29.532607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:26:13.231549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607658+0000","last_change":"2024-06-13T06:55:12.096244+0000","last_active":"2024-06-13T06:56:29.607658+0000","last_peered":"2024-06-13T06:56:29.607658+0000","last_clean":"2024-06-13T06:56:29.607658+0000","last_became_active":"2024-06-13T06:55:12.095758+0000","last_became_peered":"2024-06-13T06:55:12.095758+0000","last_unstale":"2024-06-13T06:56:29.607658+0000","last_undegraded":"2024-06-13T06:56:29.607658+0000","last_fullsized":"2024-06-13T06:56:29.607658+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:38.122282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607931+0000","last_change":"2024-06-13T06:54:54.630488+0000","last_active":"2024-06-13T06:56:29.607931+0000","last_peered":"2024-06-13T06:56:29.607931+0000","last_clean":"2024-06-13T06:56:29.607931+0000","last_became_active":"2024-06-13T06:54:54.630224+0000","last_became_peered":"2024-06-13T06:54:54.630224+0000","last_unstale":"2024-06-13T06:56:29.607931+0000","last_undegraded":"2024-06-13T06:56:29.607931+0000","last_fullsized":"2024-06-13T06:56:29.607931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:33:19.107026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522127+0000","last_change":"2024-06-13T06:54:58.894466+0000","last_active":"2024-06-13T06:56:29.522127+0000","last_peered":"2024-06-13T06:56:29.522127+0000","last_clean":"2024-06-13T06:56:29.522127+0000","last_became_active":"2024-06-13T06:54:58.894034+0000","last_became_peered":"2024-06-13T06:54:58.894034+0000","last_unstale":"2024-06-13T06:56:29.522127+0000","last_undegraded":"2024-06-13T06:56:29.522127+0000","last_fullsized":"2024-06-13T06:56:29.522127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:44.051565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533541+0000","last_change":"2024-06-13T06:55:12.022078+0000","last_active":"2024-06-13T06:56:29.533541+0000","last_peered":"2024-06-13T06:56:29.533541+0000","last_clean":"2024-06-13T06:56:29.533541+0000","last_became_active":"2024-06-13T06:55:12.021652+0000","last_became_peered":"2024-06-13T06:55:12.021652+0000","last_unstale":"2024-06-13T06:56:29.533541+0000","last_undegraded":"2024-06-13T06:56:29.533541+0000","last_fullsized":"2024-06-13T06:56:29.533541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:25:57.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524569+0000","last_change":"2024-06-13T06:54:54.336757+0000","last_active":"2024-06-13T06:56:29.524569+0000","last_peered":"2024-06-13T06:56:29.524569+0000","last_clean":"2024-06-13T06:56:29.524569+0000","last_became_active":"2024-06-13T06:54:54.336012+0000","last_became_peered":"2024-06-13T06:54:54.336012+0000","last_unstale":"2024-06-13T06:56:29.524569+0000","last_undegraded":"2024-06-13T06:56:29.524569+0000","last_fullsized":"2024-06-13T06:56:29.524569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:52:21.227464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524562+0000","last_change":"2024-06-13T06:54:58.919631+0000","last_active":"2024-06-13T06:56:29.524562+0000","last_peered":"2024-06-13T06:56:29.524562+0000","last_clean":"2024-06-13T06:56:29.524562+0000","last_became_active":"2024-06-13T06:54:58.919274+0000","last_became_peered":"2024-06-13T06:54:58.919274+0000","last_unstale":"2024-06-13T06:56:29.524562+0000","last_undegraded":"2024-06-13T06:56:29.524562+0000","last_fullsized":"2024-06-13T06:56:29.524562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:22.453797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608429+0000","last_change":"2024-06-13T06:55:12.017338+0000","last_active":"2024-06-13T06:56:29.608429+0000","last_peered":"2024-06-13T06:56:29.608429+0000","last_clean":"2024-06-13T06:56:29.608429+0000","last_became_active":"2024-06-13T06:55:12.016841+0000","last_became_peered":"2024-06-13T06:55:12.016841+0000","last_unstale":"2024-06-13T06:56:29.608429+0000","last_undegraded":"2024-06-13T06:56:29.608429+0000","last_fullsized":"2024-06-13T06:56:29.608429+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:40:13.525892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607062+0000","last_change":"2024-06-13T06:54:54.328808+0000","last_active":"2024-06-13T06:56:29.607062+0000","last_peered":"2024-06-13T06:56:29.607062+0000","last_clean":"2024-06-13T06:56:29.607062+0000","last_became_active":"2024-06-13T06:54:54.326982+0000","last_became_peered":"2024-06-13T06:54:54.326982+0000","last_unstale":"2024-06-13T06:56:29.607062+0000","last_undegraded":"2024-06-13T06:56:29.607062+0000","last_fullsized":"2024-06-13T06:56:29.607062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:22:53.854275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524893+0000","last_change":"2024-06-13T06:54:58.925728+0000","last_active":"2024-06-13T06:56:29.524893+0000","last_peered":"2024-06-13T06:56:29.524893+0000","last_clean":"2024-06-13T06:56:29.524893+0000","last_became_active":"2024-06-13T06:54:58.925407+0000","last_became_peered":"2024-06-13T06:54:58.925407+0000","last_unstale":"2024-06-13T06:56:29.524893+0000","last_undegraded":"2024-06-13T06:56:29.524893+0000","last_fullsized":"2024-06-13T06:56:29.524893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:38.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607449+0000","last_change":"2024-06-13T06:55:12.095733+0000","last_active":"2024-06-13T06:56:29.607449+0000","last_peered":"2024-06-13T06:56:29.607449+0000","last_clean":"2024-06-13T06:56:29.607449+0000","last_became_active":"2024-06-13T06:55:12.095064+0000","last_became_peered":"2024-06-13T06:55:12.095064+0000","last_unstale":"2024-06-13T06:56:29.607449+0000","last_undegraded":"2024-06-13T06:56:29.607449+0000","last_fullsized":"2024-06-13T06:56:29.607449+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:26:05.486701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556112+0000","last_change":"2024-06-13T06:54:54.331448+0000","last_active":"2024-06-13T06:56:32.556112+0000","last_peered":"2024-06-13T06:56:32.556112+0000","last_clean":"2024-06-13T06:56:32.556112+0000","last_became_active":"2024-06-13T06:54:54.330964+0000","last_became_peered":"2024-06-13T06:54:54.330964+0000","last_unstale":"2024-06-13T06:56:32.556112+0000","last_undegraded":"2024-06-13T06:56:32.556112+0000","last_fullsized":"2024-06-13T06:56:32.556112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:23:12.211412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607683+0000","last_change":"2024-06-13T06:54:58.899112+0000","last_active":"2024-06-13T06:56:29.607683+0000","last_peered":"2024-06-13T06:56:29.607683+0000","last_clean":"2024-06-13T06:56:29.607683+0000","last_became_active":"2024-06-13T06:54:58.898742+0000","last_became_peered":"2024-06-13T06:54:58.898742+0000","last_unstale":"2024-06-13T06:56:29.607683+0000","last_undegraded":"2024-06-13T06:56:29.607683+0000","last_fullsized":"2024-06-13T06:56:29.607683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:50:12.983287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608176+0000","last_change":"2024-06-13T06:55:12.014149+0000","last_active":"2024-06-13T06:56:29.608176+0000","last_peered":"2024-06-13T06:56:29.608176+0000","last_clean":"2024-06-13T06:56:29.608176+0000","last_became_active":"2024-06-13T06:55:12.013454+0000","last_became_peered":"2024-06-13T06:55:12.013454+0000","last_unstale":"2024-06-13T06:56:29.608176+0000","last_undegraded":"2024-06-13T06:56:29.608176+0000","last_fullsized":"2024-06-13T06:56:29.608176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:13:39.368540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.560296+0000","last_change":"2024-06-13T06:54:54.633268+0000","last_active":"2024-06-13T06:56:32.560296+0000","last_peered":"2024-06-13T06:56:32.560296+0000","last_clean":"2024-06-13T06:56:32.560296+0000","last_became_active":"2024-06-13T06:54:54.632840+0000","last_became_peered":"2024-06-13T06:54:54.632840+0000","last_unstale":"2024-06-13T06:56:32.560296+0000","last_undegraded":"2024-06-13T06:56:32.560296+0000","last_fullsized":"2024-06-13T06:56:32.560296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:21.653727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609371+0000","last_change":"2024-06-13T06:54:58.893417+0000","last_active":"2024-06-13T06:56:29.609371+0000","last_peered":"2024-06-13T06:56:29.609371+0000","last_clean":"2024-06-13T06:56:29.609371+0000","last_became_active":"2024-06-13T06:54:58.892381+0000","last_became_peered":"2024-06-13T06:54:58.892381+0000","last_unstale":"2024-06-13T06:56:29.609371+0000","last_undegraded":"2024-06-13T06:56:29.609371+0000","last_fullsized":"2024-06-13T06:56:29.609371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:07.979654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533388+0000","last_change":"2024-06-13T06:55:12.018336+0000","last_active":"2024-06-13T06:56:29.533388+0000","last_peered":"2024-06-13T06:56:29.533388+0000","last_clean":"2024-06-13T06:56:29.533388+0000","last_became_active":"2024-06-13T06:55:12.017861+0000","last_became_peered":"2024-06-13T06:55:12.017861+0000","last_unstale":"2024-06-13T06:56:29.533388+0000","last_undegraded":"2024-06-13T06:56:29.533388+0000","last_fullsized":"2024-06-13T06:56:29.533388+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:41.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556542+0000","last_change":"2024-06-13T06:54:54.333036+0000","last_active":"2024-06-13T06:56:32.556542+0000","last_peered":"2024-06-13T06:56:32.556542+0000","last_clean":"2024-06-13T06:56:32.556542+0000","last_became_active":"2024-06-13T06:54:54.332642+0000","last_became_peered":"2024-06-13T06:54:54.332642+0000","last_unstale":"2024-06-13T06:56:32.556542+0000","last_undegraded":"2024-06-13T06:56:32.556542+0000","last_fullsized":"2024-06-13T06:56:32.556542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:06:06.809024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607529+0000","last_change":"2024-06-13T06:54:58.881894+0000","last_active":"2024-06-13T06:56:29.607529+0000","last_peered":"2024-06-13T06:56:29.607529+0000","last_clean":"2024-06-13T06:56:29.607529+0000","last_became_active":"2024-06-13T06:54:58.881591+0000","last_became_peered":"2024-06-13T06:54:58.881591+0000","last_unstale":"2024-06-13T06:56:29.607529+0000","last_undegraded":"2024-06-13T06:56:29.607529+0000","last_fullsized":"2024-06-13T06:56:29.607529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:49:00.944212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525378+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:56:29.525378+0000","last_peered":"2024-06-13T06:56:29.525378+0000","last_clean":"2024-06-13T06:56:29.525378+0000","last_became_active":"2024-06-13T06:55:12.000748+0000","last_became_peered":"2024-06-13T06:55:12.000748+0000","last_unstale":"2024-06-13T06:56:29.525378+0000","last_undegraded":"2024-06-13T06:56:29.525378+0000","last_fullsized":"2024-06-13T06:56:29.525378+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:22.035807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600248+0000","last_change":"2024-06-13T06:54:54.342724+0000","last_active":"2024-06-13T06:56:29.600248+0000","last_peered":"2024-06-13T06:56:29.600248+0000","last_clean":"2024-06-13T06:56:29.600248+0000","last_became_active":"2024-06-13T06:54:54.342267+0000","last_became_peered":"2024-06-13T06:54:54.342267+0000","last_unstale":"2024-06-13T06:56:29.600248+0000","last_undegraded":"2024-06-13T06:56:29.600248+0000","last_fullsized":"2024-06-13T06:56:29.600248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:27:53.039202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608347+0000","last_change":"2024-06-13T06:54:58.892365+0000","last_active":"2024-06-13T06:56:29.608347+0000","last_peered":"2024-06-13T06:56:29.608347+0000","last_clean":"2024-06-13T06:56:29.608347+0000","last_became_active":"2024-06-13T06:54:58.889395+0000","last_became_peered":"2024-06-13T06:54:58.889395+0000","last_unstale":"2024-06-13T06:56:29.608347+0000","last_undegraded":"2024-06-13T06:56:29.608347+0000","last_fullsized":"2024-06-13T06:56:29.608347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:55.798293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524334+0000","last_change":"2024-06-13T06:54:58.888570+0000","last_active":"2024-06-13T06:56:29.524334+0000","last_peered":"2024-06-13T06:56:29.524334+0000","last_clean":"2024-06-13T06:56:29.524334+0000","last_became_active":"2024-06-13T06:54:58.888180+0000","last_became_peered":"2024-06-13T06:54:58.888180+0000","last_unstale":"2024-06-13T06:56:29.524334+0000","last_undegraded":"2024-06-13T06:56:29.524334+0000","last_fullsized":"2024-06-13T06:56:29.524334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:46:42.537790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607162+0000","last_change":"2024-06-13T06:54:58.893476+0000","last_active":"2024-06-13T06:56:29.607162+0000","last_peered":"2024-06-13T06:56:29.607162+0000","last_clean":"2024-06-13T06:56:29.607162+0000","last_became_active":"2024-06-13T06:54:58.893021+0000","last_became_peered":"2024-06-13T06:54:58.893021+0000","last_unstale":"2024-06-13T06:56:29.607162+0000","last_undegraded":"2024-06-13T06:56:29.607162+0000","last_fullsized":"2024-06-13T06:56:29.607162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:38.739070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607046+0000","last_change":"2024-06-13T06:54:58.911042+0000","last_active":"2024-06-13T06:56:29.607046+0000","last_peered":"2024-06-13T06:56:29.607046+0000","last_clean":"2024-06-13T06:56:29.607046+0000","last_became_active":"2024-06-13T06:54:58.910641+0000","last_became_peered":"2024-06-13T06:54:58.910641+0000","last_unstale":"2024-06-13T06:56:29.607046+0000","last_undegraded":"2024-06-13T06:56:29.607046+0000","last_fullsized":"2024-06-13T06:56:29.607046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:21:39.467223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.534304+0000","last_change":"2024-06-13T06:55:29.382730+0000","last_active":"2024-06-13T06:56:29.534304+0000","last_peered":"2024-06-13T06:56:29.534304+0000","last_clean":"2024-06-13T06:56:29.534304+0000","last_became_active":"2024-06-13T06:55:29.382162+0000","last_became_peered":"2024-06-13T06:55:29.382162+0000","last_unstale":"2024-06-13T06:56:29.534304+0000","last_undegraded":"2024-06-13T06:56:29.534304+0000","last_fullsized":"2024-06-13T06:56:29.534304+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:05.486414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.599632+0000","last_change":"2024-06-13T06:55:29.385380+0000","last_active":"2024-06-13T06:56:29.599632+0000","last_peered":"2024-06-13T06:56:29.599632+0000","last_clean":"2024-06-13T06:56:29.599632+0000","last_became_active":"2024-06-13T06:55:29.384855+0000","last_became_peered":"2024-06-13T06:55:29.384855+0000","last_unstale":"2024-06-13T06:56:29.599632+0000","last_undegraded":"2024-06-13T06:56:29.599632+0000","last_fullsized":"2024-06-13T06:56:29.599632+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:31:01.945632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525533+0000","last_change":"2024-06-13T06:54:58.889128+0000","last_active":"2024-06-13T06:56:29.525533+0000","last_peered":"2024-06-13T06:56:29.525533+0000","last_clean":"2024-06-13T06:56:29.525533+0000","last_became_active":"2024-06-13T06:54:58.888606+0000","last_became_peered":"2024-06-13T06:54:58.888606+0000","last_unstale":"2024-06-13T06:56:29.525533+0000","last_undegraded":"2024-06-13T06:56:29.525533+0000","last_fullsized":"2024-06-13T06:56:29.525533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:53:43.762968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607486+0000","last_change":"2024-06-13T06:54:58.896098+0000","last_active":"2024-06-13T06:56:29.607486+0000","last_peered":"2024-06-13T06:56:29.607486+0000","last_clean":"2024-06-13T06:56:29.607486+0000","last_became_active":"2024-06-13T06:54:58.895722+0000","last_became_peered":"2024-06-13T06:54:58.895722+0000","last_unstale":"2024-06-13T06:56:29.607486+0000","last_undegraded":"2024-06-13T06:56:29.607486+0000","last_fullsized":"2024-06-13T06:56:29.607486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:18:14.421363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526716+0000","last_change":"2024-06-13T06:54:58.905590+0000","last_active":"2024-06-13T06:56:29.526716+0000","last_peered":"2024-06-13T06:56:29.526716+0000","last_clean":"2024-06-13T06:56:29.526716+0000","last_became_active":"2024-06-13T06:54:58.905146+0000","last_became_peered":"2024-06-13T06:54:58.905146+0000","last_unstale":"2024-06-13T06:56:29.526716+0000","last_undegraded":"2024-06-13T06:56:29.526716+0000","last_fullsized":"2024-06-13T06:56:29.526716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:42:08.010906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525835+0000","last_change":"2024-06-13T06:54:58.898037+0000","last_active":"2024-06-13T06:56:29.525835+0000","last_peered":"2024-06-13T06:56:29.525835+0000","last_clean":"2024-06-13T06:56:29.525835+0000","last_became_active":"2024-06-13T06:54:58.897692+0000","last_became_peered":"2024-06-13T06:54:58.897692+0000","last_unstale":"2024-06-13T06:56:29.525835+0000","last_undegraded":"2024-06-13T06:56:29.525835+0000","last_fullsized":"2024-06-13T06:56:29.525835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:44:30.018192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524735+0000","last_change":"2024-06-13T06:54:58.903441+0000","last_active":"2024-06-13T06:56:29.524735+0000","last_peered":"2024-06-13T06:56:29.524735+0000","last_clean":"2024-06-13T06:56:29.524735+0000","last_became_active":"2024-06-13T06:54:58.893598+0000","last_became_peered":"2024-06-13T06:54:58.893598+0000","last_unstale":"2024-06-13T06:56:29.524735+0000","last_undegraded":"2024-06-13T06:56:29.524735+0000","last_fullsized":"2024-06-13T06:56:29.524735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:38:29.968619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609241+0000","last_change":"2024-06-13T06:54:58.918594+0000","last_active":"2024-06-13T06:56:29.609241+0000","last_peered":"2024-06-13T06:56:29.609241+0000","last_clean":"2024-06-13T06:56:29.609241+0000","last_became_active":"2024-06-13T06:54:58.918255+0000","last_became_peered":"2024-06-13T06:54:58.918255+0000","last_unstale":"2024-06-13T06:56:29.609241+0000","last_undegraded":"2024-06-13T06:56:29.609241+0000","last_fullsized":"2024-06-13T06:56:29.609241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:39:48.527707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.597127+0000","last_change":"2024-06-13T06:54:58.926253+0000","last_active":"2024-06-13T06:56:29.597127+0000","last_peered":"2024-06-13T06:56:29.597127+0000","last_clean":"2024-06-13T06:56:29.597127+0000","last_became_active":"2024-06-13T06:54:58.924611+0000","last_became_peered":"2024-06-13T06:54:58.924611+0000","last_unstale":"2024-06-13T06:56:29.597127+0000","last_undegraded":"2024-06-13T06:56:29.597127+0000","last_fullsized":"2024-06-13T06:56:29.597127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:29:23.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524605+0000","last_change":"2024-06-13T06:54:58.890454+0000","last_active":"2024-06-13T06:56:29.524605+0000","last_peered":"2024-06-13T06:56:29.524605+0000","last_clean":"2024-06-13T06:56:29.524605+0000","last_became_active":"2024-06-13T06:54:58.890129+0000","last_became_peered":"2024-06-13T06:54:58.890129+0000","last_unstale":"2024-06-13T06:56:29.524605+0000","last_undegraded":"2024-06-13T06:56:29.524605+0000","last_fullsized":"2024-06-13T06:56:29.524605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:10.819536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66655,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":270,"num_read_kb":207,"num_write":167,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1656,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":570102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645361,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37124,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708028,"statfs":{"total":95995035648,"available":95957020672,"internally_reserved":0,"allocated":1830912,"data_stored":1219981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":574390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070525,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":570170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332163,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1166122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495688,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1134592,"data_stored":565470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36512,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708640,"statfs":{"total":95995035648,"available":95957647360,"internally_reserved":0,"allocated":1204224,"data_stored":628035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1159168,"data_stored":566858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":566108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":565664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1228800,"data_stored":633787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1161710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:57:16.045 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-13T06:57:16.046 INFO:tasks.cephadm:Setting up client nodes... 2024-06-13T06:57:16.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-13T06:57:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:16 smithi026 ceph-mon[32041]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-13T06:57:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:16 smithi136 ceph-mon[37229]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-13T06:57:16.498 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:16 smithi135 ceph-mon[36276]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-13T06:57:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:17 smithi026 ceph-mon[32041]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:57:17.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:17 smithi136 ceph-mon[37229]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:57:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:17 smithi135 ceph-mon[36276]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:57:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:18 smithi136 ceph-mon[37229]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:57:18.411 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:18 smithi026 ceph-mon[32041]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:57:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:18 smithi135 ceph-mon[36276]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:57:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 -- 172.21.15.26:0/156977493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b2c1058d0 msgr2=0x7f7b2c107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 --2- 172.21.15.26:0/156977493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b2c1058d0 0x7f7b2c107cc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b180098f0 tx=0x7f7b18009c00 comp rx=0 tx=0).stop 2024-06-13T06:57:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 -- 172.21.15.26:0/156977493 shutdown_connections 2024-06-13T06:57:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 --2- 172.21.15.26:0/156977493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b2c102fa0 0x7f7b2c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 --2- 172.21.15.26:0/156977493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2c108200 0x7f7b2c10a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 --2- 172.21.15.26:0/156977493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b2c1058d0 0x7f7b2c107cc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 -- 172.21.15.26:0/156977493 >> 172.21.15.26:0/156977493 conn(0x7f7b2c0fc9b0 msgr2=0x7f7b2c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 -- 172.21.15.26:0/156977493 shutdown_connections 2024-06-13T06:57:18.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.987+0000 7f7b34600700 1 -- 172.21.15.26:0/156977493 wait complete. 2024-06-13T06:57:18.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.989+0000 7f7b34600700 1 Processor -- start 2024-06-13T06:57:18.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.989+0000 7f7b34600700 1 -- start start 2024-06-13T06:57:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.989+0000 7f7b34600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b2c102fa0 0x7f7b2c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.989+0000 7f7b34600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b2c1058d0 0x7f7b2c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.989+0000 7f7b34600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2c108200 0x7f7b2c1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.989+0000 7f7b34600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c10e130 con 0x7f7b2c108200 2024-06-13T06:57:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.989+0000 7f7b34600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c10dfb0 con 0x7f7b2c102fa0 2024-06-13T06:57:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.989+0000 7f7b34600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c10e2b0 con 0x7f7b2c1058d0 2024-06-13T06:57:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b2c102fa0 0x7f7b2c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b32600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2c108200 0x7f7b2c1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b2c102fa0 0x7f7b2c19a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37620/0 (socket says 172.21.15.26:37620) 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31c00700 1 -- 172.21.15.26:0/1378688025 learned_addr learned my addr 172.21.15.26:0/1378688025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31200700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b2c1058d0 0x7f7b2c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31c00700 1 -- 172.21.15.26:0/1378688025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b2c1058d0 msgr2=0x7f7b2c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31c00700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b2c1058d0 0x7f7b2c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31c00700 1 -- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2c108200 msgr2=0x7f7b2c1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31c00700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2c108200 0x7f7b2c1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b31c00700 1 -- 172.21.15.26:0/1378688025 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c1a2550 con 0x7f7b2c102fa0 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.990+0000 7f7b32600700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2c108200 0x7f7b2c1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.991+0000 7f7b31c00700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b2c102fa0 0x7f7b2c19a8f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000e9a0 tx=0x7f7b2000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.991+0000 7f7b2a000700 1 -- 172.21.15.26:0/1378688025 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000ca10 con 0x7f7b2c102fa0 2024-06-13T06:57:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.991+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c1a2840 con 0x7f7b2c102fa0 2024-06-13T06:57:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.991+0000 7f7b2a000700 1 -- 172.21.15.26:0/1378688025 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2000cbb0 con 0x7f7b2c102fa0 2024-06-13T06:57:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.991+0000 7f7b2a000700 1 -- 172.21.15.26:0/1378688025 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20018710 con 0x7f7b2c102fa0 2024-06-13T06:57:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.991+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c1a2dd0 con 0x7f7b2c102fa0 2024-06-13T06:57:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.992+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2c050a00 con 0x7f7b2c102fa0 2024-06-13T06:57:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.994+0000 7f7b2a000700 1 -- 172.21.15.26:0/1378688025 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7b200188b0 con 0x7f7b2c102fa0 2024-06-13T06:57:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.996+0000 7f7b2a000700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7b140a2d60 0x7f7b140a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.996+0000 7f7b2a000700 1 -- 172.21.15.26:0/1378688025 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7b200cf7a0 con 0x7f7b2c102fa0 2024-06-13T06:57:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.996+0000 7f7b31200700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7b140a2d60 0x7f7b140a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:18.997+0000 7f7b31200700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7b140a2d60 0x7f7b140a5220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800fb70 tx=0x7f7b18019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.000+0000 7f7b2a000700 1 -- 172.21.15.26:0/1378688025 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7b2009c150 con 0x7f7b2c102fa0 2024-06-13T06:57:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.320+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7b2c02dd70 con 0x7f7b2c102fa0 2024-06-13T06:57:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.328+0000 7f7b2a000700 1 -- 172.21.15.26:0/1378688025 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7f7b2c02dd70 con 0x7f7b2c102fa0 2024-06-13T06:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-06-13T06:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBPmGpmWyo+ExAABDQuqPQrS2c1BLrYAL2Lww== 2024-06-13T06:57:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.331+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7b140a2d60 msgr2=0x7f7b140a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.331+0000 7f7b34600700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7b140a2d60 0x7f7b140a5220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800fb70 tx=0x7f7b18019040 comp rx=0 tx=0).stop 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b2c102fa0 msgr2=0x7f7b2c19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b2c102fa0 0x7f7b2c19a8f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000e9a0 tx=0x7f7b2000ecb0 comp rx=0 tx=0).stop 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 shutdown_connections 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7b140a2d60 0x7f7b140a5220 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b2c102fa0 0x7f7b2c19a8f0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000e9a0 tx=0x7f7b2000ecb0 comp rx=0 tx=0).stop 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2c108200 0x7f7b2c1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 --2- 172.21.15.26:0/1378688025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b2c1058d0 0x7f7b2c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 >> 172.21.15.26:0/1378688025 conn(0x7f7b2c0fc9b0 msgr2=0x7f7b2c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 shutdown_connections 2024-06-13T06:57:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:19.332+0000 7f7b34600700 1 -- 172.21.15.26:0/1378688025 wait complete. 2024-06-13T06:57:20.229 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T06:57:20.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-13T06:57:20.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-13T06:57:20.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:20 smithi026 ceph-mon[32041]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:57:20.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:20 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1378688025' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:57:20.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:20 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:57:20.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:20 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-13T06:57:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:20 smithi136 ceph-mon[37229]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:57:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:20 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1378688025' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:57:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:20 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:57:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:20 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-13T06:57:20.378 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-13T06:57:20.378 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-13T06:57:20.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph mgr dump --format=json 2024-06-13T06:57:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:20 smithi135 ceph-mon[36276]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:57:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:20 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1378688025' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:57:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:20 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:57:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:20 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-13T06:57:20.898 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:22 smithi136 ceph-mon[37229]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:57:22.490 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:22 smithi026 ceph-mon[32041]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:57:22.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:22 smithi135 ceph-mon[36276]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:57:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 -- 172.21.15.26:0/371746352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4c105070 msgr2=0x7fbd4c111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 --2- 172.21.15.26:0/371746352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4c105070 0x7fbd4c111580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400b0a0 tx=0x7fbd4400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 -- 172.21.15.26:0/371746352 shutdown_connections 2024-06-13T06:57:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 --2- 172.21.15.26:0/371746352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4c105070 0x7fbd4c111580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 --2- 172.21.15.26:0/371746352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd4c10a6b0 0x7fbd4c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 --2- 172.21.15.26:0/371746352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4c1046b0 0x7fbd4c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 -- 172.21.15.26:0/371746352 >> 172.21.15.26:0/371746352 conn(0x7fbd4c1001a0 msgr2=0x7fbd4c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 -- 172.21.15.26:0/371746352 shutdown_connections 2024-06-13T06:57:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.360+0000 7fbd51e00700 1 -- 172.21.15.26:0/371746352 wait complete. 2024-06-13T06:57:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.362+0000 7fbd51e00700 1 Processor -- start 2024-06-13T06:57:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.362+0000 7fbd51e00700 1 -- start start 2024-06-13T06:57:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.362+0000 7fbd51e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4c1046b0 0x7fbd4c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.362+0000 7fbd51e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd4c105070 0x7fbd4c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.362+0000 7fbd51e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4c10a6b0 0x7fbd4c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.362+0000 7fbd51e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c114a90 con 0x7fbd4c105070 2024-06-13T06:57:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.362+0000 7fbd51e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c114910 con 0x7fbd4c1046b0 2024-06-13T06:57:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.362+0000 7fbd51e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c114c10 con 0x7fbd4c10a6b0 2024-06-13T06:57:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4c1046b0 0x7fbd4c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd4c105070 0x7fbd4c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4c1046b0 0x7fbd4c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50864/0 (socket says 172.21.15.26:50864) 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4aa00700 1 -- 172.21.15.26:0/2791216835 learned_addr learned my addr 172.21.15.26:0/2791216835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4b400700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4c10a6b0 0x7fbd4c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4aa00700 1 -- 172.21.15.26:0/2791216835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4c10a6b0 msgr2=0x7fbd4c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4aa00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4c10a6b0 0x7fbd4c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4aa00700 1 -- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd4c105070 msgr2=0x7fbd4c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4aa00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd4c105070 0x7fbd4c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.363+0000 7fbd4aa00700 1 -- 172.21.15.26:0/2791216835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c19fe10 con 0x7fbd4c1046b0 2024-06-13T06:57:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.364+0000 7fbd4aa00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4c1046b0 0x7fbd4c19ae00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00d980 tx=0x7fbd3c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.364+0000 7fbd50a00700 1 -- 172.21.15.26:0/2791216835 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c016070 con 0x7fbd4c1046b0 2024-06-13T06:57:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.364+0000 7fbd50a00700 1 -- 172.21.15.26:0/2791216835 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c004d10 con 0x7fbd4c1046b0 2024-06-13T06:57:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.364+0000 7fbd50a00700 1 -- 172.21.15.26:0/2791216835 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c005670 con 0x7fbd4c1046b0 2024-06-13T06:57:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.364+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c1a0100 con 0x7fbd4c1046b0 2024-06-13T06:57:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.364+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c1a7980 con 0x7fbd4c1046b0 2024-06-13T06:57:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.365+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c069180 con 0x7fbd4c1046b0 2024-06-13T06:57:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.367+0000 7fbd50a00700 1 -- 172.21.15.26:0/2791216835 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd3c004750 con 0x7fbd4c1046b0 2024-06-13T06:57:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.368+0000 7fbd50a00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd300a2d30 0x7fbd300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.369+0000 7fbd50a00700 1 -- 172.21.15.26:0/2791216835 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbd3c0cf2d0 con 0x7fbd4c1046b0 2024-06-13T06:57:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.369+0000 7fbd4a000700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd300a2d30 0x7fbd300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.370+0000 7fbd4a000700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd300a2d30 0x7fbd300a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c109c70 tx=0x7fbd38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:23.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.373+0000 7fbd50a00700 1 -- 172.21.15.26:0/2791216835 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd3c01f020 con 0x7fbd4c1046b0 2024-06-13T06:57:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.664+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbd4c02dca0 con 0x7fbd4c1046b0 2024-06-13T06:57:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.670+0000 7fbd50a00700 1 -- 172.21.15.26:0/2791216835 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263195 (secure 0 0 0) 0x7fbd3c022350 con 0x7fbd4c1046b0 2024-06-13T06:57:23.678 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:57:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd300a2d30 msgr2=0x7fbd300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd300a2d30 0x7fbd300a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c109c70 tx=0x7fbd38009260 comp rx=0 tx=0).stop 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4c1046b0 msgr2=0x7fbd4c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4c1046b0 0x7fbd4c19ae00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00d980 tx=0x7fbd3c00dc90 comp rx=0 tx=0).stop 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 shutdown_connections 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd300a2d30 0x7fbd300a51f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4c1046b0 0x7fbd4c19ae00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4c10a6b0 0x7fbd4c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 --2- 172.21.15.26:0/2791216835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd4c105070 0x7fbd4c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 >> 172.21.15.26:0/2791216835 conn(0x7fbd4c1001a0 msgr2=0x7fbd4c100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 shutdown_connections 2024-06-13T06:57:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:23.678+0000 7fbd51e00700 1 -- 172.21.15.26:0/2791216835 wait complete. 2024-06-13T06:57:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:24 smithi136 ceph-mon[37229]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:57:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:24 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2791216835' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-13T06:57:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:24 smithi026 ceph-mon[32041]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:57:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:24 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2791216835' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-13T06:57:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:24 smithi135 ceph-mon[36276]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:57:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:24 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2791216835' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-13T06:57:24.738 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":662324914},{"type":"v1","addr":"172.21.15.26:6801","nonce":662324914}]},"active_addr":"172.21.15.26:6801/662324914","active_change":"2024-06-13T06:46:28.110058+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.26:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1507751277}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3379045443}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3099403779}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":63559790}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":741013945}]}]} 2024-06-13T06:57:24.743 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-13T06:57:24.743 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-13T06:57:24.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:57:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:26.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:26 smithi026 ceph-mon[32041]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:57:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:26 smithi136 ceph-mon[37229]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:57:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:26 smithi135 ceph-mon[36276]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:57:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 -- 172.21.15.26:0/3783645349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb8406b3c0 msgr2=0x7fdb8410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 --2- 172.21.15.26:0/3783645349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb8406b3c0 0x7fdb8410f310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00b0a0 tx=0x7fdb7c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 -- 172.21.15.26:0/3783645349 shutdown_connections 2024-06-13T06:57:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 --2- 172.21.15.26:0/3783645349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb8406b3c0 0x7fdb8410f310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 --2- 172.21.15.26:0/3783645349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb841084f0 0x7fdb841088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 --2- 172.21.15.26:0/3783645349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8406aa00 0x7fdb8406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 -- 172.21.15.26:0/3783645349 >> 172.21.15.26:0/3783645349 conn(0x7fdb84078300 msgr2=0x7fdb84078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 -- 172.21.15.26:0/3783645349 shutdown_connections 2024-06-13T06:57:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.592+0000 7fdb88e00700 1 -- 172.21.15.26:0/3783645349 wait complete. 2024-06-13T06:57:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.594+0000 7fdb88e00700 1 Processor -- start 2024-06-13T06:57:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.594+0000 7fdb88e00700 1 -- start start 2024-06-13T06:57:27.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb88e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8406aa00 0x7fdb8419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb88e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb8406b3c0 0x7fdb8419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb88e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb841084f0 0x7fdb84198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb88e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841127a0 con 0x7fdb841084f0 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb88e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84112620 con 0x7fdb8406aa00 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb88e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84112920 con 0x7fdb8406b3c0 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb81e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8406aa00 0x7fdb8419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb81e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8406aa00 0x7fdb8419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50882/0 (socket says 172.21.15.26:50882) 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb81e00700 1 -- 172.21.15.26:0/1580693944 learned_addr learned my addr 172.21.15.26:0/1580693944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb82800700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb841084f0 0x7fdb84198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb81400700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb8406b3c0 0x7fdb8419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb81e00700 1 -- 172.21.15.26:0/1580693944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb8406b3c0 msgr2=0x7fdb8419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb81e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb8406b3c0 0x7fdb8419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.595+0000 7fdb81e00700 1 -- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb841084f0 msgr2=0x7fdb84198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.596+0000 7fdb81e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb841084f0 0x7fdb84198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.596+0000 7fdb81e00700 1 -- 172.21.15.26:0/1580693944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb84199220 con 0x7fdb8406aa00 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.596+0000 7fdb81e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8406aa00 0x7fdb8419e8b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400e9a0 tx=0x7fdb7400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.596+0000 7fdb7aa00700 1 -- 172.21.15.26:0/1580693944 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400ca10 con 0x7fdb8406aa00 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.596+0000 7fdb7aa00700 1 -- 172.21.15.26:0/1580693944 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb7400cbb0 con 0x7fdb8406aa00 2024-06-13T06:57:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.596+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb84199510 con 0x7fdb8406aa00 2024-06-13T06:57:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.596+0000 7fdb7aa00700 1 -- 172.21.15.26:0/1580693944 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74018680 con 0x7fdb8406aa00 2024-06-13T06:57:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.596+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb841a5780 con 0x7fdb8406aa00 2024-06-13T06:57:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.598+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb84069180 con 0x7fdb8406aa00 2024-06-13T06:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.599+0000 7fdb7aa00700 1 -- 172.21.15.26:0/1580693944 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdb74010430 con 0x7fdb8406aa00 2024-06-13T06:57:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.601+0000 7fdb7aa00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb680a2d30 0x7fdb680a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.601+0000 7fdb7aa00700 1 -- 172.21.15.26:0/1580693944 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdb740cf750 con 0x7fdb8406aa00 2024-06-13T06:57:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.601+0000 7fdb81400700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb680a2d30 0x7fdb680a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.602+0000 7fdb81400700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb680a2d30 0x7fdb680a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb84078950 tx=0x7fdb70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.605+0000 7fdb7aa00700 1 -- 172.21.15.26:0/1580693944 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdb7409c100 con 0x7fdb8406aa00 2024-06-13T06:57:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.843+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb8402dca0 con 0x7fdb8406aa00 2024-06-13T06:57:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.845+0000 7fdb7aa00700 1 -- 172.21.15.26:0/1580693944 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7fdb7409ba00 con 0x7fdb8406aa00 2024-06-13T06:57:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:57:27.846 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:57:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb680a2d30 msgr2=0x7fdb680a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb680a2d30 0x7fdb680a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb84078950 tx=0x7fdb70009260 comp rx=0 tx=0).stop 2024-06-13T06:57:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8406aa00 msgr2=0x7fdb8419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8406aa00 0x7fdb8419e8b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400e9a0 tx=0x7fdb7400ecb0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 shutdown_connections 2024-06-13T06:57:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb680a2d30 0x7fdb680a51f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8406aa00 0x7fdb8419e8b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb841084f0 0x7fdb84198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 --2- 172.21.15.26:0/1580693944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb8406b3c0 0x7fdb8419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 >> 172.21.15.26:0/1580693944 conn(0x7fdb84078300 msgr2=0x7fdb840fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.849+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 shutdown_connections 2024-06-13T06:57:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:27.850+0000 7fdb88e00700 1 -- 172.21.15.26:0/1580693944 wait complete. 2024-06-13T06:57:28.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:28 smithi136 ceph-mon[37229]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:28.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:28 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1580693944' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:28 smithi026 ceph-mon[32041]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:28 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1580693944' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:28 smithi135 ceph-mon[36276]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:28 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1580693944' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:28.861 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-13T06:57:28.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:57:29.327 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:29.474 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:29 smithi026 ceph-mon[32041]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:57:29.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:29 smithi135 ceph-mon[36276]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:57:29.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:29 smithi136 ceph-mon[37229]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:57:31.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:31 smithi026 ceph-mon[32041]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:31 smithi135 ceph-mon[36276]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:31.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.796+0000 7f816ae00700 1 -- 172.21.15.26:0/2482190602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8164074fe0 msgr2=0x7f816410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:31.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.796+0000 7f816ae00700 1 --2- 172.21.15.26:0/2482190602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8164074fe0 0x7f816410e9b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098f0 tx=0x7f814c009c00 comp rx=0 tx=0).stop 2024-06-13T06:57:31.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.797+0000 7f816ae00700 1 -- 172.21.15.26:0/2482190602 shutdown_connections 2024-06-13T06:57:31.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.797+0000 7f816ae00700 1 --2- 172.21.15.26:0/2482190602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816410eef0 0x7f81641112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:31.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.797+0000 7f816ae00700 1 --2- 172.21.15.26:0/2482190602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81640746c0 0x7f8164074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:31.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.797+0000 7f816ae00700 1 --2- 172.21.15.26:0/2482190602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8164074fe0 0x7f816410e9b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:31.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.797+0000 7f816ae00700 1 -- 172.21.15.26:0/2482190602 >> 172.21.15.26:0/2482190602 conn(0x7f81640795f0 msgr2=0x7f8164079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:31.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.797+0000 7f816ae00700 1 -- 172.21.15.26:0/2482190602 shutdown_connections 2024-06-13T06:57:31.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.797+0000 7f816ae00700 1 -- 172.21.15.26:0/2482190602 wait complete. 2024-06-13T06:57:31.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.798+0000 7f816ae00700 1 Processor -- start 2024-06-13T06:57:31.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.798+0000 7f816ae00700 1 -- start start 2024-06-13T06:57:31.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f816ae00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81640746c0 0x7f816419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:31.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f816ae00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8164074fe0 0x7f816419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:31.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f816ae00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816410eef0 0x7f816419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:31.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f816ae00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8164114a70 con 0x7f816410eef0 2024-06-13T06:57:31.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f816ae00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81641148f0 con 0x7f81640746c0 2024-06-13T06:57:31.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f816ae00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8164114bf0 con 0x7f8164074fe0 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8163e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81640746c0 0x7f816419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8163e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81640746c0 0x7f816419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50892/0 (socket says 172.21.15.26:50892) 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8163e00700 1 -- 172.21.15.26:0/1039399989 learned_addr learned my addr 172.21.15.26:0/1039399989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8168e00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816410eef0 0x7f816419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f815be00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8164074fe0 0x7f816419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8163e00700 1 -- 172.21.15.26:0/1039399989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8164074fe0 msgr2=0x7f816419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8163e00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8164074fe0 0x7f816419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8163e00700 1 -- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816410eef0 msgr2=0x7f816419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8163e00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816410eef0 0x7f816419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.799+0000 7f8163e00700 1 -- 172.21.15.26:0/1039399989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816419ff00 con 0x7f81640746c0 2024-06-13T06:57:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.800+0000 7f8163e00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81640746c0 0x7f816419af40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f815400e9c0 tx=0x7f815400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.800+0000 7f8161600700 1 -- 172.21.15.26:0/1039399989 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815400cbd0 con 0x7f81640746c0 2024-06-13T06:57:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.800+0000 7f8161600700 1 -- 172.21.15.26:0/1039399989 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f815400ee60 con 0x7f81640746c0 2024-06-13T06:57:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.800+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81641a01f0 con 0x7f81640746c0 2024-06-13T06:57:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.800+0000 7f8161600700 1 -- 172.21.15.26:0/1039399989 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154005630 con 0x7f81640746c0 2024-06-13T06:57:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.800+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641a7a70 con 0x7f81640746c0 2024-06-13T06:57:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.801+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8164069180 con 0x7f81640746c0 2024-06-13T06:57:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.803+0000 7f8161600700 1 -- 172.21.15.26:0/1039399989 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8154004750 con 0x7f81640746c0 2024-06-13T06:57:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.805+0000 7f8161600700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f81480a2d30 0x7f81480a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.805+0000 7f8161600700 1 -- 172.21.15.26:0/1039399989 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f81540cf800 con 0x7f81640746c0 2024-06-13T06:57:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.805+0000 7f815be00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f81480a2d30 0x7f81480a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.806+0000 7f815be00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f81480a2d30 0x7f81480a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f814c009810 tx=0x7f814c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:31.809+0000 7f8161600700 1 -- 172.21.15.26:0/1039399989 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f815409c1b0 con 0x7f81640746c0 2024-06-13T06:57:31.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:31 smithi136 ceph-mon[37229]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:32.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.039+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f816419c250 con 0x7f81640746c0 2024-06-13T06:57:32.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.041+0000 7f8161600700 1 -- 172.21.15.26:0/1039399989 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7f815409bab0 con 0x7f81640746c0 2024-06-13T06:57:32.041 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:57:32.042 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:57:32.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.044+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f81480a2d30 msgr2=0x7f81480a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:32.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.044+0000 7f816ae00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f81480a2d30 0x7f81480a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f814c009810 tx=0x7f814c019040 comp rx=0 tx=0).stop 2024-06-13T06:57:32.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.044+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81640746c0 msgr2=0x7f816419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:32.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.044+0000 7f816ae00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81640746c0 0x7f816419af40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f815400e9c0 tx=0x7f815400ecd0 comp rx=0 tx=0).stop 2024-06-13T06:57:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.045+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 shutdown_connections 2024-06-13T06:57:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.045+0000 7f816ae00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f81480a2d30 0x7f81480a51f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.045+0000 7f816ae00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81640746c0 0x7f816419af40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.045+0000 7f816ae00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816410eef0 0x7f816419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.045+0000 7f816ae00700 1 --2- 172.21.15.26:0/1039399989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8164074fe0 0x7f816419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.045+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 >> 172.21.15.26:0/1039399989 conn(0x7f81640795f0 msgr2=0x7f81640ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.045+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 shutdown_connections 2024-06-13T06:57:32.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:32.045+0000 7f816ae00700 1 -- 172.21.15.26:0/1039399989 wait complete. 2024-06-13T06:57:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:32 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1039399989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:32.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:32 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1039399989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:32.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:32 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1039399989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:57:33.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-13T06:57:33.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-13T06:57:33.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-13T06:57:33.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-13T06:57:33.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-13T06:57:33.008 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-13T06:57:33.008 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-13T06:57:33.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-13T06:57:33.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-13T06:57:33.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-13T06:57:33.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-13T06:57:33.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-13T06:57:33.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:33 smithi026 ceph-mon[32041]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:33 smithi135 ceph-mon[36276]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:33 smithi136 ceph-mon[37229]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:34.358 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:34.380 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:34.385 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:35.633 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:35 smithi026 ceph-mon[32041]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:35.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:35 smithi135 ceph-mon[36276]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:35.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:35 smithi136 ceph-mon[37229]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:37.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:37 smithi026 ceph-mon[32041]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:37 smithi135 ceph-mon[36276]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:37 smithi136 ceph-mon[37229]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:39.720 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:39 smithi026 ceph-mon[32041]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:57:39.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:39 smithi135 ceph-mon[36276]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:57:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:39 smithi136 ceph-mon[37229]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:57:41.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:41 smithi026 ceph-mon[32041]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:41 smithi135 ceph-mon[36276]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:41 smithi136 ceph-mon[37229]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:43.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:43 smithi026 ceph-mon[32041]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:43 smithi135 ceph-mon[36276]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:43 smithi136 ceph-mon[37229]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:45.653 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:45 smithi026 ceph-mon[32041]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:45.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:45 smithi135 ceph-mon[36276]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:45 smithi136 ceph-mon[37229]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 -- 172.21.15.26:0/3981209159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd39810ee90 msgr2=0x7fd398111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 --2- 172.21.15.26:0/3981209159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd39810ee90 0x7fd398111280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd39000b0a0 tx=0x7fd39000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 -- 172.21.15.26:0/3981209159 shutdown_connections 2024-06-13T06:57:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 --2- 172.21.15.26:0/3981209159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3980746c0 0x7fd398074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 --2- 172.21.15.26:0/3981209159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd39810ee90 0x7fd398111280 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 --2- 172.21.15.26:0/3981209159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398074fe0 0x7fd39810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 -- 172.21.15.26:0/3981209159 >> 172.21.15.26:0/3981209159 conn(0x7fd3980795f0 msgr2=0x7fd398079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 -- 172.21.15.26:0/3981209159 shutdown_connections 2024-06-13T06:57:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.793+0000 7fd39fe00700 1 -- 172.21.15.26:0/3981209159 wait complete. 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.795+0000 7fd39fe00700 1 Processor -- start 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.795+0000 7fd39fe00700 1 -- start start 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.795+0000 7fd39fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3980746c0 0x7fd39819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.795+0000 7fd39fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398074fe0 0x7fd39819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.795+0000 7fd39fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd39810ee90 0x7fd39819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.795+0000 7fd39fe00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981149d0 con 0x7fd39810ee90 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.795+0000 7fd39fe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd398114850 con 0x7fd3980746c0 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.795+0000 7fd39fe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd398114b50 con 0x7fd398074fe0 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd39810ee90 0x7fd39819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3980746c0 0x7fd39819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3980746c0 0x7fd39819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49456/0 (socket says 172.21.15.26:49456) 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd39810ee90 0x7fd39819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60812/0 (socket says 172.21.15.26:60812) 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39d400700 1 -- 172.21.15.26:0/4247577563 learned_addr learned my addr 172.21.15.26:0/4247577563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39ca00700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398074fe0 0x7fd39819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39d400700 1 -- 172.21.15.26:0/4247577563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398074fe0 msgr2=0x7fd39819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39d400700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398074fe0 0x7fd39819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39d400700 1 -- 172.21.15.26:0/4247577563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd39810ee90 msgr2=0x7fd39819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39d400700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd39810ee90 0x7fd39819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.796+0000 7fd39d400700 1 -- 172.21.15.26:0/4247577563 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39819fdb0 con 0x7fd3980746c0 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.797+0000 7fd39d400700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3980746c0 0x7fd39819ada0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00e9a0 tx=0x7fd38c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.797+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c00ca10 con 0x7fd3980746c0 2024-06-13T06:57:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.797+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38c00cbb0 con 0x7fd3980746c0 2024-06-13T06:57:46.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.797+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c018680 con 0x7fd3980746c0 2024-06-13T06:57:46.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.797+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3981a00a0 con 0x7fd3980746c0 2024-06-13T06:57:46.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.797+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3981a7920 con 0x7fd3980746c0 2024-06-13T06:57:46.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.798+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd398100020 con 0x7fd3980746c0 2024-06-13T06:57:46.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.800+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd38c010430 con 0x7fd3980746c0 2024-06-13T06:57:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.802+0000 7fd396000700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3800a2d30 0x7fd3800a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.802+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd38c0cff70 con 0x7fd3980746c0 2024-06-13T06:57:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.802+0000 7fd39ca00700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3800a2d30 0x7fd3800a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.802+0000 7fd396000700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] conn(0x7fd3800ae110 0x7fd3800b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.802+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 --> [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd38c001d30 con 0x7fd3800ae110 2024-06-13T06:57:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.802+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fd38c0d0320 con 0x7fd3980746c0 2024-06-13T06:57:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.803+0000 7fd39de00700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] conn(0x7fd3800ae110 0x7fd3800b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.803+0000 7fd39ca00700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3800a2d30 0x7fd3800a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd384009680 tx=0x7fd384009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:46.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.803+0000 7fd39de00700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] conn(0x7fd3800ae110 0x7fd3800b05b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:46.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.804+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 <== osd.2 v2:172.21.15.136:6800/4076851589 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd38c001d30 con 0x7fd3800ae110 2024-06-13T06:57:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.823+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 --> [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd398075db0 con 0x7fd3800ae110 2024-06-13T06:57:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.824+0000 7fd396000700 1 -- 172.21.15.26:0/4247577563 <== osd.2 v2:172.21.15.136:6800/4076851589 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd398075db0 con 0x7fd3800ae110 2024-06-13T06:57:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.824+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 >> [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] conn(0x7fd3800ae110 msgr2=0x7fd3800b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.824+0000 7fd39fe00700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] conn(0x7fd3800ae110 0x7fd3800b05b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3800a2d30 msgr2=0x7fd3800a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39fe00700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3800a2d30 0x7fd3800a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd384009680 tx=0x7fd384009260 comp rx=0 tx=0).stop 2024-06-13T06:57:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3980746c0 msgr2=0x7fd39819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39fe00700 1 --2- 172.21.15.26:0/4247577563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3980746c0 0x7fd39819ada0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00e9a0 tx=0x7fd38c00ecb0 comp rx=0 tx=0).stop 2024-06-13T06:57:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39de00700 1 -- 172.21.15.26:0/4247577563 reap_dead start 2024-06-13T06:57:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 shutdown_connections 2024-06-13T06:57:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 >> 172.21.15.26:0/4247577563 conn(0x7fd3980795f0 msgr2=0x7fd3980ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 shutdown_connections 2024-06-13T06:57:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.825+0000 7fd39fe00700 1 -- 172.21.15.26:0/4247577563 wait complete. 2024-06-13T06:57:47.659 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:47 smithi026 ceph-mon[32041]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.981+0000 7f7897600700 1 -- 172.21.15.26:0/3856783687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7890102ee0 msgr2=0x7f789010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.982+0000 7f7897600700 1 --2- 172.21.15.26:0/3856783687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7890102ee0 0x7f789010f3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7888009f90 tx=0x7f788800b330 comp rx=0 tx=0).stop 2024-06-13T06:57:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.982+0000 7f7897600700 1 -- 172.21.15.26:0/3856783687 shutdown_connections 2024-06-13T06:57:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.982+0000 7f7897600700 1 --2- 172.21.15.26:0/3856783687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890107450 0x7f7890107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.982+0000 7f7897600700 1 --2- 172.21.15.26:0/3856783687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7890102ee0 0x7f789010f3c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:47.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.982+0000 7f7897600700 1 --2- 172.21.15.26:0/3856783687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7890102540 0x7f78901029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.982+0000 7f7897600700 1 -- 172.21.15.26:0/3856783687 >> 172.21.15.26:0/3856783687 conn(0x7f78900795f0 msgr2=0x7f7890079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.982+0000 7f7897600700 1 -- 172.21.15.26:0/3856783687 shutdown_connections 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.982+0000 7f7897600700 1 -- 172.21.15.26:0/3856783687 wait complete. 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.983+0000 7f7897600700 1 Processor -- start 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.983+0000 7f7897600700 1 -- start start 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.984+0000 7f7897600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7890102540 0x7f789019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.984+0000 7f7897600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7890102ee0 0x7f789019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.984+0000 7f7897600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890107450 0x7f789019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.984+0000 7f7897600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7890112830 con 0x7f7890102540 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.984+0000 7f7897600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78901126b0 con 0x7f7890107450 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.984+0000 7f7897600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78901129b0 con 0x7f7890102ee0 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.984+0000 7f7894c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7890102540 0x7f789019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f7894c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7890102540 0x7f789019ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60756/0 (socket says 172.21.15.26:60756) 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f7894c00700 1 -- 172.21.15.26:0/778458602 learned_addr learned my addr 172.21.15.26:0/778458602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f788fe00700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7890102ee0 0x7f789019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f7895600700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890107450 0x7f789019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f7894c00700 1 -- 172.21.15.26:0/778458602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7890102ee0 msgr2=0x7f789019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f7894c00700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7890102ee0 0x7f789019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f7894c00700 1 -- 172.21.15.26:0/778458602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890107450 msgr2=0x7f789019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f7894c00700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890107450 0x7f789019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.985+0000 7f7894c00700 1 -- 172.21.15.26:0/778458602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789019fe90 con 0x7f7890102540 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.986+0000 7f7894c00700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7890102540 0x7f789019ae80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f788400e970 tx=0x7f788400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.986+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884004220 con 0x7f7890102540 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.986+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78840043c0 con 0x7f7890102540 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.986+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884003dc0 con 0x7f7890102540 2024-06-13T06:57:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.986+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78901a0180 con 0x7f7890102540 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.986+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78901a7a00 con 0x7f7890102540 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.987+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7890100d50 con 0x7f7890102540 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.990+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f788400c7e0 con 0x7f7890102540 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.991+0000 7f788d600700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f78780a2cb0 0x7f78780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.991+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f78840cf900 con 0x7f7890102540 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.991+0000 7f788fe00700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f78780a2cb0 0x7f78780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.991+0000 7f788d600700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] conn(0x7f78780ae090 0x7f78780b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.991+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 --> [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7884012ec0 con 0x7f78780ae090 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.991+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f78840cfd50 con 0x7f7890102540 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.992+0000 7f7895600700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] conn(0x7f78780ae090 0x7f78780b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.992+0000 7f788fe00700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f78780a2cb0 0x7f78780a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f787c009750 tx=0x7f787c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.992+0000 7f7895600700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] conn(0x7f78780ae090 0x7f78780b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:47.786 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:47 smithi135 ceph-mon[36276]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:47.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:43.993+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 <== osd.1 v2:172.21.15.135:6800/2669544474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7884012ec0 con 0x7f78780ae090 2024-06-13T06:57:47.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.012+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 --> [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7890103c00 con 0x7f78780ae090 2024-06-13T06:57:47.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.012+0000 7f788d600700 1 -- 172.21.15.26:0/778458602 <== osd.1 v2:172.21.15.135:6800/2669544474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7890103c00 con 0x7f78780ae090 2024-06-13T06:57:47.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 >> [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] conn(0x7f78780ae090 msgr2=0x7f78780b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:47.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] conn(0x7f78780ae090 0x7f78780b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f78780a2cb0 msgr2=0x7f78780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f78780a2cb0 0x7f78780a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f787c009750 tx=0x7f787c009310 comp rx=0 tx=0).stop 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7890102540 msgr2=0x7f789019ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 --2- 172.21.15.26:0/778458602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7890102540 0x7f789019ae80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f788400e970 tx=0x7f788400ec80 comp rx=0 tx=0).stop 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7895600700 1 -- 172.21.15.26:0/778458602 reap_dead start 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 shutdown_connections 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 >> 172.21.15.26:0/778458602 conn(0x7f78900795f0 msgr2=0x7f78901003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 shutdown_connections 2024-06-13T06:57:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:44.013+0000 7f7897600700 1 -- 172.21.15.26:0/778458602 wait complete. 2024-06-13T06:57:47.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:47 smithi136 ceph-mon[37229]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:48.905 INFO:teuthology.orchestra.run.smithi026.stdout:55834574957 2024-06-13T06:57:48.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.680+0000 7f7654600700 1 -- 172.21.15.26:0/2687063621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c1030f0 msgr2=0x7f764c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.680+0000 7f7654600700 1 --2- 172.21.15.26:0/2687063621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c1030f0 0x7f764c107110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76380099c0 tx=0x7f7638009cd0 comp rx=0 tx=0).stop 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.681+0000 7f7654600700 1 -- 172.21.15.26:0/2687063621 shutdown_connections 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.681+0000 7f7654600700 1 --2- 172.21.15.26:0/2687063621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c107840 0x7f764c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.681+0000 7f7654600700 1 --2- 172.21.15.26:0/2687063621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f764c102740 0x7f764c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.681+0000 7f7654600700 1 --2- 172.21.15.26:0/2687063621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c1030f0 0x7f764c107110 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.681+0000 7f7654600700 1 -- 172.21.15.26:0/2687063621 >> 172.21.15.26:0/2687063621 conn(0x7f764c0795f0 msgr2=0x7f764c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.681+0000 7f7654600700 1 -- 172.21.15.26:0/2687063621 shutdown_connections 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.681+0000 7f7654600700 1 -- 172.21.15.26:0/2687063621 wait complete. 2024-06-13T06:57:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.682+0000 7f7654600700 1 Processor -- start 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.683+0000 7f7654600700 1 -- start start 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.683+0000 7f7654600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102740 0x7f764c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.683+0000 7f7654600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c1030f0 0x7f764c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.683+0000 7f7654600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f764c107840 0x7f764c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.683+0000 7f7654600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f764c112770 con 0x7f764c1030f0 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.683+0000 7f7654600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1125f0 con 0x7f764c102740 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.683+0000 7f7654600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1128f0 con 0x7f764c107840 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102740 0x7f764c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102740 0x7f764c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49438/0 (socket says 172.21.15.26:49438) 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 -- 172.21.15.26:0/2178777847 learned_addr learned my addr 172.21.15.26:0/2178777847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651200700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c1030f0 0x7f764c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7652600700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f764c107840 0x7f764c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 -- 172.21.15.26:0/2178777847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f764c107840 msgr2=0x7f764c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f764c107840 0x7f764c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 -- 172.21.15.26:0/2178777847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c1030f0 msgr2=0x7f764c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c1030f0 0x7f764c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 -- 172.21.15.26:0/2178777847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c19fdb0 con 0x7f764c102740 2024-06-13T06:57:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.684+0000 7f7651c00700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102740 0x7f764c19ada0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f764000e9c0 tx=0x7f764000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.685+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764000cbd0 con 0x7f764c102740 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.685+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f764000cd70 con 0x7f764c102740 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.685+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c1a00a0 con 0x7f764c102740 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.685+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640010430 con 0x7f764c102740 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.685+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c1a7920 con 0x7f764c102740 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.686+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f764c10ea00 con 0x7f764c102740 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.688+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7640044b60 con 0x7f764c102740 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.690+0000 7f764a000700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76340a2d30 0x7f76340a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.691+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f76400cfc10 con 0x7f764c102740 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.691+0000 7f7651200700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76340a2d30 0x7f76340a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.691+0000 7f764a000700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] conn(0x7f76340ae110 0x7f76340b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.691+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 --> [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7640004500 con 0x7f76340ae110 2024-06-13T06:57:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.691+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f76400cffc0 con 0x7f764c102740 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.692+0000 7f7652600700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] conn(0x7f76340ae110 0x7f76340b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.692+0000 7f7651200700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76340a2d30 0x7f76340a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f764c102440 tx=0x7f7638005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.692+0000 7f7652600700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] conn(0x7f76340ae110 0x7f76340b05b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.693+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 <== osd.10 v2:172.21.15.135:6824/1309174884 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7640004500 con 0x7f76340ae110 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.712+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 --> [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f764c103bd0 con 0x7f76340ae110 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.713+0000 7f764a000700 1 -- 172.21.15.26:0/2178777847 <== osd.10 v2:172.21.15.135:6824/1309174884 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f764c103bd0 con 0x7f76340ae110 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 >> [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] conn(0x7f76340ae110 msgr2=0x7f76340b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] conn(0x7f76340ae110 0x7f76340b05b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76340a2d30 msgr2=0x7f76340a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76340a2d30 0x7f76340a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f764c102440 tx=0x7f7638005940 comp rx=0 tx=0).stop 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102740 msgr2=0x7f764c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 --2- 172.21.15.26:0/2178777847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102740 0x7f764c19ada0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f764000e9c0 tx=0x7f764000ed80 comp rx=0 tx=0).stop 2024-06-13T06:57:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7652600700 1 -- 172.21.15.26:0/2178777847 reap_dead start 2024-06-13T06:57:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 shutdown_connections 2024-06-13T06:57:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 >> 172.21.15.26:0/2178777847 conn(0x7f764c0795f0 msgr2=0x7f764c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 shutdown_connections 2024-06-13T06:57:49.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:46.714+0000 7f7654600700 1 -- 172.21.15.26:0/2178777847 wait complete. 2024-06-13T06:57:49.673 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:49 smithi026 ceph-mon[32041]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-06-13T06:57:49.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:49 smithi135 ceph-mon[36276]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-06-13T06:57:49.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:49 smithi136 ceph-mon[37229]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-06-13T06:57:51.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:51 smithi026 ceph-mon[32041]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:51.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:51 smithi135 ceph-mon[36276]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:51.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:51 smithi136 ceph-mon[37229]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:57:52.204 INFO:teuthology.orchestra.run.smithi026.stdout:309237645368 2024-06-13T06:57:52.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.593+0000 7faa39400700 1 -- 172.21.15.26:0/1892437261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa340764c0 msgr2=0x7faa340746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.593+0000 7faa39400700 1 --2- 172.21.15.26:0/1892437261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa340764c0 0x7faa340746c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa2400b0a0 tx=0x7faa2400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.594+0000 7faa39400700 1 -- 172.21.15.26:0/1892437261 shutdown_connections 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.594+0000 7faa39400700 1 --2- 172.21.15.26:0/1892437261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa34113120 0x7faa34115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.594+0000 7faa39400700 1 --2- 172.21.15.26:0/1892437261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa340764c0 0x7faa340746c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.594+0000 7faa39400700 1 --2- 172.21.15.26:0/1892437261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa34074c00 0x7faa34075080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.594+0000 7faa39400700 1 -- 172.21.15.26:0/1892437261 >> 172.21.15.26:0/1892437261 conn(0x7faa34079530 msgr2=0x7faa34079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.594+0000 7faa39400700 1 -- 172.21.15.26:0/1892437261 shutdown_connections 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.594+0000 7faa39400700 1 -- 172.21.15.26:0/1892437261 wait complete. 2024-06-13T06:57:52.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.595+0000 7faa39400700 1 Processor -- start 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.595+0000 7faa39400700 1 -- start start 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa39400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa34074c00 0x7faa3419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa39400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa340764c0 0x7faa3419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa39400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa34113120 0x7faa341a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa39400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa34118d80 con 0x7faa34074c00 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa39400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa34118c00 con 0x7faa34113120 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa39400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa34118f00 con 0x7faa340764c0 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa32800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa34074c00 0x7faa3419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa31e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa340764c0 0x7faa3419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa32800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa34074c00 0x7faa3419f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50792/0 (socket says 172.21.15.26:50792) 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa32800700 1 -- 172.21.15.26:0/1201837250 learned_addr learned my addr 172.21.15.26:0/1201837250 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:52.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa31e00700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa340764c0 0x7faa3419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59862/0 (socket says 172.21.15.26:59862) 2024-06-13T06:57:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.596+0000 7faa33200700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa34113120 0x7faa341a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa32800700 1 -- 172.21.15.26:0/1201837250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa340764c0 msgr2=0x7faa3419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa32800700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa340764c0 0x7faa3419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa32800700 1 -- 172.21.15.26:0/1201837250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa34113120 msgr2=0x7faa341a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa32800700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa34113120 0x7faa341a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa32800700 1 -- 172.21.15.26:0/1201837250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa341a4210 con 0x7faa34074c00 2024-06-13T06:57:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa33200700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa34113120 0x7faa341a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa32800700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa34074c00 0x7faa3419f250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faa240062a0 tx=0x7faa24004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24021070 con 0x7faa34074c00 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa24007b80 con 0x7faa34074c00 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa341a44a0 con 0x7faa34074c00 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.597+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240052c0 con 0x7faa34074c00 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.598+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa341abd80 con 0x7faa34074c00 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.599+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa34112880 con 0x7faa34074c00 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.601+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faa24024070 con 0x7faa34074c00 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.603+0000 7faa2b400700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa180a2cb0 0x7faa180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.603+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faa2400f020 con 0x7faa34074c00 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.603+0000 7faa31e00700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa180a2cb0 0x7faa180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:52.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.603+0000 7faa2b400700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] conn(0x7faa180ae090 0x7faa180b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.603+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 --> [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa240020b0 con 0x7faa180ae090 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.603+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7faa240d52a0 con 0x7faa34074c00 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.604+0000 7faa33200700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] conn(0x7faa180ae090 0x7faa180b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.604+0000 7faa31e00700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa180a2cb0 0x7faa180a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faa34079d50 tx=0x7faa20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.604+0000 7faa33200700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] conn(0x7faa180ae090 0x7faa180b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.605+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 <== osd.8 v2:172.21.15.136:6816/2605237371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa240020b0 con 0x7faa180ae090 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.625+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 --> [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa34075d70 con 0x7faa180ae090 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa2b400700 1 -- 172.21.15.26:0/1201837250 <== osd.8 v2:172.21.15.136:6816/2605237371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faa34075d70 con 0x7faa180ae090 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 >> [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] conn(0x7faa180ae090 msgr2=0x7faa180b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa39400700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] conn(0x7faa180ae090 0x7faa180b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa180a2cb0 msgr2=0x7faa180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:52.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa39400700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa180a2cb0 0x7faa180a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faa34079d50 tx=0x7faa20009260 comp rx=0 tx=0).stop 2024-06-13T06:57:52.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa34074c00 msgr2=0x7faa3419f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:52.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa39400700 1 --2- 172.21.15.26:0/1201837250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa34074c00 0x7faa3419f250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faa240062a0 tx=0x7faa24004ce0 comp rx=0 tx=0).stop 2024-06-13T06:57:52.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa33200700 1 -- 172.21.15.26:0/1201837250 reap_dead start 2024-06-13T06:57:52.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 shutdown_connections 2024-06-13T06:57:52.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.626+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 >> 172.21.15.26:0/1201837250 conn(0x7faa34079530 msgr2=0x7faa34111e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:52.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.627+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 shutdown_connections 2024-06-13T06:57:52.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:52.627+0000 7faa39400700 1 -- 172.21.15.26:0/1201837250 wait complete. 2024-06-13T06:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 -- 172.21.15.26:0/1661183712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1060102fd0 msgr2=0x7f1060107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 --2- 172.21.15.26:0/1661183712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1060102fd0 0x7f1060107020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f104c0099c0 tx=0x7f104c009cd0 comp rx=0 tx=0).stop 2024-06-13T06:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 -- 172.21.15.26:0/1661183712 shutdown_connections 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 --2- 172.21.15.26:0/1661183712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1060102620 0x7f1060102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 --2- 172.21.15.26:0/1661183712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1060107700 0x7f106010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 --2- 172.21.15.26:0/1661183712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1060102fd0 0x7f1060107020 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 -- 172.21.15.26:0/1661183712 >> 172.21.15.26:0/1661183712 conn(0x7f10600fdf10 msgr2=0x7f1060100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 -- 172.21.15.26:0/1661183712 shutdown_connections 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.832+0000 7f1067000700 1 -- 172.21.15.26:0/1661183712 wait complete. 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.833+0000 7f1067000700 1 Processor -- start 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.833+0000 7f1067000700 1 -- start start 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1067000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1060102620 0x7f106019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1067000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1060102fd0 0x7f106019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1067000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1060107700 0x7f106019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1067000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1060075110 con 0x7f1060102620 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1067000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1060074f90 con 0x7f1060102fd0 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1067000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1060075290 con 0x7f1060107700 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1065c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1060102620 0x7f106019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1065200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1060102fd0 0x7f106019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1065c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1060102620 0x7f106019afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60834/0 (socket says 172.21.15.26:60834) 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1066600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1060107700 0x7f106019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.834+0000 7f1065c00700 1 -- 172.21.15.26:0/3477297312 learned_addr learned my addr 172.21.15.26:0/3477297312 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f1066600700 1 -- 172.21.15.26:0/3477297312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1060102fd0 msgr2=0x7f106019b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f1066600700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1060102fd0 0x7f106019b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f1066600700 1 -- 172.21.15.26:0/3477297312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1060102620 msgr2=0x7f106019afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f1066600700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1060102620 0x7f106019afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f1066600700 1 -- 172.21.15.26:0/3477297312 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106019fe50 con 0x7f1060107700 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f1066600700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1060107700 0x7f106019f710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f105800e970 tx=0x7f105800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058004220 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1058004d10 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.835+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058010700 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.836+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10601a0140 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.836+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a79c0 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.837+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10601069e0 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.839+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1058004750 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.840+0000 7f105e000700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10480a2cb0 0x7f10480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.840+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f10580cf5a0 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.841+0000 7f105e000700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] conn(0x7f10480ae080 0x7f10480b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.841+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 --> [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10480b0bf0 con 0x7f10480ae080 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.841+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f10580cf950 con 0x7f1060107700 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.841+0000 7f1065c00700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10480a2cb0 0x7f10480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.841+0000 7f1065200700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] conn(0x7f10480ae080 0x7f10480b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.841+0000 7f1065c00700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10480a2cb0 0x7f10480a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1054009c10 tx=0x7f105400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.842+0000 7f1065200700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] conn(0x7f10480ae080 0x7f10480b0520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.843+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 <== osd.5 v2:172.21.15.136:6808/3459833212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10480b0bf0 con 0x7f10480ae080 2024-06-13T06:57:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.861+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 --> [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1060103b00 con 0x7f10480ae080 2024-06-13T06:57:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.862+0000 7f105e000700 1 -- 172.21.15.26:0/3477297312 <== osd.5 v2:172.21.15.136:6808/3459833212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1060103b00 con 0x7f10480ae080 2024-06-13T06:57:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.862+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 >> [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] conn(0x7f10480ae080 msgr2=0x7f10480b0520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.862+0000 7f1067000700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] conn(0x7f10480ae080 0x7f10480b0520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.862+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10480a2cb0 msgr2=0x7f10480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.862+0000 7f1067000700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f10480a2cb0 0x7f10480a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1054009c10 tx=0x7f105400b040 comp rx=0 tx=0).stop 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.863+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1060107700 msgr2=0x7f106019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.863+0000 7f1067000700 1 --2- 172.21.15.26:0/3477297312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1060107700 0x7f106019f710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f105800e970 tx=0x7f105800ec80 comp rx=0 tx=0).stop 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.863+0000 7f1066600700 1 -- 172.21.15.26:0/3477297312 reap_dead start 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.863+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 shutdown_connections 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.863+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 >> 172.21.15.26:0/3477297312 conn(0x7f10600fdf10 msgr2=0x7f1060106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.863+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 shutdown_connections 2024-06-13T06:57:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:50.863+0000 7f1067000700 1 -- 172.21.15.26:0/3477297312 wait complete. 2024-06-13T06:57:53.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:53 smithi026 ceph-mon[32041]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:53.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:53 smithi135 ceph-mon[36276]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:53.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:53 smithi136 ceph-mon[37229]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:57:54.270 INFO:teuthology.orchestra.run.smithi026.stdout:163208757336 2024-06-13T06:57:54.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-13T06:57:54.632 INFO:teuthology.orchestra.run.smithi026.stdout:77309411433 2024-06-13T06:57:54.632 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-13T06:57:55.733 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:55 smithi026 ceph-mon[32041]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:55.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:55 smithi135 ceph-mon[36276]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:55.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:55 smithi136 ceph-mon[37229]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:57:56.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.861+0000 7f4d90c00700 1 -- 172.21.15.26:0/3084092918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c113340 msgr2=0x7f4d8c115730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.861+0000 7f4d90c00700 1 --2- 172.21.15.26:0/3084092918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c113340 0x7f4d8c115730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400b0a0 tx=0x7f4d8400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.861+0000 7f4d90c00700 1 -- 172.21.15.26:0/3084092918 shutdown_connections 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.861+0000 7f4d90c00700 1 --2- 172.21.15.26:0/3084092918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c113340 0x7f4d8c115730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.861+0000 7f4d90c00700 1 --2- 172.21.15.26:0/3084092918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c0769c0 0x7f4d8c076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.861+0000 7f4d90c00700 1 --2- 172.21.15.26:0/3084092918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d8c0772e0 0x7f4d8c112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.861+0000 7f4d90c00700 1 -- 172.21.15.26:0/3084092918 >> 172.21.15.26:0/3084092918 conn(0x7f4d8c100170 msgr2=0x7f4d8c102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.862+0000 7f4d90c00700 1 -- 172.21.15.26:0/3084092918 shutdown_connections 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.862+0000 7f4d90c00700 1 -- 172.21.15.26:0/3084092918 wait complete. 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.863+0000 7f4d90c00700 1 Processor -- start 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.863+0000 7f4d90c00700 1 -- start start 2024-06-13T06:57:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.863+0000 7f4d90c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c0769c0 0x7f4d8c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.863+0000 7f4d90c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c0772e0 0x7f4d8c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.863+0000 7f4d90c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d8c113340 0x7f4d8c1a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.863+0000 7f4d90c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c118dc0 con 0x7f4d8c0772e0 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.863+0000 7f4d90c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c118c40 con 0x7f4d8c113340 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.863+0000 7f4d90c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c118f40 con 0x7f4d8c0769c0 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c0769c0 0x7f4d8c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c0772e0 0x7f4d8c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d8a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d8c113340 0x7f4d8c1a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c0769c0 0x7f4d8c19f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59886/0 (socket says 172.21.15.26:59886) 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 -- 172.21.15.26:0/872599242 learned_addr learned my addr 172.21.15.26:0/872599242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 -- 172.21.15.26:0/872599242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d8c113340 msgr2=0x7f4d8c1a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d8c113340 0x7f4d8c1a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 -- 172.21.15.26:0/872599242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c0772e0 msgr2=0x7f4d8c19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c0772e0 0x7f4d8c19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 -- 172.21.15.26:0/872599242 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c1a41e0 con 0x7f4d8c0769c0 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d8a800700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d8c113340 0x7f4d8c1a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89400700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c0772e0 0x7f4d8c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.864+0000 7f4d89e00700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c0769c0 0x7f4d8c19f1d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00e9a0 tx=0x7f4d7c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.865+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00ca10 con 0x7f4d8c0769c0 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.865+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c1a44d0 con 0x7f4d8c0769c0 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.865+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d7c00cbb0 con 0x7f4d8c0769c0 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.865+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c018680 con 0x7f4d8c0769c0 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.865+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8c1abd50 con 0x7f4d8c0769c0 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.866+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d8c062c20 con 0x7f4d8c0769c0 2024-06-13T06:57:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.868+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4d7c010430 con 0x7f4d8c0769c0 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.869+0000 7f4d82a00700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d700a2cb0 0x7f4d700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.870+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4d7c0cffe0 con 0x7f4d8c0769c0 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.870+0000 7f4d89400700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d700a2cb0 0x7f4d700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.870+0000 7f4d82a00700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] conn(0x7f4d700ae090 0x7f4d700b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.870+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 --> [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d7c01bec0 con 0x7f4d700ae090 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.870+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f4d7c0d0430 con 0x7f4d8c0769c0 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.870+0000 7f4d8a800700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] conn(0x7f4d700ae090 0x7f4d700b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.870+0000 7f4d89400700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d700a2cb0 0x7f4d700a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c103be0 tx=0x7f4d780073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.871+0000 7f4d8a800700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] conn(0x7f4d700ae090 0x7f4d700b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.872+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 <== osd.7 v2:172.21.15.135:6816/2969819310 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d7c01bec0 con 0x7f4d700ae090 2024-06-13T06:57:56.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.891+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 --> [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d8c063020 con 0x7f4d700ae090 2024-06-13T06:57:56.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.891+0000 7f4d82a00700 1 -- 172.21.15.26:0/872599242 <== osd.7 v2:172.21.15.135:6816/2969819310 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4d8c063020 con 0x7f4d700ae090 2024-06-13T06:57:56.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 >> [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] conn(0x7f4d700ae090 msgr2=0x7f4d700b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d90c00700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] conn(0x7f4d700ae090 0x7f4d700b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d700a2cb0 msgr2=0x7f4d700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d90c00700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4d700a2cb0 0x7f4d700a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c103be0 tx=0x7f4d780073d0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c0769c0 msgr2=0x7f4d8c19f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d90c00700 1 --2- 172.21.15.26:0/872599242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c0769c0 0x7f4d8c19f1d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00e9a0 tx=0x7f4d7c00ecb0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d8a800700 1 -- 172.21.15.26:0/872599242 reap_dead start 2024-06-13T06:57:56.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 shutdown_connections 2024-06-13T06:57:56.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.892+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 >> 172.21.15.26:0/872599242 conn(0x7f4d8c100170 msgr2=0x7f4d8c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:56.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.893+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 shutdown_connections 2024-06-13T06:57:56.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:53.893+0000 7f4d90c00700 1 -- 172.21.15.26:0/872599242 wait complete. 2024-06-13T06:57:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.106+0000 7f92f6000700 1 -- 172.21.15.26:0/2887638260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f006ab60 msgr2=0x7f92f006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.106+0000 7f92f6000700 1 --2- 172.21.15.26:0/2887638260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f006ab60 0x7f92f006afe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009990 tx=0x7f92dc009ca0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.107+0000 7f92f6000700 1 -- 172.21.15.26:0/2887638260 shutdown_connections 2024-06-13T06:57:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.107+0000 7f92f6000700 1 --2- 172.21.15.26:0/2887638260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92f006b520 0x7f92f010f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.107+0000 7f92f6000700 1 --2- 172.21.15.26:0/2887638260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92f0108520 0x7f92f0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.107+0000 7f92f6000700 1 --2- 172.21.15.26:0/2887638260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f006ab60 0x7f92f006afe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.107+0000 7f92f6000700 1 -- 172.21.15.26:0/2887638260 >> 172.21.15.26:0/2887638260 conn(0x7f92f0078340 msgr2=0x7f92f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.107+0000 7f92f6000700 1 -- 172.21.15.26:0/2887638260 shutdown_connections 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.107+0000 7f92f6000700 1 -- 172.21.15.26:0/2887638260 wait complete. 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.108+0000 7f92f6000700 1 Processor -- start 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.108+0000 7f92f6000700 1 -- start start 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.108+0000 7f92f6000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92f006ab60 0x7f92f0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.108+0000 7f92f6000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f006b520 0x7f92f0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.108+0000 7f92f6000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92f0108520 0x7f92f01022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.108+0000 7f92f6000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0112780 con 0x7f92f006b520 2024-06-13T06:57:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.108+0000 7f92f6000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0112600 con 0x7f92f0108520 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.108+0000 7f92f6000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0112900 con 0x7f92f006ab60 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92f0108520 0x7f92f01022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92eea00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f006b520 0x7f92f0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92f0108520 0x7f92f01022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51754/0 (socket says 172.21.15.26:51754) 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 -- 172.21.15.26:0/684605956 learned_addr learned my addr 172.21.15.26:0/684605956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92ef400700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92f006ab60 0x7f92f0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 -- 172.21.15.26:0/684605956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92f006ab60 msgr2=0x7f92f0101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92f006ab60 0x7f92f0101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 -- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f006b520 msgr2=0x7f92f0101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92f006b520 0x7f92f0101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 -- 172.21.15.26:0/684605956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f01a71e0 con 0x7f92f0108520 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.109+0000 7f92efe00700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92f0108520 0x7f92f01022c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92e800ef30 tx=0x7f92e800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.110+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80092e0 con 0x7f92f0108520 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.110+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f01a7470 con 0x7f92f0108520 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.110+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e8004bb0 con 0x7f92f0108520 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.110+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8004280 con 0x7f92f0108520 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.110+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a7980 con 0x7f92f0108520 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.111+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f92f0062ba0 con 0x7f92f0108520 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.113+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f92e801b030 con 0x7f92f0108520 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.115+0000 7f92e7e00700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f92d40a2d30 0x7f92d40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.115+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f92e80cfa00 con 0x7f92f0108520 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.115+0000 7f92ef400700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f92d40a2d30 0x7f92d40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.115+0000 7f92e7e00700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] conn(0x7f92d40ae110 0x7f92d40b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.115+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 --> [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92e801fe50 con 0x7f92d40ae110 2024-06-13T06:57:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.115+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f92e80cfdb0 con 0x7f92f0108520 2024-06-13T06:57:56.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.115+0000 7f92eea00700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] conn(0x7f92d40ae110 0x7f92d40b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.116+0000 7f92ef400700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f92d40a2d30 0x7f92d40a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009be0 tx=0x7f92e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.116+0000 7f92eea00700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] conn(0x7f92d40ae110 0x7f92d40b05b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.117+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 <== osd.0 v2:172.21.15.26:6802/4252497348 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92e801fe50 con 0x7f92d40ae110 2024-06-13T06:57:56.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.136+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 --> [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92f0062fa0 con 0x7f92d40ae110 2024-06-13T06:57:56.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.136+0000 7f92e7e00700 1 -- 172.21.15.26:0/684605956 <== osd.0 v2:172.21.15.26:6802/4252497348 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f92f0062fa0 con 0x7f92d40ae110 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.137+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] conn(0x7f92d40ae110 msgr2=0x7f92d40b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.137+0000 7f92f6000700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] conn(0x7f92d40ae110 0x7f92d40b05b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.137+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f92d40a2d30 msgr2=0x7f92d40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.137+0000 7f92f6000700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f92d40a2d30 0x7f92d40a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009be0 tx=0x7f92e0009260 comp rx=0 tx=0).stop 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.137+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92f0108520 msgr2=0x7f92f01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.137+0000 7f92f6000700 1 --2- 172.21.15.26:0/684605956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92f0108520 0x7f92f01022c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92e800ef30 tx=0x7f92e800c3a0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.137+0000 7f92efe00700 1 -- 172.21.15.26:0/684605956 reap_dead start 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.138+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 shutdown_connections 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.138+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 >> 172.21.15.26:0/684605956 conn(0x7f92f0078340 msgr2=0x7f92f01004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.138+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 shutdown_connections 2024-06-13T06:57:56.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.138+0000 7f92f6000700 1 -- 172.21.15.26:0/684605956 wait complete. 2024-06-13T06:57:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 -- 172.21.15.26:0/3469126196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b2c105090 msgr2=0x7f9b2c1115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 --2- 172.21.15.26:0/3469126196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b2c105090 0x7f9b2c1115a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400b0a0 tx=0x7f9b2400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 -- 172.21.15.26:0/3469126196 shutdown_connections 2024-06-13T06:57:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 --2- 172.21.15.26:0/3469126196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b2c10a6d0 0x7f9b2c10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 --2- 172.21.15.26:0/3469126196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b2c105090 0x7f9b2c1115a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 --2- 172.21.15.26:0/3469126196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b2c1046d0 0x7f9b2c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 -- 172.21.15.26:0/3469126196 >> 172.21.15.26:0/3469126196 conn(0x7f9b2c1001c0 msgr2=0x7f9b2c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 -- 172.21.15.26:0/3469126196 shutdown_connections 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.891+0000 7f9b32000700 1 -- 172.21.15.26:0/3469126196 wait complete. 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.893+0000 7f9b32000700 1 Processor -- start 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.893+0000 7f9b32000700 1 -- start start 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b32000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b2c1046d0 0x7f9b2c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b32000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b2c105090 0x7f9b2c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b32000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b2c10a6d0 0x7f9b2c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b32000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c114ab0 con 0x7f9b2c10a6d0 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b32000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c114930 con 0x7f9b2c1046d0 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b32000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c114c30 con 0x7f9b2c105090 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b2aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b2c105090 0x7f9b2c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b2aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b2c105090 0x7f9b2c19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59918/0 (socket says 172.21.15.26:59918) 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b2aa00700 1 -- 172.21.15.26:0/3334181715 learned_addr learned my addr 172.21.15.26:0/3334181715 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b2b400700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b2c1046d0 0x7f9b2c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.894+0000 7f9b2be00700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b2c10a6d0 0x7f9b2c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.895+0000 7f9b2b400700 1 -- 172.21.15.26:0/3334181715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b2c105090 msgr2=0x7f9b2c19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.895+0000 7f9b2b400700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b2c105090 0x7f9b2c19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.895+0000 7f9b2b400700 1 -- 172.21.15.26:0/3334181715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b2c10a6d0 msgr2=0x7f9b2c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.895+0000 7f9b2b400700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b2c10a6d0 0x7f9b2c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.895+0000 7f9b2b400700 1 -- 172.21.15.26:0/3334181715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c19ff10 con 0x7f9b2c1046d0 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.895+0000 7f9b2b400700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b2c1046d0 0x7f9b2c19af00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00e9f0 tx=0x7f9b1c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.895+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00c9c0 con 0x7f9b2c1046d0 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.896+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1c00cb60 con 0x7f9b2c1046d0 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.896+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c1a0200 con 0x7f9b2c1046d0 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.896+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c018680 con 0x7f9b2c1046d0 2024-06-13T06:57:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.896+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c1a7a80 con 0x7f9b2c1046d0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.897+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b2c062ba0 con 0x7f9b2c1046d0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.899+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9b1c010430 con 0x7f9b2c1046d0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.900+0000 7f9b23e00700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9b100a2cb0 0x7f9b100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.900+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9b1c0cfb20 con 0x7f9b2c1046d0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.901+0000 7f9b2aa00700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9b100a2cb0 0x7f9b100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.901+0000 7f9b23e00700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] conn(0x7f9b100ae090 0x7f9b100b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.901+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 --> [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b100b0c00 con 0x7f9b100ae090 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.901+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f9b1c0cfed0 con 0x7f9b2c1046d0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.901+0000 7f9b2be00700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] conn(0x7f9b100ae090 0x7f9b100b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.902+0000 7f9b2aa00700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9b100a2cb0 0x7f9b100a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c109c60 tx=0x7f9b18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.902+0000 7f9b2be00700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] conn(0x7f9b100ae090 0x7f9b100b0530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.903+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 <== osd.4 v2:172.21.15.135:6808/2020131902 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b100b0c00 con 0x7f9b100ae090 2024-06-13T06:57:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.921+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 --> [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b2c062fa0 con 0x7f9b100ae090 2024-06-13T06:57:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.922+0000 7f9b23e00700 1 -- 172.21.15.26:0/3334181715 <== osd.4 v2:172.21.15.135:6808/2020131902 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b2c062fa0 con 0x7f9b100ae090 2024-06-13T06:57:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.922+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] conn(0x7f9b100ae090 msgr2=0x7f9b100b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.922+0000 7f9b32000700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] conn(0x7f9b100ae090 0x7f9b100b0530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9b100a2cb0 msgr2=0x7f9b100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b32000700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9b100a2cb0 0x7f9b100a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c109c60 tx=0x7f9b18009260 comp rx=0 tx=0).stop 2024-06-13T06:57:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b2c1046d0 msgr2=0x7f9b2c19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b32000700 1 --2- 172.21.15.26:0/3334181715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b2c1046d0 0x7f9b2c19af00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00e9f0 tx=0x7f9b1c00ed00 comp rx=0 tx=0).stop 2024-06-13T06:57:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b2be00700 1 -- 172.21.15.26:0/3334181715 reap_dead start 2024-06-13T06:57:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 shutdown_connections 2024-06-13T06:57:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 >> 172.21.15.26:0/3334181715 conn(0x7f9b2c1001c0 msgr2=0x7f9b2c1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 shutdown_connections 2024-06-13T06:57:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:56.923+0000 7f9b32000700 1 -- 172.21.15.26:0/3334181715 wait complete. 2024-06-13T06:57:57.659 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:57 smithi026 ceph-mon[32041]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:57.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:57 smithi135 ceph-mon[36276]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:57 smithi136 ceph-mon[37229]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.002+0000 7f60ba200700 1 -- 172.21.15.26:0/1507232702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b410a660 msgr2=0x7f60b410aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.002+0000 7f60ba200700 1 --2- 172.21.15.26:0/1507232702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b410a660 0x7f60b410aa40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009990 tx=0x7f60a4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.003+0000 7f60ba200700 1 -- 172.21.15.26:0/1507232702 shutdown_connections 2024-06-13T06:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.003+0000 7f60ba200700 1 --2- 172.21.15.26:0/1507232702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b4105020 0x7f60b4111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.003+0000 7f60ba200700 1 --2- 172.21.15.26:0/1507232702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b410a660 0x7f60b410aa40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.003+0000 7f60ba200700 1 --2- 172.21.15.26:0/1507232702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b4104660 0x7f60b4104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.003+0000 7f60ba200700 1 -- 172.21.15.26:0/1507232702 >> 172.21.15.26:0/1507232702 conn(0x7f60b4100190 msgr2=0x7f60b41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.003+0000 7f60ba200700 1 -- 172.21.15.26:0/1507232702 shutdown_connections 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.003+0000 7f60ba200700 1 -- 172.21.15.26:0/1507232702 wait complete. 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.004+0000 7f60ba200700 1 Processor -- start 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60ba200700 1 -- start start 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60ba200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b4104660 0x7f60b419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60ba200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b4105020 0x7f60b419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60ba200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b410a660 0x7f60b419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60ba200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4114990 con 0x7f60b4104660 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60ba200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4114810 con 0x7f60b410a660 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60ba200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4114b10 con 0x7f60b4105020 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60b3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b4104660 0x7f60b419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60b3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b4104660 0x7f60b419aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50882/0 (socket says 172.21.15.26:50882) 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60b3400700 1 -- 172.21.15.26:0/1898308060 learned_addr learned my addr 172.21.15.26:0/1898308060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60b2a00700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b4105020 0x7f60b419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.005+0000 7f60b3e00700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b410a660 0x7f60b419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60b3400700 1 -- 172.21.15.26:0/1898308060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b4105020 msgr2=0x7f60b419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60b3400700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b4105020 0x7f60b419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60b3400700 1 -- 172.21.15.26:0/1898308060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b410a660 msgr2=0x7f60b419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60b3400700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b410a660 0x7f60b419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60b3400700 1 -- 172.21.15.26:0/1898308060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b419fe80 con 0x7f60b4104660 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60b3400700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b4104660 0x7f60b419aec0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60a400bae0 tx=0x7f60a400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4026070 con 0x7f60b4104660 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a4003d70 con 0x7f60b4104660 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b41a0110 con 0x7f60b4104660 2024-06-13T06:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.006+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a400fe50 con 0x7f60b4104660 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.007+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b41a79f0 con 0x7f60b4104660 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.007+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60b4062c20 con 0x7f60b4104660 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.010+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f60a4029070 con 0x7f60b4104660 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.012+0000 7f60abe00700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60980a2ed0 0x7f60980a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.012+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f60a40d0a10 con 0x7f60b4104660 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.012+0000 7f60b2a00700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60980a2ed0 0x7f60980a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.012+0000 7f60abe00700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] conn(0x7f60980ae180 0x7f60980b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.012+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 --> [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60a400fe50 con 0x7f60980ae180 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.012+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f60a40d0dc0 con 0x7f60b4104660 2024-06-13T06:57:58.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.012+0000 7f60b3e00700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] conn(0x7f60980ae180 0x7f60980b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.013+0000 7f60b2a00700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60980a2ed0 0x7f60980a5390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009680 tx=0x7f60a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.013+0000 7f60b3e00700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] conn(0x7f60980ae180 0x7f60980b0620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.014+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 <== osd.3 v2:172.21.15.26:6810/2922553278 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60a400fe50 con 0x7f60980ae180 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.033+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 --> [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60b4063020 con 0x7f60980ae180 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.033+0000 7f60abe00700 1 -- 172.21.15.26:0/1898308060 <== osd.3 v2:172.21.15.26:6810/2922553278 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f60b4063020 con 0x7f60980ae180 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] conn(0x7f60980ae180 msgr2=0x7f60980b0620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] conn(0x7f60980ae180 0x7f60980b0620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60980a2ed0 msgr2=0x7f60980a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60980a2ed0 0x7f60980a5390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009680 tx=0x7f60a0009260 comp rx=0 tx=0).stop 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b4104660 msgr2=0x7f60b419aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 --2- 172.21.15.26:0/1898308060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60b4104660 0x7f60b419aec0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60a400bae0 tx=0x7f60a400bb10 comp rx=0 tx=0).stop 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60b3e00700 1 -- 172.21.15.26:0/1898308060 reap_dead start 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 shutdown_connections 2024-06-13T06:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 >> 172.21.15.26:0/1898308060 conn(0x7f60b4100190 msgr2=0x7f60b4100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 shutdown_connections 2024-06-13T06:57:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:57.034+0000 7f60ba200700 1 -- 172.21.15.26:0/1898308060 wait complete. 2024-06-13T06:57:58.553 INFO:teuthology.orchestra.run.smithi026.stdout:141733920863 2024-06-13T06:57:58.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-13T06:57:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 -- 172.21.15.26:0/2356481751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 msgr2=0x7f6d9406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 --2- 172.21.15.26:0/2356481751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 0x7f6d9406a5d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6d800098f0 tx=0x7f6d80009c00 comp rx=0 tx=0).stop 2024-06-13T06:57:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 -- 172.21.15.26:0/2356481751 shutdown_connections 2024-06-13T06:57:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 --2- 172.21.15.26:0/2356481751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d9406ab10 0x7f6d9410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 --2- 172.21.15.26:0/2356481751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d941084e0 0x7f6d941088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 --2- 172.21.15.26:0/2356481751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 0x7f6d9406a5d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 -- 172.21.15.26:0/2356481751 >> 172.21.15.26:0/2356481751 conn(0x7f6d940782c0 msgr2=0x7f6d940786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 -- 172.21.15.26:0/2356481751 shutdown_connections 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.119+0000 7f6d99600700 1 -- 172.21.15.26:0/2356481751 wait complete. 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.120+0000 7f6d99600700 1 Processor -- start 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d99600700 1 -- start start 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d99600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 0x7f6d94101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d99600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9406ab10 0x7f6d941019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d99600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d941084e0 0x7f6d94101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d99600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d94112740 con 0x7f6d9406ab10 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d99600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941125c0 con 0x7f6d9406a170 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d99600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941128c0 con 0x7f6d941084e0 2024-06-13T06:57:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d92000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9406ab10 0x7f6d941019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.121+0000 7f6d92a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 0x7f6d94101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9406ab10 0x7f6d941019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50896/0 (socket says 172.21.15.26:50896) 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 0x7f6d94101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51814/0 (socket says 172.21.15.26:51814) 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92000700 1 -- 172.21.15.26:0/1075787686 learned_addr learned my addr 172.21.15.26:0/1075787686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d93400700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d941084e0 0x7f6d94101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92a00700 1 -- 172.21.15.26:0/1075787686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d941084e0 msgr2=0x7f6d94101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92a00700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d941084e0 0x7f6d94101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92a00700 1 -- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9406ab10 msgr2=0x7f6d941019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92a00700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d9406ab10 0x7f6d941019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92a00700 1 -- 172.21.15.26:0/1075787686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d941a7140 con 0x7f6d9406a170 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.122+0000 7f6d92a00700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 0x7f6d94101460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400e9c0 tx=0x7f6d8400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.123+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400c9c0 con 0x7f6d9406a170 2024-06-13T06:57:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.123+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d8400cb60 con 0x7f6d9406a170 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.123+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84010e60 con 0x7f6d9406a170 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.123+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d941a7430 con 0x7f6d9406a170 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.123+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d941a7990 con 0x7f6d9406a170 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.124+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d94062ba0 con 0x7f6d9406a170 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.126+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6d8400cd10 con 0x7f6d9406a170 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.127+0000 7f6d8b400700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d780a2d30 0x7f6d780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.128+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6d840cfd70 con 0x7f6d9406a170 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.128+0000 7f6d8b400700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] conn(0x7f6d780ae110 0x7f6d780b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.128+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 --> [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d8401bec0 con 0x7f6d780ae110 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.128+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f6d840d0120 con 0x7f6d9406a170 2024-06-13T06:57:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.128+0000 7f6d92000700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d780a2d30 0x7f6d780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.129+0000 7f6d93400700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] conn(0x7f6d780ae110 0x7f6d780b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.130+0000 7f6d92000700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d780a2d30 0x7f6d780a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d94102ef0 tx=0x7f6d8001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.130+0000 7f6d93400700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] conn(0x7f6d780ae110 0x7f6d780b05b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.131+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 <== osd.9 v2:172.21.15.26:6826/1456557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d8401bec0 con 0x7f6d780ae110 2024-06-13T06:57:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.150+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 --> [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d94062fa0 con 0x7f6d780ae110 2024-06-13T06:57:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.150+0000 7f6d8b400700 1 -- 172.21.15.26:0/1075787686 <== osd.9 v2:172.21.15.26:6826/1456557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d94062fa0 con 0x7f6d780ae110 2024-06-13T06:57:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.151+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] conn(0x7f6d780ae110 msgr2=0x7f6d780b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.151+0000 7f6d99600700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] conn(0x7f6d780ae110 0x7f6d780b05b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.151+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d780a2d30 msgr2=0x7f6d780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.151+0000 7f6d99600700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d780a2d30 0x7f6d780a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d94102ef0 tx=0x7f6d8001a040 comp rx=0 tx=0).stop 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.151+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 msgr2=0x7f6d94101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.151+0000 7f6d99600700 1 --2- 172.21.15.26:0/1075787686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d9406a170 0x7f6d94101460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400e9c0 tx=0x7f6d8400ecd0 comp rx=0 tx=0).stop 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.152+0000 7f6d93400700 1 -- 172.21.15.26:0/1075787686 reap_dead start 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.152+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 shutdown_connections 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.152+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 >> 172.21.15.26:0/1075787686 conn(0x7f6d940782c0 msgr2=0x7f6d940ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.152+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 shutdown_connections 2024-06-13T06:57:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:57:58.152+0000 7f6d99600700 1 -- 172.21.15.26:0/1075787686 wait complete. 2024-06-13T06:57:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:59.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:57:59 smithi026 ceph-mon[32041]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:57:59.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:57:59 smithi135 ceph-mon[36276]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:57:59.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:57:59 smithi136 ceph-mon[37229]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:00.298 INFO:teuthology.orchestra.run.smithi026.stdout:34359738484 2024-06-13T06:58:00.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-13T06:58:00.415 INFO:teuthology.orchestra.run.smithi026.stdout:287762808897 2024-06-13T06:58:00.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-13T06:58:00.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:00 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:58:00.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:00 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:58:00.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:00 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:58:00.879 INFO:teuthology.orchestra.run.smithi026.stdout:107374182499 2024-06-13T06:58:00.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-13T06:58:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 -- 172.21.15.26:0/2876416713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2a810a6d0 msgr2=0x7ff2a810aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 --2- 172.21.15.26:0/2876416713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2a810a6d0 0x7ff2a810aab0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff298009990 tx=0x7ff298009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 -- 172.21.15.26:0/2876416713 shutdown_connections 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 --2- 172.21.15.26:0/2876416713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2a810a6d0 0x7ff2a810aab0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 --2- 172.21.15.26:0/2876416713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2a8105090 0x7ff2a81115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 --2- 172.21.15.26:0/2876416713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a81046d0 0x7ff2a8104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 -- 172.21.15.26:0/2876416713 >> 172.21.15.26:0/2876416713 conn(0x7ff2a81001c0 msgr2=0x7ff2a81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 -- 172.21.15.26:0/2876416713 shutdown_connections 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.473+0000 7ff2ad000700 1 -- 172.21.15.26:0/2876416713 wait complete. 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.474+0000 7ff2ad000700 1 Processor -- start 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.474+0000 7ff2ad000700 1 -- start start 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2ad000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a81046d0 0x7ff2a819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2ad000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2a8105090 0x7ff2a819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2ad000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2a810a6d0 0x7ff2a819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2ad000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8114a60 con 0x7ff2a81046d0 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2ad000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a81148e0 con 0x7ff2a810a6d0 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2ad000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8114be0 con 0x7ff2a8105090 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2a8105090 0x7ff2a819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2a8105090 0x7ff2a819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59992/0 (socket says 172.21.15.26:59992) 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5400700 1 -- 172.21.15.26:0/1941982704 learned_addr learned my addr 172.21.15.26:0/1941982704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5e00700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a81046d0 0x7ff2a819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a6800700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2a810a6d0 0x7ff2a819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5400700 1 -- 172.21.15.26:0/1941982704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2a810a6d0 msgr2=0x7ff2a819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5400700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2a810a6d0 0x7ff2a819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5400700 1 -- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a81046d0 msgr2=0x7ff2a819aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5400700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a81046d0 0x7ff2a819aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.475+0000 7ff2a5400700 1 -- 172.21.15.26:0/1941982704 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a819fe90 con 0x7ff2a8105090 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.476+0000 7ff2a5e00700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a81046d0 0x7ff2a819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.476+0000 7ff2a5400700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2a8105090 0x7ff2a819b410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff29400d980 tx=0x7ff29400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.476+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294016070 con 0x7ff2a8105090 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.476+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff294004500 con 0x7ff2a8105090 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.476+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a81a0180 con 0x7ff2a8105090 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.476+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294004e20 con 0x7ff2a8105090 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.476+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a81a7a00 con 0x7ff2a8105090 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.477+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff2a8062c20 con 0x7ff2a8105090 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.480+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff294004020 con 0x7ff2a8105090 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.483+0000 7ff29ea00700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff28c0a2d30 0x7ff28c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.483+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff2940cfb10 con 0x7ff2a8105090 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.483+0000 7ff2a5e00700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff28c0a2d30 0x7ff28c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.483+0000 7ff29ea00700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] conn(0x7ff28c0ae110 0x7ff28c0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.483+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 --> [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff294012e70 con 0x7ff28c0ae110 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.483+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7ff2940cfec0 con 0x7ff2a8105090 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.484+0000 7ff2a6800700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] conn(0x7ff28c0ae110 0x7ff28c0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.484+0000 7ff2a5e00700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff28c0a2d30 0x7ff28c0a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff29800b2e0 tx=0x7ff29801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.485+0000 7ff2a6800700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] conn(0x7ff28c0ae110 0x7ff28c0b05b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.486+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 <== osd.6 v2:172.21.15.26:6818/2705956031 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff294012e70 con 0x7ff28c0ae110 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.504+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 --> [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff2a8063020 con 0x7ff28c0ae110 2024-06-13T06:58:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.505+0000 7ff29ea00700 1 -- 172.21.15.26:0/1941982704 <== osd.6 v2:172.21.15.26:6818/2705956031 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff2a8063020 con 0x7ff28c0ae110 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.505+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] conn(0x7ff28c0ae110 msgr2=0x7ff28c0b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.505+0000 7ff2ad000700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] conn(0x7ff28c0ae110 0x7ff28c0b05b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.505+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff28c0a2d30 msgr2=0x7ff28c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.505+0000 7ff2ad000700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff28c0a2d30 0x7ff28c0a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff29800b2e0 tx=0x7ff29801a040 comp rx=0 tx=0).stop 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.505+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2a8105090 msgr2=0x7ff2a819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.505+0000 7ff2ad000700 1 --2- 172.21.15.26:0/1941982704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2a8105090 0x7ff2a819b410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff29400d980 tx=0x7ff29400dd40 comp rx=0 tx=0).stop 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.505+0000 7ff2a6800700 1 -- 172.21.15.26:0/1941982704 reap_dead start 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.506+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 shutdown_connections 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.506+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 >> 172.21.15.26:0/1941982704 conn(0x7ff2a81001c0 msgr2=0x7ff2a8100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.506+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 shutdown_connections 2024-06-13T06:58:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:01.506+0000 7ff2ad000700 1 -- 172.21.15.26:0/1941982704 wait complete. 2024-06-13T06:58:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:01 smithi026 ceph-mon[32041]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-13T06:58:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:58:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:58:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:01 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:01 smithi135 ceph-mon[36276]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-13T06:58:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:58:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:58:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:01 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:01 smithi136 ceph-mon[37229]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-13T06:58:02.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:58:02.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:58:02.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:01 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:58:02.872 INFO:teuthology.orchestra.run.smithi026.stdout:219043332172 2024-06-13T06:58:02.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-13T06:58:03.713 INFO:teuthology.orchestra.run.smithi026.stdout:253403070534 2024-06-13T06:58:03.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-13T06:58:04.244 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:03 smithi026 ceph-mon[32041]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-13T06:58:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:03 smithi135 ceph-mon[36276]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-13T06:58:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:03 smithi136 ceph-mon[37229]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-13T06:58:04.492 INFO:teuthology.orchestra.run.smithi026.stdout:197568495698 2024-06-13T06:58:04.493 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-13T06:58:04.545 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:04.565 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:06.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:05 smithi026 ceph-mon[32041]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:05 smithi135 ceph-mon[36276]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:06.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 -- 172.21.15.26:0/2541322143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68107570 msgr2=0x7fdb68107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:06.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 --2- 172.21.15.26:0/2541322143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68107570 0x7fdb68107950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0098f0 tx=0x7fdb5c009c00 comp rx=0 tx=0).stop 2024-06-13T06:58:06.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 -- 172.21.15.26:0/2541322143 shutdown_connections 2024-06-13T06:58:06.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 --2- 172.21.15.26:0/2541322143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb68101810 0x7fdb6810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 --2- 172.21.15.26:0/2541322143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68107570 0x7fdb68107950 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 --2- 172.21.15.26:0/2541322143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb68100e50 0x7fdb681012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 -- 172.21.15.26:0/2541322143 >> 172.21.15.26:0/2541322143 conn(0x7fdb680766b0 msgr2=0x7fdb68076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 -- 172.21.15.26:0/2541322143 shutdown_connections 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.190+0000 7fdb6f600700 1 -- 172.21.15.26:0/2541322143 wait complete. 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6f600700 1 Processor -- start 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6f600700 1 -- start start 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6f600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68100e50 0x7fdb6819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6f600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb68101810 0x7fdb6819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6f600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb68107570 0x7fdb6819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6f600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68112800 con 0x7fdb68100e50 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6f600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68112680 con 0x7fdb68101810 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6f600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68112980 con 0x7fdb68107570 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.192+0000 7fdb6cc00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68100e50 0x7fdb6819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6cc00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68100e50 0x7fdb6819af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44742/0 (socket says 172.21.15.26:44742) 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6cc00700 1 -- 172.21.15.26:0/277573810 learned_addr learned my addr 172.21.15.26:0/277573810 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6d600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb68107570 0x7fdb6819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb67e00700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb68101810 0x7fdb6819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6cc00700 1 -- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb68107570 msgr2=0x7fdb6819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6cc00700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb68107570 0x7fdb6819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6cc00700 1 -- 172.21.15.26:0/277573810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb68101810 msgr2=0x7fdb6819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6cc00700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb68101810 0x7fdb6819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6cc00700 1 -- 172.21.15.26:0/277573810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6819ff20 con 0x7fdb68100e50 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6d600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb68107570 0x7fdb6819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.193+0000 7fdb6cc00700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68100e50 0x7fdb6819af10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0094f0 tx=0x7fdb5c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.194+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c027070 con 0x7fdb68100e50 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.194+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5c0055a0 con 0x7fdb68100e50 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.194+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb681a01b0 con 0x7fdb68100e50 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.194+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c004930 con 0x7fdb68100e50 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.194+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb681a7a90 con 0x7fdb68100e50 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.195+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb681001a0 con 0x7fdb68100e50 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.197+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdb5c02a070 con 0x7fdb68100e50 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.198+0000 7fdb65600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb500a2cb0 0x7fdb500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.198+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdb5c0d0680 con 0x7fdb68100e50 2024-06-13T06:58:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.199+0000 7fdb67e00700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb500a2cb0 0x7fdb500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.199+0000 7fdb65600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] conn(0x7fdb500ae090 0x7fdb500b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.199+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 --> [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb5c018e80 con 0x7fdb500ae090 2024-06-13T06:58:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.199+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fdb5c0d0a30 con 0x7fdb68100e50 2024-06-13T06:58:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.199+0000 7fdb6d600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] conn(0x7fdb500ae090 0x7fdb500b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.199+0000 7fdb67e00700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb500a2cb0 0x7fdb500a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009750 tx=0x7fdb54009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.200+0000 7fdb6d600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] conn(0x7fdb500ae090 0x7fdb500b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.201+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 <== osd.11 v2:172.21.15.136:6824/2902066544 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb5c018e80 con 0x7fdb500ae090 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.220+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 --> [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb6806a680 con 0x7fdb500ae090 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.221+0000 7fdb65600700 1 -- 172.21.15.26:0/277573810 <== osd.11 v2:172.21.15.136:6824/2902066544 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb6806a680 con 0x7fdb500ae090 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.221+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] conn(0x7fdb500ae090 msgr2=0x7fdb500b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.221+0000 7fdb6f600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] conn(0x7fdb500ae090 0x7fdb500b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb500a2cb0 msgr2=0x7fdb500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6f600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdb500a2cb0 0x7fdb500a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009750 tx=0x7fdb54009310 comp rx=0 tx=0).stop 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68100e50 msgr2=0x7fdb6819af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6f600700 1 --2- 172.21.15.26:0/277573810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68100e50 0x7fdb6819af10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0094f0 tx=0x7fdb5c00fa00 comp rx=0 tx=0).stop 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6d600700 1 -- 172.21.15.26:0/277573810 reap_dead start 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 shutdown_connections 2024-06-13T06:58:06.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 >> 172.21.15.26:0/277573810 conn(0x7fdb680766b0 msgr2=0x7fdb680ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:06.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 shutdown_connections 2024-06-13T06:58:06.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:06.222+0000 7fdb6f600700 1 -- 172.21.15.26:0/277573810 wait complete. 2024-06-13T06:58:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:05 smithi136 ceph-mon[37229]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:07.323 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:07.341 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:07.341 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:07 smithi026 ceph-mon[32041]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:08.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:07 smithi135 ceph-mon[36276]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:07 smithi136 ceph-mon[37229]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:10.117 INFO:teuthology.orchestra.run.smithi026.stdout:343597383736 2024-06-13T06:58:10.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-13T06:58:10.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:09 smithi026 ceph-mon[32041]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:10.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:09 smithi135 ceph-mon[36276]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:09 smithi136 ceph-mon[37229]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:10.414 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:11 smithi026 ceph-mon[32041]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:12.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:11 smithi135 ceph-mon[36276]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:11 smithi136 ceph-mon[37229]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:14.193 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:13 smithi136 ceph-mon[37229]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:14.222 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:13 smithi026 ceph-mon[32041]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:14.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.274+0000 7f6cad600700 1 -- 172.21.15.26:0/1916005062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca8102ee0 msgr2=0x7f6ca810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.274+0000 7f6cad600700 1 --2- 172.21.15.26:0/1916005062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca8102ee0 0x7f6ca810f3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000b0a0 tx=0x7f6ca000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.275+0000 7f6cad600700 1 -- 172.21.15.26:0/1916005062 shutdown_connections 2024-06-13T06:58:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.275+0000 7f6cad600700 1 --2- 172.21.15.26:0/1916005062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca8102ee0 0x7f6ca810f3c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.275+0000 7f6cad600700 1 --2- 172.21.15.26:0/1916005062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ca8107450 0x7f6ca8107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.275+0000 7f6cad600700 1 --2- 172.21.15.26:0/1916005062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ca8102540 0x7f6ca81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.275+0000 7f6cad600700 1 -- 172.21.15.26:0/1916005062 >> 172.21.15.26:0/1916005062 conn(0x7f6ca80795f0 msgr2=0x7f6ca8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.275+0000 7f6cad600700 1 -- 172.21.15.26:0/1916005062 shutdown_connections 2024-06-13T06:58:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.275+0000 7f6cad600700 1 -- 172.21.15.26:0/1916005062 wait complete. 2024-06-13T06:58:14.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.276+0000 7f6cad600700 1 Processor -- start 2024-06-13T06:58:14.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.276+0000 7f6cad600700 1 -- start start 2024-06-13T06:58:14.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.276+0000 7f6cad600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ca8102540 0x7f6ca8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.276+0000 7f6cad600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca8102ee0 0x7f6ca8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.276+0000 7f6cad600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ca8107450 0x7f6ca8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.276+0000 7f6cad600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca8112830 con 0x7f6ca8107450 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.276+0000 7f6cad600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81126b0 con 0x7f6ca8102ee0 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.276+0000 7f6cad600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81129b0 con 0x7f6ca8102540 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ca8102540 0x7f6ca8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ca8102540 0x7f6ca8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:32804/0 (socket says 172.21.15.26:32804) 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6a00700 1 -- 172.21.15.26:0/1468744679 learned_addr learned my addr 172.21.15.26:0/1468744679 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca7400700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ca8107450 0x7f6ca8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6000700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca8102ee0 0x7f6ca8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:14.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6a00700 1 -- 172.21.15.26:0/1468744679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca8102ee0 msgr2=0x7f6ca8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:14.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6a00700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca8102ee0 0x7f6ca8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6a00700 1 -- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ca8107450 msgr2=0x7f6ca8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6a00700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ca8107450 0x7f6ca8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.277+0000 7f6ca6a00700 1 -- 172.21.15.26:0/1468744679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca81a71c0 con 0x7f6ca8102540 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.278+0000 7f6ca6a00700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ca8102540 0x7f6ca8074790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800d980 tx=0x7f6c9800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.278+0000 7f6c9f400700 1 -- 172.21.15.26:0/1468744679 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98016070 con 0x7f6ca8102540 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.278+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca81a74b0 con 0x7f6ca8102540 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.278+0000 7f6c9f400700 1 -- 172.21.15.26:0/1468744679 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c9800fd70 con 0x7f6ca8102540 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.278+0000 7f6c9f400700 1 -- 172.21.15.26:0/1468744679 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98019360 con 0x7f6ca8102540 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.278+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca81a7a40 con 0x7f6ca8102540 2024-06-13T06:58:14.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.279+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca8103c00 con 0x7f6ca8102540 2024-06-13T06:58:14.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.281+0000 7f6c9f400700 1 -- 172.21.15.26:0/1468744679 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6c98019070 con 0x7f6ca8102540 2024-06-13T06:58:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.283+0000 7f6c9f400700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c8c0a2d30 0x7f6c8c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.283+0000 7f6c9f400700 1 -- 172.21.15.26:0/1468744679 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6c980cebf0 con 0x7f6ca8102540 2024-06-13T06:58:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.283+0000 7f6ca6000700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c8c0a2d30 0x7f6c8c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:13 smithi135 ceph-mon[36276]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.284+0000 7f6ca6000700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c8c0a2d30 0x7f6c8c0a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8079af0 tx=0x7f6c94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.287+0000 7f6c9f400700 1 -- 172.21.15.26:0/1468744679 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6c9809b5a0 con 0x7f6ca8102540 2024-06-13T06:58:14.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.527+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6ca8075c70 con 0x7f6ca8102540 2024-06-13T06:58:14.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.528+0000 7f6c9f400700 1 -- 172.21.15.26:0/1468744679 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6c9809aea0 con 0x7f6ca8102540 2024-06-13T06:58:14.527 INFO:teuthology.orchestra.run.smithi026.stdout:141733920866 2024-06-13T06:58:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c8c0a2d30 msgr2=0x7f6c8c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c8c0a2d30 0x7f6c8c0a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8079af0 tx=0x7f6c94009260 comp rx=0 tx=0).stop 2024-06-13T06:58:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ca8102540 msgr2=0x7f6ca8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ca8102540 0x7f6ca8074790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800d980 tx=0x7f6c9800dc90 comp rx=0 tx=0).stop 2024-06-13T06:58:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 shutdown_connections 2024-06-13T06:58:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6c8c0a2d30 0x7f6c8c0a51f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ca8107450 0x7f6ca8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ca8102540 0x7f6ca8074790 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 --2- 172.21.15.26:0/1468744679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca8102ee0 0x7f6ca8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:14.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 >> 172.21.15.26:0/1468744679 conn(0x7f6ca80795f0 msgr2=0x7f6ca8100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:14.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 shutdown_connections 2024-06-13T06:58:14.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:14.531+0000 7f6cad600700 1 -- 172.21.15.26:0/1468744679 wait complete. 2024-06-13T06:58:15.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:14 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1468744679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-13T06:58:15.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:14 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1468744679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-13T06:58:15.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:14 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1468744679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-13T06:58:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:15 smithi026 ceph-mon[32041]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-13T06:58:16.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:15 smithi135 ceph-mon[36276]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-13T06:58:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:15 smithi136 ceph-mon[37229]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-13T06:58:16.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.658+0000 7fb010000700 1 -- 172.21.15.26:0/3630526568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 msgr2=0x7fb00810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.658+0000 7fb010000700 1 --2- 172.21.15.26:0/3630526568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 0x7fb00810f320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb000009f90 tx=0x7fb00000b330 comp rx=0 tx=0).stop 2024-06-13T06:58:16.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.661+0000 7fb010000700 1 -- 172.21.15.26:0/3630526568 shutdown_connections 2024-06-13T06:58:16.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.661+0000 7fb010000700 1 --2- 172.21.15.26:0/3630526568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 0x7fb00810f320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.661+0000 7fb010000700 1 --2- 172.21.15.26:0/3630526568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb008108500 0x7fb0081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.661+0000 7fb010000700 1 --2- 172.21.15.26:0/3630526568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb00806a170 0x7fb00806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.662+0000 7fb010000700 1 -- 172.21.15.26:0/3630526568 >> 172.21.15.26:0/3630526568 conn(0x7fb008078340 msgr2=0x7fb008078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:16.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.664+0000 7fb010000700 1 -- 172.21.15.26:0/3630526568 shutdown_connections 2024-06-13T06:58:16.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.664+0000 7fb010000700 1 -- 172.21.15.26:0/3630526568 wait complete. 2024-06-13T06:58:16.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.665+0000 7fb010000700 1 Processor -- start 2024-06-13T06:58:16.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.666+0000 7fb010000700 1 -- start start 2024-06-13T06:58:16.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.670+0000 7fb010000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00806a170 0x7fb00819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.670+0000 7fb010000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 0x7fb00819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.670+0000 7fb010000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb008108500 0x7fb00819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.670+0000 7fb010000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112760 con 0x7fb00806ab30 2024-06-13T06:58:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.670+0000 7fb010000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081125e0 con 0x7fb008108500 2024-06-13T06:58:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.670+0000 7fb010000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081128e0 con 0x7fb00806a170 2024-06-13T06:58:16.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00e000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb008108500 0x7fb00819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00e000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb008108500 0x7fb00819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55620/0 (socket says 172.21.15.26:55620) 2024-06-13T06:58:16.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00cc00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 0x7fb00819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00e000700 1 -- 172.21.15.26:0/1803048027 learned_addr learned my addr 172.21.15.26:0/1803048027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:16.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00e000700 1 -- 172.21.15.26:0/1803048027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00806a170 msgr2=0x7fb00819ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T06:58:16.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00e000700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00806a170 0x7fb00819ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00e000700 1 -- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 msgr2=0x7fb00819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00e000700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 0x7fb00819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00e000700 1 -- 172.21.15.26:0/1803048027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00819fd90 con 0x7fb008108500 2024-06-13T06:58:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.671+0000 7fb00cc00700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 0x7fb00819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.672+0000 7fb00e000700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb008108500 0x7fb00819f650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb000009ae0 tx=0x7fb000009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.672+0000 7fb006000700 1 -- 172.21.15.26:0/1803048027 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000021070 con 0x7fb008108500 2024-06-13T06:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.672+0000 7fb010000700 1 -- 172.21.15.26:0/1803048027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0081a0020 con 0x7fb008108500 2024-06-13T06:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.672+0000 7fb006000700 1 -- 172.21.15.26:0/1803048027 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb000007bc0 con 0x7fb008108500 2024-06-13T06:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.672+0000 7fb006000700 1 -- 172.21.15.26:0/1803048027 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000005330 con 0x7fb008108500 2024-06-13T06:58:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.673+0000 7fb010000700 1 -- 172.21.15.26:0/1803048027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0081a7900 con 0x7fb008108500 2024-06-13T06:58:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.675+0000 7faffb400700 1 -- 172.21.15.26:0/1803048027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe4005330 con 0x7fb008108500 2024-06-13T06:58:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.679+0000 7fb006000700 1 -- 172.21.15.26:0/1803048027 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb000024070 con 0x7fb008108500 2024-06-13T06:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.681+0000 7fb006000700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faff00a2d30 0x7faff00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.681+0000 7fb006000700 1 -- 172.21.15.26:0/1803048027 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb00000f020 con 0x7fb008108500 2024-06-13T06:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.681+0000 7fb00d600700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faff00a2d30 0x7faff00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.682+0000 7fb00d600700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faff00a2d30 0x7faff00a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faffc00b990 tx=0x7faffc00b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.682+0000 7fb006000700 1 -- 172.21.15.26:0/1803048027 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb00002b070 con 0x7fb008108500 2024-06-13T06:58:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.934+0000 7faffb400700 1 -- 172.21.15.26:0/1803048027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fafe40058f0 con 0x7fb008108500 2024-06-13T06:58:16.934 INFO:teuthology.orchestra.run.smithi026.stdout:309237645374 2024-06-13T06:58:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.934+0000 7fb006000700 1 -- 172.21.15.26:0/1803048027 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb00009d6a0 con 0x7fb008108500 2024-06-13T06:58:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 -- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faff00a2d30 msgr2=0x7faff00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faff00a2d30 0x7faff00a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faffc00b990 tx=0x7faffc00b370 comp rx=0 tx=0).stop 2024-06-13T06:58:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 -- 172.21.15.26:0/1803048027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb008108500 msgr2=0x7fb00819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb008108500 0x7fb00819f650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb000009ae0 tx=0x7fb000009bc0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 -- 172.21.15.26:0/1803048027 shutdown_connections 2024-06-13T06:58:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faff00a2d30 0x7faff00a51f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb008108500 0x7fb00819f650 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb00806a170 0x7fb00819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 --2- 172.21.15.26:0/1803048027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00806ab30 0x7fb00819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 -- 172.21.15.26:0/1803048027 >> 172.21.15.26:0/1803048027 conn(0x7fb008078340 msgr2=0x7fb00810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 -- 172.21.15.26:0/1803048027 shutdown_connections 2024-06-13T06:58:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:16.938+0000 7faffb400700 1 -- 172.21.15.26:0/1803048027 wait complete. 2024-06-13T06:58:17.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:16 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1803048027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-13T06:58:17.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:16 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1803048027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-13T06:58:17.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:16 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1803048027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-13T06:58:18.117 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:17 smithi026 ceph-mon[32041]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:17 smithi135 ceph-mon[36276]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 -- 172.21.15.26:0/1150053988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc1046b0 msgr2=0x7fd2cc104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 --2- 172.21.15.26:0/1150053988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc1046b0 0x7fd2cc104b30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009990 tx=0x7fd2b8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 -- 172.21.15.26:0/1150053988 shutdown_connections 2024-06-13T06:58:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 --2- 172.21.15.26:0/1150053988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2cc105070 0x7fd2cc111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 --2- 172.21.15.26:0/1150053988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2cc10a6b0 0x7fd2cc10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 --2- 172.21.15.26:0/1150053988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc1046b0 0x7fd2cc104b30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 -- 172.21.15.26:0/1150053988 >> 172.21.15.26:0/1150053988 conn(0x7fd2cc1001a0 msgr2=0x7fd2cc1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 -- 172.21.15.26:0/1150053988 shutdown_connections 2024-06-13T06:58:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.117+0000 7fd2d4400700 1 -- 172.21.15.26:0/1150053988 wait complete. 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.118+0000 7fd2d4400700 1 Processor -- start 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.118+0000 7fd2d4400700 1 -- start start 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d4400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2cc1046b0 0x7fd2cc19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d4400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc105070 0x7fd2cc19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d4400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2cc10a6b0 0x7fd2cc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d4400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc114a90 con 0x7fd2cc105070 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d4400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc114910 con 0x7fd2cc1046b0 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d4400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc114c10 con 0x7fd2cc10a6b0 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d1000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc105070 0x7fd2cc19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d1000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc105070 0x7fd2cc19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37984/0 (socket says 172.21.15.26:37984) 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d1000700 1 -- 172.21.15.26:0/3459147213 learned_addr learned my addr 172.21.15.26:0/3459147213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d1a00700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2cc1046b0 0x7fd2cc19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.119+0000 7fd2d2400700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2cc10a6b0 0x7fd2cc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d1a00700 1 -- 172.21.15.26:0/3459147213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2cc10a6b0 msgr2=0x7fd2cc19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d1a00700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2cc10a6b0 0x7fd2cc19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d1a00700 1 -- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc105070 msgr2=0x7fd2cc19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d1a00700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc105070 0x7fd2cc19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d1a00700 1 -- 172.21.15.26:0/3459147213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc19fe70 con 0x7fd2cc1046b0 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d1000700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc105070 0x7fd2cc19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d1a00700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2cc1046b0 0x7fd2cc19ae60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000e9f0 tx=0x7fd2c000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2ca000700 1 -- 172.21.15.26:0/3459147213 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0004220 con 0x7fd2cc1046b0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc1a0160 con 0x7fd2cc1046b0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2ca000700 1 -- 172.21.15.26:0/3459147213 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2c0004d10 con 0x7fd2cc1046b0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2ca000700 1 -- 172.21.15.26:0/3459147213 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0005020 con 0x7fd2cc1046b0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.120+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc1a79e0 con 0x7fd2cc1046b0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.122+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc10cc40 con 0x7fd2cc1046b0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.123+0000 7fd2ca000700 1 -- 172.21.15.26:0/3459147213 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd2c0004750 con 0x7fd2cc1046b0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.125+0000 7fd2ca000700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd2b40a2cb0 0x7fd2b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.125+0000 7fd2ca000700 1 -- 172.21.15.26:0/3459147213 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd2c00cf460 con 0x7fd2cc1046b0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.125+0000 7fd2d1000700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd2b40a2cb0 0x7fd2b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.126+0000 7fd2d1000700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd2b40a2cb0 0x7fd2b40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009750 tx=0x7fd2b800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.129+0000 7fd2ca000700 1 -- 172.21.15.26:0/3459147213 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd2c009be10 con 0x7fd2cc1046b0 2024-06-13T06:58:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.367+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd2cc19c1c0 con 0x7fd2cc1046b0 2024-06-13T06:58:18.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.367+0000 7fd2ca000700 1 -- 172.21.15.26:0/3459147213 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd2c009b710 con 0x7fd2cc1046b0 2024-06-13T06:58:18.367 INFO:teuthology.orchestra.run.smithi026.stdout:219043332177 2024-06-13T06:58:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.370+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd2b40a2cb0 msgr2=0x7fd2b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.370+0000 7fd2d4400700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd2b40a2cb0 0x7fd2b40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009750 tx=0x7fd2b800b280 comp rx=0 tx=0).stop 2024-06-13T06:58:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.370+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2cc1046b0 msgr2=0x7fd2cc19ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:18.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.370+0000 7fd2d4400700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2cc1046b0 0x7fd2cc19ae60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000e9f0 tx=0x7fd2c000ed00 comp rx=0 tx=0).stop 2024-06-13T06:58:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.371+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 shutdown_connections 2024-06-13T06:58:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.371+0000 7fd2d4400700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd2b40a2cb0 0x7fd2b40a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.371+0000 7fd2d4400700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2cc1046b0 0x7fd2cc19ae60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.371+0000 7fd2d4400700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2cc10a6b0 0x7fd2cc19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.371+0000 7fd2d4400700 1 --2- 172.21.15.26:0/3459147213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2cc105070 0x7fd2cc19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.371+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 >> 172.21.15.26:0/3459147213 conn(0x7fd2cc1001a0 msgr2=0x7fd2cc100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.371+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 shutdown_connections 2024-06-13T06:58:18.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:18.371+0000 7fd2d4400700 1 -- 172.21.15.26:0/3459147213 wait complete. 2024-06-13T06:58:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:17 smithi136 ceph-mon[37229]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:18.511 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920863 got 141733920866 for osd.4 2024-06-13T06:58:18.511 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:19.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:18 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3459147213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-13T06:58:19.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:18 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3459147213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-13T06:58:19.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:18 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3459147213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-13T06:58:19.841 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332172 got 219043332177 for osd.7 2024-06-13T06:58:19.841 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:20.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:19 smithi026 ceph-mon[32041]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:20.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:19 smithi135 ceph-mon[36276]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:19 smithi136 ceph-mon[37229]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.526+0000 7f2e30a00700 1 -- 172.21.15.26:0/1292589339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e2c0746c0 msgr2=0x7f2e2c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.526+0000 7f2e30a00700 1 --2- 172.21.15.26:0/1292589339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e2c0746c0 0x7f2e2c074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009990 tx=0x7f2e1c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.526+0000 7f2e30a00700 1 -- 172.21.15.26:0/1292589339 shutdown_connections 2024-06-13T06:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.526+0000 7f2e30a00700 1 --2- 172.21.15.26:0/1292589339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c10ee90 0x7f2e2c111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.526+0000 7f2e30a00700 1 --2- 172.21.15.26:0/1292589339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e2c0746c0 0x7f2e2c074aa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.526+0000 7f2e30a00700 1 --2- 172.21.15.26:0/1292589339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e2c074fe0 0x7f2e2c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.526+0000 7f2e30a00700 1 -- 172.21.15.26:0/1292589339 >> 172.21.15.26:0/1292589339 conn(0x7f2e2c0795f0 msgr2=0x7f2e2c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.527+0000 7f2e30a00700 1 -- 172.21.15.26:0/1292589339 shutdown_connections 2024-06-13T06:58:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.527+0000 7f2e30a00700 1 -- 172.21.15.26:0/1292589339 wait complete. 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.528+0000 7f2e30a00700 1 Processor -- start 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.528+0000 7f2e30a00700 1 -- start start 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.528+0000 7f2e30a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e2c0746c0 0x7f2e2c19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.528+0000 7f2e30a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c074fe0 0x7f2e2c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.528+0000 7f2e30a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e2c10ee90 0x7f2e2c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.528+0000 7f2e30a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c114a10 con 0x7f2e2c074fe0 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.528+0000 7f2e30a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c114890 con 0x7f2e2c0746c0 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.528+0000 7f2e30a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c114b90 con 0x7f2e2c10ee90 2024-06-13T06:58:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e2c0746c0 0x7f2e2c19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c074fe0 0x7f2e2c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e2c0746c0 0x7f2e2c19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55662/0 (socket says 172.21.15.26:55662) 2024-06-13T06:58:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29e00700 1 -- 172.21.15.26:0/2968865056 learned_addr learned my addr 172.21.15.26:0/2968865056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e2a800700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e2c10ee90 0x7f2e2c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29e00700 1 -- 172.21.15.26:0/2968865056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e2c10ee90 msgr2=0x7f2e2c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29e00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e2c10ee90 0x7f2e2c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29e00700 1 -- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c074fe0 msgr2=0x7f2e2c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29e00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c074fe0 0x7f2e2c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.529+0000 7f2e29e00700 1 -- 172.21.15.26:0/2968865056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2c19fe60 con 0x7f2e2c0746c0 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.530+0000 7f2e29e00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e2c0746c0 0x7f2e2c19aea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c000c00 tx=0x7f2e1c00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.530+0000 7f2e22a00700 1 -- 172.21.15.26:0/2968865056 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c027070 con 0x7f2e2c0746c0 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.530+0000 7f2e22a00700 1 -- 172.21.15.26:0/2968865056 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e1c0056d0 con 0x7f2e2c0746c0 2024-06-13T06:58:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.530+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2c1a00f0 con 0x7f2e2c0746c0 2024-06-13T06:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.530+0000 7f2e22a00700 1 -- 172.21.15.26:0/2968865056 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c022e10 con 0x7f2e2c0746c0 2024-06-13T06:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.530+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2c1a79d0 con 0x7f2e2c0746c0 2024-06-13T06:58:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.531+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e2c069180 con 0x7f2e2c0746c0 2024-06-13T06:58:21.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.533+0000 7f2e22a00700 1 -- 172.21.15.26:0/2968865056 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2e1c02a070 con 0x7f2e2c0746c0 2024-06-13T06:58:21.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.535+0000 7f2e22a00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e100a2cb0 0x7f2e100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.535+0000 7f2e22a00700 1 -- 172.21.15.26:0/2968865056 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2e1c0d08c0 con 0x7f2e2c0746c0 2024-06-13T06:58:21.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.535+0000 7f2e29400700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e100a2cb0 0x7f2e100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.536+0000 7f2e29400700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e100a2cb0 0x7f2e100a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c110c00 tx=0x7f2e18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:21.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.537+0000 7f2e22a00700 1 -- 172.21.15.26:0/2968865056 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2e1c013070 con 0x7f2e2c0746c0 2024-06-13T06:58:21.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.776+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2e2c02dca0 con 0x7f2e2c0746c0 2024-06-13T06:58:21.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.777+0000 7f2e22a00700 1 -- 172.21.15.26:0/2968865056 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2e1c09d300 con 0x7f2e2c0746c0 2024-06-13T06:58:21.778 INFO:teuthology.orchestra.run.smithi026.stdout:163208757342 2024-06-13T06:58:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e100a2cb0 msgr2=0x7f2e100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e100a2cb0 0x7f2e100a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c110c00 tx=0x7f2e18009260 comp rx=0 tx=0).stop 2024-06-13T06:58:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e2c0746c0 msgr2=0x7f2e2c19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e2c0746c0 0x7f2e2c19aea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c000c00 tx=0x7f2e1c00bab0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 shutdown_connections 2024-06-13T06:58:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2e100a2cb0 0x7f2e100a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e2c0746c0 0x7f2e2c19aea0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e2c10ee90 0x7f2e2c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 --2- 172.21.15.26:0/2968865056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c074fe0 0x7f2e2c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.780+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 >> 172.21.15.26:0/2968865056 conn(0x7f2e2c0795f0 msgr2=0x7f2e2c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.781+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 shutdown_connections 2024-06-13T06:58:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:21.781+0000 7f2e30a00700 1 -- 172.21.15.26:0/2968865056 wait complete. 2024-06-13T06:58:22.156 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:21 smithi026 ceph-mon[32041]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:22.156 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:21 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2968865056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-13T06:58:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:21 smithi135 ceph-mon[36276]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:21 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2968865056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-13T06:58:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:21 smithi136 ceph-mon[37229]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:21 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2968865056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 -- 172.21.15.26:0/3092559792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 msgr2=0x7fc2d4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 --2- 172.21.15.26:0/3092559792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 0x7fc2d4102940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009990 tx=0x7fc2c0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 -- 172.21.15.26:0/3092559792 shutdown_connections 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 --2- 172.21.15.26:0/3092559792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2d4102e80 0x7fc2d410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 --2- 172.21.15.26:0/3092559792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2d4108650 0x7fc2d4108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 --2- 172.21.15.26:0/3092559792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 0x7fc2d4102940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 -- 172.21.15.26:0/3092559792 >> 172.21.15.26:0/3092559792 conn(0x7fc2d40795f0 msgr2=0x7fc2d4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 -- 172.21.15.26:0/3092559792 shutdown_connections 2024-06-13T06:58:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.632+0000 7fc2dc400700 1 -- 172.21.15.26:0/3092559792 wait complete. 2024-06-13T06:58:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.633+0000 7fc2dc400700 1 Processor -- start 2024-06-13T06:58:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.633+0000 7fc2dc400700 1 -- start start 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2dc400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 0x7fc2d419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2dc400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2d4102e80 0x7fc2d419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2dc400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2d4108650 0x7fc2d419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2dc400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41127f0 con 0x7fc2d41024e0 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2dc400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d4112670 con 0x7fc2d4102e80 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2dc400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d4112970 con 0x7fc2d4108650 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2d9a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 0x7fc2d419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2d9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2d4102e80 0x7fc2d419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2d9a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 0x7fc2d419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52950/0 (socket says 172.21.15.26:52950) 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2d9a00700 1 -- 172.21.15.26:0/2140444981 learned_addr learned my addr 172.21.15.26:0/2140444981 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.634+0000 7fc2da400700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2d4108650 0x7fc2d419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d9000700 1 -- 172.21.15.26:0/2140444981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2d4108650 msgr2=0x7fc2d419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d9000700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2d4108650 0x7fc2d419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d9000700 1 -- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 msgr2=0x7fc2d419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d9000700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 0x7fc2d419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d9000700 1 -- 172.21.15.26:0/2140444981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d419fe40 con 0x7fc2d4102e80 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d9a00700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 0x7fc2d419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2da400700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2d4108650 0x7fc2d419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d9000700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2d4102e80 0x7fc2d419b370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0000c00 tx=0x7fc2c000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d2000700 1 -- 172.21.15.26:0/2140444981 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0026070 con 0x7fc2d4102e80 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d2000700 1 -- 172.21.15.26:0/2140444981 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c0003d70 con 0x7fc2d4102e80 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d41a00d0 con 0x7fc2d4102e80 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2d2000700 1 -- 172.21.15.26:0/2140444981 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000fdc0 con 0x7fc2d4102e80 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.636+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d41a79b0 con 0x7fc2d4102e80 2024-06-13T06:58:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.638+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d4069180 con 0x7fc2d4102e80 2024-06-13T06:58:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.641+0000 7fc2d2000700 1 -- 172.21.15.26:0/2140444981 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc2c0029070 con 0x7fc2d4102e80 2024-06-13T06:58:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.641+0000 7fc2d2000700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc2bc0a2cb0 0x7fc2bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.641+0000 7fc2d2000700 1 -- 172.21.15.26:0/2140444981 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc2c00d0670 con 0x7fc2d4102e80 2024-06-13T06:58:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.641+0000 7fc2d9a00700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc2bc0a2cb0 0x7fc2bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.642+0000 7fc2d9a00700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc2bc0a2cb0 0x7fc2bc0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8011550 tx=0x7fc2c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.644+0000 7fc2d2000700 1 -- 172.21.15.26:0/2140444981 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc2c0013070 con 0x7fc2d4102e80 2024-06-13T06:58:23.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.909+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fc2d41a7b80 con 0x7fc2d4102e80 2024-06-13T06:58:23.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.910+0000 7fc2d2000700 1 -- 172.21.15.26:0/2140444981 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc2c009d0b0 con 0x7fc2d4102e80 2024-06-13T06:58:23.910 INFO:teuthology.orchestra.run.smithi026.stdout:287762808902 2024-06-13T06:58:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc2bc0a2cb0 msgr2=0x7fc2bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc2bc0a2cb0 0x7fc2bc0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8011550 tx=0x7fc2c8009260 comp rx=0 tx=0).stop 2024-06-13T06:58:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2d4102e80 msgr2=0x7fc2d419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:23.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2d4102e80 0x7fc2d419b370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0000c00 tx=0x7fc2c000bab0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 shutdown_connections 2024-06-13T06:58:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc2bc0a2cb0 0x7fc2bc0a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2d4108650 0x7fc2d419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2d41024e0 0x7fc2d419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 --2- 172.21.15.26:0/2140444981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2d4102e80 0x7fc2d419b370 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 >> 172.21.15.26:0/2140444981 conn(0x7fc2d40795f0 msgr2=0x7fc2d40747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 shutdown_connections 2024-06-13T06:58:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:23.913+0000 7fc2dc400700 1 -- 172.21.15.26:0/2140444981 wait complete. 2024-06-13T06:58:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:23 smithi026 ceph-mon[32041]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:23 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2140444981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-13T06:58:24.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:23 smithi135 ceph-mon[36276]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:24.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:23 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2140444981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-13T06:58:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:23 smithi136 ceph-mon[37229]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:23 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2140444981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-13T06:58:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.664+0000 7f76eca00700 1 -- 172.21.15.26:0/3006427531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806b610 msgr2=0x7f76e8107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.664+0000 7f76eca00700 1 --2- 172.21.15.26:0/3006427531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806b610 0x7f76e8107090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009990 tx=0x7f76d4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.664+0000 7f76eca00700 1 -- 172.21.15.26:0/3006427531 shutdown_connections 2024-06-13T06:58:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.664+0000 7f76eca00700 1 --2- 172.21.15.26:0/3006427531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76e81077c0 0x7f76e810f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.664+0000 7f76eca00700 1 --2- 172.21.15.26:0/3006427531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e806ac60 0x7f76e806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.664+0000 7f76eca00700 1 --2- 172.21.15.26:0/3006427531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806b610 0x7f76e8107090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.664+0000 7f76eca00700 1 -- 172.21.15.26:0/3006427531 >> 172.21.15.26:0/3006427531 conn(0x7f76e80781d0 msgr2=0x7f76e80785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.664+0000 7f76eca00700 1 -- 172.21.15.26:0/3006427531 shutdown_connections 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.665+0000 7f76eca00700 1 -- 172.21.15.26:0/3006427531 wait complete. 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.666+0000 7f76eca00700 1 Processor -- start 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.666+0000 7f76eca00700 1 -- start start 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.666+0000 7f76eca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806ac60 0x7f76e8101620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.666+0000 7f76eca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76e806b610 0x7f76e8101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.666+0000 7f76eca00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e81077c0 0x7f76e81020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.666+0000 7f76eca00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112700 con 0x7f76e81077c0 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.666+0000 7f76eca00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112580 con 0x7f76e806b610 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.666+0000 7f76eca00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112880 con 0x7f76e806ac60 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76e806b610 0x7f76e8101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e7e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e81077c0 0x7f76e81020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806ac60 0x7f76e8101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76e806b610 0x7f76e8101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46738/0 (socket says 172.21.15.26:46738) 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 -- 172.21.15.26:0/1371809848 learned_addr learned my addr 172.21.15.26:0/1371809848 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 -- 172.21.15.26:0/1371809848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806ac60 msgr2=0x7f76e8101620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806ac60 0x7f76e8101620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 -- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e81077c0 msgr2=0x7f76e81020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e81077c0 0x7f76e81020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 -- 172.21.15.26:0/1371809848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81a7080 con 0x7f76e806b610 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e7400700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806ac60 0x7f76e8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.667+0000 7f76e6a00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76e806b610 0x7f76e8101b60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f76d400f690 tx=0x7f76d400f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.668+0000 7f76dfe00700 1 -- 172.21.15.26:0/1371809848 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4027070 con 0x7f76e806b610 2024-06-13T06:58:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.668+0000 7f76dfe00700 1 -- 172.21.15.26:0/1371809848 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d400be70 con 0x7f76e806b610 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.668+0000 7f76dfe00700 1 -- 172.21.15.26:0/1371809848 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4022d50 con 0x7f76e806b610 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.668+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81a7310 con 0x7f76e806b610 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.668+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a7710 con 0x7f76e806b610 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.669+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8069180 con 0x7f76e806b610 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.672+0000 7f76dfe00700 1 -- 172.21.15.26:0/1371809848 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f76d402a070 con 0x7f76e806b610 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.673+0000 7f76dfe00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76cc0a2d30 0x7f76cc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.673+0000 7f76dfe00700 1 -- 172.21.15.26:0/1371809848 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f76d40cfff0 con 0x7f76e806b610 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.673+0000 7f76e7400700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76cc0a2d30 0x7f76cc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.674+0000 7f76e7400700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76cc0a2d30 0x7f76cc0a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f76d8005950 tx=0x7f76d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.675+0000 7f76dfe00700 1 -- 172.21.15.26:0/1371809848 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f76d4013030 con 0x7f76e806b610 2024-06-13T06:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.913+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f76e810a970 con 0x7f76e806b610 2024-06-13T06:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.914+0000 7f76dfe00700 1 -- 172.21.15.26:0/1371809848 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f76d409ca90 con 0x7f76e806b610 2024-06-13T06:58:24.913 INFO:teuthology.orchestra.run.smithi026.stdout:107374182504 2024-06-13T06:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76cc0a2d30 msgr2=0x7f76cc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76cc0a2d30 0x7f76cc0a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f76d8005950 tx=0x7f76d800b410 comp rx=0 tx=0).stop 2024-06-13T06:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76e806b610 msgr2=0x7f76e8101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76e806b610 0x7f76e8101b60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f76d400f690 tx=0x7f76d400f6c0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 shutdown_connections 2024-06-13T06:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76cc0a2d30 0x7f76cc0a51f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e81077c0 0x7f76e81020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e806ac60 0x7f76e8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 --2- 172.21.15.26:0/1371809848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76e806b610 0x7f76e8101b60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.917+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 >> 172.21.15.26:0/1371809848 conn(0x7f76e80781d0 msgr2=0x7f76e81088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.918+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 shutdown_connections 2024-06-13T06:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.918+0000 7f76eca00700 1 -- 172.21.15.26:0/1371809848 wait complete. 2024-06-13T06:58:25.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:25 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2961768341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-13T06:58:25.213 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:25 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1371809848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-13T06:58:25.264 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645368 got 309237645374 for osd.10 2024-06-13T06:58:25.264 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:25.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:25 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2961768341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-13T06:58:25.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:25 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1371809848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-13T06:58:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 -- 172.21.15.26:0/3573955661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa68006a170 msgr2=0x7fa68006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 --2- 172.21.15.26:0/3573955661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa68006a170 0x7fa68006a5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0098f0 tx=0x7fa66c009c00 comp rx=0 tx=0).stop 2024-06-13T06:58:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 -- 172.21.15.26:0/3573955661 shutdown_connections 2024-06-13T06:58:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 --2- 172.21.15.26:0/3573955661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa68006ab30 0x7fa68010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 --2- 172.21.15.26:0/3573955661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa680108500 0x7fa6801088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 --2- 172.21.15.26:0/3573955661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa68006a170 0x7fa68006a5f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 -- 172.21.15.26:0/3573955661 >> 172.21.15.26:0/3573955661 conn(0x7fa680078340 msgr2=0x7fa680078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 -- 172.21.15.26:0/3573955661 shutdown_connections 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.845+0000 7fa686200700 1 -- 172.21.15.26:0/3573955661 wait complete. 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.847+0000 7fa686200700 1 Processor -- start 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.847+0000 7fa686200700 1 -- start start 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa686200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68006a170 0x7fa680101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa686200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa68006ab30 0x7fa6801019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa686200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa680108500 0x7fa680101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa686200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801127d0 con 0x7fa68006a170 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa686200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa680112650 con 0x7fa68006ab30 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa686200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa680112950 con 0x7fa680108500 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa67f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68006a170 0x7fa680101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa67f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68006a170 0x7fa680101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53016/0 (socket says 172.21.15.26:53016) 2024-06-13T06:58:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa67f400700 1 -- 172.21.15.26:0/492755818 learned_addr learned my addr 172.21.15.26:0/492755818 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa67fe00700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa680108500 0x7fa680101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.848+0000 7fa67ea00700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa68006ab30 0x7fa6801019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.849+0000 7fa67ea00700 1 -- 172.21.15.26:0/492755818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa680108500 msgr2=0x7fa680101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.849+0000 7fa67ea00700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa680108500 0x7fa680101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.849+0000 7fa67ea00700 1 -- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68006a170 msgr2=0x7fa680101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.849+0000 7fa67ea00700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68006a170 0x7fa680101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.849+0000 7fa67ea00700 1 -- 172.21.15.26:0/492755818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6801a71f0 con 0x7fa68006ab30 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.849+0000 7fa67f400700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68006a170 0x7fa680101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.849+0000 7fa67ea00700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa68006ab30 0x7fa6801019a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000c00 tx=0x7fa66c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.850+0000 7fa677e00700 1 -- 172.21.15.26:0/492755818 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c005040 con 0x7fa68006ab30 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.850+0000 7fa677e00700 1 -- 172.21.15.26:0/492755818 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c0056c0 con 0x7fa68006ab30 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.850+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6801a7480 con 0x7fa68006ab30 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.850+0000 7fa677e00700 1 -- 172.21.15.26:0/492755818 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c003a00 con 0x7fa68006ab30 2024-06-13T06:58:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.850+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801a7930 con 0x7fa68006ab30 2024-06-13T06:58:25.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.851+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa680069180 con 0x7fa68006ab30 2024-06-13T06:58:25.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.853+0000 7fa677e00700 1 -- 172.21.15.26:0/492755818 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa66c00b560 con 0x7fa68006ab30 2024-06-13T06:58:25.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.854+0000 7fa677e00700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2d30 0x7fa6640a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:25.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.855+0000 7fa677e00700 1 -- 172.21.15.26:0/492755818 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa66c0d0720 con 0x7fa68006ab30 2024-06-13T06:58:25.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.855+0000 7fa67f400700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2d30 0x7fa6640a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:25.324 INFO:teuthology.orchestra.run.smithi026.stdout:34359738490 2024-06-13T06:58:25.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.855+0000 7fa67f400700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2d30 0x7fa6640a51f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa670009750 tx=0x7fa670009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.857+0000 7fa677e00700 1 -- 172.21.15.26:0/492755818 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa66c013030 con 0x7fa68006ab30 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.114+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa68002dca0 con 0x7fa68006ab30 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.115+0000 7fa677e00700 1 -- 172.21.15.26:0/492755818 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa66c09d160 con 0x7fa68006ab30 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2d30 msgr2=0x7fa6640a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2d30 0x7fa6640a51f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa670009750 tx=0x7fa670009310 comp rx=0 tx=0).stop 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa68006ab30 msgr2=0x7fa6801019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa68006ab30 0x7fa6801019a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000c00 tx=0x7fa66c00fa00 comp rx=0 tx=0).stop 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 shutdown_connections 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2d30 0x7fa6640a51f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa680108500 0x7fa680101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68006a170 0x7fa680101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 --2- 172.21.15.26:0/492755818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa68006ab30 0x7fa6801019a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 >> 172.21.15.26:0/492755818 conn(0x7fa680078340 msgr2=0x7fa680100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 shutdown_connections 2024-06-13T06:58:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:25.119+0000 7fa686200700 1 -- 172.21.15.26:0/492755818 wait complete. 2024-06-13T06:58:25.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:25 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2961768341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-13T06:58:25.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:25 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1371809848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-13T06:58:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.153+0000 7f0f39200700 1 -- 172.21.15.26:0/2552292573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f341076d0 msgr2=0x7f0f3410f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.153+0000 7f0f39200700 1 --2- 172.21.15.26:0/2552292573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f341076d0 0x7f0f3410f2a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00b0a0 tx=0x7f0f2c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.154+0000 7f0f39200700 1 -- 172.21.15.26:0/2552292573 shutdown_connections 2024-06-13T06:58:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.154+0000 7f0f39200700 1 --2- 172.21.15.26:0/2552292573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f341076d0 0x7f0f3410f2a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.154+0000 7f0f39200700 1 --2- 172.21.15.26:0/2552292573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f341025f0 0x7f0f341029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.154+0000 7f0f39200700 1 --2- 172.21.15.26:0/2552292573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f34102fa0 0x7f0f34106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.154+0000 7f0f39200700 1 -- 172.21.15.26:0/2552292573 >> 172.21.15.26:0/2552292573 conn(0x7f0f340fdea0 msgr2=0x7f0f341002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.154+0000 7f0f39200700 1 -- 172.21.15.26:0/2552292573 shutdown_connections 2024-06-13T06:58:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.154+0000 7f0f39200700 1 -- 172.21.15.26:0/2552292573 wait complete. 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.155+0000 7f0f39200700 1 Processor -- start 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.155+0000 7f0f39200700 1 -- start start 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f39200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f341025f0 0x7f0f3419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f39200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f34102fa0 0x7f0f3419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f39200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f341076d0 0x7f0f34198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f39200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341126c0 con 0x7f0f341025f0 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f39200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34112540 con 0x7f0f34102fa0 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f39200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34112840 con 0x7f0f341076d0 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f33400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f341025f0 0x7f0f3419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f33400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f341025f0 0x7f0f3419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52976/0 (socket says 172.21.15.26:52976) 2024-06-13T06:58:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f33400700 1 -- 172.21.15.26:0/2961768341 learned_addr learned my addr 172.21.15.26:0/2961768341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f33e00700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f341076d0 0x7f0f34198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.156+0000 7f0f32a00700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f34102fa0 0x7f0f3419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f33400700 1 -- 172.21.15.26:0/2961768341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f341076d0 msgr2=0x7f0f34198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f33400700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f341076d0 0x7f0f34198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f33400700 1 -- 172.21.15.26:0/2961768341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f34102fa0 msgr2=0x7f0f3419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f33400700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f34102fa0 0x7f0f3419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f33400700 1 -- 172.21.15.26:0/2961768341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f34199080 con 0x7f0f341025f0 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f32a00700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f34102fa0 0x7f0f3419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f33400700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f341025f0 0x7f0f3419e760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400d980 tx=0x7f0f2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f2be00700 1 -- 172.21.15.26:0/2961768341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24016070 con 0x7f0f341025f0 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f34199370 con 0x7f0f341025f0 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.157+0000 7f0f2be00700 1 -- 172.21.15.26:0/2961768341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f24004d10 con 0x7f0f341025f0 2024-06-13T06:58:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.158+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f340746c0 con 0x7f0f341025f0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.158+0000 7f0f2be00700 1 -- 172.21.15.26:0/2961768341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24005610 con 0x7f0f341025f0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.159+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f34069180 con 0x7f0f341025f0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.161+0000 7f0f2be00700 1 -- 172.21.15.26:0/2961768341 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0f24004750 con 0x7f0f341025f0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.162+0000 7f0f2be00700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0f180a2cb0 0x7f0f180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.163+0000 7f0f2be00700 1 -- 172.21.15.26:0/2961768341 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0f240cf370 con 0x7f0f341025f0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.163+0000 7f0f32a00700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0f180a2cb0 0x7f0f180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.164+0000 7f0f32a00700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0f180a2cb0 0x7f0f180a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0f34101590 tx=0x7f0f2000f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.167+0000 7f0f2be00700 1 -- 172.21.15.26:0/2961768341 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0f2401f020 con 0x7f0f341025f0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.421+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0f3402dca0 con 0x7f0f341025f0 2024-06-13T06:58:25.651 INFO:teuthology.orchestra.run.smithi026.stdout:77309411440 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.421+0000 7f0f2be00700 1 -- 172.21.15.26:0/2961768341 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0f2409bd20 con 0x7f0f341025f0 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0f180a2cb0 msgr2=0x7f0f180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0f180a2cb0 0x7f0f180a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0f34101590 tx=0x7f0f2000f820 comp rx=0 tx=0).stop 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f341025f0 msgr2=0x7f0f3419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f341025f0 0x7f0f3419e760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400d980 tx=0x7f0f2400dc90 comp rx=0 tx=0).stop 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 shutdown_connections 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0f180a2cb0 0x7f0f180a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f341076d0 0x7f0f34198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f341025f0 0x7f0f3419e760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 --2- 172.21.15.26:0/2961768341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f34102fa0 0x7f0f3419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.424+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 >> 172.21.15.26:0/2961768341 conn(0x7f0f340fdea0 msgr2=0x7f0f340fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.425+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 shutdown_connections 2024-06-13T06:58:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:24.425+0000 7f0f39200700 1 -- 172.21.15.26:0/2961768341 wait complete. 2024-06-13T06:58:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:26 smithi135 ceph-mon[36276]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:26 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/492755818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-13T06:58:26.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:26 smithi026 ceph-mon[32041]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:26.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:26 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/492755818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-13T06:58:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:26 smithi136 ceph-mon[37229]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:26 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/492755818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-13T06:58:26.972 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411433 got 77309411440 for osd.2 2024-06-13T06:58:26.972 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:27.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 -- 172.21.15.26:0/1772720139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14a4105090 msgr2=0x7f14a41115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 --2- 172.21.15.26:0/1772720139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14a4105090 0x7f14a41115a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f149c00b0a0 tx=0x7f149c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 -- 172.21.15.26:0/1772720139 shutdown_connections 2024-06-13T06:58:27.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 --2- 172.21.15.26:0/1772720139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14a4105090 0x7f14a41115a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 --2- 172.21.15.26:0/1772720139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14a410a6d0 0x7f14a410aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 --2- 172.21.15.26:0/1772720139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a41046d0 0x7f14a4104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 -- 172.21.15.26:0/1772720139 >> 172.21.15.26:0/1772720139 conn(0x7f14a41001c0 msgr2=0x7f14a41025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 -- 172.21.15.26:0/1772720139 shutdown_connections 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.564+0000 7f14a9200700 1 -- 172.21.15.26:0/1772720139 wait complete. 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.565+0000 7f14a9200700 1 Processor -- start 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.565+0000 7f14a9200700 1 -- start start 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a9200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14a41046d0 0x7f14a419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a9200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14a4105090 0x7f14a419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a9200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a410a6d0 0x7f14a419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a9200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4114a20 con 0x7f14a410a6d0 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a9200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41148a0 con 0x7f14a41046d0 2024-06-13T06:58:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a9200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4114ba0 con 0x7f14a4105090 2024-06-13T06:58:27.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a2800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a410a6d0 0x7f14a419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14a4105090 0x7f14a419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a2800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a410a6d0 0x7f14a419f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53038/0 (socket says 172.21.15.26:53038) 2024-06-13T06:58:27.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a2800700 1 -- 172.21.15.26:0/3176258344 learned_addr learned my addr 172.21.15.26:0/3176258344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:27.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.566+0000 7f14a1e00700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14a41046d0 0x7f14a419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f14a1400700 1 -- 172.21.15.26:0/3176258344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14a41046d0 msgr2=0x7f14a419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f14a1400700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14a41046d0 0x7f14a419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f14a1400700 1 -- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a410a6d0 msgr2=0x7f14a419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f14a1400700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a410a6d0 0x7f14a419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f14a1400700 1 -- 172.21.15.26:0/3176258344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a419fdc0 con 0x7f14a4105090 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f14a2800700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a410a6d0 0x7f14a419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f14a1400700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14a4105090 0x7f14a419b2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f149000e9f0 tx=0x7f149000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f149aa00700 1 -- 172.21.15.26:0/3176258344 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149000c9c0 con 0x7f14a4105090 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a41a00b0 con 0x7f14a4105090 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f149aa00700 1 -- 172.21.15.26:0/3176258344 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149000cb60 con 0x7f14a4105090 2024-06-13T06:58:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.567+0000 7f149aa00700 1 -- 172.21.15.26:0/3176258344 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490018680 con 0x7f14a4105090 2024-06-13T06:58:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.568+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a41a7930 con 0x7f14a4105090 2024-06-13T06:58:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.569+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a4069180 con 0x7f14a4105090 2024-06-13T06:58:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.571+0000 7f149aa00700 1 -- 172.21.15.26:0/3176258344 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1490010430 con 0x7f14a4105090 2024-06-13T06:58:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.573+0000 7f149aa00700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14880a2cb0 0x7f14880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.573+0000 7f149aa00700 1 -- 172.21.15.26:0/3176258344 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f14900cf730 con 0x7f14a4105090 2024-06-13T06:58:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.573+0000 7f14a1e00700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14880a2cb0 0x7f14880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.574+0000 7f14a1e00700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14880a2cb0 0x7f14880a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14940045b0 tx=0x7f149400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.577+0000 7f149aa00700 1 -- 172.21.15.26:0/3176258344 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f149009c0e0 con 0x7f14a4105090 2024-06-13T06:58:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.810+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f14a41a7b60 con 0x7f14a4105090 2024-06-13T06:58:27.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.811+0000 7f149aa00700 1 -- 172.21.15.26:0/3176258344 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f149009b9e0 con 0x7f14a4105090 2024-06-13T06:58:27.810 INFO:teuthology.orchestra.run.smithi026.stdout:55834574965 2024-06-13T06:58:27.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14880a2cb0 msgr2=0x7f14880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14880a2cb0 0x7f14880a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14940045b0 tx=0x7f149400b3c0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14a4105090 msgr2=0x7f14a419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14a4105090 0x7f14a419b2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f149000e9f0 tx=0x7f149000ed00 comp rx=0 tx=0).stop 2024-06-13T06:58:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 shutdown_connections 2024-06-13T06:58:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14880a2cb0 0x7f14880a5170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14a41046d0 0x7f14a419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a410a6d0 0x7f14a419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 --2- 172.21.15.26:0/3176258344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14a4105090 0x7f14a419b2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.814+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 >> 172.21.15.26:0/3176258344 conn(0x7f14a41001c0 msgr2=0x7f14a41006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.815+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 shutdown_connections 2024-06-13T06:58:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:27.815+0000 7f14a9200700 1 -- 172.21.15.26:0/3176258344 wait complete. 2024-06-13T06:58:27.917 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182499 got 107374182504 for osd.3 2024-06-13T06:58:27.918 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:28.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:28 smithi135 ceph-mon[36276]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:28.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:28 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3176258344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-13T06:58:28.292 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:28 smithi026 ceph-mon[32041]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:28.292 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:28 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3176258344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-13T06:58:28.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:28 smithi136 ceph-mon[37229]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:28.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:28 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3176258344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-13T06:58:28.494 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757336 got 163208757342 for osd.5 2024-06-13T06:58:28.494 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:28.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574957 got 55834574965 for osd.1 2024-06-13T06:58:28.895 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:29.550 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808897 got 287762808902 for osd.9 2024-06-13T06:58:29.551 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:30.141 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:30 smithi026 ceph-mon[32041]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:30.142 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:30 smithi026 ceph-mon[32041]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-06-13T06:58:30.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:30 smithi135 ceph-mon[36276]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:30.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:30 smithi135 ceph-mon[36276]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-06-13T06:58:30.360 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738484 got 34359738490 for osd.0 2024-06-13T06:58:30.361 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:30.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:30 smithi136 ceph-mon[37229]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:30.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:30 smithi136 ceph-mon[37229]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-06-13T06:58:32.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:32 smithi136 ceph-mon[37229]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-13T06:58:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:32 smithi026 ceph-mon[32041]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-13T06:58:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:32 smithi135 ceph-mon[36276]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-13T06:58:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 -- 172.21.15.26:0/98774649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806a170 msgr2=0x7f00e806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 --2- 172.21.15.26:0/98774649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806a170 0x7f00e806a5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009990 tx=0x7f00d0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 -- 172.21.15.26:0/98774649 shutdown_connections 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 --2- 172.21.15.26:0/98774649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e806ab30 0x7f00e810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 --2- 172.21.15.26:0/98774649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e8108500 0x7f00e81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 --2- 172.21.15.26:0/98774649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806a170 0x7f00e806a5f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 -- 172.21.15.26:0/98774649 >> 172.21.15.26:0/98774649 conn(0x7f00e8078340 msgr2=0x7f00e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 -- 172.21.15.26:0/98774649 shutdown_connections 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.067+0000 7f00ee200700 1 -- 172.21.15.26:0/98774649 wait complete. 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.069+0000 7f00ee200700 1 Processor -- start 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.069+0000 7f00ee200700 1 -- start start 2024-06-13T06:58:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.069+0000 7f00ee200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e806a170 0x7f00e819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.069+0000 7f00ee200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806ab30 0x7f00e819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.069+0000 7f00ee200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e8108500 0x7f00e8198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.069+0000 7f00ee200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81127b0 con 0x7f00e8108500 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.069+0000 7f00ee200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112630 con 0x7f00e806ab30 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.069+0000 7f00ee200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112930 con 0x7f00e806a170 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806ab30 0x7f00e819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00e7e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e8108500 0x7f00e8198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806ab30 0x7f00e819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51734/0 (socket says 172.21.15.26:51734) 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00e7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e806a170 0x7f00e819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 -- 172.21.15.26:0/4184832274 learned_addr learned my addr 172.21.15.26:0/4184832274 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 -- 172.21.15.26:0/4184832274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e806a170 msgr2=0x7f00e819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e806a170 0x7f00e819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 -- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e8108500 msgr2=0x7f00e8198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e8108500 0x7f00e8198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 -- 172.21.15.26:0/4184832274 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e8199450 con 0x7f00e806ab30 2024-06-13T06:58:33.787 INFO:teuthology.orchestra.run.smithi026.stdout:253403070542 2024-06-13T06:58:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.070+0000 7f00dea00700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806ab30 0x7f00e819ef90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005910 tx=0x7f00d000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.071+0000 7f00e4c00700 1 -- 172.21.15.26:0/4184832274 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0027070 con 0x7f00e806ab30 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.071+0000 7f00e4c00700 1 -- 172.21.15.26:0/4184832274 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d0004650 con 0x7f00e806ab30 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.071+0000 7f00e4c00700 1 -- 172.21.15.26:0/4184832274 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d000f460 con 0x7f00e806ab30 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.071+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81996e0 con 0x7f00e806ab30 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.071+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a5800 con 0x7f00e806ab30 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.072+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8103bc0 con 0x7f00e806ab30 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.074+0000 7f00e4c00700 1 -- 172.21.15.26:0/4184832274 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00d002a070 con 0x7f00e806ab30 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.075+0000 7f00e4c00700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f00cc0a2c20 0x7f00cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.075+0000 7f00e4c00700 1 -- 172.21.15.26:0/4184832274 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f00d00d0360 con 0x7f00e806ab30 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.075+0000 7f00e7400700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f00cc0a2c20 0x7f00cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.076+0000 7f00e7400700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f00cc0a2c20 0x7f00cc0a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00d800f540 tx=0x7f00d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.080+0000 7f00e4c00700 1 -- 172.21.15.26:0/4184832274 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f00d005e850 con 0x7f00e806ab30 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.312+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f00e8199fe0 con 0x7f00e806ab30 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.313+0000 7f00e4c00700 1 -- 172.21.15.26:0/4184832274 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f00d0013070 con 0x7f00e806ab30 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f00cc0a2c20 msgr2=0x7f00cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f00cc0a2c20 0x7f00cc0a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00d800f540 tx=0x7f00d800b040 comp rx=0 tx=0).stop 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806ab30 msgr2=0x7f00e819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806ab30 0x7f00e819ef90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005910 tx=0x7f00d000bab0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 shutdown_connections 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f00cc0a2c20 0x7f00cc0a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e8108500 0x7f00e8198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e806a170 0x7f00e819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 --2- 172.21.15.26:0/4184832274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e806ab30 0x7f00e819ef90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 >> 172.21.15.26:0/4184832274 conn(0x7f00e8078340 msgr2=0x7f00e8100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 shutdown_connections 2024-06-13T06:58:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:33.316+0000 7f00ee200700 1 -- 172.21.15.26:0/4184832274 wait complete. 2024-06-13T06:58:34.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:34 smithi136 ceph-mon[37229]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:34.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:34 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/4184832274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-13T06:58:34.464 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:34 smithi026 ceph-mon[32041]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:34.465 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:34 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/4184832274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-13T06:58:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:34 smithi135 ceph-mon[36276]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:34 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/4184832274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-13T06:58:35.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 -- 172.21.15.26:0/1969179491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38080746c0 msgr2=0x7f3808074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 --2- 172.21.15.26:0/1969179491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38080746c0 0x7f3808074aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009990 tx=0x7f37f8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 -- 172.21.15.26:0/1969179491 shutdown_connections 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 --2- 172.21.15.26:0/1969179491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808113660 0x7f3808115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 --2- 172.21.15.26:0/1969179491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38080746c0 0x7f3808074aa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 --2- 172.21.15.26:0/1969179491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3808074fe0 0x7f3808113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 -- 172.21.15.26:0/1969179491 >> 172.21.15.26:0/1969179491 conn(0x7f38080795f0 msgr2=0x7f3808079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 -- 172.21.15.26:0/1969179491 shutdown_connections 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.374+0000 7f380e200700 1 -- 172.21.15.26:0/1969179491 wait complete. 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.376+0000 7f380e200700 1 Processor -- start 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.376+0000 7f380e200700 1 -- start start 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.376+0000 7f380e200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38080746c0 0x7f380819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.376+0000 7f380e200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3808074fe0 0x7f380819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.376+0000 7f380e200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808113660 0x7f38081a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.376+0000 7f380e200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3808118e00 con 0x7f3808074fe0 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.376+0000 7f380e200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3808118c80 con 0x7f38080746c0 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.376+0000 7f380e200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3808118f80 con 0x7f3808113660 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3808074fe0 0x7f380819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f3807400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38080746c0 0x7f380819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3808074fe0 0x7f380819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50860/0 (socket says 172.21.15.26:50860) 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f37ffe00700 1 -- 172.21.15.26:0/3151795779 learned_addr learned my addr 172.21.15.26:0/3151795779 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f3807e00700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808113660 0x7f38081a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f3807400700 1 -- 172.21.15.26:0/3151795779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808113660 msgr2=0x7f38081a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f3807400700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808113660 0x7f38081a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f3807400700 1 -- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3808074fe0 msgr2=0x7f380819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f3807400700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3808074fe0 0x7f380819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.377+0000 7f3807400700 1 -- 172.21.15.26:0/3151795779 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38081a42c0 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.378+0000 7f3807400700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38080746c0 0x7f380819f2b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37f8006010 tx=0x7f37f800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.378+0000 7f3804c00700 1 -- 172.21.15.26:0/3151795779 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8027070 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.378+0000 7f3804c00700 1 -- 172.21.15.26:0/3151795779 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f8003d70 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.378+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38081a4550 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.378+0000 7f3804c00700 1 -- 172.21.15.26:0/3151795779 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f800f460 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.378+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38081abe30 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.379+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3808075db0 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.381+0000 7f3804c00700 1 -- 172.21.15.26:0/3151795779 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37f802a070 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.382+0000 7f3804c00700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37ec0a2c20 0x7f37ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.382+0000 7f3804c00700 1 -- 172.21.15.26:0/3151795779 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f37f80d0310 con 0x7f38080746c0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.383+0000 7f37ffe00700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37ec0a2c20 0x7f37ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:35.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.384+0000 7f37ffe00700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37ec0a2c20 0x7f37ec0a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f380810feb0 tx=0x7f37f0007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:35.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.387+0000 7f3804c00700 1 -- 172.21.15.26:0/3151795779 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37f805e850 con 0x7f38080746c0 2024-06-13T06:58:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 -- 172.21.15.26:0/234177238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf581046b0 msgr2=0x7fdf58104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 --2- 172.21.15.26:0/234177238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf581046b0 0x7fdf58104b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009990 tx=0x7fdf44009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 -- 172.21.15.26:0/234177238 shutdown_connections 2024-06-13T06:58:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 --2- 172.21.15.26:0/234177238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf5810a6b0 0x7fdf5810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 --2- 172.21.15.26:0/234177238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf58105070 0x7fdf58111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 --2- 172.21.15.26:0/234177238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf581046b0 0x7fdf58104b30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 -- 172.21.15.26:0/234177238 >> 172.21.15.26:0/234177238 conn(0x7fdf581001a0 msgr2=0x7fdf581025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 -- 172.21.15.26:0/234177238 shutdown_connections 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.501+0000 7fdf5e400700 1 -- 172.21.15.26:0/234177238 wait complete. 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.502+0000 7fdf5e400700 1 Processor -- start 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.502+0000 7fdf5e400700 1 -- start start 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf5e400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf581046b0 0x7fdf5819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf5e400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf58105070 0x7fdf5819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf5e400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5810a6b0 0x7fdf5819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf5e400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581149f0 con 0x7fdf5810a6b0 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf5e400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58114870 con 0x7fdf581046b0 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf5e400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58114b70 con 0x7fdf58105070 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf57e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5810a6b0 0x7fdf5819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:35.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf57400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf581046b0 0x7fdf5819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf57e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5810a6b0 0x7fdf5819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50884/0 (socket says 172.21.15.26:50884) 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf57e00700 1 -- 172.21.15.26:0/1318829100 learned_addr learned my addr 172.21.15.26:0/1318829100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.503+0000 7fdf56a00700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf58105070 0x7fdf5819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf57e00700 1 -- 172.21.15.26:0/1318829100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf58105070 msgr2=0x7fdf5819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf57e00700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf58105070 0x7fdf5819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf57e00700 1 -- 172.21.15.26:0/1318829100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf581046b0 msgr2=0x7fdf5819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf57e00700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf581046b0 0x7fdf5819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf57e00700 1 -- 172.21.15.26:0/1318829100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5819fe10 con 0x7fdf5810a6b0 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf57400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf581046b0 0x7fdf5819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf57e00700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5810a6b0 0x7fdf5819f6d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000ef30 tx=0x7fdf5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf4fe00700 1 -- 172.21.15.26:0/1318829100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500092e0 con 0x7fdf5810a6b0 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf4fe00700 1 -- 172.21.15.26:0/1318829100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf50004bb0 con 0x7fdf5810a6b0 2024-06-13T06:58:35.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf581a00a0 con 0x7fdf5810a6b0 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.504+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf581a7980 con 0x7fdf5810a6b0 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.505+0000 7fdf4fe00700 1 -- 172.21.15.26:0/1318829100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50004280 con 0x7fdf5810a6b0 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.506+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf58069180 con 0x7fdf5810a6b0 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.509+0000 7fdf4fe00700 1 -- 172.21.15.26:0/1318829100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf5001b030 con 0x7fdf5810a6b0 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.511+0000 7fdf4fe00700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdf3c0a2c20 0x7fdf3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.511+0000 7fdf4fe00700 1 -- 172.21.15.26:0/1318829100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdf500cf900 con 0x7fdf5810a6b0 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.511+0000 7fdf57400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdf3c0a2c20 0x7fdf3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.512+0000 7fdf57400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdf3c0a2c20 0x7fdf3c0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009680 tx=0x7fdf48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:35.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.514+0000 7fdf4fe00700 1 -- 172.21.15.26:0/1318829100 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf50022020 con 0x7fdf5810a6b0 2024-06-13T06:58:35.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.632+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f38081ac000 con 0x7f38080746c0 2024-06-13T06:58:35.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.633+0000 7f3804c00700 1 -- 172.21.15.26:0/3151795779 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f37f8013070 con 0x7f38080746c0 2024-06-13T06:58:35.633 INFO:teuthology.orchestra.run.smithi026.stdout:197568495705 2024-06-13T06:58:35.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.636+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37ec0a2c20 msgr2=0x7f37ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.636+0000 7f380e200700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37ec0a2c20 0x7f37ec0a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f380810feb0 tx=0x7f37f0007480 comp rx=0 tx=0).stop 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.636+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38080746c0 msgr2=0x7f380819f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.636+0000 7f380e200700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38080746c0 0x7f380819f2b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37f8006010 tx=0x7f37f800bab0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.637+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 shutdown_connections 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.637+0000 7f380e200700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f37ec0a2c20 0x7f37ec0a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.637+0000 7f380e200700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38080746c0 0x7f380819f2b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.637+0000 7f380e200700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808113660 0x7f38081a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.637+0000 7f380e200700 1 --2- 172.21.15.26:0/3151795779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3808074fe0 0x7f380819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.637+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 >> 172.21.15.26:0/3151795779 conn(0x7f38080795f0 msgr2=0x7f3808112430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.637+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 shutdown_connections 2024-06-13T06:58:35.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.637+0000 7f380e200700 1 -- 172.21.15.26:0/3151795779 wait complete. 2024-06-13T06:58:35.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.768+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fdf5802dca0 con 0x7fdf5810a6b0 2024-06-13T06:58:35.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.769+0000 7fdf4fe00700 1 -- 172.21.15.26:0/1318829100 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdf50010040 con 0x7fdf5810a6b0 2024-06-13T06:58:35.769 INFO:teuthology.orchestra.run.smithi026.stdout:343597383741 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdf3c0a2c20 msgr2=0x7fdf3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdf3c0a2c20 0x7fdf3c0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009680 tx=0x7fdf48009260 comp rx=0 tx=0).stop 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5810a6b0 msgr2=0x7fdf5819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5810a6b0 0x7fdf5819f6d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000ef30 tx=0x7fdf5000c3a0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 shutdown_connections 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdf3c0a2c20 0x7fdf3c0a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf581046b0 0x7fdf5819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5810a6b0 0x7fdf5819f6d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 --2- 172.21.15.26:0/1318829100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf58105070 0x7fdf5819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.772+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 >> 172.21.15.26:0/1318829100 conn(0x7fdf581001a0 msgr2=0x7fdf58100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.773+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 shutdown_connections 2024-06-13T06:58:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:35.773+0000 7fdf5e400700 1 -- 172.21.15.26:0/1318829100 wait complete. 2024-06-13T06:58:36.321 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:36 smithi026 ceph-mon[32041]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:58:36.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:36 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3151795779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-13T06:58:36.322 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:36 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1318829100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-13T06:58:36.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:36 smithi136 ceph-mon[37229]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:58:36.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:36 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3151795779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-13T06:58:36.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:36 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1318829100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-13T06:58:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:36 smithi135 ceph-mon[36276]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:58:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:36 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3151795779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-13T06:58:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:36 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1318829100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-13T06:58:37.134 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070534 got 253403070542 for osd.8 2024-06-13T06:58:37.134 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:37.614 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383736 got 343597383741 for osd.11 2024-06-13T06:58:37.614 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:37.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495698 got 197568495705 for osd.6 2024-06-13T06:58:37.738 DEBUG:teuthology.parallel:result is None 2024-06-13T06:58:37.738 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-13T06:58:37.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph pg dump --format=json 2024-06-13T06:58:38.211 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:38 smithi026 ceph-mon[32041]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:58:38.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:38 smithi136 ceph-mon[37229]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:58:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:38 smithi135 ceph-mon[36276]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:58:40.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:40 smithi026 ceph-mon[32041]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:40.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:40 smithi136 ceph-mon[37229]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:40 smithi135 ceph-mon[36276]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:40.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 -- 172.21.15.26:0/1210026853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c0410c9a0 msgr2=0x7f5c0410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:40.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 --2- 172.21.15.26:0/1210026853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c0410c9a0 0x7f5c0410ee60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b0a0 tx=0x7f5bfc00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:58:40.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 -- 172.21.15.26:0/1210026853 shutdown_connections 2024-06-13T06:58:40.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 --2- 172.21.15.26:0/1210026853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c0410c9a0 0x7f5c0410ee60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:40.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 --2- 172.21.15.26:0/1210026853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c04103570 0x7f5c04103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:40.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 --2- 172.21.15.26:0/1210026853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c04103e90 0x7f5c0410c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:40.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 -- 172.21.15.26:0/1210026853 >> 172.21.15.26:0/1210026853 conn(0x7f5c040fcda0 msgr2=0x7f5c040ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:40.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 -- 172.21.15.26:0/1210026853 shutdown_connections 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.792+0000 7f5c0a000700 1 -- 172.21.15.26:0/1210026853 wait complete. 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.793+0000 7f5c0a000700 1 Processor -- start 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.793+0000 7f5c0a000700 1 -- start start 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.793+0000 7f5c0a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c04103570 0x7f5c041011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.793+0000 7f5c0a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c04103e90 0x7f5c04101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.793+0000 7f5c0a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c0410c9a0 0x7f5c04101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.793+0000 7f5c0a000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04112750 con 0x7f5c0410c9a0 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c0a000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c041125d0 con 0x7f5c04103e90 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c0a000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c041128d0 con 0x7f5c04103570 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c08c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c04103570 0x7f5c041011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c09600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c0410c9a0 0x7f5c04101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c08c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c04103570 0x7f5c041011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43432/0 (socket says 172.21.15.26:43432) 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c09600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c0410c9a0 0x7f5c04101c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50904/0 (socket says 172.21.15.26:50904) 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c08c00700 1 -- 172.21.15.26:0/3370806058 learned_addr learned my addr 172.21.15.26:0/3370806058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:40.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5bfbe00700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c04103e90 0x7f5c04101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c08c00700 1 -- 172.21.15.26:0/3370806058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c04103e90 msgr2=0x7f5c04101710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c08c00700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c04103e90 0x7f5c04101710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c08c00700 1 -- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c0410c9a0 msgr2=0x7f5c04101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c08c00700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c0410c9a0 0x7f5c04101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c08c00700 1 -- 172.21.15.26:0/3370806058 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c041a7070 con 0x7f5c04103570 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.794+0000 7f5c09600700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c0410c9a0 0x7f5c04101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.795+0000 7f5c08c00700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c04103570 0x7f5c041011d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400e9c0 tx=0x7f5bf400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.795+0000 7f5c02000700 1 -- 172.21.15.26:0/3370806058 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf400cbd0 con 0x7f5c04103570 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.795+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c041a7360 con 0x7f5c04103570 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.795+0000 7f5c02000700 1 -- 172.21.15.26:0/3370806058 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf400ee60 con 0x7f5c04103570 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.795+0000 7f5c02000700 1 -- 172.21.15.26:0/3370806058 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4005630 con 0x7f5c04103570 2024-06-13T06:58:40.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.795+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c041a78a0 con 0x7f5c04103570 2024-06-13T06:58:40.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.796+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c04069180 con 0x7f5c04103570 2024-06-13T06:58:40.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.798+0000 7f5c02000700 1 -- 172.21.15.26:0/3370806058 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5bf4004750 con 0x7f5c04103570 2024-06-13T06:58:40.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.800+0000 7f5c02000700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5be80a2ca0 0x7f5be80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:40.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.800+0000 7f5c02000700 1 -- 172.21.15.26:0/3370806058 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5bf40cf6a0 con 0x7f5c04103570 2024-06-13T06:58:40.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.800+0000 7f5bfbe00700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5be80a2ca0 0x7f5be80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:40.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.801+0000 7f5bfbe00700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5be80a2ca0 0x7f5be80a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c040ff820 tx=0x7f5bec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:40.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:40.804+0000 7f5c02000700 1 -- 172.21.15.26:0/3370806058 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5bf409c1b0 con 0x7f5c04103570 2024-06-13T06:58:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.030+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c040fea20 con 0x7f5be80a2ca0 2024-06-13T06:58:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.061+0000 7f5c02000700 1 -- 172.21.15.26:0/3370806058 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340497 (secure 0 0 0) 0x7f5c040fea20 con 0x7f5be80a2ca0 2024-06-13T06:58:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:58:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5be80a2ca0 msgr2=0x7f5be80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5be80a2ca0 0x7f5be80a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c040ff820 tx=0x7f5bec009260 comp rx=0 tx=0).stop 2024-06-13T06:58:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c04103570 msgr2=0x7f5c041011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c04103570 0x7f5c041011d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400e9c0 tx=0x7f5bf400ecd0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 shutdown_connections 2024-06-13T06:58:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5be80a2ca0 0x7f5be80a5160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c0410c9a0 0x7f5c04101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c04103570 0x7f5c041011d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 --2- 172.21.15.26:0/3370806058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c04103e90 0x7f5c04101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 >> 172.21.15.26:0/3370806058 conn(0x7f5c040fcda0 msgr2=0x7f5c0410ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 shutdown_connections 2024-06-13T06:58:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:41.069+0000 7f5c0a000700 1 -- 172.21.15.26:0/3370806058 wait complete. 2024-06-13T06:58:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-06-13T06:58:42.095 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-06-13T06:58:40.370274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657647,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":908,"num_read_kb":573,"num_write":357,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182392,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402032,"kb_used_data":15440,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539792,"statfs":{"total":1151940427776,"available":1151528747008,"internally_reserved":0,"allocated":15810560,"data_stored":8789567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":42,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005887"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293106+0000","last_change":"2024-06-13T06:54:58.919669+0000","last_active":"2024-06-13T06:58:28.293106+0000","last_peered":"2024-06-13T06:58:28.293106+0000","last_clean":"2024-06-13T06:58:28.293106+0000","last_became_active":"2024-06-13T06:54:58.919334+0000","last_became_peered":"2024-06-13T06:54:58.919334+0000","last_unstale":"2024-06-13T06:58:28.293106+0000","last_undegraded":"2024-06-13T06:58:28.293106+0000","last_fullsized":"2024-06-13T06:58:28.293106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:45:40.831640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295026+0000","last_change":"2024-06-13T06:54:58.909508+0000","last_active":"2024-06-13T06:58:28.295026+0000","last_peered":"2024-06-13T06:58:28.295026+0000","last_clean":"2024-06-13T06:58:28.295026+0000","last_became_active":"2024-06-13T06:54:58.909221+0000","last_became_peered":"2024-06-13T06:54:58.909221+0000","last_unstale":"2024-06-13T06:58:28.295026+0000","last_undegraded":"2024-06-13T06:58:28.295026+0000","last_fullsized":"2024-06-13T06:58:28.295026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:17:07.476858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.303909+0000","last_change":"2024-06-13T06:54:58.909283+0000","last_active":"2024-06-13T06:58:28.303909+0000","last_peered":"2024-06-13T06:58:28.303909+0000","last_clean":"2024-06-13T06:58:28.303909+0000","last_became_active":"2024-06-13T06:54:58.908912+0000","last_became_peered":"2024-06-13T06:54:58.908912+0000","last_unstale":"2024-06-13T06:58:28.303909+0000","last_undegraded":"2024-06-13T06:58:28.303909+0000","last_fullsized":"2024-06-13T06:58:28.303909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:48:26.937971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.262650+0000","last_change":"2024-06-13T06:54:58.902821+0000","last_active":"2024-06-13T06:58:28.262650+0000","last_peered":"2024-06-13T06:58:28.262650+0000","last_clean":"2024-06-13T06:58:28.262650+0000","last_became_active":"2024-06-13T06:54:58.902215+0000","last_became_peered":"2024-06-13T06:54:58.902215+0000","last_unstale":"2024-06-13T06:58:28.262650+0000","last_undegraded":"2024-06-13T06:58:28.262650+0000","last_fullsized":"2024-06-13T06:58:28.262650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:11:57.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296512+0000","last_change":"2024-06-13T06:54:58.897038+0000","last_active":"2024-06-13T06:58:28.296512+0000","last_peered":"2024-06-13T06:58:28.296512+0000","last_clean":"2024-06-13T06:58:28.296512+0000","last_became_active":"2024-06-13T06:54:58.896665+0000","last_became_peered":"2024-06-13T06:54:58.896665+0000","last_unstale":"2024-06-13T06:58:28.296512+0000","last_undegraded":"2024-06-13T06:58:28.296512+0000","last_fullsized":"2024-06-13T06:58:28.296512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:24:19.216932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267357+0000","last_change":"2024-06-13T06:54:58.895501+0000","last_active":"2024-06-13T06:58:28.267357+0000","last_peered":"2024-06-13T06:58:28.267357+0000","last_clean":"2024-06-13T06:58:28.267357+0000","last_became_active":"2024-06-13T06:54:58.895144+0000","last_became_peered":"2024-06-13T06:54:58.895144+0000","last_unstale":"2024-06-13T06:58:28.267357+0000","last_undegraded":"2024-06-13T06:58:28.267357+0000","last_fullsized":"2024-06-13T06:58:28.267357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:50.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272231+0000","last_change":"2024-06-13T06:54:58.901097+0000","last_active":"2024-06-13T06:58:28.272231+0000","last_peered":"2024-06-13T06:58:28.272231+0000","last_clean":"2024-06-13T06:58:28.272231+0000","last_became_active":"2024-06-13T06:54:58.900727+0000","last_became_peered":"2024-06-13T06:54:58.900727+0000","last_unstale":"2024-06-13T06:58:28.272231+0000","last_undegraded":"2024-06-13T06:58:28.272231+0000","last_fullsized":"2024-06-13T06:58:28.272231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:09:38.349517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278377+0000","last_change":"2024-06-13T06:54:58.910740+0000","last_active":"2024-06-13T06:58:28.278377+0000","last_peered":"2024-06-13T06:58:28.278377+0000","last_clean":"2024-06-13T06:58:28.278377+0000","last_became_active":"2024-06-13T06:54:58.910351+0000","last_became_peered":"2024-06-13T06:54:58.910351+0000","last_unstale":"2024-06-13T06:58:28.278377+0000","last_undegraded":"2024-06-13T06:58:28.278377+0000","last_fullsized":"2024-06-13T06:58:28.278377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:45:12.854996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281724+0000","last_change":"2024-06-13T06:55:29.364516+0000","last_active":"2024-06-13T06:58:28.281724+0000","last_peered":"2024-06-13T06:58:28.281724+0000","last_clean":"2024-06-13T06:58:28.281724+0000","last_became_active":"2024-06-13T06:55:29.364079+0000","last_became_peered":"2024-06-13T06:55:29.364079+0000","last_unstale":"2024-06-13T06:58:28.281724+0000","last_undegraded":"2024-06-13T06:58:28.281724+0000","last_fullsized":"2024-06-13T06:58:28.281724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:51:28.624359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.266852+0000","last_change":"2024-06-13T06:54:58.903882+0000","last_active":"2024-06-13T06:58:28.266852+0000","last_peered":"2024-06-13T06:58:28.266852+0000","last_clean":"2024-06-13T06:58:28.266852+0000","last_became_active":"2024-06-13T06:54:58.903495+0000","last_became_peered":"2024-06-13T06:54:58.903495+0000","last_unstale":"2024-06-13T06:58:28.266852+0000","last_undegraded":"2024-06-13T06:58:28.266852+0000","last_fullsized":"2024-06-13T06:58:28.266852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:23.682993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281281+0000","last_change":"2024-06-13T06:54:58.900080+0000","last_active":"2024-06-13T06:58:28.281281+0000","last_peered":"2024-06-13T06:58:28.281281+0000","last_clean":"2024-06-13T06:58:28.281281+0000","last_became_active":"2024-06-13T06:54:58.899736+0000","last_became_peered":"2024-06-13T06:54:58.899736+0000","last_unstale":"2024-06-13T06:58:28.281281+0000","last_undegraded":"2024-06-13T06:58:28.281281+0000","last_fullsized":"2024-06-13T06:58:28.281281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:51:45.808821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.261193+0000","last_change":"2024-06-13T06:54:58.900238+0000","last_active":"2024-06-13T06:58:28.261193+0000","last_peered":"2024-06-13T06:58:28.261193+0000","last_clean":"2024-06-13T06:58:28.261193+0000","last_became_active":"2024-06-13T06:54:58.899872+0000","last_became_peered":"2024-06-13T06:54:58.899872+0000","last_unstale":"2024-06-13T06:58:28.261193+0000","last_undegraded":"2024-06-13T06:58:28.261193+0000","last_fullsized":"2024-06-13T06:58:28.261193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:12:40.780894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296753+0000","last_change":"2024-06-13T06:54:58.901981+0000","last_active":"2024-06-13T06:58:28.296753+0000","last_peered":"2024-06-13T06:58:28.296753+0000","last_clean":"2024-06-13T06:58:28.296753+0000","last_became_active":"2024-06-13T06:54:58.898704+0000","last_became_peered":"2024-06-13T06:54:58.898704+0000","last_unstale":"2024-06-13T06:58:28.296753+0000","last_undegraded":"2024-06-13T06:58:28.296753+0000","last_fullsized":"2024-06-13T06:58:28.296753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:03:56.658091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.299012+0000","last_change":"2024-06-13T06:54:58.909022+0000","last_active":"2024-06-13T06:58:28.299012+0000","last_peered":"2024-06-13T06:58:28.299012+0000","last_clean":"2024-06-13T06:58:28.299012+0000","last_became_active":"2024-06-13T06:54:58.907438+0000","last_became_peered":"2024-06-13T06:54:58.907438+0000","last_unstale":"2024-06-13T06:58:28.299012+0000","last_undegraded":"2024-06-13T06:58:28.299012+0000","last_fullsized":"2024-06-13T06:58:28.299012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:40:34.190510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305315+0000","last_change":"2024-06-13T06:54:58.897691+0000","last_active":"2024-06-13T06:58:28.305315+0000","last_peered":"2024-06-13T06:58:28.305315+0000","last_clean":"2024-06-13T06:58:28.305315+0000","last_became_active":"2024-06-13T06:54:58.897381+0000","last_became_peered":"2024-06-13T06:54:58.897381+0000","last_unstale":"2024-06-13T06:58:28.305315+0000","last_undegraded":"2024-06-13T06:58:28.305315+0000","last_fullsized":"2024-06-13T06:58:28.305315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:10.052052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295084+0000","last_change":"2024-06-13T06:54:58.907676+0000","last_active":"2024-06-13T06:58:28.295084+0000","last_peered":"2024-06-13T06:58:28.295084+0000","last_clean":"2024-06-13T06:58:28.295084+0000","last_became_active":"2024-06-13T06:54:58.907304+0000","last_became_peered":"2024-06-13T06:54:58.907304+0000","last_unstale":"2024-06-13T06:58:28.295084+0000","last_undegraded":"2024-06-13T06:58:28.295084+0000","last_fullsized":"2024-06-13T06:58:28.295084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:08:20.292516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293255+0000","last_change":"2024-06-13T06:54:58.920012+0000","last_active":"2024-06-13T06:58:28.293255+0000","last_peered":"2024-06-13T06:58:28.293255+0000","last_clean":"2024-06-13T06:58:28.293255+0000","last_became_active":"2024-06-13T06:54:58.919473+0000","last_became_peered":"2024-06-13T06:54:58.919473+0000","last_unstale":"2024-06-13T06:58:28.293255+0000","last_undegraded":"2024-06-13T06:58:28.293255+0000","last_fullsized":"2024-06-13T06:58:28.293255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:49:13.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.280038+0000","last_change":"2024-06-13T06:54:58.902576+0000","last_active":"2024-06-13T06:58:28.280038+0000","last_peered":"2024-06-13T06:58:28.280038+0000","last_clean":"2024-06-13T06:58:28.280038+0000","last_became_active":"2024-06-13T06:54:58.902205+0000","last_became_peered":"2024-06-13T06:54:58.902205+0000","last_unstale":"2024-06-13T06:58:28.280038+0000","last_undegraded":"2024-06-13T06:58:28.280038+0000","last_fullsized":"2024-06-13T06:58:28.280038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:13:24.107081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292246+0000","last_change":"2024-06-13T06:54:58.905392+0000","last_active":"2024-06-13T06:58:28.292246+0000","last_peered":"2024-06-13T06:58:28.292246+0000","last_clean":"2024-06-13T06:58:28.292246+0000","last_became_active":"2024-06-13T06:54:58.905017+0000","last_became_peered":"2024-06-13T06:54:58.905017+0000","last_unstale":"2024-06-13T06:58:28.292246+0000","last_undegraded":"2024-06-13T06:58:28.292246+0000","last_fullsized":"2024-06-13T06:58:28.292246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:05.931653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606898+0000","last_change":"2024-06-13T06:55:12.021359+0000","last_active":"2024-06-13T06:56:29.606898+0000","last_peered":"2024-06-13T06:56:29.606898+0000","last_clean":"2024-06-13T06:56:29.606898+0000","last_became_active":"2024-06-13T06:55:12.020860+0000","last_became_peered":"2024-06-13T06:55:12.020860+0000","last_unstale":"2024-06-13T06:56:29.606898+0000","last_undegraded":"2024-06-13T06:56:29.606898+0000","last_fullsized":"2024-06-13T06:56:29.606898+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:05:00.833905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.561282+0000","last_change":"2024-06-13T06:54:54.339220+0000","last_active":"2024-06-13T06:56:32.561282+0000","last_peered":"2024-06-13T06:56:32.561282+0000","last_clean":"2024-06-13T06:56:32.561282+0000","last_became_active":"2024-06-13T06:54:54.338573+0000","last_became_peered":"2024-06-13T06:54:54.338573+0000","last_unstale":"2024-06-13T06:56:32.561282+0000","last_undegraded":"2024-06-13T06:56:32.561282+0000","last_fullsized":"2024-06-13T06:56:32.561282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:53:50.258097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.277033+0000","last_change":"2024-06-13T06:54:58.903920+0000","last_active":"2024-06-13T06:58:28.277033+0000","last_peered":"2024-06-13T06:58:28.277033+0000","last_clean":"2024-06-13T06:58:28.277033+0000","last_became_active":"2024-06-13T06:54:58.895364+0000","last_became_peered":"2024-06-13T06:54:58.895364+0000","last_unstale":"2024-06-13T06:58:28.277033+0000","last_undegraded":"2024-06-13T06:58:28.277033+0000","last_fullsized":"2024-06-13T06:58:28.277033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:04:35.109799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607026+0000","last_change":"2024-06-13T06:55:12.017561+0000","last_active":"2024-06-13T06:56:29.607026+0000","last_peered":"2024-06-13T06:56:29.607026+0000","last_clean":"2024-06-13T06:56:29.607026+0000","last_became_active":"2024-06-13T06:55:12.017104+0000","last_became_peered":"2024-06-13T06:55:12.017104+0000","last_unstale":"2024-06-13T06:56:29.607026+0000","last_undegraded":"2024-06-13T06:56:29.607026+0000","last_fullsized":"2024-06-13T06:56:29.607026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:08:22.570491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526871+0000","last_change":"2024-06-13T06:54:54.631599+0000","last_active":"2024-06-13T06:56:29.526871+0000","last_peered":"2024-06-13T06:56:29.526871+0000","last_clean":"2024-06-13T06:56:29.526871+0000","last_became_active":"2024-06-13T06:54:54.631007+0000","last_became_peered":"2024-06-13T06:54:54.631007+0000","last_unstale":"2024-06-13T06:56:29.526871+0000","last_undegraded":"2024-06-13T06:56:29.526871+0000","last_fullsized":"2024-06-13T06:56:29.526871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:49:56.602192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304403+0000","last_change":"2024-06-13T06:54:58.917255+0000","last_active":"2024-06-13T06:58:28.304403+0000","last_peered":"2024-06-13T06:58:28.304403+0000","last_clean":"2024-06-13T06:58:28.304403+0000","last_became_active":"2024-06-13T06:54:58.916875+0000","last_became_peered":"2024-06-13T06:54:58.916875+0000","last_unstale":"2024-06-13T06:58:28.304403+0000","last_undegraded":"2024-06-13T06:58:28.304403+0000","last_fullsized":"2024-06-13T06:58:28.304403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:09:47.638802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608521+0000","last_change":"2024-06-13T06:55:12.018209+0000","last_active":"2024-06-13T06:56:29.608521+0000","last_peered":"2024-06-13T06:56:29.608521+0000","last_clean":"2024-06-13T06:56:29.608521+0000","last_became_active":"2024-06-13T06:55:12.017038+0000","last_became_peered":"2024-06-13T06:55:12.017038+0000","last_unstale":"2024-06-13T06:56:29.608521+0000","last_undegraded":"2024-06-13T06:56:29.608521+0000","last_fullsized":"2024-06-13T06:56:29.608521+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:36:26.476454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522812+0000","last_change":"2024-06-13T06:54:54.328849+0000","last_active":"2024-06-13T06:56:29.522812+0000","last_peered":"2024-06-13T06:56:29.522812+0000","last_clean":"2024-06-13T06:56:29.522812+0000","last_became_active":"2024-06-13T06:54:54.328388+0000","last_became_peered":"2024-06-13T06:54:54.328388+0000","last_unstale":"2024-06-13T06:56:29.522812+0000","last_undegraded":"2024-06-13T06:56:29.522812+0000","last_fullsized":"2024-06-13T06:56:29.522812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:41.264403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282933+0000","last_change":"2024-06-13T06:54:58.897588+0000","last_active":"2024-06-13T06:58:28.282933+0000","last_peered":"2024-06-13T06:58:28.282933+0000","last_clean":"2024-06-13T06:58:28.282933+0000","last_became_active":"2024-06-13T06:54:58.897236+0000","last_became_peered":"2024-06-13T06:54:58.897236+0000","last_unstale":"2024-06-13T06:58:28.282933+0000","last_undegraded":"2024-06-13T06:58:28.282933+0000","last_fullsized":"2024-06-13T06:58:28.282933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:20:38.460445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607292+0000","last_change":"2024-06-13T06:55:12.014080+0000","last_active":"2024-06-13T06:56:29.607292+0000","last_peered":"2024-06-13T06:56:29.607292+0000","last_clean":"2024-06-13T06:56:29.607292+0000","last_became_active":"2024-06-13T06:55:12.013627+0000","last_became_peered":"2024-06-13T06:55:12.013627+0000","last_unstale":"2024-06-13T06:56:29.607292+0000","last_undegraded":"2024-06-13T06:56:29.607292+0000","last_fullsized":"2024-06-13T06:56:29.607292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:29:54.081593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522259+0000","last_change":"2024-06-13T06:54:54.328249+0000","last_active":"2024-06-13T06:56:29.522259+0000","last_peered":"2024-06-13T06:56:29.522259+0000","last_clean":"2024-06-13T06:56:29.522259+0000","last_became_active":"2024-06-13T06:54:54.327832+0000","last_became_peered":"2024-06-13T06:54:54.327832+0000","last_unstale":"2024-06-13T06:56:29.522259+0000","last_undegraded":"2024-06-13T06:56:29.522259+0000","last_fullsized":"2024-06-13T06:56:29.522259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:21:05.624075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281353+0000","last_change":"2024-06-13T06:55:29.383305+0000","last_active":"2024-06-13T06:58:28.281353+0000","last_peered":"2024-06-13T06:58:28.281353+0000","last_clean":"2024-06-13T06:58:28.281353+0000","last_became_active":"2024-06-13T06:55:29.381491+0000","last_became_peered":"2024-06-13T06:55:29.381491+0000","last_unstale":"2024-06-13T06:58:28.281353+0000","last_undegraded":"2024-06-13T06:58:28.281353+0000","last_fullsized":"2024-06-13T06:58:28.281353+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:52:26.437237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607257+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:56:29.607257+0000","last_peered":"2024-06-13T06:56:29.607257+0000","last_clean":"2024-06-13T06:56:29.607257+0000","last_became_active":"2024-06-13T06:55:12.000759+0000","last_became_peered":"2024-06-13T06:55:12.000759+0000","last_unstale":"2024-06-13T06:56:29.607257+0000","last_undegraded":"2024-06-13T06:56:29.607257+0000","last_fullsized":"2024-06-13T06:56:29.607257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:11:35.805338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'7","reported_seq":186,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:34.805344+0000","last_change":"2024-06-13T06:55:29.367538+0000","last_active":"2024-06-13T06:58:34.805344+0000","last_peered":"2024-06-13T06:58:34.805344+0000","last_clean":"2024-06-13T06:58:34.805344+0000","last_became_active":"2024-06-13T06:55:29.367149+0000","last_became_peered":"2024-06-13T06:55:29.367149+0000","last_unstale":"2024-06-13T06:58:34.805344+0000","last_undegraded":"2024-06-13T06:58:34.805344+0000","last_fullsized":"2024-06-13T06:58:34.805344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:30:55.632799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.268814+0000","last_change":"2024-06-13T06:54:58.896258+0000","last_active":"2024-06-13T06:58:28.268814+0000","last_peered":"2024-06-13T06:58:28.268814+0000","last_clean":"2024-06-13T06:58:28.268814+0000","last_became_active":"2024-06-13T06:54:58.895904+0000","last_became_peered":"2024-06-13T06:54:58.895904+0000","last_unstale":"2024-06-13T06:58:28.268814+0000","last_undegraded":"2024-06-13T06:58:28.268814+0000","last_fullsized":"2024-06-13T06:58:28.268814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:28:33.615338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607271+0000","last_change":"2024-06-13T06:55:12.018499+0000","last_active":"2024-06-13T06:56:29.607271+0000","last_peered":"2024-06-13T06:56:29.607271+0000","last_clean":"2024-06-13T06:56:29.607271+0000","last_became_active":"2024-06-13T06:55:12.018073+0000","last_became_peered":"2024-06-13T06:55:12.018073+0000","last_unstale":"2024-06-13T06:56:29.607271+0000","last_undegraded":"2024-06-13T06:56:29.607271+0000","last_fullsized":"2024-06-13T06:56:29.607271+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:59:53.206259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523149+0000","last_change":"2024-06-13T06:54:54.339482+0000","last_active":"2024-06-13T06:56:29.523149+0000","last_peered":"2024-06-13T06:56:29.523149+0000","last_clean":"2024-06-13T06:56:29.523149+0000","last_became_active":"2024-06-13T06:54:54.339066+0000","last_became_peered":"2024-06-13T06:54:54.339066+0000","last_unstale":"2024-06-13T06:56:29.523149+0000","last_undegraded":"2024-06-13T06:56:29.523149+0000","last_fullsized":"2024-06-13T06:56:29.523149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:31:04.136530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279127+0000","last_change":"2024-06-13T06:54:58.891154+0000","last_active":"2024-06-13T06:58:28.279127+0000","last_peered":"2024-06-13T06:58:28.279127+0000","last_clean":"2024-06-13T06:58:28.279127+0000","last_became_active":"2024-06-13T06:54:58.890810+0000","last_became_peered":"2024-06-13T06:54:58.890810+0000","last_unstale":"2024-06-13T06:58:28.279127+0000","last_undegraded":"2024-06-13T06:58:28.279127+0000","last_fullsized":"2024-06-13T06:58:28.279127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:14.618233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524803+0000","last_change":"2024-06-13T06:55:12.025619+0000","last_active":"2024-06-13T06:56:29.524803+0000","last_peered":"2024-06-13T06:56:29.524803+0000","last_clean":"2024-06-13T06:56:29.524803+0000","last_became_active":"2024-06-13T06:55:12.025153+0000","last_became_peered":"2024-06-13T06:55:12.025153+0000","last_unstale":"2024-06-13T06:56:29.524803+0000","last_undegraded":"2024-06-13T06:56:29.524803+0000","last_fullsized":"2024-06-13T06:56:29.524803+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:12.124550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.596897+0000","last_change":"2024-06-13T06:54:54.341698+0000","last_active":"2024-06-13T06:56:29.596897+0000","last_peered":"2024-06-13T06:56:29.596897+0000","last_clean":"2024-06-13T06:56:29.596897+0000","last_became_active":"2024-06-13T06:54:54.341113+0000","last_became_peered":"2024-06-13T06:54:54.341113+0000","last_unstale":"2024-06-13T06:56:29.596897+0000","last_undegraded":"2024-06-13T06:56:29.596897+0000","last_fullsized":"2024-06-13T06:56:29.596897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:18:23.368705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279404+0000","last_change":"2024-06-13T06:54:58.913688+0000","last_active":"2024-06-13T06:58:28.279404+0000","last_peered":"2024-06-13T06:58:28.279404+0000","last_clean":"2024-06-13T06:58:28.279404+0000","last_became_active":"2024-06-13T06:54:58.910493+0000","last_became_peered":"2024-06-13T06:54:58.910493+0000","last_unstale":"2024-06-13T06:58:28.279404+0000","last_undegraded":"2024-06-13T06:58:28.279404+0000","last_fullsized":"2024-06-13T06:58:28.279404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:29:18.081743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532034+0000","last_change":"2024-06-13T06:55:12.016257+0000","last_active":"2024-06-13T06:56:29.532034+0000","last_peered":"2024-06-13T06:56:29.532034+0000","last_clean":"2024-06-13T06:56:29.532034+0000","last_became_active":"2024-06-13T06:55:12.015594+0000","last_became_peered":"2024-06-13T06:55:12.015594+0000","last_unstale":"2024-06-13T06:56:29.532034+0000","last_undegraded":"2024-06-13T06:56:29.532034+0000","last_fullsized":"2024-06-13T06:56:29.532034+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:02:44.642075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525725+0000","last_change":"2024-06-13T06:54:54.330366+0000","last_active":"2024-06-13T06:56:29.525725+0000","last_peered":"2024-06-13T06:56:29.525725+0000","last_clean":"2024-06-13T06:56:29.525725+0000","last_became_active":"2024-06-13T06:54:54.330010+0000","last_became_peered":"2024-06-13T06:54:54.330010+0000","last_unstale":"2024-06-13T06:56:29.525725+0000","last_undegraded":"2024-06-13T06:56:29.525725+0000","last_fullsized":"2024-06-13T06:56:29.525725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:36:01.488766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276429+0000","last_change":"2024-06-13T06:54:58.918076+0000","last_active":"2024-06-13T06:58:28.276429+0000","last_peered":"2024-06-13T06:58:28.276429+0000","last_clean":"2024-06-13T06:58:28.276429+0000","last_became_active":"2024-06-13T06:54:58.917631+0000","last_became_peered":"2024-06-13T06:54:58.917631+0000","last_unstale":"2024-06-13T06:58:28.276429+0000","last_undegraded":"2024-06-13T06:58:28.276429+0000","last_fullsized":"2024-06-13T06:58:28.276429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:31:00.056837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.613299+0000","last_change":"2024-06-13T06:56:29.611761+0000","last_active":"2024-06-13T06:56:29.613299+0000","last_peered":"2024-06-13T06:56:29.613299+0000","last_clean":"2024-06-13T06:56:29.613299+0000","last_became_active":"2024-06-13T06:56:29.610385+0000","last_became_peered":"2024-06-13T06:56:29.610385+0000","last_unstale":"2024-06-13T06:56:29.613299+0000","last_undegraded":"2024-06-13T06:56:29.613299+0000","last_fullsized":"2024-06-13T06:56:29.613299+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:26:10.678918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":175,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:36.808301+0000","last_change":"2024-06-13T06:54:54.335324+0000","last_active":"2024-06-13T06:58:36.808301+0000","last_peered":"2024-06-13T06:58:36.808301+0000","last_clean":"2024-06-13T06:58:36.808301+0000","last_became_active":"2024-06-13T06:54:54.334936+0000","last_became_peered":"2024-06-13T06:54:54.334936+0000","last_unstale":"2024-06-13T06:58:36.808301+0000","last_undegraded":"2024-06-13T06:58:36.808301+0000","last_fullsized":"2024-06-13T06:58:36.808301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:41:52.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281052+0000","last_change":"2024-06-13T06:54:58.902146+0000","last_active":"2024-06-13T06:58:28.281052+0000","last_peered":"2024-06-13T06:58:28.281052+0000","last_clean":"2024-06-13T06:58:28.281052+0000","last_became_active":"2024-06-13T06:54:58.901807+0000","last_became_peered":"2024-06-13T06:54:58.901807+0000","last_unstale":"2024-06-13T06:58:28.281052+0000","last_undegraded":"2024-06-13T06:58:28.281052+0000","last_fullsized":"2024-06-13T06:58:28.281052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:59:53.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533203+0000","last_change":"2024-06-13T06:55:12.025418+0000","last_active":"2024-06-13T06:56:29.533203+0000","last_peered":"2024-06-13T06:56:29.533203+0000","last_clean":"2024-06-13T06:56:29.533203+0000","last_became_active":"2024-06-13T06:55:12.024859+0000","last_became_peered":"2024-06-13T06:55:12.024859+0000","last_unstale":"2024-06-13T06:56:29.533203+0000","last_undegraded":"2024-06-13T06:56:29.533203+0000","last_fullsized":"2024-06-13T06:56:29.533203+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:05:17.553169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609432+0000","last_change":"2024-06-13T06:54:54.326011+0000","last_active":"2024-06-13T06:56:29.609432+0000","last_peered":"2024-06-13T06:56:29.609432+0000","last_clean":"2024-06-13T06:56:29.609432+0000","last_became_active":"2024-06-13T06:54:54.325664+0000","last_became_peered":"2024-06-13T06:54:54.325664+0000","last_unstale":"2024-06-13T06:56:29.609432+0000","last_undegraded":"2024-06-13T06:56:29.609432+0000","last_fullsized":"2024-06-13T06:56:29.609432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:32.973094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301619+0000","last_change":"2024-06-13T06:55:29.383612+0000","last_active":"2024-06-13T06:58:28.301619+0000","last_peered":"2024-06-13T06:58:28.301619+0000","last_clean":"2024-06-13T06:58:28.301619+0000","last_became_active":"2024-06-13T06:55:29.382174+0000","last_became_peered":"2024-06-13T06:55:29.382174+0000","last_unstale":"2024-06-13T06:58:28.301619+0000","last_undegraded":"2024-06-13T06:58:28.301619+0000","last_fullsized":"2024-06-13T06:58:28.301619+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:09:35.178209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608743+0000","last_change":"2024-06-13T06:55:12.018654+0000","last_active":"2024-06-13T06:56:29.608743+0000","last_peered":"2024-06-13T06:56:29.608743+0000","last_clean":"2024-06-13T06:56:29.608743+0000","last_became_active":"2024-06-13T06:55:12.017219+0000","last_became_peered":"2024-06-13T06:55:12.017219+0000","last_unstale":"2024-06-13T06:56:29.608743+0000","last_undegraded":"2024-06-13T06:56:29.608743+0000","last_fullsized":"2024-06-13T06:56:29.608743+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:46:54.161912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.562063+0000","last_change":"2024-06-13T06:54:54.632154+0000","last_active":"2024-06-13T06:56:32.562063+0000","last_peered":"2024-06-13T06:56:32.562063+0000","last_clean":"2024-06-13T06:56:32.562063+0000","last_became_active":"2024-06-13T06:54:54.631721+0000","last_became_peered":"2024-06-13T06:54:54.631721+0000","last_unstale":"2024-06-13T06:56:32.562063+0000","last_undegraded":"2024-06-13T06:56:32.562063+0000","last_fullsized":"2024-06-13T06:56:32.562063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:16:10.492144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278120+0000","last_change":"2024-06-13T06:54:58.898961+0000","last_active":"2024-06-13T06:58:28.278120+0000","last_peered":"2024-06-13T06:58:28.278120+0000","last_clean":"2024-06-13T06:58:28.278120+0000","last_became_active":"2024-06-13T06:54:58.898599+0000","last_became_peered":"2024-06-13T06:54:58.898599+0000","last_unstale":"2024-06-13T06:58:28.278120+0000","last_undegraded":"2024-06-13T06:58:28.278120+0000","last_fullsized":"2024-06-13T06:58:28.278120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:21:53.044205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.527672+0000","last_change":"2024-06-13T06:56:29.527242+0000","last_active":"2024-06-13T06:56:29.527672+0000","last_peered":"2024-06-13T06:56:29.527672+0000","last_clean":"2024-06-13T06:56:29.527672+0000","last_became_active":"2024-06-13T06:56:29.526816+0000","last_became_peered":"2024-06-13T06:56:29.526816+0000","last_unstale":"2024-06-13T06:56:29.527672+0000","last_undegraded":"2024-06-13T06:56:29.527672+0000","last_fullsized":"2024-06-13T06:56:29.527672+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:45:17.770029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'36","reported_seq":358,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:37.765144+0000","last_change":"2024-06-13T06:55:29.376808+0000","last_active":"2024-06-13T06:58:37.765144+0000","last_peered":"2024-06-13T06:58:37.765144+0000","last_clean":"2024-06-13T06:58:37.765144+0000","last_became_active":"2024-06-13T06:55:29.367134+0000","last_became_peered":"2024-06-13T06:55:29.367134+0000","last_unstale":"2024-06-13T06:58:37.765144+0000","last_undegraded":"2024-06-13T06:58:37.765144+0000","last_fullsized":"2024-06-13T06:58:37.765144+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:19:08.784438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62904,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":261,"num_read_kb":193,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.265076+0000","last_change":"2024-06-13T06:54:58.897686+0000","last_active":"2024-06-13T06:58:28.265076+0000","last_peered":"2024-06-13T06:58:28.265076+0000","last_clean":"2024-06-13T06:58:28.265076+0000","last_became_active":"2024-06-13T06:54:58.897313+0000","last_became_peered":"2024-06-13T06:54:58.897313+0000","last_unstale":"2024-06-13T06:58:28.265076+0000","last_undegraded":"2024-06-13T06:58:28.265076+0000","last_fullsized":"2024-06-13T06:58:28.265076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:54:49.558542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526786+0000","last_change":"2024-06-13T06:55:12.013918+0000","last_active":"2024-06-13T06:56:29.526786+0000","last_peered":"2024-06-13T06:56:29.526786+0000","last_clean":"2024-06-13T06:56:29.526786+0000","last_became_active":"2024-06-13T06:55:12.013522+0000","last_became_peered":"2024-06-13T06:55:12.013522+0000","last_unstale":"2024-06-13T06:56:29.526786+0000","last_undegraded":"2024-06-13T06:56:29.526786+0000","last_fullsized":"2024-06-13T06:56:29.526786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:51:46.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556331+0000","last_change":"2024-06-13T06:54:54.328674+0000","last_active":"2024-06-13T06:56:32.556331+0000","last_peered":"2024-06-13T06:56:32.556331+0000","last_clean":"2024-06-13T06:56:32.556331+0000","last_became_active":"2024-06-13T06:54:54.326789+0000","last_became_peered":"2024-06-13T06:54:54.326789+0000","last_unstale":"2024-06-13T06:56:32.556331+0000","last_undegraded":"2024-06-13T06:56:32.556331+0000","last_fullsized":"2024-06-13T06:56:32.556331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:18.280514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.290908+0000","last_change":"2024-06-13T06:54:58.918598+0000","last_active":"2024-06-13T06:58:28.290908+0000","last_peered":"2024-06-13T06:58:28.290908+0000","last_clean":"2024-06-13T06:58:28.290908+0000","last_became_active":"2024-06-13T06:54:58.918247+0000","last_became_peered":"2024-06-13T06:54:58.918247+0000","last_unstale":"2024-06-13T06:58:28.290908+0000","last_undegraded":"2024-06-13T06:58:28.290908+0000","last_fullsized":"2024-06-13T06:58:28.290908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:08:56.738806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524314+0000","last_change":"2024-06-13T06:55:12.017074+0000","last_active":"2024-06-13T06:56:29.524314+0000","last_peered":"2024-06-13T06:56:29.524314+0000","last_clean":"2024-06-13T06:56:29.524314+0000","last_became_active":"2024-06-13T06:55:12.016675+0000","last_became_peered":"2024-06-13T06:55:12.016675+0000","last_unstale":"2024-06-13T06:56:29.524314+0000","last_undegraded":"2024-06-13T06:56:29.524314+0000","last_fullsized":"2024-06-13T06:56:29.524314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:20:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525973+0000","last_change":"2024-06-13T06:54:54.334994+0000","last_active":"2024-06-13T06:56:29.525973+0000","last_peered":"2024-06-13T06:56:29.525973+0000","last_clean":"2024-06-13T06:56:29.525973+0000","last_became_active":"2024-06-13T06:54:54.334647+0000","last_became_peered":"2024-06-13T06:54:54.334647+0000","last_unstale":"2024-06-13T06:56:29.525973+0000","last_undegraded":"2024-06-13T06:56:29.525973+0000","last_fullsized":"2024-06-13T06:56:29.525973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:59:15.317976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275875+0000","last_change":"2024-06-13T06:54:58.899107+0000","last_active":"2024-06-13T06:58:28.275875+0000","last_peered":"2024-06-13T06:58:28.275875+0000","last_clean":"2024-06-13T06:58:28.275875+0000","last_became_active":"2024-06-13T06:54:58.896034+0000","last_became_peered":"2024-06-13T06:54:58.896034+0000","last_unstale":"2024-06-13T06:58:28.275875+0000","last_undegraded":"2024-06-13T06:58:28.275875+0000","last_fullsized":"2024-06-13T06:58:28.275875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:00:01.132565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533277+0000","last_change":"2024-06-13T06:55:12.025681+0000","last_active":"2024-06-13T06:56:29.533277+0000","last_peered":"2024-06-13T06:56:29.533277+0000","last_clean":"2024-06-13T06:56:29.533277+0000","last_became_active":"2024-06-13T06:55:12.025267+0000","last_became_peered":"2024-06-13T06:55:12.025267+0000","last_unstale":"2024-06-13T06:56:29.533277+0000","last_undegraded":"2024-06-13T06:56:29.533277+0000","last_fullsized":"2024-06-13T06:56:29.533277+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:58:23.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":176,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:36.810961+0000","last_change":"2024-06-13T06:54:54.325404+0000","last_active":"2024-06-13T06:58:36.810961+0000","last_peered":"2024-06-13T06:58:36.810961+0000","last_clean":"2024-06-13T06:58:36.810961+0000","last_became_active":"2024-06-13T06:54:54.325024+0000","last_became_peered":"2024-06-13T06:54:54.325024+0000","last_unstale":"2024-06-13T06:58:36.810961+0000","last_undegraded":"2024-06-13T06:58:36.810961+0000","last_fullsized":"2024-06-13T06:58:36.810961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:23.726350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272642+0000","last_change":"2024-06-13T06:54:58.894737+0000","last_active":"2024-06-13T06:58:28.272642+0000","last_peered":"2024-06-13T06:58:28.272642+0000","last_clean":"2024-06-13T06:58:28.272642+0000","last_became_active":"2024-06-13T06:54:58.894224+0000","last_became_peered":"2024-06-13T06:54:58.894224+0000","last_unstale":"2024-06-13T06:58:28.272642+0000","last_undegraded":"2024-06-13T06:58:28.272642+0000","last_fullsized":"2024-06-13T06:58:28.272642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:03.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600221+0000","last_change":"2024-06-13T06:55:12.026587+0000","last_active":"2024-06-13T06:56:29.600221+0000","last_peered":"2024-06-13T06:56:29.600221+0000","last_clean":"2024-06-13T06:56:29.600221+0000","last_became_active":"2024-06-13T06:55:12.026114+0000","last_became_peered":"2024-06-13T06:55:12.026114+0000","last_unstale":"2024-06-13T06:56:29.600221+0000","last_undegraded":"2024-06-13T06:56:29.600221+0000","last_fullsized":"2024-06-13T06:56:29.600221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:43:09.245641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556229+0000","last_change":"2024-06-13T06:54:54.632069+0000","last_active":"2024-06-13T06:56:32.556229+0000","last_peered":"2024-06-13T06:56:32.556229+0000","last_clean":"2024-06-13T06:56:32.556229+0000","last_became_active":"2024-06-13T06:54:54.631692+0000","last_became_peered":"2024-06-13T06:54:54.631692+0000","last_unstale":"2024-06-13T06:56:32.556229+0000","last_undegraded":"2024-06-13T06:56:32.556229+0000","last_fullsized":"2024-06-13T06:56:32.556229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:07.109507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.258901+0000","last_change":"2024-06-13T06:55:29.366286+0000","last_active":"2024-06-13T06:58:28.258901+0000","last_peered":"2024-06-13T06:58:28.258901+0000","last_clean":"2024-06-13T06:58:28.258901+0000","last_became_active":"2024-06-13T06:55:29.365854+0000","last_became_peered":"2024-06-13T06:55:29.365854+0000","last_unstale":"2024-06-13T06:58:28.258901+0000","last_undegraded":"2024-06-13T06:58:28.258901+0000","last_fullsized":"2024-06-13T06:58:28.258901+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:04.209244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526226+0000","last_change":"2024-06-13T06:55:12.021621+0000","last_active":"2024-06-13T06:56:29.526226+0000","last_peered":"2024-06-13T06:56:29.526226+0000","last_clean":"2024-06-13T06:56:29.526226+0000","last_became_active":"2024-06-13T06:55:12.021150+0000","last_became_peered":"2024-06-13T06:55:12.021150+0000","last_unstale":"2024-06-13T06:56:29.526226+0000","last_undegraded":"2024-06-13T06:56:29.526226+0000","last_fullsized":"2024-06-13T06:56:29.526226+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:48.318918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":22,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556254+0000","last_change":"2024-06-13T06:55:29.369972+0000","last_active":"2024-06-13T06:56:32.556254+0000","last_peered":"2024-06-13T06:56:32.556254+0000","last_clean":"2024-06-13T06:56:32.556254+0000","last_became_active":"2024-06-13T06:55:29.366390+0000","last_became_peered":"2024-06-13T06:55:29.366390+0000","last_unstale":"2024-06-13T06:56:32.556254+0000","last_undegraded":"2024-06-13T06:56:32.556254+0000","last_fullsized":"2024-06-13T06:56:32.556254+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:18.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271750+0000","last_change":"2024-06-13T06:54:58.891309+0000","last_active":"2024-06-13T06:58:28.271750+0000","last_peered":"2024-06-13T06:58:28.271750+0000","last_clean":"2024-06-13T06:58:28.271750+0000","last_became_active":"2024-06-13T06:54:58.890972+0000","last_became_peered":"2024-06-13T06:54:58.890972+0000","last_unstale":"2024-06-13T06:58:28.271750+0000","last_undegraded":"2024-06-13T06:58:28.271750+0000","last_fullsized":"2024-06-13T06:58:28.271750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:41.240061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.611267+0000","last_change":"2024-06-13T06:56:29.610713+0000","last_active":"2024-06-13T06:56:29.611267+0000","last_peered":"2024-06-13T06:56:29.611267+0000","last_clean":"2024-06-13T06:56:29.611267+0000","last_became_active":"2024-06-13T06:56:29.610139+0000","last_became_peered":"2024-06-13T06:56:29.610139+0000","last_unstale":"2024-06-13T06:56:29.611267+0000","last_undegraded":"2024-06-13T06:56:29.611267+0000","last_fullsized":"2024-06-13T06:56:29.611267+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:14:36.601243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.534462+0000","last_change":"2024-06-13T06:54:54.633893+0000","last_active":"2024-06-13T06:56:29.534462+0000","last_peered":"2024-06-13T06:56:29.534462+0000","last_clean":"2024-06-13T06:56:29.534462+0000","last_became_active":"2024-06-13T06:54:54.632946+0000","last_became_peered":"2024-06-13T06:54:54.632946+0000","last_unstale":"2024-06-13T06:56:29.534462+0000","last_undegraded":"2024-06-13T06:56:29.534462+0000","last_fullsized":"2024-06-13T06:56:29.534462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:16:54.812477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279749+0000","last_change":"2024-06-13T06:54:58.907182+0000","last_active":"2024-06-13T06:58:28.279749+0000","last_peered":"2024-06-13T06:58:28.279749+0000","last_clean":"2024-06-13T06:58:28.279749+0000","last_became_active":"2024-06-13T06:54:58.906824+0000","last_became_peered":"2024-06-13T06:54:58.906824+0000","last_unstale":"2024-06-13T06:58:28.279749+0000","last_undegraded":"2024-06-13T06:58:28.279749+0000","last_fullsized":"2024-06-13T06:58:28.279749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:34:49.626991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.597238+0000","last_change":"2024-06-13T06:55:12.014803+0000","last_active":"2024-06-13T06:56:29.597238+0000","last_peered":"2024-06-13T06:56:29.597238+0000","last_clean":"2024-06-13T06:56:29.597238+0000","last_became_active":"2024-06-13T06:55:12.014398+0000","last_became_peered":"2024-06-13T06:55:12.014398+0000","last_unstale":"2024-06-13T06:56:29.597238+0000","last_undegraded":"2024-06-13T06:56:29.597238+0000","last_fullsized":"2024-06-13T06:56:29.597238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:07:44.960446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607834+0000","last_change":"2024-06-13T06:54:54.339056+0000","last_active":"2024-06-13T06:56:29.607834+0000","last_peered":"2024-06-13T06:56:29.607834+0000","last_clean":"2024-06-13T06:56:29.607834+0000","last_became_active":"2024-06-13T06:54:54.338683+0000","last_became_peered":"2024-06-13T06:54:54.338683+0000","last_unstale":"2024-06-13T06:56:29.607834+0000","last_undegraded":"2024-06-13T06:56:29.607834+0000","last_fullsized":"2024-06-13T06:56:29.607834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:05:17.060471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296931+0000","last_change":"2024-06-13T06:54:58.919843+0000","last_active":"2024-06-13T06:58:28.296931+0000","last_peered":"2024-06-13T06:58:28.296931+0000","last_clean":"2024-06-13T06:58:28.296931+0000","last_became_active":"2024-06-13T06:54:58.919405+0000","last_became_peered":"2024-06-13T06:54:58.919405+0000","last_unstale":"2024-06-13T06:58:28.296931+0000","last_undegraded":"2024-06-13T06:58:28.296931+0000","last_fullsized":"2024-06-13T06:58:28.296931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:00:20.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600188+0000","last_change":"2024-06-13T06:55:12.097651+0000","last_active":"2024-06-13T06:56:29.600188+0000","last_peered":"2024-06-13T06:56:29.600188+0000","last_clean":"2024-06-13T06:56:29.600188+0000","last_became_active":"2024-06-13T06:55:12.095644+0000","last_became_peered":"2024-06-13T06:55:12.095644+0000","last_unstale":"2024-06-13T06:56:29.600188+0000","last_undegraded":"2024-06-13T06:56:29.600188+0000","last_fullsized":"2024-06-13T06:56:29.600188+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:09:54.328428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.257770+0000","last_change":"2024-06-13T06:54:58.891187+0000","last_active":"2024-06-13T06:58:28.257770+0000","last_peered":"2024-06-13T06:58:28.257770+0000","last_clean":"2024-06-13T06:58:28.257770+0000","last_became_active":"2024-06-13T06:54:58.890848+0000","last_became_peered":"2024-06-13T06:54:58.890848+0000","last_unstale":"2024-06-13T06:58:28.257770+0000","last_undegraded":"2024-06-13T06:58:28.257770+0000","last_fullsized":"2024-06-13T06:58:28.257770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:13:31.709650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":174,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:35.808172+0000","last_change":"2024-06-13T06:54:54.631482+0000","last_active":"2024-06-13T06:58:35.808172+0000","last_peered":"2024-06-13T06:58:35.808172+0000","last_clean":"2024-06-13T06:58:35.808172+0000","last_became_active":"2024-06-13T06:54:54.630818+0000","last_became_peered":"2024-06-13T06:54:54.630818+0000","last_unstale":"2024-06-13T06:58:35.808172+0000","last_undegraded":"2024-06-13T06:58:35.808172+0000","last_fullsized":"2024-06-13T06:58:35.808172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:34:18.801727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:45.996461+0000","last_change":"2024-06-13T06:55:12.018856+0000","last_active":"2024-06-13T06:56:45.996461+0000","last_peered":"2024-06-13T06:56:45.996461+0000","last_clean":"2024-06-13T06:56:45.996461+0000","last_became_active":"2024-06-13T06:55:12.018488+0000","last_became_peered":"2024-06-13T06:55:12.018488+0000","last_unstale":"2024-06-13T06:56:45.996461+0000","last_undegraded":"2024-06-13T06:56:45.996461+0000","last_fullsized":"2024-06-13T06:56:45.996461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:14.973197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281746+0000","last_change":"2024-06-13T06:55:29.377203+0000","last_active":"2024-06-13T06:58:28.281746+0000","last_peered":"2024-06-13T06:58:28.281746+0000","last_clean":"2024-06-13T06:58:28.281746+0000","last_became_active":"2024-06-13T06:55:29.375263+0000","last_became_peered":"2024-06-13T06:55:29.375263+0000","last_unstale":"2024-06-13T06:58:28.281746+0000","last_undegraded":"2024-06-13T06:58:28.281746+0000","last_fullsized":"2024-06-13T06:58:28.281746+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:06:07.935172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558452+0000","last_change":"2024-06-13T06:54:54.335319+0000","last_active":"2024-06-13T06:56:32.558452+0000","last_peered":"2024-06-13T06:56:32.558452+0000","last_clean":"2024-06-13T06:56:32.558452+0000","last_became_active":"2024-06-13T06:54:54.334896+0000","last_became_peered":"2024-06-13T06:54:54.334896+0000","last_unstale":"2024-06-13T06:56:32.558452+0000","last_undegraded":"2024-06-13T06:56:32.558452+0000","last_fullsized":"2024-06-13T06:56:32.558452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:40.331535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607040+0000","last_change":"2024-06-13T06:53:16.968830+0000","last_active":"2024-06-13T06:56:29.607040+0000","last_peered":"2024-06-13T06:56:29.607040+0000","last_clean":"2024-06-13T06:56:29.607040+0000","last_became_active":"2024-06-13T06:53:16.968361+0000","last_became_peered":"2024-06-13T06:53:16.968361+0000","last_unstale":"2024-06-13T06:56:29.607040+0000","last_undegraded":"2024-06-13T06:56:29.607040+0000","last_fullsized":"2024-06-13T06:56:29.607040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_clean_scrub_stamp":"2024-06-13T06:49:18.858540+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:32:26.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.602642+0000","last_change":"2024-06-13T06:56:29.602124+0000","last_active":"2024-06-13T06:56:29.602642+0000","last_peered":"2024-06-13T06:56:29.602642+0000","last_clean":"2024-06-13T06:56:29.602642+0000","last_became_active":"2024-06-13T06:56:29.601650+0000","last_became_peered":"2024-06-13T06:56:29.601650+0000","last_unstale":"2024-06-13T06:56:29.602642+0000","last_undegraded":"2024-06-13T06:56:29.602642+0000","last_fullsized":"2024-06-13T06:56:29.602642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:37:23.903614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.294492+0000","last_change":"2024-06-13T06:54:58.898930+0000","last_active":"2024-06-13T06:58:28.294492+0000","last_peered":"2024-06-13T06:58:28.294492+0000","last_clean":"2024-06-13T06:58:28.294492+0000","last_became_active":"2024-06-13T06:54:58.898384+0000","last_became_peered":"2024-06-13T06:54:58.898384+0000","last_unstale":"2024-06-13T06:58:28.294492+0000","last_undegraded":"2024-06-13T06:58:28.294492+0000","last_fullsized":"2024-06-13T06:58:28.294492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:06:02.210336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607221+0000","last_change":"2024-06-13T06:54:54.328776+0000","last_active":"2024-06-13T06:56:29.607221+0000","last_peered":"2024-06-13T06:56:29.607221+0000","last_clean":"2024-06-13T06:56:29.607221+0000","last_became_active":"2024-06-13T06:54:54.327773+0000","last_became_peered":"2024-06-13T06:54:54.327773+0000","last_unstale":"2024-06-13T06:56:29.607221+0000","last_undegraded":"2024-06-13T06:56:29.607221+0000","last_fullsized":"2024-06-13T06:56:29.607221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:23:02.240774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524508+0000","last_change":"2024-06-13T06:55:12.013862+0000","last_active":"2024-06-13T06:56:29.524508+0000","last_peered":"2024-06-13T06:56:29.524508+0000","last_clean":"2024-06-13T06:56:29.524508+0000","last_became_active":"2024-06-13T06:55:12.013389+0000","last_became_peered":"2024-06-13T06:55:12.013389+0000","last_unstale":"2024-06-13T06:56:29.524508+0000","last_undegraded":"2024-06-13T06:56:29.524508+0000","last_fullsized":"2024-06-13T06:56:29.524508+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:49.701202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558954+0000","last_change":"2024-06-13T06:56:29.539862+0000","last_active":"2024-06-13T06:56:32.558954+0000","last_peered":"2024-06-13T06:56:32.558954+0000","last_clean":"2024-06-13T06:56:32.558954+0000","last_became_active":"2024-06-13T06:56:29.536109+0000","last_became_peered":"2024-06-13T06:56:29.536109+0000","last_unstale":"2024-06-13T06:56:32.558954+0000","last_undegraded":"2024-06-13T06:56:32.558954+0000","last_fullsized":"2024-06-13T06:56:32.558954+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:23.307850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.287464+0000","last_change":"2024-06-13T06:54:58.901773+0000","last_active":"2024-06-13T06:58:28.287464+0000","last_peered":"2024-06-13T06:58:28.287464+0000","last_clean":"2024-06-13T06:58:28.287464+0000","last_became_active":"2024-06-13T06:54:58.901168+0000","last_became_peered":"2024-06-13T06:54:58.901168+0000","last_unstale":"2024-06-13T06:58:28.287464+0000","last_undegraded":"2024-06-13T06:58:28.287464+0000","last_fullsized":"2024-06-13T06:58:28.287464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:57:15.848785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524234+0000","last_change":"2024-06-13T06:55:12.095890+0000","last_active":"2024-06-13T06:56:29.524234+0000","last_peered":"2024-06-13T06:56:29.524234+0000","last_clean":"2024-06-13T06:56:29.524234+0000","last_became_active":"2024-06-13T06:55:12.095397+0000","last_became_peered":"2024-06-13T06:55:12.095397+0000","last_unstale":"2024-06-13T06:56:29.524234+0000","last_undegraded":"2024-06-13T06:56:29.524234+0000","last_fullsized":"2024-06-13T06:56:29.524234+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:56:18.211036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'12","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:05.552121+0000","last_change":"2024-06-13T06:54:54.337946+0000","last_active":"2024-06-13T06:57:05.552121+0000","last_peered":"2024-06-13T06:57:05.552121+0000","last_clean":"2024-06-13T06:57:05.552121+0000","last_became_active":"2024-06-13T06:54:54.337028+0000","last_became_peered":"2024-06-13T06:54:54.337028+0000","last_unstale":"2024-06-13T06:57:05.552121+0000","last_undegraded":"2024-06-13T06:57:05.552121+0000","last_fullsized":"2024-06-13T06:57:05.552121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:08:52.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275581+0000","last_change":"2024-06-13T06:54:58.895110+0000","last_active":"2024-06-13T06:58:28.275581+0000","last_peered":"2024-06-13T06:58:28.275581+0000","last_clean":"2024-06-13T06:58:28.275581+0000","last_became_active":"2024-06-13T06:54:58.893481+0000","last_became_peered":"2024-06-13T06:54:58.893481+0000","last_unstale":"2024-06-13T06:58:28.275581+0000","last_undegraded":"2024-06-13T06:58:28.275581+0000","last_fullsized":"2024-06-13T06:58:28.275581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:54.932110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606884+0000","last_change":"2024-06-13T06:55:11.997712+0000","last_active":"2024-06-13T06:56:29.606884+0000","last_peered":"2024-06-13T06:56:29.606884+0000","last_clean":"2024-06-13T06:56:29.606884+0000","last_became_active":"2024-06-13T06:55:11.997193+0000","last_became_peered":"2024-06-13T06:55:11.997193+0000","last_unstale":"2024-06-13T06:56:29.606884+0000","last_undegraded":"2024-06-13T06:56:29.606884+0000","last_fullsized":"2024-06-13T06:56:29.606884+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:15.168070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608714+0000","last_change":"2024-06-13T06:54:54.520121+0000","last_active":"2024-06-13T06:56:29.608714+0000","last_peered":"2024-06-13T06:56:29.608714+0000","last_clean":"2024-06-13T06:56:29.608714+0000","last_became_active":"2024-06-13T06:54:54.519723+0000","last_became_peered":"2024-06-13T06:54:54.519723+0000","last_unstale":"2024-06-13T06:56:29.608714+0000","last_undegraded":"2024-06-13T06:56:29.608714+0000","last_fullsized":"2024-06-13T06:56:29.608714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:19:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304621+0000","last_change":"2024-06-13T06:54:58.923626+0000","last_active":"2024-06-13T06:58:28.304621+0000","last_peered":"2024-06-13T06:58:28.304621+0000","last_clean":"2024-06-13T06:58:28.304621+0000","last_became_active":"2024-06-13T06:54:58.923192+0000","last_became_peered":"2024-06-13T06:54:58.923192+0000","last_unstale":"2024-06-13T06:58:28.304621+0000","last_undegraded":"2024-06-13T06:58:28.304621+0000","last_fullsized":"2024-06-13T06:58:28.304621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:26:13.231549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607709+0000","last_change":"2024-06-13T06:55:12.096244+0000","last_active":"2024-06-13T06:56:29.607709+0000","last_peered":"2024-06-13T06:56:29.607709+0000","last_clean":"2024-06-13T06:56:29.607709+0000","last_became_active":"2024-06-13T06:55:12.095758+0000","last_became_peered":"2024-06-13T06:55:12.095758+0000","last_unstale":"2024-06-13T06:56:29.607709+0000","last_undegraded":"2024-06-13T06:56:29.607709+0000","last_fullsized":"2024-06-13T06:56:29.607709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:38.122282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607931+0000","last_change":"2024-06-13T06:54:54.630488+0000","last_active":"2024-06-13T06:56:29.607931+0000","last_peered":"2024-06-13T06:56:29.607931+0000","last_clean":"2024-06-13T06:56:29.607931+0000","last_became_active":"2024-06-13T06:54:54.630224+0000","last_became_peered":"2024-06-13T06:54:54.630224+0000","last_unstale":"2024-06-13T06:56:29.607931+0000","last_undegraded":"2024-06-13T06:56:29.607931+0000","last_fullsized":"2024-06-13T06:56:29.607931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:33:19.107026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.231934+0000","last_change":"2024-06-13T06:54:58.894466+0000","last_active":"2024-06-13T06:58:28.231934+0000","last_peered":"2024-06-13T06:58:28.231934+0000","last_clean":"2024-06-13T06:58:28.231934+0000","last_became_active":"2024-06-13T06:54:58.894034+0000","last_became_peered":"2024-06-13T06:54:58.894034+0000","last_unstale":"2024-06-13T06:58:28.231934+0000","last_undegraded":"2024-06-13T06:58:28.231934+0000","last_fullsized":"2024-06-13T06:58:28.231934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:44.051565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533647+0000","last_change":"2024-06-13T06:55:12.022078+0000","last_active":"2024-06-13T06:56:29.533647+0000","last_peered":"2024-06-13T06:56:29.533647+0000","last_clean":"2024-06-13T06:56:29.533647+0000","last_became_active":"2024-06-13T06:55:12.021652+0000","last_became_peered":"2024-06-13T06:55:12.021652+0000","last_unstale":"2024-06-13T06:56:29.533647+0000","last_undegraded":"2024-06-13T06:56:29.533647+0000","last_fullsized":"2024-06-13T06:56:29.533647+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:25:57.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524569+0000","last_change":"2024-06-13T06:54:54.336757+0000","last_active":"2024-06-13T06:56:29.524569+0000","last_peered":"2024-06-13T06:56:29.524569+0000","last_clean":"2024-06-13T06:56:29.524569+0000","last_became_active":"2024-06-13T06:54:54.336012+0000","last_became_peered":"2024-06-13T06:54:54.336012+0000","last_unstale":"2024-06-13T06:56:29.524569+0000","last_undegraded":"2024-06-13T06:56:29.524569+0000","last_fullsized":"2024-06-13T06:56:29.524569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:52:21.227464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292719+0000","last_change":"2024-06-13T06:54:58.919631+0000","last_active":"2024-06-13T06:58:28.292719+0000","last_peered":"2024-06-13T06:58:28.292719+0000","last_clean":"2024-06-13T06:58:28.292719+0000","last_became_active":"2024-06-13T06:54:58.919274+0000","last_became_peered":"2024-06-13T06:54:58.919274+0000","last_unstale":"2024-06-13T06:58:28.292719+0000","last_undegraded":"2024-06-13T06:58:28.292719+0000","last_fullsized":"2024-06-13T06:58:28.292719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:22.453797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:54.172963+0000","last_change":"2024-06-13T06:55:12.017338+0000","last_active":"2024-06-13T06:56:54.172963+0000","last_peered":"2024-06-13T06:56:54.172963+0000","last_clean":"2024-06-13T06:56:54.172963+0000","last_became_active":"2024-06-13T06:55:12.016841+0000","last_became_peered":"2024-06-13T06:55:12.016841+0000","last_unstale":"2024-06-13T06:56:54.172963+0000","last_undegraded":"2024-06-13T06:56:54.172963+0000","last_fullsized":"2024-06-13T06:56:54.172963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:40:13.525892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607062+0000","last_change":"2024-06-13T06:54:54.328808+0000","last_active":"2024-06-13T06:56:29.607062+0000","last_peered":"2024-06-13T06:56:29.607062+0000","last_clean":"2024-06-13T06:56:29.607062+0000","last_became_active":"2024-06-13T06:54:54.326982+0000","last_became_peered":"2024-06-13T06:54:54.326982+0000","last_unstale":"2024-06-13T06:56:29.607062+0000","last_undegraded":"2024-06-13T06:56:29.607062+0000","last_fullsized":"2024-06-13T06:56:29.607062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:22:53.854275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.302930+0000","last_change":"2024-06-13T06:54:58.925728+0000","last_active":"2024-06-13T06:58:28.302930+0000","last_peered":"2024-06-13T06:58:28.302930+0000","last_clean":"2024-06-13T06:58:28.302930+0000","last_became_active":"2024-06-13T06:54:58.925407+0000","last_became_peered":"2024-06-13T06:54:58.925407+0000","last_unstale":"2024-06-13T06:58:28.302930+0000","last_undegraded":"2024-06-13T06:58:28.302930+0000","last_fullsized":"2024-06-13T06:58:28.302930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:38.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607500+0000","last_change":"2024-06-13T06:55:12.095733+0000","last_active":"2024-06-13T06:56:29.607500+0000","last_peered":"2024-06-13T06:56:29.607500+0000","last_clean":"2024-06-13T06:56:29.607500+0000","last_became_active":"2024-06-13T06:55:12.095064+0000","last_became_peered":"2024-06-13T06:55:12.095064+0000","last_unstale":"2024-06-13T06:56:29.607500+0000","last_undegraded":"2024-06-13T06:56:29.607500+0000","last_fullsized":"2024-06-13T06:56:29.607500+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:26:05.486701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556112+0000","last_change":"2024-06-13T06:54:54.331448+0000","last_active":"2024-06-13T06:56:32.556112+0000","last_peered":"2024-06-13T06:56:32.556112+0000","last_clean":"2024-06-13T06:56:32.556112+0000","last_became_active":"2024-06-13T06:54:54.330964+0000","last_became_peered":"2024-06-13T06:54:54.330964+0000","last_unstale":"2024-06-13T06:56:32.556112+0000","last_undegraded":"2024-06-13T06:56:32.556112+0000","last_fullsized":"2024-06-13T06:56:32.556112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:23:12.211412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281344+0000","last_change":"2024-06-13T06:54:58.899112+0000","last_active":"2024-06-13T06:58:28.281344+0000","last_peered":"2024-06-13T06:58:28.281344+0000","last_clean":"2024-06-13T06:58:28.281344+0000","last_became_active":"2024-06-13T06:54:58.898742+0000","last_became_peered":"2024-06-13T06:54:58.898742+0000","last_unstale":"2024-06-13T06:58:28.281344+0000","last_undegraded":"2024-06-13T06:58:28.281344+0000","last_fullsized":"2024-06-13T06:58:28.281344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:50:12.983287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608239+0000","last_change":"2024-06-13T06:55:12.014149+0000","last_active":"2024-06-13T06:56:29.608239+0000","last_peered":"2024-06-13T06:56:29.608239+0000","last_clean":"2024-06-13T06:56:29.608239+0000","last_became_active":"2024-06-13T06:55:12.013454+0000","last_became_peered":"2024-06-13T06:55:12.013454+0000","last_unstale":"2024-06-13T06:56:29.608239+0000","last_undegraded":"2024-06-13T06:56:29.608239+0000","last_fullsized":"2024-06-13T06:56:29.608239+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:13:39.368540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.560296+0000","last_change":"2024-06-13T06:54:54.633268+0000","last_active":"2024-06-13T06:56:32.560296+0000","last_peered":"2024-06-13T06:56:32.560296+0000","last_clean":"2024-06-13T06:56:32.560296+0000","last_became_active":"2024-06-13T06:54:54.632840+0000","last_became_peered":"2024-06-13T06:54:54.632840+0000","last_unstale":"2024-06-13T06:56:32.560296+0000","last_undegraded":"2024-06-13T06:56:32.560296+0000","last_fullsized":"2024-06-13T06:56:32.560296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:21.653727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281293+0000","last_change":"2024-06-13T06:54:58.893417+0000","last_active":"2024-06-13T06:58:28.281293+0000","last_peered":"2024-06-13T06:58:28.281293+0000","last_clean":"2024-06-13T06:58:28.281293+0000","last_became_active":"2024-06-13T06:54:58.892381+0000","last_became_peered":"2024-06-13T06:54:58.892381+0000","last_unstale":"2024-06-13T06:58:28.281293+0000","last_undegraded":"2024-06-13T06:58:28.281293+0000","last_fullsized":"2024-06-13T06:58:28.281293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:07.979654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533526+0000","last_change":"2024-06-13T06:55:12.018336+0000","last_active":"2024-06-13T06:56:29.533526+0000","last_peered":"2024-06-13T06:56:29.533526+0000","last_clean":"2024-06-13T06:56:29.533526+0000","last_became_active":"2024-06-13T06:55:12.017861+0000","last_became_peered":"2024-06-13T06:55:12.017861+0000","last_unstale":"2024-06-13T06:56:29.533526+0000","last_undegraded":"2024-06-13T06:56:29.533526+0000","last_fullsized":"2024-06-13T06:56:29.533526+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:41.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556542+0000","last_change":"2024-06-13T06:54:54.333036+0000","last_active":"2024-06-13T06:56:32.556542+0000","last_peered":"2024-06-13T06:56:32.556542+0000","last_clean":"2024-06-13T06:56:32.556542+0000","last_became_active":"2024-06-13T06:54:54.332642+0000","last_became_peered":"2024-06-13T06:54:54.332642+0000","last_unstale":"2024-06-13T06:56:32.556542+0000","last_undegraded":"2024-06-13T06:56:32.556542+0000","last_fullsized":"2024-06-13T06:56:32.556542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:06:06.809024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279764+0000","last_change":"2024-06-13T06:54:58.881894+0000","last_active":"2024-06-13T06:58:28.279764+0000","last_peered":"2024-06-13T06:58:28.279764+0000","last_clean":"2024-06-13T06:58:28.279764+0000","last_became_active":"2024-06-13T06:54:58.881591+0000","last_became_peered":"2024-06-13T06:54:58.881591+0000","last_unstale":"2024-06-13T06:58:28.279764+0000","last_undegraded":"2024-06-13T06:58:28.279764+0000","last_fullsized":"2024-06-13T06:58:28.279764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:49:00.944212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525430+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:56:29.525430+0000","last_peered":"2024-06-13T06:56:29.525430+0000","last_clean":"2024-06-13T06:56:29.525430+0000","last_became_active":"2024-06-13T06:55:12.000748+0000","last_became_peered":"2024-06-13T06:55:12.000748+0000","last_unstale":"2024-06-13T06:56:29.525430+0000","last_undegraded":"2024-06-13T06:56:29.525430+0000","last_fullsized":"2024-06-13T06:56:29.525430+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:22.035807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600248+0000","last_change":"2024-06-13T06:54:54.342724+0000","last_active":"2024-06-13T06:56:29.600248+0000","last_peered":"2024-06-13T06:56:29.600248+0000","last_clean":"2024-06-13T06:56:29.600248+0000","last_became_active":"2024-06-13T06:54:54.342267+0000","last_became_peered":"2024-06-13T06:54:54.342267+0000","last_unstale":"2024-06-13T06:56:29.600248+0000","last_undegraded":"2024-06-13T06:56:29.600248+0000","last_fullsized":"2024-06-13T06:56:29.600248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:27:53.039202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271410+0000","last_change":"2024-06-13T06:54:58.892365+0000","last_active":"2024-06-13T06:58:28.271410+0000","last_peered":"2024-06-13T06:58:28.271410+0000","last_clean":"2024-06-13T06:58:28.271410+0000","last_became_active":"2024-06-13T06:54:58.889395+0000","last_became_peered":"2024-06-13T06:54:58.889395+0000","last_unstale":"2024-06-13T06:58:28.271410+0000","last_undegraded":"2024-06-13T06:58:28.271410+0000","last_fullsized":"2024-06-13T06:58:28.271410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:55.798293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272092+0000","last_change":"2024-06-13T06:54:58.888570+0000","last_active":"2024-06-13T06:58:28.272092+0000","last_peered":"2024-06-13T06:58:28.272092+0000","last_clean":"2024-06-13T06:58:28.272092+0000","last_became_active":"2024-06-13T06:54:58.888180+0000","last_became_peered":"2024-06-13T06:54:58.888180+0000","last_unstale":"2024-06-13T06:58:28.272092+0000","last_undegraded":"2024-06-13T06:58:28.272092+0000","last_fullsized":"2024-06-13T06:58:28.272092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:46:42.537790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.270977+0000","last_change":"2024-06-13T06:54:58.893476+0000","last_active":"2024-06-13T06:58:28.270977+0000","last_peered":"2024-06-13T06:58:28.270977+0000","last_clean":"2024-06-13T06:58:28.270977+0000","last_became_active":"2024-06-13T06:54:58.893021+0000","last_became_peered":"2024-06-13T06:54:58.893021+0000","last_unstale":"2024-06-13T06:58:28.270977+0000","last_undegraded":"2024-06-13T06:58:28.270977+0000","last_fullsized":"2024-06-13T06:58:28.270977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:38.739070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301473+0000","last_change":"2024-06-13T06:54:58.911042+0000","last_active":"2024-06-13T06:58:28.301473+0000","last_peered":"2024-06-13T06:58:28.301473+0000","last_clean":"2024-06-13T06:58:28.301473+0000","last_became_active":"2024-06-13T06:54:58.910641+0000","last_became_peered":"2024-06-13T06:54:58.910641+0000","last_unstale":"2024-06-13T06:58:28.301473+0000","last_undegraded":"2024-06-13T06:58:28.301473+0000","last_fullsized":"2024-06-13T06:58:28.301473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:21:39.467223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301939+0000","last_change":"2024-06-13T06:55:29.382730+0000","last_active":"2024-06-13T06:58:28.301939+0000","last_peered":"2024-06-13T06:58:28.301939+0000","last_clean":"2024-06-13T06:58:28.301939+0000","last_became_active":"2024-06-13T06:55:29.382162+0000","last_became_peered":"2024-06-13T06:55:29.382162+0000","last_unstale":"2024-06-13T06:58:28.301939+0000","last_undegraded":"2024-06-13T06:58:28.301939+0000","last_fullsized":"2024-06-13T06:58:28.301939+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:05.486414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.302996+0000","last_change":"2024-06-13T06:55:29.385380+0000","last_active":"2024-06-13T06:58:28.302996+0000","last_peered":"2024-06-13T06:58:28.302996+0000","last_clean":"2024-06-13T06:58:28.302996+0000","last_became_active":"2024-06-13T06:55:29.384855+0000","last_became_peered":"2024-06-13T06:55:29.384855+0000","last_unstale":"2024-06-13T06:58:28.302996+0000","last_undegraded":"2024-06-13T06:58:28.302996+0000","last_fullsized":"2024-06-13T06:58:28.302996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:31:01.945632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271515+0000","last_change":"2024-06-13T06:54:58.889128+0000","last_active":"2024-06-13T06:58:28.271515+0000","last_peered":"2024-06-13T06:58:28.271515+0000","last_clean":"2024-06-13T06:58:28.271515+0000","last_became_active":"2024-06-13T06:54:58.888606+0000","last_became_peered":"2024-06-13T06:54:58.888606+0000","last_unstale":"2024-06-13T06:58:28.271515+0000","last_undegraded":"2024-06-13T06:58:28.271515+0000","last_fullsized":"2024-06-13T06:58:28.271515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:53:43.762968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.269593+0000","last_change":"2024-06-13T06:54:58.896098+0000","last_active":"2024-06-13T06:58:28.269593+0000","last_peered":"2024-06-13T06:58:28.269593+0000","last_clean":"2024-06-13T06:58:28.269593+0000","last_became_active":"2024-06-13T06:54:58.895722+0000","last_became_peered":"2024-06-13T06:54:58.895722+0000","last_unstale":"2024-06-13T06:58:28.269593+0000","last_undegraded":"2024-06-13T06:58:28.269593+0000","last_fullsized":"2024-06-13T06:58:28.269593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:18:14.421363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279703+0000","last_change":"2024-06-13T06:54:58.905590+0000","last_active":"2024-06-13T06:58:28.279703+0000","last_peered":"2024-06-13T06:58:28.279703+0000","last_clean":"2024-06-13T06:58:28.279703+0000","last_became_active":"2024-06-13T06:54:58.905146+0000","last_became_peered":"2024-06-13T06:54:58.905146+0000","last_unstale":"2024-06-13T06:58:28.279703+0000","last_undegraded":"2024-06-13T06:58:28.279703+0000","last_fullsized":"2024-06-13T06:58:28.279703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:42:08.010906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.270240+0000","last_change":"2024-06-13T06:54:58.898037+0000","last_active":"2024-06-13T06:58:28.270240+0000","last_peered":"2024-06-13T06:58:28.270240+0000","last_clean":"2024-06-13T06:58:28.270240+0000","last_became_active":"2024-06-13T06:54:58.897692+0000","last_became_peered":"2024-06-13T06:54:58.897692+0000","last_unstale":"2024-06-13T06:58:28.270240+0000","last_undegraded":"2024-06-13T06:58:28.270240+0000","last_fullsized":"2024-06-13T06:58:28.270240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:44:30.018192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275550+0000","last_change":"2024-06-13T06:54:58.903441+0000","last_active":"2024-06-13T06:58:28.275550+0000","last_peered":"2024-06-13T06:58:28.275550+0000","last_clean":"2024-06-13T06:58:28.275550+0000","last_became_active":"2024-06-13T06:54:58.893598+0000","last_became_peered":"2024-06-13T06:54:58.893598+0000","last_unstale":"2024-06-13T06:58:28.275550+0000","last_undegraded":"2024-06-13T06:58:28.275550+0000","last_fullsized":"2024-06-13T06:58:28.275550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:38:29.968619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282381+0000","last_change":"2024-06-13T06:54:58.918594+0000","last_active":"2024-06-13T06:58:28.282381+0000","last_peered":"2024-06-13T06:58:28.282381+0000","last_clean":"2024-06-13T06:58:28.282381+0000","last_became_active":"2024-06-13T06:54:58.918255+0000","last_became_peered":"2024-06-13T06:54:58.918255+0000","last_unstale":"2024-06-13T06:58:28.282381+0000","last_undegraded":"2024-06-13T06:58:28.282381+0000","last_fullsized":"2024-06-13T06:58:28.282381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:39:48.527707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304459+0000","last_change":"2024-06-13T06:54:58.926253+0000","last_active":"2024-06-13T06:58:28.304459+0000","last_peered":"2024-06-13T06:58:28.304459+0000","last_clean":"2024-06-13T06:58:28.304459+0000","last_became_active":"2024-06-13T06:54:58.924611+0000","last_became_peered":"2024-06-13T06:54:58.924611+0000","last_unstale":"2024-06-13T06:58:28.304459+0000","last_undegraded":"2024-06-13T06:58:28.304459+0000","last_fullsized":"2024-06-13T06:58:28.304459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:29:23.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267122+0000","last_change":"2024-06-13T06:54:58.890454+0000","last_active":"2024-06-13T06:58:28.267122+0000","last_peered":"2024-06-13T06:58:28.267122+0000","last_clean":"2024-06-13T06:58:28.267122+0000","last_became_active":"2024-06-13T06:54:58.890129+0000","last_became_peered":"2024-06-13T06:54:58.890129+0000","last_unstale":"2024-06-13T06:58:28.267122+0000","last_undegraded":"2024-06-13T06:58:28.267122+0000","last_fullsized":"2024-06-13T06:58:28.267122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:10.819536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67045,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":782,"num_read_kb":464,"num_write":174,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1656,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383743,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":570102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645379,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37124,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708028,"statfs":{"total":95995035648,"available":95957020672,"internally_reserved":0,"allocated":1830912,"data_stored":1220230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808905,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":574390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070544,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":570170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332182,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1166263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495706,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":565470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36512,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708640,"statfs":{"total":95995035648,"available":95957647360,"internally_reserved":0,"allocated":1204224,"data_stored":628284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":566858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":566108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":565664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920872,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1228800,"data_stored":634177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757346,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1161851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:58:42.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph pg dump --format=json 2024-06-13T06:58:42.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:42 smithi136 ceph-mon[37229]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:42.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:42 smithi136 ceph-mon[37229]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:58:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:42 smithi026 ceph-mon[32041]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:42 smithi026 ceph-mon[32041]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:58:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:42 smithi135 ceph-mon[36276]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:58:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:42 smithi135 ceph-mon[36276]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:58:42.574 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:44.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:44 smithi136 ceph-mon[37229]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T06:58:44.482 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:44 smithi026 ceph-mon[32041]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T06:58:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:44 smithi135 ceph-mon[36276]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T06:58:45.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 -- 172.21.15.26:0/2697319067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6410ee90 msgr2=0x7f2b64111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:45.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 --2- 172.21.15.26:0/2697319067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6410ee90 0x7f2b64111280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0098f0 tx=0x7f2b5c009c00 comp rx=0 tx=0).stop 2024-06-13T06:58:45.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 -- 172.21.15.26:0/2697319067 shutdown_connections 2024-06-13T06:58:45.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 --2- 172.21.15.26:0/2697319067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b640746c0 0x7f2b64074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 --2- 172.21.15.26:0/2697319067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b6410ee90 0x7f2b64111280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 --2- 172.21.15.26:0/2697319067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b64074fe0 0x7f2b6410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 -- 172.21.15.26:0/2697319067 >> 172.21.15.26:0/2697319067 conn(0x7f2b640795f0 msgr2=0x7f2b64079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:45.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 -- 172.21.15.26:0/2697319067 shutdown_connections 2024-06-13T06:58:45.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.072+0000 7f2b6a200700 1 -- 172.21.15.26:0/2697319067 wait complete. 2024-06-13T06:58:45.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.073+0000 7f2b6a200700 1 Processor -- start 2024-06-13T06:58:45.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.073+0000 7f2b6a200700 1 -- start start 2024-06-13T06:58:45.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.073+0000 7f2b6a200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b640746c0 0x7f2b6419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:45.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.073+0000 7f2b6a200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b64074fe0 0x7f2b6419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:45.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b6a200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6410ee90 0x7f2b6419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:45.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b6a200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641149d0 con 0x7f2b6410ee90 2024-06-13T06:58:45.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b6a200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64114850 con 0x7f2b64074fe0 2024-06-13T06:58:45.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b6a200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64114b50 con 0x7f2b640746c0 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b5aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b64074fe0 0x7f2b6419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b5aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b64074fe0 0x7f2b6419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:45766/0 (socket says 172.21.15.26:45766) 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b5aa00700 1 -- 172.21.15.26:0/3422350767 learned_addr learned my addr 172.21.15.26:0/3422350767 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b63e00700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6410ee90 0x7f2b6419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b63400700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b640746c0 0x7f2b6419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b63400700 1 -- 172.21.15.26:0/3422350767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b64074fe0 msgr2=0x7f2b6419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b63400700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b64074fe0 0x7f2b6419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b63400700 1 -- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6410ee90 msgr2=0x7f2b6419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b63400700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6410ee90 0x7f2b6419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.074+0000 7f2b63400700 1 -- 172.21.15.26:0/3422350767 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6419fe10 con 0x7f2b640746c0 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.075+0000 7f2b63400700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b640746c0 0x7f2b6419ae00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400e9f0 tx=0x7f2b5400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:45.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.075+0000 7f2b60c00700 1 -- 172.21.15.26:0/3422350767 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400cb70 con 0x7f2b640746c0 2024-06-13T06:58:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.075+0000 7f2b60c00700 1 -- 172.21.15.26:0/3422350767 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5400ee90 con 0x7f2b640746c0 2024-06-13T06:58:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.075+0000 7f2b60c00700 1 -- 172.21.15.26:0/3422350767 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54003dc0 con 0x7f2b640746c0 2024-06-13T06:58:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.075+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b641a0100 con 0x7f2b640746c0 2024-06-13T06:58:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.075+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641a7980 con 0x7f2b640746c0 2024-06-13T06:58:45.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.077+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b64069180 con 0x7f2b640746c0 2024-06-13T06:58:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.078+0000 7f2b60c00700 1 -- 172.21.15.26:0/3422350767 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b54003fa0 con 0x7f2b640746c0 2024-06-13T06:58:45.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.080+0000 7f2b60c00700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b480a2ca0 0x7f2b480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:45.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.080+0000 7f2b60c00700 1 -- 172.21.15.26:0/3422350767 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2b540cf590 con 0x7f2b640746c0 2024-06-13T06:58:45.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.080+0000 7f2b5aa00700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b480a2ca0 0x7f2b480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:45.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.081+0000 7f2b5aa00700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b480a2ca0 0x7f2b480a5160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b64110a50 tx=0x7f2b4c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.084+0000 7f2b60c00700 1 -- 172.21.15.26:0/3422350767 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2b5409c0a0 con 0x7f2b640746c0 2024-06-13T06:58:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.310+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b640ffcd0 con 0x7f2b480a2ca0 2024-06-13T06:58:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.329+0000 7f2b60c00700 1 -- 172.21.15.26:0/3422350767 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340495 (secure 0 0 0) 0x7f2b640ffcd0 con 0x7f2b480a2ca0 2024-06-13T06:58:45.331 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:58:45.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.340+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b480a2ca0 msgr2=0x7f2b480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:45.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.340+0000 7f2b6a200700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b480a2ca0 0x7f2b480a5160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b64110a50 tx=0x7f2b4c00b040 comp rx=0 tx=0).stop 2024-06-13T06:58:45.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.340+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b640746c0 msgr2=0x7f2b6419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:45.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.340+0000 7f2b6a200700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b640746c0 0x7f2b6419ae00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400e9f0 tx=0x7f2b5400ed00 comp rx=0 tx=0).stop 2024-06-13T06:58:45.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.341+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 shutdown_connections 2024-06-13T06:58:45.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.341+0000 7f2b6a200700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2b480a2ca0 0x7f2b480a5160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.341+0000 7f2b6a200700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6410ee90 0x7f2b6419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.341+0000 7f2b6a200700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b640746c0 0x7f2b6419ae00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.341+0000 7f2b6a200700 1 --2- 172.21.15.26:0/3422350767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b64074fe0 0x7f2b6419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.341+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 >> 172.21.15.26:0/3422350767 conn(0x7f2b640795f0 msgr2=0x7f2b640ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.341+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 shutdown_connections 2024-06-13T06:58:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:45.341+0000 7f2b6a200700 1 -- 172.21.15.26:0/3422350767 wait complete. 2024-06-13T06:58:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-06-13T06:58:46.287 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-06-13T06:58:44.372385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657647,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":938,"num_read_kb":588,"num_write":357,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182392,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402032,"kb_used_data":15440,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539792,"statfs":{"total":1151940427776,"available":1151528747008,"internally_reserved":0,"allocated":15810560,"data_stored":8789567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":46,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006360"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293106+0000","last_change":"2024-06-13T06:54:58.919669+0000","last_active":"2024-06-13T06:58:28.293106+0000","last_peered":"2024-06-13T06:58:28.293106+0000","last_clean":"2024-06-13T06:58:28.293106+0000","last_became_active":"2024-06-13T06:54:58.919334+0000","last_became_peered":"2024-06-13T06:54:58.919334+0000","last_unstale":"2024-06-13T06:58:28.293106+0000","last_undegraded":"2024-06-13T06:58:28.293106+0000","last_fullsized":"2024-06-13T06:58:28.293106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:45:40.831640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295026+0000","last_change":"2024-06-13T06:54:58.909508+0000","last_active":"2024-06-13T06:58:28.295026+0000","last_peered":"2024-06-13T06:58:28.295026+0000","last_clean":"2024-06-13T06:58:28.295026+0000","last_became_active":"2024-06-13T06:54:58.909221+0000","last_became_peered":"2024-06-13T06:54:58.909221+0000","last_unstale":"2024-06-13T06:58:28.295026+0000","last_undegraded":"2024-06-13T06:58:28.295026+0000","last_fullsized":"2024-06-13T06:58:28.295026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:17:07.476858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.303909+0000","last_change":"2024-06-13T06:54:58.909283+0000","last_active":"2024-06-13T06:58:28.303909+0000","last_peered":"2024-06-13T06:58:28.303909+0000","last_clean":"2024-06-13T06:58:28.303909+0000","last_became_active":"2024-06-13T06:54:58.908912+0000","last_became_peered":"2024-06-13T06:54:58.908912+0000","last_unstale":"2024-06-13T06:58:28.303909+0000","last_undegraded":"2024-06-13T06:58:28.303909+0000","last_fullsized":"2024-06-13T06:58:28.303909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:48:26.937971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.262650+0000","last_change":"2024-06-13T06:54:58.902821+0000","last_active":"2024-06-13T06:58:28.262650+0000","last_peered":"2024-06-13T06:58:28.262650+0000","last_clean":"2024-06-13T06:58:28.262650+0000","last_became_active":"2024-06-13T06:54:58.902215+0000","last_became_peered":"2024-06-13T06:54:58.902215+0000","last_unstale":"2024-06-13T06:58:28.262650+0000","last_undegraded":"2024-06-13T06:58:28.262650+0000","last_fullsized":"2024-06-13T06:58:28.262650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:11:57.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296512+0000","last_change":"2024-06-13T06:54:58.897038+0000","last_active":"2024-06-13T06:58:28.296512+0000","last_peered":"2024-06-13T06:58:28.296512+0000","last_clean":"2024-06-13T06:58:28.296512+0000","last_became_active":"2024-06-13T06:54:58.896665+0000","last_became_peered":"2024-06-13T06:54:58.896665+0000","last_unstale":"2024-06-13T06:58:28.296512+0000","last_undegraded":"2024-06-13T06:58:28.296512+0000","last_fullsized":"2024-06-13T06:58:28.296512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:24:19.216932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267357+0000","last_change":"2024-06-13T06:54:58.895501+0000","last_active":"2024-06-13T06:58:28.267357+0000","last_peered":"2024-06-13T06:58:28.267357+0000","last_clean":"2024-06-13T06:58:28.267357+0000","last_became_active":"2024-06-13T06:54:58.895144+0000","last_became_peered":"2024-06-13T06:54:58.895144+0000","last_unstale":"2024-06-13T06:58:28.267357+0000","last_undegraded":"2024-06-13T06:58:28.267357+0000","last_fullsized":"2024-06-13T06:58:28.267357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:50.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272231+0000","last_change":"2024-06-13T06:54:58.901097+0000","last_active":"2024-06-13T06:58:28.272231+0000","last_peered":"2024-06-13T06:58:28.272231+0000","last_clean":"2024-06-13T06:58:28.272231+0000","last_became_active":"2024-06-13T06:54:58.900727+0000","last_became_peered":"2024-06-13T06:54:58.900727+0000","last_unstale":"2024-06-13T06:58:28.272231+0000","last_undegraded":"2024-06-13T06:58:28.272231+0000","last_fullsized":"2024-06-13T06:58:28.272231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:09:38.349517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278377+0000","last_change":"2024-06-13T06:54:58.910740+0000","last_active":"2024-06-13T06:58:28.278377+0000","last_peered":"2024-06-13T06:58:28.278377+0000","last_clean":"2024-06-13T06:58:28.278377+0000","last_became_active":"2024-06-13T06:54:58.910351+0000","last_became_peered":"2024-06-13T06:54:58.910351+0000","last_unstale":"2024-06-13T06:58:28.278377+0000","last_undegraded":"2024-06-13T06:58:28.278377+0000","last_fullsized":"2024-06-13T06:58:28.278377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:45:12.854996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281724+0000","last_change":"2024-06-13T06:55:29.364516+0000","last_active":"2024-06-13T06:58:28.281724+0000","last_peered":"2024-06-13T06:58:28.281724+0000","last_clean":"2024-06-13T06:58:28.281724+0000","last_became_active":"2024-06-13T06:55:29.364079+0000","last_became_peered":"2024-06-13T06:55:29.364079+0000","last_unstale":"2024-06-13T06:58:28.281724+0000","last_undegraded":"2024-06-13T06:58:28.281724+0000","last_fullsized":"2024-06-13T06:58:28.281724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:51:28.624359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.266852+0000","last_change":"2024-06-13T06:54:58.903882+0000","last_active":"2024-06-13T06:58:28.266852+0000","last_peered":"2024-06-13T06:58:28.266852+0000","last_clean":"2024-06-13T06:58:28.266852+0000","last_became_active":"2024-06-13T06:54:58.903495+0000","last_became_peered":"2024-06-13T06:54:58.903495+0000","last_unstale":"2024-06-13T06:58:28.266852+0000","last_undegraded":"2024-06-13T06:58:28.266852+0000","last_fullsized":"2024-06-13T06:58:28.266852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:23.682993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281281+0000","last_change":"2024-06-13T06:54:58.900080+0000","last_active":"2024-06-13T06:58:28.281281+0000","last_peered":"2024-06-13T06:58:28.281281+0000","last_clean":"2024-06-13T06:58:28.281281+0000","last_became_active":"2024-06-13T06:54:58.899736+0000","last_became_peered":"2024-06-13T06:54:58.899736+0000","last_unstale":"2024-06-13T06:58:28.281281+0000","last_undegraded":"2024-06-13T06:58:28.281281+0000","last_fullsized":"2024-06-13T06:58:28.281281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:51:45.808821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.261193+0000","last_change":"2024-06-13T06:54:58.900238+0000","last_active":"2024-06-13T06:58:28.261193+0000","last_peered":"2024-06-13T06:58:28.261193+0000","last_clean":"2024-06-13T06:58:28.261193+0000","last_became_active":"2024-06-13T06:54:58.899872+0000","last_became_peered":"2024-06-13T06:54:58.899872+0000","last_unstale":"2024-06-13T06:58:28.261193+0000","last_undegraded":"2024-06-13T06:58:28.261193+0000","last_fullsized":"2024-06-13T06:58:28.261193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:12:40.780894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296753+0000","last_change":"2024-06-13T06:54:58.901981+0000","last_active":"2024-06-13T06:58:28.296753+0000","last_peered":"2024-06-13T06:58:28.296753+0000","last_clean":"2024-06-13T06:58:28.296753+0000","last_became_active":"2024-06-13T06:54:58.898704+0000","last_became_peered":"2024-06-13T06:54:58.898704+0000","last_unstale":"2024-06-13T06:58:28.296753+0000","last_undegraded":"2024-06-13T06:58:28.296753+0000","last_fullsized":"2024-06-13T06:58:28.296753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:03:56.658091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.299012+0000","last_change":"2024-06-13T06:54:58.909022+0000","last_active":"2024-06-13T06:58:28.299012+0000","last_peered":"2024-06-13T06:58:28.299012+0000","last_clean":"2024-06-13T06:58:28.299012+0000","last_became_active":"2024-06-13T06:54:58.907438+0000","last_became_peered":"2024-06-13T06:54:58.907438+0000","last_unstale":"2024-06-13T06:58:28.299012+0000","last_undegraded":"2024-06-13T06:58:28.299012+0000","last_fullsized":"2024-06-13T06:58:28.299012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:40:34.190510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305315+0000","last_change":"2024-06-13T06:54:58.897691+0000","last_active":"2024-06-13T06:58:28.305315+0000","last_peered":"2024-06-13T06:58:28.305315+0000","last_clean":"2024-06-13T06:58:28.305315+0000","last_became_active":"2024-06-13T06:54:58.897381+0000","last_became_peered":"2024-06-13T06:54:58.897381+0000","last_unstale":"2024-06-13T06:58:28.305315+0000","last_undegraded":"2024-06-13T06:58:28.305315+0000","last_fullsized":"2024-06-13T06:58:28.305315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:10.052052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295084+0000","last_change":"2024-06-13T06:54:58.907676+0000","last_active":"2024-06-13T06:58:28.295084+0000","last_peered":"2024-06-13T06:58:28.295084+0000","last_clean":"2024-06-13T06:58:28.295084+0000","last_became_active":"2024-06-13T06:54:58.907304+0000","last_became_peered":"2024-06-13T06:54:58.907304+0000","last_unstale":"2024-06-13T06:58:28.295084+0000","last_undegraded":"2024-06-13T06:58:28.295084+0000","last_fullsized":"2024-06-13T06:58:28.295084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:08:20.292516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293255+0000","last_change":"2024-06-13T06:54:58.920012+0000","last_active":"2024-06-13T06:58:28.293255+0000","last_peered":"2024-06-13T06:58:28.293255+0000","last_clean":"2024-06-13T06:58:28.293255+0000","last_became_active":"2024-06-13T06:54:58.919473+0000","last_became_peered":"2024-06-13T06:54:58.919473+0000","last_unstale":"2024-06-13T06:58:28.293255+0000","last_undegraded":"2024-06-13T06:58:28.293255+0000","last_fullsized":"2024-06-13T06:58:28.293255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:49:13.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.280038+0000","last_change":"2024-06-13T06:54:58.902576+0000","last_active":"2024-06-13T06:58:28.280038+0000","last_peered":"2024-06-13T06:58:28.280038+0000","last_clean":"2024-06-13T06:58:28.280038+0000","last_became_active":"2024-06-13T06:54:58.902205+0000","last_became_peered":"2024-06-13T06:54:58.902205+0000","last_unstale":"2024-06-13T06:58:28.280038+0000","last_undegraded":"2024-06-13T06:58:28.280038+0000","last_fullsized":"2024-06-13T06:58:28.280038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:13:24.107081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292246+0000","last_change":"2024-06-13T06:54:58.905392+0000","last_active":"2024-06-13T06:58:28.292246+0000","last_peered":"2024-06-13T06:58:28.292246+0000","last_clean":"2024-06-13T06:58:28.292246+0000","last_became_active":"2024-06-13T06:54:58.905017+0000","last_became_peered":"2024-06-13T06:54:58.905017+0000","last_unstale":"2024-06-13T06:58:28.292246+0000","last_undegraded":"2024-06-13T06:58:28.292246+0000","last_fullsized":"2024-06-13T06:58:28.292246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:05.931653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606898+0000","last_change":"2024-06-13T06:55:12.021359+0000","last_active":"2024-06-13T06:56:29.606898+0000","last_peered":"2024-06-13T06:56:29.606898+0000","last_clean":"2024-06-13T06:56:29.606898+0000","last_became_active":"2024-06-13T06:55:12.020860+0000","last_became_peered":"2024-06-13T06:55:12.020860+0000","last_unstale":"2024-06-13T06:56:29.606898+0000","last_undegraded":"2024-06-13T06:56:29.606898+0000","last_fullsized":"2024-06-13T06:56:29.606898+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:05:00.833905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.561282+0000","last_change":"2024-06-13T06:54:54.339220+0000","last_active":"2024-06-13T06:56:32.561282+0000","last_peered":"2024-06-13T06:56:32.561282+0000","last_clean":"2024-06-13T06:56:32.561282+0000","last_became_active":"2024-06-13T06:54:54.338573+0000","last_became_peered":"2024-06-13T06:54:54.338573+0000","last_unstale":"2024-06-13T06:56:32.561282+0000","last_undegraded":"2024-06-13T06:56:32.561282+0000","last_fullsized":"2024-06-13T06:56:32.561282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:53:50.258097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.277033+0000","last_change":"2024-06-13T06:54:58.903920+0000","last_active":"2024-06-13T06:58:28.277033+0000","last_peered":"2024-06-13T06:58:28.277033+0000","last_clean":"2024-06-13T06:58:28.277033+0000","last_became_active":"2024-06-13T06:54:58.895364+0000","last_became_peered":"2024-06-13T06:54:58.895364+0000","last_unstale":"2024-06-13T06:58:28.277033+0000","last_undegraded":"2024-06-13T06:58:28.277033+0000","last_fullsized":"2024-06-13T06:58:28.277033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:04:35.109799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607026+0000","last_change":"2024-06-13T06:55:12.017561+0000","last_active":"2024-06-13T06:56:29.607026+0000","last_peered":"2024-06-13T06:56:29.607026+0000","last_clean":"2024-06-13T06:56:29.607026+0000","last_became_active":"2024-06-13T06:55:12.017104+0000","last_became_peered":"2024-06-13T06:55:12.017104+0000","last_unstale":"2024-06-13T06:56:29.607026+0000","last_undegraded":"2024-06-13T06:56:29.607026+0000","last_fullsized":"2024-06-13T06:56:29.607026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:08:22.570491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526871+0000","last_change":"2024-06-13T06:54:54.631599+0000","last_active":"2024-06-13T06:56:29.526871+0000","last_peered":"2024-06-13T06:56:29.526871+0000","last_clean":"2024-06-13T06:56:29.526871+0000","last_became_active":"2024-06-13T06:54:54.631007+0000","last_became_peered":"2024-06-13T06:54:54.631007+0000","last_unstale":"2024-06-13T06:56:29.526871+0000","last_undegraded":"2024-06-13T06:56:29.526871+0000","last_fullsized":"2024-06-13T06:56:29.526871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:49:56.602192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304403+0000","last_change":"2024-06-13T06:54:58.917255+0000","last_active":"2024-06-13T06:58:28.304403+0000","last_peered":"2024-06-13T06:58:28.304403+0000","last_clean":"2024-06-13T06:58:28.304403+0000","last_became_active":"2024-06-13T06:54:58.916875+0000","last_became_peered":"2024-06-13T06:54:58.916875+0000","last_unstale":"2024-06-13T06:58:28.304403+0000","last_undegraded":"2024-06-13T06:58:28.304403+0000","last_fullsized":"2024-06-13T06:58:28.304403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:09:47.638802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608521+0000","last_change":"2024-06-13T06:55:12.018209+0000","last_active":"2024-06-13T06:56:29.608521+0000","last_peered":"2024-06-13T06:56:29.608521+0000","last_clean":"2024-06-13T06:56:29.608521+0000","last_became_active":"2024-06-13T06:55:12.017038+0000","last_became_peered":"2024-06-13T06:55:12.017038+0000","last_unstale":"2024-06-13T06:56:29.608521+0000","last_undegraded":"2024-06-13T06:56:29.608521+0000","last_fullsized":"2024-06-13T06:56:29.608521+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:36:26.476454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522812+0000","last_change":"2024-06-13T06:54:54.328849+0000","last_active":"2024-06-13T06:56:29.522812+0000","last_peered":"2024-06-13T06:56:29.522812+0000","last_clean":"2024-06-13T06:56:29.522812+0000","last_became_active":"2024-06-13T06:54:54.328388+0000","last_became_peered":"2024-06-13T06:54:54.328388+0000","last_unstale":"2024-06-13T06:56:29.522812+0000","last_undegraded":"2024-06-13T06:56:29.522812+0000","last_fullsized":"2024-06-13T06:56:29.522812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:41.264403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282933+0000","last_change":"2024-06-13T06:54:58.897588+0000","last_active":"2024-06-13T06:58:28.282933+0000","last_peered":"2024-06-13T06:58:28.282933+0000","last_clean":"2024-06-13T06:58:28.282933+0000","last_became_active":"2024-06-13T06:54:58.897236+0000","last_became_peered":"2024-06-13T06:54:58.897236+0000","last_unstale":"2024-06-13T06:58:28.282933+0000","last_undegraded":"2024-06-13T06:58:28.282933+0000","last_fullsized":"2024-06-13T06:58:28.282933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:20:38.460445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607292+0000","last_change":"2024-06-13T06:55:12.014080+0000","last_active":"2024-06-13T06:56:29.607292+0000","last_peered":"2024-06-13T06:56:29.607292+0000","last_clean":"2024-06-13T06:56:29.607292+0000","last_became_active":"2024-06-13T06:55:12.013627+0000","last_became_peered":"2024-06-13T06:55:12.013627+0000","last_unstale":"2024-06-13T06:56:29.607292+0000","last_undegraded":"2024-06-13T06:56:29.607292+0000","last_fullsized":"2024-06-13T06:56:29.607292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:29:54.081593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.522259+0000","last_change":"2024-06-13T06:54:54.328249+0000","last_active":"2024-06-13T06:56:29.522259+0000","last_peered":"2024-06-13T06:56:29.522259+0000","last_clean":"2024-06-13T06:56:29.522259+0000","last_became_active":"2024-06-13T06:54:54.327832+0000","last_became_peered":"2024-06-13T06:54:54.327832+0000","last_unstale":"2024-06-13T06:56:29.522259+0000","last_undegraded":"2024-06-13T06:56:29.522259+0000","last_fullsized":"2024-06-13T06:56:29.522259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:21:05.624075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281353+0000","last_change":"2024-06-13T06:55:29.383305+0000","last_active":"2024-06-13T06:58:28.281353+0000","last_peered":"2024-06-13T06:58:28.281353+0000","last_clean":"2024-06-13T06:58:28.281353+0000","last_became_active":"2024-06-13T06:55:29.381491+0000","last_became_peered":"2024-06-13T06:55:29.381491+0000","last_unstale":"2024-06-13T06:58:28.281353+0000","last_undegraded":"2024-06-13T06:58:28.281353+0000","last_fullsized":"2024-06-13T06:58:28.281353+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:52:26.437237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607257+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:56:29.607257+0000","last_peered":"2024-06-13T06:56:29.607257+0000","last_clean":"2024-06-13T06:56:29.607257+0000","last_became_active":"2024-06-13T06:55:12.000759+0000","last_became_peered":"2024-06-13T06:55:12.000759+0000","last_unstale":"2024-06-13T06:56:29.607257+0000","last_undegraded":"2024-06-13T06:56:29.607257+0000","last_fullsized":"2024-06-13T06:56:29.607257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:11:35.805338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'7","reported_seq":191,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:39.815500+0000","last_change":"2024-06-13T06:55:29.367538+0000","last_active":"2024-06-13T06:58:39.815500+0000","last_peered":"2024-06-13T06:58:39.815500+0000","last_clean":"2024-06-13T06:58:39.815500+0000","last_became_active":"2024-06-13T06:55:29.367149+0000","last_became_peered":"2024-06-13T06:55:29.367149+0000","last_unstale":"2024-06-13T06:58:39.815500+0000","last_undegraded":"2024-06-13T06:58:39.815500+0000","last_fullsized":"2024-06-13T06:58:39.815500+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:30:55.632799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.268814+0000","last_change":"2024-06-13T06:54:58.896258+0000","last_active":"2024-06-13T06:58:28.268814+0000","last_peered":"2024-06-13T06:58:28.268814+0000","last_clean":"2024-06-13T06:58:28.268814+0000","last_became_active":"2024-06-13T06:54:58.895904+0000","last_became_peered":"2024-06-13T06:54:58.895904+0000","last_unstale":"2024-06-13T06:58:28.268814+0000","last_undegraded":"2024-06-13T06:58:28.268814+0000","last_fullsized":"2024-06-13T06:58:28.268814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:28:33.615338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607271+0000","last_change":"2024-06-13T06:55:12.018499+0000","last_active":"2024-06-13T06:56:29.607271+0000","last_peered":"2024-06-13T06:56:29.607271+0000","last_clean":"2024-06-13T06:56:29.607271+0000","last_became_active":"2024-06-13T06:55:12.018073+0000","last_became_peered":"2024-06-13T06:55:12.018073+0000","last_unstale":"2024-06-13T06:56:29.607271+0000","last_undegraded":"2024-06-13T06:56:29.607271+0000","last_fullsized":"2024-06-13T06:56:29.607271+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:59:53.206259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.523149+0000","last_change":"2024-06-13T06:54:54.339482+0000","last_active":"2024-06-13T06:56:29.523149+0000","last_peered":"2024-06-13T06:56:29.523149+0000","last_clean":"2024-06-13T06:56:29.523149+0000","last_became_active":"2024-06-13T06:54:54.339066+0000","last_became_peered":"2024-06-13T06:54:54.339066+0000","last_unstale":"2024-06-13T06:56:29.523149+0000","last_undegraded":"2024-06-13T06:56:29.523149+0000","last_fullsized":"2024-06-13T06:56:29.523149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:31:04.136530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279127+0000","last_change":"2024-06-13T06:54:58.891154+0000","last_active":"2024-06-13T06:58:28.279127+0000","last_peered":"2024-06-13T06:58:28.279127+0000","last_clean":"2024-06-13T06:58:28.279127+0000","last_became_active":"2024-06-13T06:54:58.890810+0000","last_became_peered":"2024-06-13T06:54:58.890810+0000","last_unstale":"2024-06-13T06:58:28.279127+0000","last_undegraded":"2024-06-13T06:58:28.279127+0000","last_fullsized":"2024-06-13T06:58:28.279127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:14.618233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524803+0000","last_change":"2024-06-13T06:55:12.025619+0000","last_active":"2024-06-13T06:56:29.524803+0000","last_peered":"2024-06-13T06:56:29.524803+0000","last_clean":"2024-06-13T06:56:29.524803+0000","last_became_active":"2024-06-13T06:55:12.025153+0000","last_became_peered":"2024-06-13T06:55:12.025153+0000","last_unstale":"2024-06-13T06:56:29.524803+0000","last_undegraded":"2024-06-13T06:56:29.524803+0000","last_fullsized":"2024-06-13T06:56:29.524803+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:12.124550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.596897+0000","last_change":"2024-06-13T06:54:54.341698+0000","last_active":"2024-06-13T06:56:29.596897+0000","last_peered":"2024-06-13T06:56:29.596897+0000","last_clean":"2024-06-13T06:56:29.596897+0000","last_became_active":"2024-06-13T06:54:54.341113+0000","last_became_peered":"2024-06-13T06:54:54.341113+0000","last_unstale":"2024-06-13T06:56:29.596897+0000","last_undegraded":"2024-06-13T06:56:29.596897+0000","last_fullsized":"2024-06-13T06:56:29.596897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:18:23.368705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279404+0000","last_change":"2024-06-13T06:54:58.913688+0000","last_active":"2024-06-13T06:58:28.279404+0000","last_peered":"2024-06-13T06:58:28.279404+0000","last_clean":"2024-06-13T06:58:28.279404+0000","last_became_active":"2024-06-13T06:54:58.910493+0000","last_became_peered":"2024-06-13T06:54:58.910493+0000","last_unstale":"2024-06-13T06:58:28.279404+0000","last_undegraded":"2024-06-13T06:58:28.279404+0000","last_fullsized":"2024-06-13T06:58:28.279404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:29:18.081743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.532034+0000","last_change":"2024-06-13T06:55:12.016257+0000","last_active":"2024-06-13T06:56:29.532034+0000","last_peered":"2024-06-13T06:56:29.532034+0000","last_clean":"2024-06-13T06:56:29.532034+0000","last_became_active":"2024-06-13T06:55:12.015594+0000","last_became_peered":"2024-06-13T06:55:12.015594+0000","last_unstale":"2024-06-13T06:56:29.532034+0000","last_undegraded":"2024-06-13T06:56:29.532034+0000","last_fullsized":"2024-06-13T06:56:29.532034+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:02:44.642075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525725+0000","last_change":"2024-06-13T06:54:54.330366+0000","last_active":"2024-06-13T06:56:29.525725+0000","last_peered":"2024-06-13T06:56:29.525725+0000","last_clean":"2024-06-13T06:56:29.525725+0000","last_became_active":"2024-06-13T06:54:54.330010+0000","last_became_peered":"2024-06-13T06:54:54.330010+0000","last_unstale":"2024-06-13T06:56:29.525725+0000","last_undegraded":"2024-06-13T06:56:29.525725+0000","last_fullsized":"2024-06-13T06:56:29.525725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:36:01.488766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276429+0000","last_change":"2024-06-13T06:54:58.918076+0000","last_active":"2024-06-13T06:58:28.276429+0000","last_peered":"2024-06-13T06:58:28.276429+0000","last_clean":"2024-06-13T06:58:28.276429+0000","last_became_active":"2024-06-13T06:54:58.917631+0000","last_became_peered":"2024-06-13T06:54:58.917631+0000","last_unstale":"2024-06-13T06:58:28.276429+0000","last_undegraded":"2024-06-13T06:58:28.276429+0000","last_fullsized":"2024-06-13T06:58:28.276429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:31:00.056837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.613299+0000","last_change":"2024-06-13T06:56:29.611761+0000","last_active":"2024-06-13T06:56:29.613299+0000","last_peered":"2024-06-13T06:56:29.613299+0000","last_clean":"2024-06-13T06:56:29.613299+0000","last_became_active":"2024-06-13T06:56:29.610385+0000","last_became_peered":"2024-06-13T06:56:29.610385+0000","last_unstale":"2024-06-13T06:56:29.613299+0000","last_undegraded":"2024-06-13T06:56:29.613299+0000","last_fullsized":"2024-06-13T06:56:29.613299+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:26:10.678918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":180,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:41.818594+0000","last_change":"2024-06-13T06:54:54.335324+0000","last_active":"2024-06-13T06:58:41.818594+0000","last_peered":"2024-06-13T06:58:41.818594+0000","last_clean":"2024-06-13T06:58:41.818594+0000","last_became_active":"2024-06-13T06:54:54.334936+0000","last_became_peered":"2024-06-13T06:54:54.334936+0000","last_unstale":"2024-06-13T06:58:41.818594+0000","last_undegraded":"2024-06-13T06:58:41.818594+0000","last_fullsized":"2024-06-13T06:58:41.818594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:41:52.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281052+0000","last_change":"2024-06-13T06:54:58.902146+0000","last_active":"2024-06-13T06:58:28.281052+0000","last_peered":"2024-06-13T06:58:28.281052+0000","last_clean":"2024-06-13T06:58:28.281052+0000","last_became_active":"2024-06-13T06:54:58.901807+0000","last_became_peered":"2024-06-13T06:54:58.901807+0000","last_unstale":"2024-06-13T06:58:28.281052+0000","last_undegraded":"2024-06-13T06:58:28.281052+0000","last_fullsized":"2024-06-13T06:58:28.281052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:59:53.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533203+0000","last_change":"2024-06-13T06:55:12.025418+0000","last_active":"2024-06-13T06:56:29.533203+0000","last_peered":"2024-06-13T06:56:29.533203+0000","last_clean":"2024-06-13T06:56:29.533203+0000","last_became_active":"2024-06-13T06:55:12.024859+0000","last_became_peered":"2024-06-13T06:55:12.024859+0000","last_unstale":"2024-06-13T06:56:29.533203+0000","last_undegraded":"2024-06-13T06:56:29.533203+0000","last_fullsized":"2024-06-13T06:56:29.533203+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:05:17.553169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.609432+0000","last_change":"2024-06-13T06:54:54.326011+0000","last_active":"2024-06-13T06:56:29.609432+0000","last_peered":"2024-06-13T06:56:29.609432+0000","last_clean":"2024-06-13T06:56:29.609432+0000","last_became_active":"2024-06-13T06:54:54.325664+0000","last_became_peered":"2024-06-13T06:54:54.325664+0000","last_unstale":"2024-06-13T06:56:29.609432+0000","last_undegraded":"2024-06-13T06:56:29.609432+0000","last_fullsized":"2024-06-13T06:56:29.609432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:32.973094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301619+0000","last_change":"2024-06-13T06:55:29.383612+0000","last_active":"2024-06-13T06:58:28.301619+0000","last_peered":"2024-06-13T06:58:28.301619+0000","last_clean":"2024-06-13T06:58:28.301619+0000","last_became_active":"2024-06-13T06:55:29.382174+0000","last_became_peered":"2024-06-13T06:55:29.382174+0000","last_unstale":"2024-06-13T06:58:28.301619+0000","last_undegraded":"2024-06-13T06:58:28.301619+0000","last_fullsized":"2024-06-13T06:58:28.301619+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:09:35.178209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608743+0000","last_change":"2024-06-13T06:55:12.018654+0000","last_active":"2024-06-13T06:56:29.608743+0000","last_peered":"2024-06-13T06:56:29.608743+0000","last_clean":"2024-06-13T06:56:29.608743+0000","last_became_active":"2024-06-13T06:55:12.017219+0000","last_became_peered":"2024-06-13T06:55:12.017219+0000","last_unstale":"2024-06-13T06:56:29.608743+0000","last_undegraded":"2024-06-13T06:56:29.608743+0000","last_fullsized":"2024-06-13T06:56:29.608743+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:46:54.161912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.562063+0000","last_change":"2024-06-13T06:54:54.632154+0000","last_active":"2024-06-13T06:56:32.562063+0000","last_peered":"2024-06-13T06:56:32.562063+0000","last_clean":"2024-06-13T06:56:32.562063+0000","last_became_active":"2024-06-13T06:54:54.631721+0000","last_became_peered":"2024-06-13T06:54:54.631721+0000","last_unstale":"2024-06-13T06:56:32.562063+0000","last_undegraded":"2024-06-13T06:56:32.562063+0000","last_fullsized":"2024-06-13T06:56:32.562063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:16:10.492144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278120+0000","last_change":"2024-06-13T06:54:58.898961+0000","last_active":"2024-06-13T06:58:28.278120+0000","last_peered":"2024-06-13T06:58:28.278120+0000","last_clean":"2024-06-13T06:58:28.278120+0000","last_became_active":"2024-06-13T06:54:58.898599+0000","last_became_peered":"2024-06-13T06:54:58.898599+0000","last_unstale":"2024-06-13T06:58:28.278120+0000","last_undegraded":"2024-06-13T06:58:28.278120+0000","last_fullsized":"2024-06-13T06:58:28.278120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:21:53.044205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.527672+0000","last_change":"2024-06-13T06:56:29.527242+0000","last_active":"2024-06-13T06:56:29.527672+0000","last_peered":"2024-06-13T06:56:29.527672+0000","last_clean":"2024-06-13T06:56:29.527672+0000","last_became_active":"2024-06-13T06:56:29.526816+0000","last_became_peered":"2024-06-13T06:56:29.526816+0000","last_unstale":"2024-06-13T06:56:29.527672+0000","last_undegraded":"2024-06-13T06:56:29.527672+0000","last_fullsized":"2024-06-13T06:56:29.527672+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:45:17.770029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'36","reported_seq":368,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:42.772989+0000","last_change":"2024-06-13T06:55:29.376808+0000","last_active":"2024-06-13T06:58:42.772989+0000","last_peered":"2024-06-13T06:58:42.772989+0000","last_clean":"2024-06-13T06:58:42.772989+0000","last_became_active":"2024-06-13T06:55:29.367134+0000","last_became_peered":"2024-06-13T06:55:29.367134+0000","last_unstale":"2024-06-13T06:58:42.772989+0000","last_undegraded":"2024-06-13T06:58:42.772989+0000","last_fullsized":"2024-06-13T06:58:42.772989+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:19:08.784438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62904,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":271,"num_read_kb":198,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.265076+0000","last_change":"2024-06-13T06:54:58.897686+0000","last_active":"2024-06-13T06:58:28.265076+0000","last_peered":"2024-06-13T06:58:28.265076+0000","last_clean":"2024-06-13T06:58:28.265076+0000","last_became_active":"2024-06-13T06:54:58.897313+0000","last_became_peered":"2024-06-13T06:54:58.897313+0000","last_unstale":"2024-06-13T06:58:28.265076+0000","last_undegraded":"2024-06-13T06:58:28.265076+0000","last_fullsized":"2024-06-13T06:58:28.265076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:54:49.558542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526786+0000","last_change":"2024-06-13T06:55:12.013918+0000","last_active":"2024-06-13T06:56:29.526786+0000","last_peered":"2024-06-13T06:56:29.526786+0000","last_clean":"2024-06-13T06:56:29.526786+0000","last_became_active":"2024-06-13T06:55:12.013522+0000","last_became_peered":"2024-06-13T06:55:12.013522+0000","last_unstale":"2024-06-13T06:56:29.526786+0000","last_undegraded":"2024-06-13T06:56:29.526786+0000","last_fullsized":"2024-06-13T06:56:29.526786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:51:46.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556331+0000","last_change":"2024-06-13T06:54:54.328674+0000","last_active":"2024-06-13T06:56:32.556331+0000","last_peered":"2024-06-13T06:56:32.556331+0000","last_clean":"2024-06-13T06:56:32.556331+0000","last_became_active":"2024-06-13T06:54:54.326789+0000","last_became_peered":"2024-06-13T06:54:54.326789+0000","last_unstale":"2024-06-13T06:56:32.556331+0000","last_undegraded":"2024-06-13T06:56:32.556331+0000","last_fullsized":"2024-06-13T06:56:32.556331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:18.280514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.290908+0000","last_change":"2024-06-13T06:54:58.918598+0000","last_active":"2024-06-13T06:58:28.290908+0000","last_peered":"2024-06-13T06:58:28.290908+0000","last_clean":"2024-06-13T06:58:28.290908+0000","last_became_active":"2024-06-13T06:54:58.918247+0000","last_became_peered":"2024-06-13T06:54:58.918247+0000","last_unstale":"2024-06-13T06:58:28.290908+0000","last_undegraded":"2024-06-13T06:58:28.290908+0000","last_fullsized":"2024-06-13T06:58:28.290908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:08:56.738806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524314+0000","last_change":"2024-06-13T06:55:12.017074+0000","last_active":"2024-06-13T06:56:29.524314+0000","last_peered":"2024-06-13T06:56:29.524314+0000","last_clean":"2024-06-13T06:56:29.524314+0000","last_became_active":"2024-06-13T06:55:12.016675+0000","last_became_peered":"2024-06-13T06:55:12.016675+0000","last_unstale":"2024-06-13T06:56:29.524314+0000","last_undegraded":"2024-06-13T06:56:29.524314+0000","last_fullsized":"2024-06-13T06:56:29.524314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:20:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525973+0000","last_change":"2024-06-13T06:54:54.334994+0000","last_active":"2024-06-13T06:56:29.525973+0000","last_peered":"2024-06-13T06:56:29.525973+0000","last_clean":"2024-06-13T06:56:29.525973+0000","last_became_active":"2024-06-13T06:54:54.334647+0000","last_became_peered":"2024-06-13T06:54:54.334647+0000","last_unstale":"2024-06-13T06:56:29.525973+0000","last_undegraded":"2024-06-13T06:56:29.525973+0000","last_fullsized":"2024-06-13T06:56:29.525973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:59:15.317976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275875+0000","last_change":"2024-06-13T06:54:58.899107+0000","last_active":"2024-06-13T06:58:28.275875+0000","last_peered":"2024-06-13T06:58:28.275875+0000","last_clean":"2024-06-13T06:58:28.275875+0000","last_became_active":"2024-06-13T06:54:58.896034+0000","last_became_peered":"2024-06-13T06:54:58.896034+0000","last_unstale":"2024-06-13T06:58:28.275875+0000","last_undegraded":"2024-06-13T06:58:28.275875+0000","last_fullsized":"2024-06-13T06:58:28.275875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:00:01.132565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533277+0000","last_change":"2024-06-13T06:55:12.025681+0000","last_active":"2024-06-13T06:56:29.533277+0000","last_peered":"2024-06-13T06:56:29.533277+0000","last_clean":"2024-06-13T06:56:29.533277+0000","last_became_active":"2024-06-13T06:55:12.025267+0000","last_became_peered":"2024-06-13T06:55:12.025267+0000","last_unstale":"2024-06-13T06:56:29.533277+0000","last_undegraded":"2024-06-13T06:56:29.533277+0000","last_fullsized":"2024-06-13T06:56:29.533277+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:58:23.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":181,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:41.819931+0000","last_change":"2024-06-13T06:54:54.325404+0000","last_active":"2024-06-13T06:58:41.819931+0000","last_peered":"2024-06-13T06:58:41.819931+0000","last_clean":"2024-06-13T06:58:41.819931+0000","last_became_active":"2024-06-13T06:54:54.325024+0000","last_became_peered":"2024-06-13T06:54:54.325024+0000","last_unstale":"2024-06-13T06:58:41.819931+0000","last_undegraded":"2024-06-13T06:58:41.819931+0000","last_fullsized":"2024-06-13T06:58:41.819931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:23.726350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272642+0000","last_change":"2024-06-13T06:54:58.894737+0000","last_active":"2024-06-13T06:58:28.272642+0000","last_peered":"2024-06-13T06:58:28.272642+0000","last_clean":"2024-06-13T06:58:28.272642+0000","last_became_active":"2024-06-13T06:54:58.894224+0000","last_became_peered":"2024-06-13T06:54:58.894224+0000","last_unstale":"2024-06-13T06:58:28.272642+0000","last_undegraded":"2024-06-13T06:58:28.272642+0000","last_fullsized":"2024-06-13T06:58:28.272642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:03.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600221+0000","last_change":"2024-06-13T06:55:12.026587+0000","last_active":"2024-06-13T06:56:29.600221+0000","last_peered":"2024-06-13T06:56:29.600221+0000","last_clean":"2024-06-13T06:56:29.600221+0000","last_became_active":"2024-06-13T06:55:12.026114+0000","last_became_peered":"2024-06-13T06:55:12.026114+0000","last_unstale":"2024-06-13T06:56:29.600221+0000","last_undegraded":"2024-06-13T06:56:29.600221+0000","last_fullsized":"2024-06-13T06:56:29.600221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:43:09.245641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556229+0000","last_change":"2024-06-13T06:54:54.632069+0000","last_active":"2024-06-13T06:56:32.556229+0000","last_peered":"2024-06-13T06:56:32.556229+0000","last_clean":"2024-06-13T06:56:32.556229+0000","last_became_active":"2024-06-13T06:54:54.631692+0000","last_became_peered":"2024-06-13T06:54:54.631692+0000","last_unstale":"2024-06-13T06:56:32.556229+0000","last_undegraded":"2024-06-13T06:56:32.556229+0000","last_fullsized":"2024-06-13T06:56:32.556229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:07.109507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.258901+0000","last_change":"2024-06-13T06:55:29.366286+0000","last_active":"2024-06-13T06:58:28.258901+0000","last_peered":"2024-06-13T06:58:28.258901+0000","last_clean":"2024-06-13T06:58:28.258901+0000","last_became_active":"2024-06-13T06:55:29.365854+0000","last_became_peered":"2024-06-13T06:55:29.365854+0000","last_unstale":"2024-06-13T06:58:28.258901+0000","last_undegraded":"2024-06-13T06:58:28.258901+0000","last_fullsized":"2024-06-13T06:58:28.258901+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:04.209244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.526226+0000","last_change":"2024-06-13T06:55:12.021621+0000","last_active":"2024-06-13T06:56:29.526226+0000","last_peered":"2024-06-13T06:56:29.526226+0000","last_clean":"2024-06-13T06:56:29.526226+0000","last_became_active":"2024-06-13T06:55:12.021150+0000","last_became_peered":"2024-06-13T06:55:12.021150+0000","last_unstale":"2024-06-13T06:56:29.526226+0000","last_undegraded":"2024-06-13T06:56:29.526226+0000","last_fullsized":"2024-06-13T06:56:29.526226+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:48.318918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":22,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556254+0000","last_change":"2024-06-13T06:55:29.369972+0000","last_active":"2024-06-13T06:56:32.556254+0000","last_peered":"2024-06-13T06:56:32.556254+0000","last_clean":"2024-06-13T06:56:32.556254+0000","last_became_active":"2024-06-13T06:55:29.366390+0000","last_became_peered":"2024-06-13T06:55:29.366390+0000","last_unstale":"2024-06-13T06:56:32.556254+0000","last_undegraded":"2024-06-13T06:56:32.556254+0000","last_fullsized":"2024-06-13T06:56:32.556254+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:18.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271750+0000","last_change":"2024-06-13T06:54:58.891309+0000","last_active":"2024-06-13T06:58:28.271750+0000","last_peered":"2024-06-13T06:58:28.271750+0000","last_clean":"2024-06-13T06:58:28.271750+0000","last_became_active":"2024-06-13T06:54:58.890972+0000","last_became_peered":"2024-06-13T06:54:58.890972+0000","last_unstale":"2024-06-13T06:58:28.271750+0000","last_undegraded":"2024-06-13T06:58:28.271750+0000","last_fullsized":"2024-06-13T06:58:28.271750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:41.240061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.611267+0000","last_change":"2024-06-13T06:56:29.610713+0000","last_active":"2024-06-13T06:56:29.611267+0000","last_peered":"2024-06-13T06:56:29.611267+0000","last_clean":"2024-06-13T06:56:29.611267+0000","last_became_active":"2024-06-13T06:56:29.610139+0000","last_became_peered":"2024-06-13T06:56:29.610139+0000","last_unstale":"2024-06-13T06:56:29.611267+0000","last_undegraded":"2024-06-13T06:56:29.611267+0000","last_fullsized":"2024-06-13T06:56:29.611267+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:14:36.601243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.534462+0000","last_change":"2024-06-13T06:54:54.633893+0000","last_active":"2024-06-13T06:56:29.534462+0000","last_peered":"2024-06-13T06:56:29.534462+0000","last_clean":"2024-06-13T06:56:29.534462+0000","last_became_active":"2024-06-13T06:54:54.632946+0000","last_became_peered":"2024-06-13T06:54:54.632946+0000","last_unstale":"2024-06-13T06:56:29.534462+0000","last_undegraded":"2024-06-13T06:56:29.534462+0000","last_fullsized":"2024-06-13T06:56:29.534462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:16:54.812477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279749+0000","last_change":"2024-06-13T06:54:58.907182+0000","last_active":"2024-06-13T06:58:28.279749+0000","last_peered":"2024-06-13T06:58:28.279749+0000","last_clean":"2024-06-13T06:58:28.279749+0000","last_became_active":"2024-06-13T06:54:58.906824+0000","last_became_peered":"2024-06-13T06:54:58.906824+0000","last_unstale":"2024-06-13T06:58:28.279749+0000","last_undegraded":"2024-06-13T06:58:28.279749+0000","last_fullsized":"2024-06-13T06:58:28.279749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:34:49.626991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.597238+0000","last_change":"2024-06-13T06:55:12.014803+0000","last_active":"2024-06-13T06:56:29.597238+0000","last_peered":"2024-06-13T06:56:29.597238+0000","last_clean":"2024-06-13T06:56:29.597238+0000","last_became_active":"2024-06-13T06:55:12.014398+0000","last_became_peered":"2024-06-13T06:55:12.014398+0000","last_unstale":"2024-06-13T06:56:29.597238+0000","last_undegraded":"2024-06-13T06:56:29.597238+0000","last_fullsized":"2024-06-13T06:56:29.597238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:07:44.960446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607834+0000","last_change":"2024-06-13T06:54:54.339056+0000","last_active":"2024-06-13T06:56:29.607834+0000","last_peered":"2024-06-13T06:56:29.607834+0000","last_clean":"2024-06-13T06:56:29.607834+0000","last_became_active":"2024-06-13T06:54:54.338683+0000","last_became_peered":"2024-06-13T06:54:54.338683+0000","last_unstale":"2024-06-13T06:56:29.607834+0000","last_undegraded":"2024-06-13T06:56:29.607834+0000","last_fullsized":"2024-06-13T06:56:29.607834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:05:17.060471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296931+0000","last_change":"2024-06-13T06:54:58.919843+0000","last_active":"2024-06-13T06:58:28.296931+0000","last_peered":"2024-06-13T06:58:28.296931+0000","last_clean":"2024-06-13T06:58:28.296931+0000","last_became_active":"2024-06-13T06:54:58.919405+0000","last_became_peered":"2024-06-13T06:54:58.919405+0000","last_unstale":"2024-06-13T06:58:28.296931+0000","last_undegraded":"2024-06-13T06:58:28.296931+0000","last_fullsized":"2024-06-13T06:58:28.296931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:00:20.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600188+0000","last_change":"2024-06-13T06:55:12.097651+0000","last_active":"2024-06-13T06:56:29.600188+0000","last_peered":"2024-06-13T06:56:29.600188+0000","last_clean":"2024-06-13T06:56:29.600188+0000","last_became_active":"2024-06-13T06:55:12.095644+0000","last_became_peered":"2024-06-13T06:55:12.095644+0000","last_unstale":"2024-06-13T06:56:29.600188+0000","last_undegraded":"2024-06-13T06:56:29.600188+0000","last_fullsized":"2024-06-13T06:56:29.600188+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:09:54.328428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.257770+0000","last_change":"2024-06-13T06:54:58.891187+0000","last_active":"2024-06-13T06:58:28.257770+0000","last_peered":"2024-06-13T06:58:28.257770+0000","last_clean":"2024-06-13T06:58:28.257770+0000","last_became_active":"2024-06-13T06:54:58.890848+0000","last_became_peered":"2024-06-13T06:54:58.890848+0000","last_unstale":"2024-06-13T06:58:28.257770+0000","last_undegraded":"2024-06-13T06:58:28.257770+0000","last_fullsized":"2024-06-13T06:58:28.257770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:13:31.709650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":179,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:40.817212+0000","last_change":"2024-06-13T06:54:54.631482+0000","last_active":"2024-06-13T06:58:40.817212+0000","last_peered":"2024-06-13T06:58:40.817212+0000","last_clean":"2024-06-13T06:58:40.817212+0000","last_became_active":"2024-06-13T06:54:54.630818+0000","last_became_peered":"2024-06-13T06:54:54.630818+0000","last_unstale":"2024-06-13T06:58:40.817212+0000","last_undegraded":"2024-06-13T06:58:40.817212+0000","last_fullsized":"2024-06-13T06:58:40.817212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:34:18.801727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:45.996461+0000","last_change":"2024-06-13T06:55:12.018856+0000","last_active":"2024-06-13T06:56:45.996461+0000","last_peered":"2024-06-13T06:56:45.996461+0000","last_clean":"2024-06-13T06:56:45.996461+0000","last_became_active":"2024-06-13T06:55:12.018488+0000","last_became_peered":"2024-06-13T06:55:12.018488+0000","last_unstale":"2024-06-13T06:56:45.996461+0000","last_undegraded":"2024-06-13T06:56:45.996461+0000","last_fullsized":"2024-06-13T06:56:45.996461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:14.973197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281746+0000","last_change":"2024-06-13T06:55:29.377203+0000","last_active":"2024-06-13T06:58:28.281746+0000","last_peered":"2024-06-13T06:58:28.281746+0000","last_clean":"2024-06-13T06:58:28.281746+0000","last_became_active":"2024-06-13T06:55:29.375263+0000","last_became_peered":"2024-06-13T06:55:29.375263+0000","last_unstale":"2024-06-13T06:58:28.281746+0000","last_undegraded":"2024-06-13T06:58:28.281746+0000","last_fullsized":"2024-06-13T06:58:28.281746+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:06:07.935172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558452+0000","last_change":"2024-06-13T06:54:54.335319+0000","last_active":"2024-06-13T06:56:32.558452+0000","last_peered":"2024-06-13T06:56:32.558452+0000","last_clean":"2024-06-13T06:56:32.558452+0000","last_became_active":"2024-06-13T06:54:54.334896+0000","last_became_peered":"2024-06-13T06:54:54.334896+0000","last_unstale":"2024-06-13T06:56:32.558452+0000","last_undegraded":"2024-06-13T06:56:32.558452+0000","last_fullsized":"2024-06-13T06:56:32.558452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:40.331535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607040+0000","last_change":"2024-06-13T06:53:16.968830+0000","last_active":"2024-06-13T06:56:29.607040+0000","last_peered":"2024-06-13T06:56:29.607040+0000","last_clean":"2024-06-13T06:56:29.607040+0000","last_became_active":"2024-06-13T06:53:16.968361+0000","last_became_peered":"2024-06-13T06:53:16.968361+0000","last_unstale":"2024-06-13T06:56:29.607040+0000","last_undegraded":"2024-06-13T06:56:29.607040+0000","last_fullsized":"2024-06-13T06:56:29.607040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_clean_scrub_stamp":"2024-06-13T06:49:18.858540+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:32:26.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.602642+0000","last_change":"2024-06-13T06:56:29.602124+0000","last_active":"2024-06-13T06:56:29.602642+0000","last_peered":"2024-06-13T06:56:29.602642+0000","last_clean":"2024-06-13T06:56:29.602642+0000","last_became_active":"2024-06-13T06:56:29.601650+0000","last_became_peered":"2024-06-13T06:56:29.601650+0000","last_unstale":"2024-06-13T06:56:29.602642+0000","last_undegraded":"2024-06-13T06:56:29.602642+0000","last_fullsized":"2024-06-13T06:56:29.602642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:37:23.903614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.294492+0000","last_change":"2024-06-13T06:54:58.898930+0000","last_active":"2024-06-13T06:58:28.294492+0000","last_peered":"2024-06-13T06:58:28.294492+0000","last_clean":"2024-06-13T06:58:28.294492+0000","last_became_active":"2024-06-13T06:54:58.898384+0000","last_became_peered":"2024-06-13T06:54:58.898384+0000","last_unstale":"2024-06-13T06:58:28.294492+0000","last_undegraded":"2024-06-13T06:58:28.294492+0000","last_fullsized":"2024-06-13T06:58:28.294492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:06:02.210336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607221+0000","last_change":"2024-06-13T06:54:54.328776+0000","last_active":"2024-06-13T06:56:29.607221+0000","last_peered":"2024-06-13T06:56:29.607221+0000","last_clean":"2024-06-13T06:56:29.607221+0000","last_became_active":"2024-06-13T06:54:54.327773+0000","last_became_peered":"2024-06-13T06:54:54.327773+0000","last_unstale":"2024-06-13T06:56:29.607221+0000","last_undegraded":"2024-06-13T06:56:29.607221+0000","last_fullsized":"2024-06-13T06:56:29.607221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:23:02.240774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524508+0000","last_change":"2024-06-13T06:55:12.013862+0000","last_active":"2024-06-13T06:56:29.524508+0000","last_peered":"2024-06-13T06:56:29.524508+0000","last_clean":"2024-06-13T06:56:29.524508+0000","last_became_active":"2024-06-13T06:55:12.013389+0000","last_became_peered":"2024-06-13T06:55:12.013389+0000","last_unstale":"2024-06-13T06:56:29.524508+0000","last_undegraded":"2024-06-13T06:56:29.524508+0000","last_fullsized":"2024-06-13T06:56:29.524508+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:49.701202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558954+0000","last_change":"2024-06-13T06:56:29.539862+0000","last_active":"2024-06-13T06:56:32.558954+0000","last_peered":"2024-06-13T06:56:32.558954+0000","last_clean":"2024-06-13T06:56:32.558954+0000","last_became_active":"2024-06-13T06:56:29.536109+0000","last_became_peered":"2024-06-13T06:56:29.536109+0000","last_unstale":"2024-06-13T06:56:32.558954+0000","last_undegraded":"2024-06-13T06:56:32.558954+0000","last_fullsized":"2024-06-13T06:56:32.558954+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:23.307850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.287464+0000","last_change":"2024-06-13T06:54:58.901773+0000","last_active":"2024-06-13T06:58:28.287464+0000","last_peered":"2024-06-13T06:58:28.287464+0000","last_clean":"2024-06-13T06:58:28.287464+0000","last_became_active":"2024-06-13T06:54:58.901168+0000","last_became_peered":"2024-06-13T06:54:58.901168+0000","last_unstale":"2024-06-13T06:58:28.287464+0000","last_undegraded":"2024-06-13T06:58:28.287464+0000","last_fullsized":"2024-06-13T06:58:28.287464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:57:15.848785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524234+0000","last_change":"2024-06-13T06:55:12.095890+0000","last_active":"2024-06-13T06:56:29.524234+0000","last_peered":"2024-06-13T06:56:29.524234+0000","last_clean":"2024-06-13T06:56:29.524234+0000","last_became_active":"2024-06-13T06:55:12.095397+0000","last_became_peered":"2024-06-13T06:55:12.095397+0000","last_unstale":"2024-06-13T06:56:29.524234+0000","last_undegraded":"2024-06-13T06:56:29.524234+0000","last_fullsized":"2024-06-13T06:56:29.524234+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:56:18.211036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'12","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:05.552121+0000","last_change":"2024-06-13T06:54:54.337946+0000","last_active":"2024-06-13T06:57:05.552121+0000","last_peered":"2024-06-13T06:57:05.552121+0000","last_clean":"2024-06-13T06:57:05.552121+0000","last_became_active":"2024-06-13T06:54:54.337028+0000","last_became_peered":"2024-06-13T06:54:54.337028+0000","last_unstale":"2024-06-13T06:57:05.552121+0000","last_undegraded":"2024-06-13T06:57:05.552121+0000","last_fullsized":"2024-06-13T06:57:05.552121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:08:52.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275581+0000","last_change":"2024-06-13T06:54:58.895110+0000","last_active":"2024-06-13T06:58:28.275581+0000","last_peered":"2024-06-13T06:58:28.275581+0000","last_clean":"2024-06-13T06:58:28.275581+0000","last_became_active":"2024-06-13T06:54:58.893481+0000","last_became_peered":"2024-06-13T06:54:58.893481+0000","last_unstale":"2024-06-13T06:58:28.275581+0000","last_undegraded":"2024-06-13T06:58:28.275581+0000","last_fullsized":"2024-06-13T06:58:28.275581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:54.932110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.606884+0000","last_change":"2024-06-13T06:55:11.997712+0000","last_active":"2024-06-13T06:56:29.606884+0000","last_peered":"2024-06-13T06:56:29.606884+0000","last_clean":"2024-06-13T06:56:29.606884+0000","last_became_active":"2024-06-13T06:55:11.997193+0000","last_became_peered":"2024-06-13T06:55:11.997193+0000","last_unstale":"2024-06-13T06:56:29.606884+0000","last_undegraded":"2024-06-13T06:56:29.606884+0000","last_fullsized":"2024-06-13T06:56:29.606884+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:15.168070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608714+0000","last_change":"2024-06-13T06:54:54.520121+0000","last_active":"2024-06-13T06:56:29.608714+0000","last_peered":"2024-06-13T06:56:29.608714+0000","last_clean":"2024-06-13T06:56:29.608714+0000","last_became_active":"2024-06-13T06:54:54.519723+0000","last_became_peered":"2024-06-13T06:54:54.519723+0000","last_unstale":"2024-06-13T06:56:29.608714+0000","last_undegraded":"2024-06-13T06:56:29.608714+0000","last_fullsized":"2024-06-13T06:56:29.608714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:19:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304621+0000","last_change":"2024-06-13T06:54:58.923626+0000","last_active":"2024-06-13T06:58:28.304621+0000","last_peered":"2024-06-13T06:58:28.304621+0000","last_clean":"2024-06-13T06:58:28.304621+0000","last_became_active":"2024-06-13T06:54:58.923192+0000","last_became_peered":"2024-06-13T06:54:58.923192+0000","last_unstale":"2024-06-13T06:58:28.304621+0000","last_undegraded":"2024-06-13T06:58:28.304621+0000","last_fullsized":"2024-06-13T06:58:28.304621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:26:13.231549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607709+0000","last_change":"2024-06-13T06:55:12.096244+0000","last_active":"2024-06-13T06:56:29.607709+0000","last_peered":"2024-06-13T06:56:29.607709+0000","last_clean":"2024-06-13T06:56:29.607709+0000","last_became_active":"2024-06-13T06:55:12.095758+0000","last_became_peered":"2024-06-13T06:55:12.095758+0000","last_unstale":"2024-06-13T06:56:29.607709+0000","last_undegraded":"2024-06-13T06:56:29.607709+0000","last_fullsized":"2024-06-13T06:56:29.607709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:38.122282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607931+0000","last_change":"2024-06-13T06:54:54.630488+0000","last_active":"2024-06-13T06:56:29.607931+0000","last_peered":"2024-06-13T06:56:29.607931+0000","last_clean":"2024-06-13T06:56:29.607931+0000","last_became_active":"2024-06-13T06:54:54.630224+0000","last_became_peered":"2024-06-13T06:54:54.630224+0000","last_unstale":"2024-06-13T06:56:29.607931+0000","last_undegraded":"2024-06-13T06:56:29.607931+0000","last_fullsized":"2024-06-13T06:56:29.607931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:33:19.107026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.231934+0000","last_change":"2024-06-13T06:54:58.894466+0000","last_active":"2024-06-13T06:58:28.231934+0000","last_peered":"2024-06-13T06:58:28.231934+0000","last_clean":"2024-06-13T06:58:28.231934+0000","last_became_active":"2024-06-13T06:54:58.894034+0000","last_became_peered":"2024-06-13T06:54:58.894034+0000","last_unstale":"2024-06-13T06:58:28.231934+0000","last_undegraded":"2024-06-13T06:58:28.231934+0000","last_fullsized":"2024-06-13T06:58:28.231934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:44.051565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533647+0000","last_change":"2024-06-13T06:55:12.022078+0000","last_active":"2024-06-13T06:56:29.533647+0000","last_peered":"2024-06-13T06:56:29.533647+0000","last_clean":"2024-06-13T06:56:29.533647+0000","last_became_active":"2024-06-13T06:55:12.021652+0000","last_became_peered":"2024-06-13T06:55:12.021652+0000","last_unstale":"2024-06-13T06:56:29.533647+0000","last_undegraded":"2024-06-13T06:56:29.533647+0000","last_fullsized":"2024-06-13T06:56:29.533647+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:25:57.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.524569+0000","last_change":"2024-06-13T06:54:54.336757+0000","last_active":"2024-06-13T06:56:29.524569+0000","last_peered":"2024-06-13T06:56:29.524569+0000","last_clean":"2024-06-13T06:56:29.524569+0000","last_became_active":"2024-06-13T06:54:54.336012+0000","last_became_peered":"2024-06-13T06:54:54.336012+0000","last_unstale":"2024-06-13T06:56:29.524569+0000","last_undegraded":"2024-06-13T06:56:29.524569+0000","last_fullsized":"2024-06-13T06:56:29.524569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:52:21.227464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292719+0000","last_change":"2024-06-13T06:54:58.919631+0000","last_active":"2024-06-13T06:58:28.292719+0000","last_peered":"2024-06-13T06:58:28.292719+0000","last_clean":"2024-06-13T06:58:28.292719+0000","last_became_active":"2024-06-13T06:54:58.919274+0000","last_became_peered":"2024-06-13T06:54:58.919274+0000","last_unstale":"2024-06-13T06:58:28.292719+0000","last_undegraded":"2024-06-13T06:58:28.292719+0000","last_fullsized":"2024-06-13T06:58:28.292719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:22.453797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:54.172963+0000","last_change":"2024-06-13T06:55:12.017338+0000","last_active":"2024-06-13T06:56:54.172963+0000","last_peered":"2024-06-13T06:56:54.172963+0000","last_clean":"2024-06-13T06:56:54.172963+0000","last_became_active":"2024-06-13T06:55:12.016841+0000","last_became_peered":"2024-06-13T06:55:12.016841+0000","last_unstale":"2024-06-13T06:56:54.172963+0000","last_undegraded":"2024-06-13T06:56:54.172963+0000","last_fullsized":"2024-06-13T06:56:54.172963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:40:13.525892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607062+0000","last_change":"2024-06-13T06:54:54.328808+0000","last_active":"2024-06-13T06:56:29.607062+0000","last_peered":"2024-06-13T06:56:29.607062+0000","last_clean":"2024-06-13T06:56:29.607062+0000","last_became_active":"2024-06-13T06:54:54.326982+0000","last_became_peered":"2024-06-13T06:54:54.326982+0000","last_unstale":"2024-06-13T06:56:29.607062+0000","last_undegraded":"2024-06-13T06:56:29.607062+0000","last_fullsized":"2024-06-13T06:56:29.607062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:22:53.854275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.302930+0000","last_change":"2024-06-13T06:54:58.925728+0000","last_active":"2024-06-13T06:58:28.302930+0000","last_peered":"2024-06-13T06:58:28.302930+0000","last_clean":"2024-06-13T06:58:28.302930+0000","last_became_active":"2024-06-13T06:54:58.925407+0000","last_became_peered":"2024-06-13T06:54:58.925407+0000","last_unstale":"2024-06-13T06:58:28.302930+0000","last_undegraded":"2024-06-13T06:58:28.302930+0000","last_fullsized":"2024-06-13T06:58:28.302930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:38.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.607500+0000","last_change":"2024-06-13T06:55:12.095733+0000","last_active":"2024-06-13T06:56:29.607500+0000","last_peered":"2024-06-13T06:56:29.607500+0000","last_clean":"2024-06-13T06:56:29.607500+0000","last_became_active":"2024-06-13T06:55:12.095064+0000","last_became_peered":"2024-06-13T06:55:12.095064+0000","last_unstale":"2024-06-13T06:56:29.607500+0000","last_undegraded":"2024-06-13T06:56:29.607500+0000","last_fullsized":"2024-06-13T06:56:29.607500+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:26:05.486701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556112+0000","last_change":"2024-06-13T06:54:54.331448+0000","last_active":"2024-06-13T06:56:32.556112+0000","last_peered":"2024-06-13T06:56:32.556112+0000","last_clean":"2024-06-13T06:56:32.556112+0000","last_became_active":"2024-06-13T06:54:54.330964+0000","last_became_peered":"2024-06-13T06:54:54.330964+0000","last_unstale":"2024-06-13T06:56:32.556112+0000","last_undegraded":"2024-06-13T06:56:32.556112+0000","last_fullsized":"2024-06-13T06:56:32.556112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:23:12.211412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281344+0000","last_change":"2024-06-13T06:54:58.899112+0000","last_active":"2024-06-13T06:58:28.281344+0000","last_peered":"2024-06-13T06:58:28.281344+0000","last_clean":"2024-06-13T06:58:28.281344+0000","last_became_active":"2024-06-13T06:54:58.898742+0000","last_became_peered":"2024-06-13T06:54:58.898742+0000","last_unstale":"2024-06-13T06:58:28.281344+0000","last_undegraded":"2024-06-13T06:58:28.281344+0000","last_fullsized":"2024-06-13T06:58:28.281344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:50:12.983287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.608239+0000","last_change":"2024-06-13T06:55:12.014149+0000","last_active":"2024-06-13T06:56:29.608239+0000","last_peered":"2024-06-13T06:56:29.608239+0000","last_clean":"2024-06-13T06:56:29.608239+0000","last_became_active":"2024-06-13T06:55:12.013454+0000","last_became_peered":"2024-06-13T06:55:12.013454+0000","last_unstale":"2024-06-13T06:56:29.608239+0000","last_undegraded":"2024-06-13T06:56:29.608239+0000","last_fullsized":"2024-06-13T06:56:29.608239+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:13:39.368540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.560296+0000","last_change":"2024-06-13T06:54:54.633268+0000","last_active":"2024-06-13T06:56:32.560296+0000","last_peered":"2024-06-13T06:56:32.560296+0000","last_clean":"2024-06-13T06:56:32.560296+0000","last_became_active":"2024-06-13T06:54:54.632840+0000","last_became_peered":"2024-06-13T06:54:54.632840+0000","last_unstale":"2024-06-13T06:56:32.560296+0000","last_undegraded":"2024-06-13T06:56:32.560296+0000","last_fullsized":"2024-06-13T06:56:32.560296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:21.653727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281293+0000","last_change":"2024-06-13T06:54:58.893417+0000","last_active":"2024-06-13T06:58:28.281293+0000","last_peered":"2024-06-13T06:58:28.281293+0000","last_clean":"2024-06-13T06:58:28.281293+0000","last_became_active":"2024-06-13T06:54:58.892381+0000","last_became_peered":"2024-06-13T06:54:58.892381+0000","last_unstale":"2024-06-13T06:58:28.281293+0000","last_undegraded":"2024-06-13T06:58:28.281293+0000","last_fullsized":"2024-06-13T06:58:28.281293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:07.979654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.533526+0000","last_change":"2024-06-13T06:55:12.018336+0000","last_active":"2024-06-13T06:56:29.533526+0000","last_peered":"2024-06-13T06:56:29.533526+0000","last_clean":"2024-06-13T06:56:29.533526+0000","last_became_active":"2024-06-13T06:55:12.017861+0000","last_became_peered":"2024-06-13T06:55:12.017861+0000","last_unstale":"2024-06-13T06:56:29.533526+0000","last_undegraded":"2024-06-13T06:56:29.533526+0000","last_fullsized":"2024-06-13T06:56:29.533526+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:41.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.556542+0000","last_change":"2024-06-13T06:54:54.333036+0000","last_active":"2024-06-13T06:56:32.556542+0000","last_peered":"2024-06-13T06:56:32.556542+0000","last_clean":"2024-06-13T06:56:32.556542+0000","last_became_active":"2024-06-13T06:54:54.332642+0000","last_became_peered":"2024-06-13T06:54:54.332642+0000","last_unstale":"2024-06-13T06:56:32.556542+0000","last_undegraded":"2024-06-13T06:56:32.556542+0000","last_fullsized":"2024-06-13T06:56:32.556542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:06:06.809024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279764+0000","last_change":"2024-06-13T06:54:58.881894+0000","last_active":"2024-06-13T06:58:28.279764+0000","last_peered":"2024-06-13T06:58:28.279764+0000","last_clean":"2024-06-13T06:58:28.279764+0000","last_became_active":"2024-06-13T06:54:58.881591+0000","last_became_peered":"2024-06-13T06:54:58.881591+0000","last_unstale":"2024-06-13T06:58:28.279764+0000","last_undegraded":"2024-06-13T06:58:28.279764+0000","last_fullsized":"2024-06-13T06:58:28.279764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:49:00.944212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.525430+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:56:29.525430+0000","last_peered":"2024-06-13T06:56:29.525430+0000","last_clean":"2024-06-13T06:56:29.525430+0000","last_became_active":"2024-06-13T06:55:12.000748+0000","last_became_peered":"2024-06-13T06:55:12.000748+0000","last_unstale":"2024-06-13T06:56:29.525430+0000","last_undegraded":"2024-06-13T06:56:29.525430+0000","last_fullsized":"2024-06-13T06:56:29.525430+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:22.035807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:29.600248+0000","last_change":"2024-06-13T06:54:54.342724+0000","last_active":"2024-06-13T06:56:29.600248+0000","last_peered":"2024-06-13T06:56:29.600248+0000","last_clean":"2024-06-13T06:56:29.600248+0000","last_became_active":"2024-06-13T06:54:54.342267+0000","last_became_peered":"2024-06-13T06:54:54.342267+0000","last_unstale":"2024-06-13T06:56:29.600248+0000","last_undegraded":"2024-06-13T06:56:29.600248+0000","last_fullsized":"2024-06-13T06:56:29.600248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:27:53.039202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271410+0000","last_change":"2024-06-13T06:54:58.892365+0000","last_active":"2024-06-13T06:58:28.271410+0000","last_peered":"2024-06-13T06:58:28.271410+0000","last_clean":"2024-06-13T06:58:28.271410+0000","last_became_active":"2024-06-13T06:54:58.889395+0000","last_became_peered":"2024-06-13T06:54:58.889395+0000","last_unstale":"2024-06-13T06:58:28.271410+0000","last_undegraded":"2024-06-13T06:58:28.271410+0000","last_fullsized":"2024-06-13T06:58:28.271410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:55.798293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272092+0000","last_change":"2024-06-13T06:54:58.888570+0000","last_active":"2024-06-13T06:58:28.272092+0000","last_peered":"2024-06-13T06:58:28.272092+0000","last_clean":"2024-06-13T06:58:28.272092+0000","last_became_active":"2024-06-13T06:54:58.888180+0000","last_became_peered":"2024-06-13T06:54:58.888180+0000","last_unstale":"2024-06-13T06:58:28.272092+0000","last_undegraded":"2024-06-13T06:58:28.272092+0000","last_fullsized":"2024-06-13T06:58:28.272092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:46:42.537790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.270977+0000","last_change":"2024-06-13T06:54:58.893476+0000","last_active":"2024-06-13T06:58:28.270977+0000","last_peered":"2024-06-13T06:58:28.270977+0000","last_clean":"2024-06-13T06:58:28.270977+0000","last_became_active":"2024-06-13T06:54:58.893021+0000","last_became_peered":"2024-06-13T06:54:58.893021+0000","last_unstale":"2024-06-13T06:58:28.270977+0000","last_undegraded":"2024-06-13T06:58:28.270977+0000","last_fullsized":"2024-06-13T06:58:28.270977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:38.739070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301473+0000","last_change":"2024-06-13T06:54:58.911042+0000","last_active":"2024-06-13T06:58:28.301473+0000","last_peered":"2024-06-13T06:58:28.301473+0000","last_clean":"2024-06-13T06:58:28.301473+0000","last_became_active":"2024-06-13T06:54:58.910641+0000","last_became_peered":"2024-06-13T06:54:58.910641+0000","last_unstale":"2024-06-13T06:58:28.301473+0000","last_undegraded":"2024-06-13T06:58:28.301473+0000","last_fullsized":"2024-06-13T06:58:28.301473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:21:39.467223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301939+0000","last_change":"2024-06-13T06:55:29.382730+0000","last_active":"2024-06-13T06:58:28.301939+0000","last_peered":"2024-06-13T06:58:28.301939+0000","last_clean":"2024-06-13T06:58:28.301939+0000","last_became_active":"2024-06-13T06:55:29.382162+0000","last_became_peered":"2024-06-13T06:55:29.382162+0000","last_unstale":"2024-06-13T06:58:28.301939+0000","last_undegraded":"2024-06-13T06:58:28.301939+0000","last_fullsized":"2024-06-13T06:58:28.301939+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:05.486414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.302996+0000","last_change":"2024-06-13T06:55:29.385380+0000","last_active":"2024-06-13T06:58:28.302996+0000","last_peered":"2024-06-13T06:58:28.302996+0000","last_clean":"2024-06-13T06:58:28.302996+0000","last_became_active":"2024-06-13T06:55:29.384855+0000","last_became_peered":"2024-06-13T06:55:29.384855+0000","last_unstale":"2024-06-13T06:58:28.302996+0000","last_undegraded":"2024-06-13T06:58:28.302996+0000","last_fullsized":"2024-06-13T06:58:28.302996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:31:01.945632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271515+0000","last_change":"2024-06-13T06:54:58.889128+0000","last_active":"2024-06-13T06:58:28.271515+0000","last_peered":"2024-06-13T06:58:28.271515+0000","last_clean":"2024-06-13T06:58:28.271515+0000","last_became_active":"2024-06-13T06:54:58.888606+0000","last_became_peered":"2024-06-13T06:54:58.888606+0000","last_unstale":"2024-06-13T06:58:28.271515+0000","last_undegraded":"2024-06-13T06:58:28.271515+0000","last_fullsized":"2024-06-13T06:58:28.271515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:53:43.762968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.269593+0000","last_change":"2024-06-13T06:54:58.896098+0000","last_active":"2024-06-13T06:58:28.269593+0000","last_peered":"2024-06-13T06:58:28.269593+0000","last_clean":"2024-06-13T06:58:28.269593+0000","last_became_active":"2024-06-13T06:54:58.895722+0000","last_became_peered":"2024-06-13T06:54:58.895722+0000","last_unstale":"2024-06-13T06:58:28.269593+0000","last_undegraded":"2024-06-13T06:58:28.269593+0000","last_fullsized":"2024-06-13T06:58:28.269593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:18:14.421363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279703+0000","last_change":"2024-06-13T06:54:58.905590+0000","last_active":"2024-06-13T06:58:28.279703+0000","last_peered":"2024-06-13T06:58:28.279703+0000","last_clean":"2024-06-13T06:58:28.279703+0000","last_became_active":"2024-06-13T06:54:58.905146+0000","last_became_peered":"2024-06-13T06:54:58.905146+0000","last_unstale":"2024-06-13T06:58:28.279703+0000","last_undegraded":"2024-06-13T06:58:28.279703+0000","last_fullsized":"2024-06-13T06:58:28.279703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:42:08.010906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.270240+0000","last_change":"2024-06-13T06:54:58.898037+0000","last_active":"2024-06-13T06:58:28.270240+0000","last_peered":"2024-06-13T06:58:28.270240+0000","last_clean":"2024-06-13T06:58:28.270240+0000","last_became_active":"2024-06-13T06:54:58.897692+0000","last_became_peered":"2024-06-13T06:54:58.897692+0000","last_unstale":"2024-06-13T06:58:28.270240+0000","last_undegraded":"2024-06-13T06:58:28.270240+0000","last_fullsized":"2024-06-13T06:58:28.270240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:44:30.018192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275550+0000","last_change":"2024-06-13T06:54:58.903441+0000","last_active":"2024-06-13T06:58:28.275550+0000","last_peered":"2024-06-13T06:58:28.275550+0000","last_clean":"2024-06-13T06:58:28.275550+0000","last_became_active":"2024-06-13T06:54:58.893598+0000","last_became_peered":"2024-06-13T06:54:58.893598+0000","last_unstale":"2024-06-13T06:58:28.275550+0000","last_undegraded":"2024-06-13T06:58:28.275550+0000","last_fullsized":"2024-06-13T06:58:28.275550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:38:29.968619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282381+0000","last_change":"2024-06-13T06:54:58.918594+0000","last_active":"2024-06-13T06:58:28.282381+0000","last_peered":"2024-06-13T06:58:28.282381+0000","last_clean":"2024-06-13T06:58:28.282381+0000","last_became_active":"2024-06-13T06:54:58.918255+0000","last_became_peered":"2024-06-13T06:54:58.918255+0000","last_unstale":"2024-06-13T06:58:28.282381+0000","last_undegraded":"2024-06-13T06:58:28.282381+0000","last_fullsized":"2024-06-13T06:58:28.282381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:39:48.527707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304459+0000","last_change":"2024-06-13T06:54:58.926253+0000","last_active":"2024-06-13T06:58:28.304459+0000","last_peered":"2024-06-13T06:58:28.304459+0000","last_clean":"2024-06-13T06:58:28.304459+0000","last_became_active":"2024-06-13T06:54:58.924611+0000","last_became_peered":"2024-06-13T06:54:58.924611+0000","last_unstale":"2024-06-13T06:58:28.304459+0000","last_undegraded":"2024-06-13T06:58:28.304459+0000","last_fullsized":"2024-06-13T06:58:28.304459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:29:23.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267122+0000","last_change":"2024-06-13T06:54:58.890454+0000","last_active":"2024-06-13T06:58:28.267122+0000","last_peered":"2024-06-13T06:58:28.267122+0000","last_clean":"2024-06-13T06:58:28.267122+0000","last_became_active":"2024-06-13T06:54:58.890129+0000","last_became_peered":"2024-06-13T06:54:58.890129+0000","last_unstale":"2024-06-13T06:58:28.267122+0000","last_undegraded":"2024-06-13T06:58:28.267122+0000","last_fullsized":"2024-06-13T06:58:28.267122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:10.819536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67045,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":812,"num_read_kb":479,"num_write":174,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1656,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383743,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":570102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645380,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37124,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708028,"statfs":{"total":95995035648,"available":95957020672,"internally_reserved":0,"allocated":1830912,"data_stored":1220230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":574390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070544,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":570170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332182,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1166263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":565470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36512,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708640,"statfs":{"total":95995035648,"available":95957647360,"internally_reserved":0,"allocated":1204224,"data_stored":628284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":566858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":566108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":565664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920873,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1228800,"data_stored":634177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757347,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1161851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:58:46.293 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-13T06:58:46.293 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-13T06:58:46.293 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-13T06:58:46.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph health --format=json 2024-06-13T06:58:46.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:46 smithi136 ceph-mon[37229]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-06-13T06:58:46.410 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:46 smithi026 ceph-mon[32041]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-06-13T06:58:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:46 smithi135 ceph-mon[36276]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-06-13T06:58:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:47.221 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:47 smithi026 ceph-mon[32041]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:58:47.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:47 smithi136 ceph-mon[37229]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:58:47.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:47 smithi135 ceph-mon[36276]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:58:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:48 smithi026 ceph-mon[32041]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:48 smithi136 ceph-mon[37229]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:48 smithi135 ceph-mon[36276]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 -- 172.21.15.26:0/2641696466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 msgr2=0x7f7c6010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 --2- 172.21.15.26:0/2641696466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 0x7f7c6010eee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009f90 tx=0x7f7c5800b330 comp rx=0 tx=0).stop 2024-06-13T06:58:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 -- 172.21.15.26:0/2641696466 shutdown_connections 2024-06-13T06:58:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 --2- 172.21.15.26:0/2641696466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 0x7f7c6010eee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 --2- 172.21.15.26:0/2641696466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c601035f0 0x7f7c601039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 --2- 172.21.15.26:0/2641696466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c60103f10 0x7f7c6010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 -- 172.21.15.26:0/2641696466 >> 172.21.15.26:0/2641696466 conn(0x7f7c600fce40 msgr2=0x7f7c600ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 -- 172.21.15.26:0/2641696466 shutdown_connections 2024-06-13T06:58:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.273+0000 7f7c65400700 1 -- 172.21.15.26:0/2641696466 wait complete. 2024-06-13T06:58:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.274+0000 7f7c65400700 1 Processor -- start 2024-06-13T06:58:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c65400700 1 -- start start 2024-06-13T06:58:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c65400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c601035f0 0x7f7c60101250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c65400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60103f10 0x7f7c60101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c65400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 0x7f7c60101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c65400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601127e0 con 0x7f7c60103f10 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c65400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60112660 con 0x7f7c6010ca20 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c65400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60112960 con 0x7f7c601035f0 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c5f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60103f10 0x7f7c60101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c5fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c601035f0 0x7f7c60101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c5fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c601035f0 0x7f7c60101250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45558/0 (socket says 172.21.15.26:45558) 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c5fe00700 1 -- 172.21.15.26:0/3913144705 learned_addr learned my addr 172.21.15.26:0/3913144705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.275+0000 7f7c64a00700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 0x7f7c60101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c64a00700 1 -- 172.21.15.26:0/3913144705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c601035f0 msgr2=0x7f7c60101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c64a00700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c601035f0 0x7f7c60101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c64a00700 1 -- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60103f10 msgr2=0x7f7c60101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c64a00700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60103f10 0x7f7c60101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c64a00700 1 -- 172.21.15.26:0/3913144705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c601a7160 con 0x7f7c6010ca20 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c5f400700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60103f10 0x7f7c60101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c64a00700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 0x7f7c60101cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009f90 tx=0x7f7c5800cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c5cc00700 1 -- 172.21.15.26:0/3913144705 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58021070 con 0x7f7c6010ca20 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c5cc00700 1 -- 172.21.15.26:0/3913144705 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c58007ac0 con 0x7f7c6010ca20 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c5cc00700 1 -- 172.21.15.26:0/3913144705 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580052a0 con 0x7f7c6010ca20 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.276+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c601a73f0 con 0x7f7c6010ca20 2024-06-13T06:58:49.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.277+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c601a78e0 con 0x7f7c6010ca20 2024-06-13T06:58:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.278+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c60069180 con 0x7f7c6010ca20 2024-06-13T06:58:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.280+0000 7f7c5cc00700 1 -- 172.21.15.26:0/3913144705 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7c58024070 con 0x7f7c6010ca20 2024-06-13T06:58:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.281+0000 7f7c5cc00700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7c440a2c20 0x7f7c440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.281+0000 7f7c5cc00700 1 -- 172.21.15.26:0/3913144705 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7c5800f020 con 0x7f7c6010ca20 2024-06-13T06:58:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.281+0000 7f7c5fe00700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7c440a2c20 0x7f7c440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.282+0000 7f7c5fe00700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7c440a2c20 0x7f7c440a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005950 tx=0x7f7c50009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:49.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.283+0000 7f7c5cc00700 1 -- 172.21.15.26:0/3913144705 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c5802b020 con 0x7f7c6010ca20 2024-06-13T06:58:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.566+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7c60102780 con 0x7f7c6010ca20 2024-06-13T06:58:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.567+0000 7f7c5cc00700 1 -- 172.21.15.26:0/3913144705 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7c58007c70 con 0x7f7c6010ca20 2024-06-13T06:58:49.567 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:58:49.567 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-13T06:58:49.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.570+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7c440a2c20 msgr2=0x7f7c440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:49.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.570+0000 7f7c65400700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7c440a2c20 0x7f7c440a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005950 tx=0x7f7c50009310 comp rx=0 tx=0).stop 2024-06-13T06:58:49.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.570+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 msgr2=0x7f7c60101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:49.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.570+0000 7f7c65400700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 0x7f7c60101cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009f90 tx=0x7f7c5800cfa0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.571+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 shutdown_connections 2024-06-13T06:58:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.571+0000 7f7c65400700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7c440a2c20 0x7f7c440a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.571+0000 7f7c65400700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c6010ca20 0x7f7c60101cd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.571+0000 7f7c65400700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c601035f0 0x7f7c60101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.571+0000 7f7c65400700 1 --2- 172.21.15.26:0/3913144705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c60103f10 0x7f7c60101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.571+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 >> 172.21.15.26:0/3913144705 conn(0x7f7c600fce40 msgr2=0x7f7c600fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.571+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 shutdown_connections 2024-06-13T06:58:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:49.571+0000 7f7c65400700 1 -- 172.21.15.26:0/3913144705 wait complete. 2024-06-13T06:58:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:50 smithi026 ceph-mon[32041]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:50 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3913144705' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-13T06:58:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:50 smithi136 ceph-mon[37229]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:50.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:50 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3913144705' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-13T06:58:50.414 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-13T06:58:50.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:58:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:50 smithi135 ceph-mon[36276]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:58:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:50 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3913144705' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-13T06:58:50.888 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:52 smithi026 ceph-mon[32041]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:52 smithi136 ceph-mon[37229]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:52 smithi135 ceph-mon[36276]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:58:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 -- 172.21.15.26:0/690930296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806a170 msgr2=0x7f9f0806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 --2- 172.21.15.26:0/690930296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806a170 0x7f9f0806a5f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009990 tx=0x7f9ef0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 -- 172.21.15.26:0/690930296 shutdown_connections 2024-06-13T06:58:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 --2- 172.21.15.26:0/690930296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0806ab30 0x7f9f0810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 --2- 172.21.15.26:0/690930296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f08108500 0x7f9f081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 --2- 172.21.15.26:0/690930296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806a170 0x7f9f0806a5f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 -- 172.21.15.26:0/690930296 >> 172.21.15.26:0/690930296 conn(0x7f9f08078340 msgr2=0x7f9f08078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 -- 172.21.15.26:0/690930296 shutdown_connections 2024-06-13T06:58:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.233+0000 7f9f0e600700 1 -- 172.21.15.26:0/690930296 wait complete. 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.234+0000 7f9f0e600700 1 Processor -- start 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.234+0000 7f9f0e600700 1 -- start start 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f0e600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0806a170 0x7f9f0819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f0e600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806ab30 0x7f9f0819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f0e600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f08108500 0x7f9f0819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f0e600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9f081127b0 con 0x7f9f08108500 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f0e600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f08112630 con 0x7f9f0806ab30 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f0e600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9f08112930 con 0x7f9f0806a170 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f07e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f08108500 0x7f9f0819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f07400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0806a170 0x7f9f0819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f07e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f08108500 0x7f9f0819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46832/0 (socket says 172.21.15.26:46832) 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9f07e00700 1 -- 172.21.15.26:0/2984482044 learned_addr learned my addr 172.21.15.26:0/2984482044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.235+0000 7f9efea00700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806ab30 0x7f9f0819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9efea00700 1 -- 172.21.15.26:0/2984482044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0806a170 msgr2=0x7f9f0819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9efea00700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0806a170 0x7f9f0819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9efea00700 1 -- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f08108500 msgr2=0x7f9f0819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9efea00700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f08108500 0x7f9f0819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9efea00700 1 -- 172.21.15.26:0/2984482044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0819ff10 con 0x7f9f0806ab30 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9f07400700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0806a170 0x7f9f0819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9efea00700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806ab30 0x7f9f0819b440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00098e0 tx=0x7f9ef000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9f04c00700 1 -- 172.21.15.26:0/2984482044 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0027070 con 0x7f9f0806ab30 2024-06-13T06:58:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9f04c00700 1 -- 172.21.15.26:0/2984482044 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef0003b60 con 0x7f9f0806ab30 2024-06-13T06:58:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.236+0000 7f9f04c00700 1 -- 172.21.15.26:0/2984482044 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0022d50 con 0x7f9f0806ab30 2024-06-13T06:58:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.237+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f081a01a0 con 0x7f9f0806ab30 2024-06-13T06:58:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.237+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f081a7a80 con 0x7f9f0806ab30 2024-06-13T06:58:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.238+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f08069180 con 0x7f9f0806ab30 2024-06-13T06:58:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.240+0000 7f9f04c00700 1 -- 172.21.15.26:0/2984482044 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ef002a070 con 0x7f9f0806ab30 2024-06-13T06:58:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.241+0000 7f9f04c00700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9eec0a2ca0 0x7f9eec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.241+0000 7f9f04c00700 1 -- 172.21.15.26:0/2984482044 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9ef00d0340 con 0x7f9f0806ab30 2024-06-13T06:58:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.241+0000 7f9f07400700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9eec0a2ca0 0x7f9eec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.242+0000 7f9f07400700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9eec0a2ca0 0x7f9eec0a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009480 tx=0x7f9ef800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.245+0000 7f9f04c00700 1 -- 172.21.15.26:0/2984482044 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ef005e850 con 0x7f9f0806ab30 2024-06-13T06:58:53.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.526+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f081a7c50 con 0x7f9f0806ab30 2024-06-13T06:58:53.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.527+0000 7f9f04c00700 1 -- 172.21.15.26:0/2984482044 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f9ef0013070 con 0x7f9f0806ab30 2024-06-13T06:58:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:58:53.528 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T06:56:36.605518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24511,24517,24506]},"id":1}]} 2024-06-13T06:58:53.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9eec0a2ca0 msgr2=0x7f9eec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9eec0a2ca0 0x7f9eec0a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009480 tx=0x7f9ef800b040 comp rx=0 tx=0).stop 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806ab30 msgr2=0x7f9f0819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806ab30 0x7f9f0819b440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00098e0 tx=0x7f9ef000b8a0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 shutdown_connections 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9eec0a2ca0 0x7f9eec0a5160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f08108500 0x7f9f0819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0806a170 0x7f9f0819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 --2- 172.21.15.26:0/2984482044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0806ab30 0x7f9f0819b440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 >> 172.21.15.26:0/2984482044 conn(0x7f9f08078340 msgr2=0x7f9f0810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:53.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.531+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 shutdown_connections 2024-06-13T06:58:53.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:53.532+0000 7f9f0e600700 1 -- 172.21.15.26:0/2984482044 wait complete. 2024-06-13T06:58:53.532 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-06-13T06:58:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:54 smithi026 ceph-mon[32041]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:54 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2984482044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:54.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:54 smithi136 ceph-mon[37229]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:54.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:54 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2984482044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:54 smithi135 ceph-mon[36276]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:58:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:54 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2984482044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:54.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:58:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:56.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:56 smithi136 ceph-mon[37229]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:56 smithi026 ceph-mon[32041]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:56.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:56 smithi135 ceph-mon[36276]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:58:57.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.723+0000 7fd74d000700 1 -- 172.21.15.26:0/3156541003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748108520 msgr2=0x7fd748108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.723+0000 7fd74d000700 1 --2- 172.21.15.26:0/3156541003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748108520 0x7fd748108900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd738009990 tx=0x7fd738009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.723+0000 7fd74d000700 1 -- 172.21.15.26:0/3156541003 shutdown_connections 2024-06-13T06:58:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.723+0000 7fd74d000700 1 --2- 172.21.15.26:0/3156541003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd748102ed0 0x7fd74810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.723+0000 7fd74d000700 1 --2- 172.21.15.26:0/3156541003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748108520 0x7fd748108900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.723+0000 7fd74d000700 1 --2- 172.21.15.26:0/3156541003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd748102510 0x7fd748102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.723+0000 7fd74d000700 1 -- 172.21.15.26:0/3156541003 >> 172.21.15.26:0/3156541003 conn(0x7fd748079560 msgr2=0x7fd748079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.724+0000 7fd74d000700 1 -- 172.21.15.26:0/3156541003 shutdown_connections 2024-06-13T06:58:57.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.724+0000 7fd74d000700 1 -- 172.21.15.26:0/3156541003 wait complete. 2024-06-13T06:58:57.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.725+0000 7fd74d000700 1 Processor -- start 2024-06-13T06:58:57.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.725+0000 7fd74d000700 1 -- start start 2024-06-13T06:58:57.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd74d000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd748102510 0x7fd74819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:57.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd74d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748102ed0 0x7fd74819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:57.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd74d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd748108520 0x7fd74819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:57.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd74d000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481127b0 con 0x7fd748102510 2024-06-13T06:58:57.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd74d000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd748112630 con 0x7fd748108520 2024-06-13T06:58:57.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd74d000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd748112930 con 0x7fd748102ed0 2024-06-13T06:58:57.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd745e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd748102510 0x7fd74819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:57.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd746800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd748108520 0x7fd74819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd746800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd748108520 0x7fd74819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39064/0 (socket says 172.21.15.26:39064) 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.726+0000 7fd746800700 1 -- 172.21.15.26:0/2753052140 learned_addr learned my addr 172.21.15.26:0/2753052140 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.727+0000 7fd745400700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748102ed0 0x7fd74819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.727+0000 7fd745400700 1 -- 172.21.15.26:0/2753052140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd748108520 msgr2=0x7fd74819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.727+0000 7fd745400700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd748108520 0x7fd74819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.727+0000 7fd745400700 1 -- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd748102510 msgr2=0x7fd74819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.727+0000 7fd745400700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd748102510 0x7fd74819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.727+0000 7fd745400700 1 -- 172.21.15.26:0/2753052140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74819fd50 con 0x7fd748102ed0 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.727+0000 7fd746800700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd748108520 0x7fd74819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.727+0000 7fd745400700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748102ed0 0x7fd74819b280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e9f0 tx=0x7fd73400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.728+0000 7fd73ea00700 1 -- 172.21.15.26:0/2753052140 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400cb70 con 0x7fd748102ed0 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.728+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7481a0040 con 0x7fd748102ed0 2024-06-13T06:58:57.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.728+0000 7fd73ea00700 1 -- 172.21.15.26:0/2753052140 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd73400ee90 con 0x7fd748102ed0 2024-06-13T06:58:57.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.728+0000 7fd73ea00700 1 -- 172.21.15.26:0/2753052140 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734003dc0 con 0x7fd748102ed0 2024-06-13T06:58:57.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.728+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7481a78c0 con 0x7fd748102ed0 2024-06-13T06:58:57.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.729+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd748069180 con 0x7fd748102ed0 2024-06-13T06:58:57.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.732+0000 7fd73ea00700 1 -- 172.21.15.26:0/2753052140 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd734043ae0 con 0x7fd748102ed0 2024-06-13T06:58:57.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.733+0000 7fd73ea00700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd72c0a2c20 0x7fd72c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:57.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.733+0000 7fd73ea00700 1 -- 172.21.15.26:0/2753052140 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd7340cf560 con 0x7fd748102ed0 2024-06-13T06:58:57.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.734+0000 7fd745e00700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd72c0a2c20 0x7fd72c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:57.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.734+0000 7fd745e00700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd72c0a2c20 0x7fd72c0a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd73800b2e0 tx=0x7fd738005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:57.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:57.735+0000 7fd73ea00700 1 -- 172.21.15.26:0/2753052140 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd73400f3c0 con 0x7fd748102ed0 2024-06-13T06:58:58.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.013+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd74819c0a0 con 0x7fd748102ed0 2024-06-13T06:58:58.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.014+0000 7fd73ea00700 1 -- 172.21.15.26:0/2753052140 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7fd74819c0a0 con 0x7fd748102ed0 2024-06-13T06:58:58.014 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:58:58.015 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T06:56:36.605518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24511,24517,24506]},"id":1}]} 2024-06-13T06:58:58.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.017+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd72c0a2c20 msgr2=0x7fd72c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:58.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.017+0000 7fd74d000700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd72c0a2c20 0x7fd72c0a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd73800b2e0 tx=0x7fd738005960 comp rx=0 tx=0).stop 2024-06-13T06:58:58.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748102ed0 msgr2=0x7fd74819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748102ed0 0x7fd74819b280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e9f0 tx=0x7fd73400ed00 comp rx=0 tx=0).stop 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 shutdown_connections 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd72c0a2c20 0x7fd72c0a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd748108520 0x7fd74819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd748102510 0x7fd74819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 --2- 172.21.15.26:0/2753052140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd748102ed0 0x7fd74819b280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 >> 172.21.15.26:0/2753052140 conn(0x7fd748079560 msgr2=0x7fd748074950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 shutdown_connections 2024-06-13T06:58:58.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:58:58.018+0000 7fd74d000700 1 -- 172.21.15.26:0/2753052140 wait complete. 2024-06-13T06:58:58.019 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-06-13T06:58:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:58 smithi136 ceph-mon[37229]: pgmap v463: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:58:58 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2753052140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:58.462 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:58 smithi026 ceph-mon[32041]: pgmap v463: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:58.462 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:58:58 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2753052140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:58 smithi135 ceph-mon[36276]: pgmap v463: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:58:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:58:58 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2753052140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:58.938 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:58:59.404 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:00 smithi026 ceph-mon[32041]: pgmap v464: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:00 smithi136 ceph-mon[37229]: pgmap v464: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:00 smithi135 ceph-mon[36276]: pgmap v464: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:01.359 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:01 smithi026 ceph-mon[32041]: pgmap v465: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:01.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:01 smithi136 ceph-mon[37229]: pgmap v465: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:01.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:01 smithi135 ceph-mon[36276]: pgmap v465: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 -- 172.21.15.26:0/978961293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09980751e0 msgr2=0x7f099810e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 --2- 172.21.15.26:0/978961293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09980751e0 0x7f099810e6c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f09880098f0 tx=0x7f0988009c00 comp rx=0 tx=0).stop 2024-06-13T06:59:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 -- 172.21.15.26:0/978961293 shutdown_connections 2024-06-13T06:59:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 --2- 172.21.15.26:0/978961293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f099810ec00 0x7f0998111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 --2- 172.21.15.26:0/978961293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09980748c0 0x7f0998074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 --2- 172.21.15.26:0/978961293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09980751e0 0x7f099810e6c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 -- 172.21.15.26:0/978961293 >> 172.21.15.26:0/978961293 conn(0x7f09980fdf10 msgr2=0x7f0998100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 -- 172.21.15.26:0/978961293 shutdown_connections 2024-06-13T06:59:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.917+0000 7f099f200700 1 -- 172.21.15.26:0/978961293 wait complete. 2024-06-13T06:59:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.919+0000 7f099f200700 1 Processor -- start 2024-06-13T06:59:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.919+0000 7f099f200700 1 -- start start 2024-06-13T06:59:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.919+0000 7f099f200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09980748c0 0x7f099819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.919+0000 7f099f200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09980751e0 0x7f099819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.919+0000 7f099f200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f099810ec00 0x7f099819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.919+0000 7f099f200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f09981148d0 con 0x7f099810ec00 2024-06-13T06:59:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.919+0000 7f099f200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0998114750 con 0x7f09980748c0 2024-06-13T06:59:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.919+0000 7f099f200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0998114a50 con 0x7f09980751e0 2024-06-13T06:59:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09980748c0 0x7f099819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09980751e0 0x7f099819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09980748c0 0x7f099819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39088/0 (socket says 172.21.15.26:39088) 2024-06-13T06:59:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 -- 172.21.15.26:0/983818116 learned_addr learned my addr 172.21.15.26:0/983818116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 -- 172.21.15.26:0/983818116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09980751e0 msgr2=0x7f099819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09980751e0 0x7f099819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 -- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f099810ec00 msgr2=0x7f099819f650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f099810ec00 0x7f099819f650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 -- 172.21.15.26:0/983818116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099819fd90 con 0x7f09980748c0 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.920+0000 7f099de00700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09980748c0 0x7f099819ad80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f098c00ec10 tx=0x7f098c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.921+0000 7f0992a00700 1 -- 172.21.15.26:0/983818116 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c00cd40 con 0x7f09980748c0 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.921+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09981a0080 con 0x7f09980748c0 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.921+0000 7f0992a00700 1 -- 172.21.15.26:0/983818116 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098c004500 con 0x7f09980748c0 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.921+0000 7f0992a00700 1 -- 172.21.15.26:0/983818116 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c010690 con 0x7f09980748c0 2024-06-13T06:59:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.921+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09981a7900 con 0x7f09980748c0 2024-06-13T06:59:01.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.922+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0998069180 con 0x7f09980748c0 2024-06-13T06:59:01.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.924+0000 7f0992a00700 1 -- 172.21.15.26:0/983818116 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f098c003ee0 con 0x7f09980748c0 2024-06-13T06:59:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.927+0000 7f0992a00700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f09800a2c20 0x7f09800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.927+0000 7f0992a00700 1 -- 172.21.15.26:0/983818116 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f098c0cf890 con 0x7f09980748c0 2024-06-13T06:59:01.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.927+0000 7f099d400700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f09800a2c20 0x7f09800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:01.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.928+0000 7f099d400700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f09800a2c20 0x7f09800a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f09880098c0 tx=0x7f0988019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:01.928+0000 7f0992a00700 1 -- 172.21.15.26:0/983818116 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f098c01f340 con 0x7f09980748c0 2024-06-13T06:59:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.171+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f099819c0e0 con 0x7f09980748c0 2024-06-13T06:59:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.173+0000 7f0992a00700 1 -- 172.21.15.26:0/983818116 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7f098c09c430 con 0x7f09980748c0 2024-06-13T06:59:02.173 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:59:02.174 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f09800a2c20 msgr2=0x7f09800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f09800a2c20 0x7f09800a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f09880098c0 tx=0x7f0988019040 comp rx=0 tx=0).stop 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09980748c0 msgr2=0x7f099819ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09980748c0 0x7f099819ad80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f098c00ec10 tx=0x7f098c00ef20 comp rx=0 tx=0).stop 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 shutdown_connections 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f09800a2c20 0x7f09800a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09980748c0 0x7f099819ad80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f099810ec00 0x7f099819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 --2- 172.21.15.26:0/983818116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09980751e0 0x7f099819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 >> 172.21.15.26:0/983818116 conn(0x7f09980fdf10 msgr2=0x7f09980fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 shutdown_connections 2024-06-13T06:59:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:02.177+0000 7f099f200700 1 -- 172.21.15.26:0/983818116 wait complete. 2024-06-13T06:59:02.434 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:02 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:59:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:02 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:59:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:02 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:59:03.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:59:03.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:03 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/983818116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:59:03.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:03 smithi026 ceph-mon[32041]: pgmap v466: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:03.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:03 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/983818116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:59:03.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:03 smithi135 ceph-mon[36276]: pgmap v466: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:03.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:03 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/983818116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:59:03.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:03 smithi136 ceph-mon[37229]: pgmap v466: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:03.642 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:04.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:59:04.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:59:04.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:04 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:59:04.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:04 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:59:04.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:04 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:59:04.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:04 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:59:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:04 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:59:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:04 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:59:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:04 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T06:59:05.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:05 smithi026 ceph-mon[32041]: pgmap v467: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:05.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:05 smithi135 ceph-mon[36276]: pgmap v467: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:05.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:05 smithi136 ceph-mon[37229]: pgmap v467: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.112+0000 7f1da1600700 1 -- 172.21.15.26:0/2949623297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d9c06a170 msgr2=0x7f1d9c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.112+0000 7f1da1600700 1 --2- 172.21.15.26:0/2949623297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d9c06a170 0x7f1d9c06a5f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009990 tx=0x7f1d84009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.113+0000 7f1da1600700 1 -- 172.21.15.26:0/2949623297 shutdown_connections 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.113+0000 7f1da1600700 1 --2- 172.21.15.26:0/2949623297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d9c06ab30 0x7f1d9c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.113+0000 7f1da1600700 1 --2- 172.21.15.26:0/2949623297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d9c108500 0x7f1d9c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.113+0000 7f1da1600700 1 --2- 172.21.15.26:0/2949623297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d9c06a170 0x7f1d9c06a5f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.113+0000 7f1da1600700 1 -- 172.21.15.26:0/2949623297 >> 172.21.15.26:0/2949623297 conn(0x7f1d9c078340 msgr2=0x7f1d9c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.113+0000 7f1da1600700 1 -- 172.21.15.26:0/2949623297 shutdown_connections 2024-06-13T06:59:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.113+0000 7f1da1600700 1 -- 172.21.15.26:0/2949623297 wait complete. 2024-06-13T06:59:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.114+0000 7f1da1600700 1 Processor -- start 2024-06-13T06:59:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.114+0000 7f1da1600700 1 -- start start 2024-06-13T06:59:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.114+0000 7f1da1600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d9c06a170 0x7f1d9c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.114+0000 7f1da1600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d9c06ab30 0x7f1d9c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.114+0000 7f1da1600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d9c108500 0x7f1d9c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.114+0000 7f1da1600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1127b0 con 0x7f1d9c108500 2024-06-13T06:59:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.114+0000 7f1da1600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c112630 con 0x7f1d9c06ab30 2024-06-13T06:59:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.114+0000 7f1da1600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c112930 con 0x7f1d9c06a170 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d92000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d9c06ab30 0x7f1d9c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d9b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d9c108500 0x7f1d9c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d92000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d9c06ab30 0x7f1d9c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:58702/0 (socket says 172.21.15.26:58702) 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d92000700 1 -- 172.21.15.26:0/46368370 learned_addr learned my addr 172.21.15.26:0/46368370 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d9b400700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d9c108500 0x7f1d9c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43386/0 (socket says 172.21.15.26:43386) 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d9aa00700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d9c06a170 0x7f1d9c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d92000700 1 -- 172.21.15.26:0/46368370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d9c06a170 msgr2=0x7f1d9c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d92000700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d9c06a170 0x7f1d9c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d92000700 1 -- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d9c108500 msgr2=0x7f1d9c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d92000700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d9c108500 0x7f1d9c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.115+0000 7f1d92000700 1 -- 172.21.15.26:0/46368370 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c1991f0 con 0x7f1d9c06ab30 2024-06-13T06:59:06.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.116+0000 7f1d92000700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d9c06ab30 0x7f1d9c19ee10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005910 tx=0x7f1d8400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.116+0000 7f1d93e00700 1 -- 172.21.15.26:0/46368370 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84027070 con 0x7f1d9c06ab30 2024-06-13T06:59:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.116+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c199480 con 0x7f1d9c06ab30 2024-06-13T06:59:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.116+0000 7f1d93e00700 1 -- 172.21.15.26:0/46368370 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d84004650 con 0x7f1d9c06ab30 2024-06-13T06:59:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.116+0000 7f1d93e00700 1 -- 172.21.15.26:0/46368370 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400f460 con 0x7f1d9c06ab30 2024-06-13T06:59:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.116+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c101390 con 0x7f1d9c06ab30 2024-06-13T06:59:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.117+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c103bc0 con 0x7f1d9c06ab30 2024-06-13T06:59:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.120+0000 7f1d93e00700 1 -- 172.21.15.26:0/46368370 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d8402a070 con 0x7f1d9c06ab30 2024-06-13T06:59:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.121+0000 7f1d93e00700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1d800a2c20 0x7f1d800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.121+0000 7f1d93e00700 1 -- 172.21.15.26:0/46368370 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1d840d07a0 con 0x7f1d9c06ab30 2024-06-13T06:59:06.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.121+0000 7f1d9aa00700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1d800a2c20 0x7f1d800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:06.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.122+0000 7f1d9aa00700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1d800a2c20 0x7f1d800a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c199bd0 tx=0x7f1d8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:06.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.123+0000 7f1d93e00700 1 -- 172.21.15.26:0/46368370 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d84031070 con 0x7f1d9c06ab30 2024-06-13T06:59:06.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.406+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d9c02dca0 con 0x7f1d9c06ab30 2024-06-13T06:59:06.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.408+0000 7f1d93e00700 1 -- 172.21.15.26:0/46368370 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f1d84013070 con 0x7f1d9c06ab30 2024-06-13T06:59:06.408 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:59:06.408 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T06:56:36.605518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24511,24517,24506]},"id":1}]} 2024-06-13T06:59:06.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1d800a2c20 msgr2=0x7f1d800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:06.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1d800a2c20 0x7f1d800a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c199bd0 tx=0x7f1d8c009260 comp rx=0 tx=0).stop 2024-06-13T06:59:06.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d9c06ab30 msgr2=0x7f1d9c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:06.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d9c06ab30 0x7f1d9c19ee10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005910 tx=0x7f1d8400bab0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 shutdown_connections 2024-06-13T06:59:06.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1d800a2c20 0x7f1d800a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d9c108500 0x7f1d9c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d9c06a170 0x7f1d9c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 --2- 172.21.15.26:0/46368370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d9c06ab30 0x7f1d9c19ee10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:06.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 >> 172.21.15.26:0/46368370 conn(0x7f1d9c078340 msgr2=0x7f1d9c0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:06.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 shutdown_connections 2024-06-13T06:59:06.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:06.411+0000 7f1da1600700 1 -- 172.21.15.26:0/46368370 wait complete. 2024-06-13T06:59:06.412 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-06-13T06:59:06.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:06 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/46368370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:06.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:06 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/46368370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:06.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:06 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/46368370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:07.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-13T06:55:01.881338+0000', 'modified': '2024-06-13T06:56:36.605518+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24517, 'mds_1': 24511, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/582335478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 582335478}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 582335478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'a', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6841/3560957238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 3560957238}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 3560957238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6837/1512917649', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 1512917649}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 1512917649}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.135:6839/1905980530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 1905980530}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 1905980530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.136:6839/2531149732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 2531149732}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 2531149732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'g', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.26:6839/173910348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 173910348}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 173910348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24517, 'qdb_cluster': [24511, 24517, 24506]} 2024-06-13T06:59:07.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-13T06:59:07.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.b status 2024-06-13T06:59:07.468 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:07 smithi026 ceph-mon[32041]: pgmap v468: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:07.786 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:07 smithi135 ceph-mon[36276]: pgmap v468: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:07.870 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:07.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:07 smithi136 ceph-mon[37229]: pgmap v468: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:09.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:09 smithi026 ceph-mon[32041]: pgmap v469: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:09.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:09 smithi135 ceph-mon[36276]: pgmap v469: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:09.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:09 smithi136 ceph-mon[37229]: pgmap v469: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 -- 172.21.15.26:0/1313363014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c103640 msgr2=0x7f281c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 --2- 172.21.15.26:0/1313363014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c103640 0x7f281c103a20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f280c009990 tx=0x7f280c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 -- 172.21.15.26:0/1313363014 shutdown_connections 2024-06-13T06:59:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 --2- 172.21.15.26:0/1313363014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c103640 0x7f281c103a20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 --2- 172.21.15.26:0/1313363014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281c10ca20 0x7f281c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 --2- 172.21.15.26:0/1313363014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c103f60 0x7f281c10c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 -- 172.21.15.26:0/1313363014 >> 172.21.15.26:0/1313363014 conn(0x7f281c0fcdb0 msgr2=0x7f281c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 -- 172.21.15.26:0/1313363014 shutdown_connections 2024-06-13T06:59:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.282+0000 7f2820e00700 1 -- 172.21.15.26:0/1313363014 wait complete. 2024-06-13T06:59:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.283+0000 7f2820e00700 1 Processor -- start 2024-06-13T06:59:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f2820e00700 1 -- start start 2024-06-13T06:59:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f2820e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c103640 0x7f281c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f2820e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281c103f60 0x7f281c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f2820e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c10ca20 0x7f281c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f2820e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f281c112770 con 0x7f281c103f60 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f2820e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f281c1125f0 con 0x7f281c103640 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f2820e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f281c1128f0 con 0x7f281c10ca20 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f281aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281c103f60 0x7f281c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f281aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281c103f60 0x7f281c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43402/0 (socket says 172.21.15.26:43402) 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f281aa00700 1 -- 172.21.15.26:0/2187061997 learned_addr learned my addr 172.21.15.26:0/2187061997 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.284+0000 7f281b400700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c103640 0x7f281c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.285+0000 7f281be00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c10ca20 0x7f281c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.285+0000 7f281aa00700 1 -- 172.21.15.26:0/2187061997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c10ca20 msgr2=0x7f281c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.285+0000 7f281aa00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c10ca20 0x7f281c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.285+0000 7f281aa00700 1 -- 172.21.15.26:0/2187061997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c103640 msgr2=0x7f281c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.285+0000 7f281aa00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c103640 0x7f281c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.285+0000 7f281aa00700 1 -- 172.21.15.26:0/2187061997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c1991d0 con 0x7f281c103f60 2024-06-13T06:59:10.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.285+0000 7f281aa00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281c103f60 0x7f281c19edf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9f0 tx=0x7f280800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:10.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.286+0000 7f2813e00700 1 -- 172.21.15.26:0/2187061997 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808004a30 con 0x7f281c103f60 2024-06-13T06:59:10.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.286+0000 7f2813e00700 1 -- 172.21.15.26:0/2187061997 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280800cca0 con 0x7f281c103f60 2024-06-13T06:59:10.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.286+0000 7f2813e00700 1 -- 172.21.15.26:0/2187061997 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080056c0 con 0x7f281c103f60 2024-06-13T06:59:10.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.286+0000 7f2820e00700 1 -- 172.21.15.26:0/2187061997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c1994c0 con 0x7f281c103f60 2024-06-13T06:59:10.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.286+0000 7f2820e00700 1 -- 172.21.15.26:0/2187061997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c1a5780 con 0x7f281c103f60 2024-06-13T06:59:10.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.290+0000 7f2813e00700 1 -- 172.21.15.26:0/2187061997 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f280800c7c0 con 0x7f281c103f60 2024-06-13T06:59:10.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.291+0000 7f282cdf5b80 1 Processor -- start 2024-06-13T06:59:10.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.291+0000 7f282cdf5b80 1 -- start start 2024-06-13T06:59:10.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.291+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903d960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.291+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903da53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.291+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903da7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.291+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e903ca59c0 con 0x55e903d96630 2024-06-13T06:59:10.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.291+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e903ce8ab0 con 0x55e903da58f0 2024-06-13T06:59:10.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.291+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e903ca38a0 con 0x55e903d95d10 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903da53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f2813e00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28000a2ca0 0x7f28000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903da53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43416/0 (socket says 172.21.15.26:43416) 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281b400700 1 -- 172.21.15.26:0/1342866479 learned_addr learned my addr 172.21.15.26:0/1342866479 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281be00700 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903d960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281aa00700 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903da7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f2813e00700 1 -- 172.21.15.26:0/2187061997 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f28080cf640 con 0x7f281c103f60 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281be00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28000a2ca0 0x7f28000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281aa00700 1 -- 172.21.15.26:0/1342866479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 msgr2=0x55e903d960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:10.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281aa00700 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903d960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281aa00700 1 -- 172.21.15.26:0/1342866479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 msgr2=0x55e903da53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:10.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281aa00700 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903da53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281aa00700 1 -- 172.21.15.26:0/1342866479 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e903c90f40 con 0x55e903da58f0 2024-06-13T06:59:10.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.292+0000 7f281b400700 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903da53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:10.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.293+0000 7f281aa00700 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903da7ce0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2808004ca0 tx=0x7f2808005120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8026b82fdb73db39 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:10.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.293+0000 7f281be00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28000a2ca0 0x7f28000a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2814009ec0 tx=0x7f2814004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:10.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.293+0000 7f2810c00700 1 -- 172.21.15.26:0/1342866479 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280809b3b0 con 0x55e903da58f0 2024-06-13T06:59:10.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.293+0000 7f2810c00700 1 -- 172.21.15.26:0/1342866479 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280800ce50 con 0x55e903da58f0 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.293+0000 7f2810c00700 1 -- 172.21.15.26:0/1342866479 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808099d50 con 0x55e903da58f0 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 -- 172.21.15.26:0/1342866479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 msgr2=0x55e903da7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903da7ce0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2808004ca0 tx=0x7f2808005120 comp rx=0 tx=0).stop 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 -- 172.21.15.26:0/1342866479 shutdown_connections 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903da7ce0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903d960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/1342866479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903da53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 -- 172.21.15.26:0/1342866479 >> 172.21.15.26:0/1342866479 conn(0x55e903cc3120 msgr2=0x55e903d92230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 -- 172.21.15.26:0/1342866479 shutdown_connections 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.294+0000 7f282cdf5b80 1 -- 172.21.15.26:0/1342866479 wait complete. 2024-06-13T06:59:10.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.295+0000 7f282cdf5b80 1 -- start start 2024-06-13T06:59:10.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.295+0000 7f282cdf5b80 1 Processor -- start 2024-06-13T06:59:10.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903da5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d96630 0x55e903da2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903da58f0 0x55e903da29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e903c84f10 con 0x55e903da58f0 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e903ce4a70 con 0x55e903d96630 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e903da8320 con 0x55e903d95d10 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f282cdf5b80 10 client.0 _ll_register_callbacks cb 0x55e903c7e330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281be00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903da58f0 0x55e903da29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d96630 0x55e903da2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903da5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281be00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903da58f0 0x55e903da29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43428/0 (socket says 172.21.15.26:43428) 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281be00700 1 -- 172.21.15.26:0/468250815 learned_addr learned my addr 172.21.15.26:0/468250815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281b400700 1 -- 172.21.15.26:0/468250815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d96630 msgr2=0x55e903da2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281b400700 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d96630 0x55e903da2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281b400700 1 -- 172.21.15.26:0/468250815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903da58f0 msgr2=0x55e903da29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281b400700 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903da58f0 0x55e903da29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281b400700 1 -- 172.21.15.26:0/468250815 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e903c37880 con 0x55e903d95d10 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.296+0000 7f281aa00700 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d96630 0x55e903da2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.297+0000 7f281b400700 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903da5190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f280c009990 tx=0x7f280c00bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81ee58e80666bbe2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.297+0000 7f2805600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.297+0000 7f2805600700 1 -- 172.21.15.26:0/468250815 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c005490 con 0x55e903d95d10 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.297+0000 7f282cdf5b80 1 -- 172.21.15.26:0/468250815 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280800ce50 con 0x55e903d95d10 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.297+0000 7f2805600700 1 -- 172.21.15.26:0/468250815 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f280c005630 con 0x55e903d95d10 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.297+0000 7f2805600700 1 -- 172.21.15.26:0/468250815 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f280c031050 con 0x55e903d95d10 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f282cdf5b80 10 client.24979 fetch_fsmap learned FSMap version 26 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f282cdf5b80 1 -- 172.21.15.26:0/468250815 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e903db2b70 con 0x55e903d95d10 2024-06-13T06:59:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f2805600700 1 -- 172.21.15.26:0/468250815 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280800ce50 con 0x55e903d95d10 2024-06-13T06:59:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f2805600700 1 -- 172.21.15.26:0/468250815 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f280c02c030 con 0x55e903d95d10 2024-06-13T06:59:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f282cdf5b80 10 client.24979 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:59:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f282cdf5b80 10 client.24979 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T06:59:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f282cdf5b80 20 client.24979 populate_metadata read hostname 'smithi026' 2024-06-13T06:59:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x55e903db51a0 0x55e903db6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f282cdf5b80 4 client.24979 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.298+0000 7f282cdf5b80 1 -- 172.21.15.26:0/468250815 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e903cf5fb0 con 0x55e903db51a0 2024-06-13T06:59:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.299+0000 7f281aa00700 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x55e903db51a0 0x55e903db6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.299+0000 7f281aa00700 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x55e903db51a0 0x55e903db6610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=585caafb7649806f server_cookie=98e6ac5bbce34efa in_seq=0 out_seq=0 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.299+0000 7f2805600700 10 client.24979 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.300+0000 7f2805600700 1 -- 172.21.15.26:0/468250815 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e903cf5fb0 con 0x55e903db51a0 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.300+0000 7f2805600700 10 client.24979 handle_command_reply: tid=0 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.300+0000 7f282cdf5b80 1 client.24979 shutdown 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 -- 172.21.15.26:0/468250815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 msgr2=0x55e903da5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903da5190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f280c009990 tx=0x7f280c00bf80 comp rx=0 tx=0).stop 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 -- 172.21.15.26:0/468250815 shutdown_connections 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x55e903db51a0 0x55e903db6610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903da58f0 0x55e903da29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d95d10 0x55e903da5190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/468250815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d96630 0x55e903da2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f2805600700 0 client.24979 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T06:59:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 -- 172.21.15.26:0/468250815 >> 172.21.15.26:0/468250815 conn(0x55e903cc3120 msgr2=0x55e903d91e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 -- 172.21.15.26:0/468250815 shutdown_connections 2024-06-13T06:59:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 1 -- 172.21.15.26:0/468250815 wait complete. 2024-06-13T06:59:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.301+0000 7f282cdf5b80 20 client.24979 trim_cache size 0 max 16384 2024-06-13T06:59:10.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f282cdf5b80 1 Processor -- start 2024-06-13T06:59:10.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f282cdf5b80 1 -- start start 2024-06-13T06:59:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d95d10 0x55e903d9dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903d984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903da58f0 0x55e903d98a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e903db4950 con 0x55e903d96630 2024-06-13T06:59:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e903db5020 con 0x55e903d95d10 2024-06-13T06:59:10.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e903db56f0 con 0x55e903da58f0 2024-06-13T06:59:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f281aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903d984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f281aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903d984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43432/0 (socket says 172.21.15.26:43432) 2024-06-13T06:59:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.325+0000 7f281aa00700 1 -- 172.21.15.26:0/387742302 learned_addr learned my addr 172.21.15.26:0/387742302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.326+0000 7f281b400700 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d95d10 0x55e903d9dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.326+0000 7f281be00700 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903da58f0 0x55e903d98a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.326+0000 7f281aa00700 1 -- 172.21.15.26:0/387742302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903da58f0 msgr2=0x55e903d98a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.326+0000 7f281aa00700 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903da58f0 0x55e903d98a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.326+0000 7f281aa00700 1 -- 172.21.15.26:0/387742302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d95d10 msgr2=0x55e903d9dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.326+0000 7f281aa00700 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d95d10 0x55e903d9dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.326+0000 7f281aa00700 1 -- 172.21.15.26:0/387742302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e903da0a00 con 0x55e903d96630 2024-06-13T06:59:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.327+0000 7f281aa00700 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903d984c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2808097eb0 tx=0x7f280809b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814b75cefd1b4b01 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.327+0000 7f2807e00700 1 -- 172.21.15.26:0/387742302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080a46c0 con 0x55e903d96630 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.327+0000 7f2807e00700 1 -- 172.21.15.26:0/387742302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280809c150 con 0x55e903d96630 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.327+0000 7f2807e00700 1 -- 172.21.15.26:0/387742302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808004cf0 con 0x55e903d96630 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 -- 172.21.15.26:0/387742302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 msgr2=0x55e903d984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903d984c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2808097eb0 tx=0x7f280809b3b0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 -- 172.21.15.26:0/387742302 shutdown_connections 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903d95d10 0x55e903d9dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903da58f0 0x55e903d98a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/387742302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d96630 0x55e903d984c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 -- 172.21.15.26:0/387742302 >> 172.21.15.26:0/387742302 conn(0x55e903cc3120 msgr2=0x55e903d97db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 -- 172.21.15.26:0/387742302 shutdown_connections 2024-06-13T06:59:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.328+0000 7f282cdf5b80 1 -- 172.21.15.26:0/387742302 wait complete. 2024-06-13T06:59:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.329+0000 7f282cdf5b80 1 -- start start 2024-06-13T06:59:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.329+0000 7f282cdf5b80 1 Processor -- start 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d95d10 0x55e903dbfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d96630 0x55e903dc00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f282cdf5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903db9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e903db6490 con 0x55e903d95d10 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e903db5dc0 con 0x55e903da58f0 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f282cdf5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e903db6b60 con 0x55e903d96630 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f282cdf5b80 10 client.0 _ll_register_callbacks cb 0x55e903c7e330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f281b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d95d10 0x55e903dbfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f281b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d95d10 0x55e903dbfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43444/0 (socket says 172.21.15.26:43444) 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f281b400700 1 -- 172.21.15.26:0/2643350470 learned_addr learned my addr 172.21.15.26:0/2643350470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f281aa00700 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d96630 0x55e903dc00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.330+0000 7f281be00700 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903db9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f281b400700 1 -- 172.21.15.26:0/2643350470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d96630 msgr2=0x55e903dc00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f281b400700 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d96630 0x55e903dc00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f281b400700 1 -- 172.21.15.26:0/2643350470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 msgr2=0x55e903db9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f281b400700 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903db9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f281b400700 1 -- 172.21.15.26:0/2643350470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e903db9850 con 0x55e903d95d10 2024-06-13T06:59:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f281aa00700 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d96630 0x55e903dc00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f281b400700 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d95d10 0x55e903dbfbb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f280c00b820 tx=0x7f280c00b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1e8b6b6c2da547 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f2805600700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f2805600700 1 -- 172.21.15.26:0/2643350470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c024cf0 con 0x55e903d95d10 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f2805600700 1 -- 172.21.15.26:0/2643350470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f280c005390 con 0x55e903d95d10 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.331+0000 7f282cdf5b80 1 -- 172.21.15.26:0/2643350470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280809c150 con 0x55e903d95d10 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f2805600700 1 -- 172.21.15.26:0/2643350470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f280c024690 con 0x55e903d95d10 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f282cdf5b80 10 client.14862 fetch_fsmap learned FSMap version 26 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f282cdf5b80 1 -- 172.21.15.26:0/2643350470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e903dbe1d0 con 0x55e903d95d10 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f2805600700 1 -- 172.21.15.26:0/2643350470 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280809c150 con 0x55e903d95d10 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f2805600700 1 -- 172.21.15.26:0/2643350470 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f280c02c040 con 0x55e903d95d10 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f282cdf5b80 10 client.14862 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f282cdf5b80 10 client.14862 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f282cdf5b80 20 client.14862 populate_metadata read hostname 'smithi026' 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.332+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x55e903e25060 0x55e903e23d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.333+0000 7f282cdf5b80 4 client.14862 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:59:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.333+0000 7f282cdf5b80 1 -- 172.21.15.26:0/2643350470 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "status"}) -- 0x55e903cf0530 con 0x55e903e25060 2024-06-13T06:59:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.333+0000 7f281aa00700 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x55e903e25060 0x55e903e23d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.333+0000 7f281aa00700 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x55e903e25060 0x55e903e23d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a87bbbf4050b7ba server_cookie=5325307d10915704 in_seq=0 out_seq=0 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.333+0000 7f2805600700 10 client.14862 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f2805600700 1 -- 172.21.15.26:0/2643350470 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e903cf0530 con 0x55e903e25060 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f2805600700 10 client.14862 handle_command_reply: tid=0 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 client.14862 shutdown 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 -- 172.21.15.26:0/2643350470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d95d10 msgr2=0x55e903dbfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d95d10 0x55e903dbfbb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f280c00b820 tx=0x7f280c00b280 comp rx=0 tx=0).stop 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 -- 172.21.15.26:0/2643350470 shutdown_connections 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x55e903e25060 0x55e903e23d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e903da58f0 0x55e903db9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e903d95d10 0x55e903dbfbb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 --2- 172.21.15.26:0/2643350470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e903d96630 0x55e903dc00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 -- 172.21.15.26:0/2643350470 >> 172.21.15.26:0/2643350470 conn(0x55e903cc3120 msgr2=0x55e903d9da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f2805600700 0 client.14862 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 -- 172.21.15.26:0/2643350470 shutdown_connections 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 1 -- 172.21.15.26:0/2643350470 wait complete. 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.334+0000 7f282cdf5b80 20 client.14862 trim_cache size 0 max 16384 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24506, 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-06-13T06:59:10.336 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-06-13T06:59:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 161.832020492, 2024-06-13T06:59:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-06-13T06:59:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 99, 2024-06-13T06:59:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 99, 2024-06-13T06:59:10.337 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 272.93263165000002 2024-06-13T06:59:10.337 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T06:59:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 -- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28000a2ca0 msgr2=0x7f28000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28000a2ca0 0x7f28000a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2814009ec0 tx=0x7f2814004b40 comp rx=0 tx=0).stop 2024-06-13T06:59:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 -- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281c103f60 msgr2=0x7f281c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281c103f60 0x7f281c19edf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9f0 tx=0x7f280800ed00 comp rx=0 tx=0).stop 2024-06-13T06:59:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 -- 172.21.15.26:0/2187061997 shutdown_connections 2024-06-13T06:59:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28000a2ca0 0x7f28000a5160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c103640 0x7f281c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c10ca20 0x7f281c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 --2- 172.21.15.26:0/2187061997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281c103f60 0x7f281c19edf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.335+0000 7f2820e00700 1 -- 172.21.15.26:0/2187061997 >> 172.21.15.26:0/2187061997 conn(0x7f281c0fcdb0 msgr2=0x7f281c0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.336+0000 7f2820e00700 1 -- 172.21.15.26:0/2187061997 shutdown_connections 2024-06-13T06:59:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:10.336+0000 7f2820e00700 1 -- 172.21.15.26:0/2187061997 wait complete. 2024-06-13T06:59:11.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.i status 2024-06-13T06:59:11.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:11 smithi026 ceph-mon[32041]: pgmap v470: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:11.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:11 smithi135 ceph-mon[36276]: pgmap v470: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:11.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:11 smithi136 ceph-mon[37229]: pgmap v470: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:13.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:13 smithi026 ceph-mon[32041]: pgmap v471: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:13.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:13 smithi135 ceph-mon[36276]: pgmap v471: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:13.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:13 smithi136 ceph-mon[37229]: pgmap v471: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 -- 172.21.15.26:0/694618769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99101046b0 msgr2=0x7f9910104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 --2- 172.21.15.26:0/694618769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99101046b0 0x7f9910104b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009990 tx=0x7f98fc009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 -- 172.21.15.26:0/694618769 shutdown_connections 2024-06-13T06:59:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 --2- 172.21.15.26:0/694618769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9910105070 0x7f9910111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 --2- 172.21.15.26:0/694618769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991010a6b0 0x7f991010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 --2- 172.21.15.26:0/694618769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99101046b0 0x7f9910104b30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 -- 172.21.15.26:0/694618769 >> 172.21.15.26:0/694618769 conn(0x7f99101001a0 msgr2=0x7f99101025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 -- 172.21.15.26:0/694618769 shutdown_connections 2024-06-13T06:59:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.328+0000 7f9915200700 1 -- 172.21.15.26:0/694618769 wait complete. 2024-06-13T06:59:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.329+0000 7f9915200700 1 Processor -- start 2024-06-13T06:59:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.329+0000 7f9915200700 1 -- start start 2024-06-13T06:59:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f9915200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99101046b0 0x7f991019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f9915200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9910105070 0x7f991019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f9915200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991010a6b0 0x7f991019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f9915200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9910114a40 con 0x7f991010a6b0 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f9915200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99101148c0 con 0x7f9910105070 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f9915200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9910114bc0 con 0x7f99101046b0 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99101046b0 0x7f991019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99101046b0 0x7f991019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53634/0 (socket says 172.21.15.26:53634) 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990de00700 1 -- 172.21.15.26:0/2570342812 learned_addr learned my addr 172.21.15.26:0/2570342812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990d400700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9910105070 0x7f991019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990e800700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991010a6b0 0x7f991019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990de00700 1 -- 172.21.15.26:0/2570342812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9910105070 msgr2=0x7f991019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990de00700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9910105070 0x7f991019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990de00700 1 -- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991010a6b0 msgr2=0x7f991019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990de00700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991010a6b0 0x7f991019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.330+0000 7f990de00700 1 -- 172.21.15.26:0/2570342812 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991019fe60 con 0x7f99101046b0 2024-06-13T06:59:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.331+0000 7f990de00700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99101046b0 0x7f991019ae50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f990000eb30 tx=0x7f990000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.331+0000 7f9906a00700 1 -- 172.21.15.26:0/2570342812 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990000cc90 con 0x7f99101046b0 2024-06-13T06:59:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.331+0000 7f9906a00700 1 -- 172.21.15.26:0/2570342812 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9900004500 con 0x7f99101046b0 2024-06-13T06:59:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.331+0000 7f9906a00700 1 -- 172.21.15.26:0/2570342812 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900010430 con 0x7f99101046b0 2024-06-13T06:59:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.331+0000 7f9915200700 1 -- 172.21.15.26:0/2570342812 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101a0150 con 0x7f99101046b0 2024-06-13T06:59:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.331+0000 7f9915200700 1 -- 172.21.15.26:0/2570342812 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99101a79d0 con 0x7f99101046b0 2024-06-13T06:59:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.336+0000 7f9906a00700 1 -- 172.21.15.26:0/2570342812 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9900003f70 con 0x7f99101046b0 2024-06-13T06:59:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.337+0000 7f991fb36b80 1 Processor -- start 2024-06-13T06:59:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.337+0000 7f991fb36b80 1 -- start start 2024-06-13T06:59:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f9906a00700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98f40a2ca0 0x7f98f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f9906a00700 1 -- 172.21.15.26:0/2570342812 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f99000cf6e0 con 0x7f99101046b0 2024-06-13T06:59:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d585b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d585b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d58662b0 0x55c2d58686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f991fb36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c2d576a9c0 con 0x55c2d58662b0 2024-06-13T06:59:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f991fb36b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c2d57adab0 con 0x55c2d57d7df0 2024-06-13T06:59:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f991fb36b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c2d57688a0 con 0x55c2d585b560 2024-06-13T06:59:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f990e800700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98f40a2ca0 0x7f98f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f990d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d585b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f990de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d585b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f990d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d585b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53646/0 (socket says 172.21.15.26:53646) 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f990d400700 1 -- 172.21.15.26:0/3373549399 learned_addr learned my addr 172.21.15.26:0/3373549399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.338+0000 7f990e800700 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d58662b0 0x55c2d58686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f990d400700 1 -- 172.21.15.26:0/3373549399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 msgr2=0x55c2d585b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f990d400700 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d585b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f990d400700 1 -- 172.21.15.26:0/3373549399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d58662b0 msgr2=0x55c2d58686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f990d400700 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d58662b0 0x55c2d58686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f990d400700 1 -- 172.21.15.26:0/3373549399 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d5755f40 con 0x55c2d585b560 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f990e800700 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d58662b0 0x55c2d58686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f990d400700 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d585b9e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98fc015000 tx=0x7f98fc00b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cc38b5160d15937 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f990e800700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98f40a2ca0 0x7f98f40a5160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9908015b80 tx=0x7f9908009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.339+0000 7f98fb400700 1 -- 172.21.15.26:0/3373549399 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc027070 con 0x55c2d585b560 2024-06-13T06:59:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f98fb400700 1 -- 172.21.15.26:0/3373549399 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98fc00bd10 con 0x55c2d585b560 2024-06-13T06:59:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f98fb400700 1 -- 172.21.15.26:0/3373549399 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc003be0 con 0x55c2d585b560 2024-06-13T06:59:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f991fb36b80 1 -- 172.21.15.26:0/3373549399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 msgr2=0x55c2d585b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f991fb36b80 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d585b9e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98fc015000 tx=0x7f98fc00b7a0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f991fb36b80 1 -- 172.21.15.26:0/3373549399 shutdown_connections 2024-06-13T06:59:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f991fb36b80 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d585b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f991fb36b80 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d58662b0 0x55c2d58686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f991fb36b80 1 --2- 172.21.15.26:0/3373549399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d585b9e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f991fb36b80 1 -- 172.21.15.26:0/3373549399 >> 172.21.15.26:0/3373549399 conn(0x55c2d5788120 msgr2=0x55c2d5857460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.340+0000 7f991fb36b80 1 -- 172.21.15.26:0/3373549399 shutdown_connections 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.341+0000 7f991fb36b80 1 -- 172.21.15.26:0/3373549399 wait complete. 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.341+0000 7f991fb36b80 1 -- start start 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.341+0000 7f991fb36b80 1 Processor -- start 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d5862d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d58662b0 0x55c2d5863250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f991fb36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5749f10 con 0x55c2d57d7df0 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f991fb36b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c2d57a9a70 con 0x55c2d58662b0 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f991fb36b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5868f10 con 0x55c2d585b560 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f991fb36b80 10 client.0 _ll_register_callbacks cb 0x55c2d57476d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f990de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f990de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58659a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55982/0 (socket says 172.21.15.26:55982) 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f990d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d5862d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f990de00700 1 -- 172.21.15.26:0/961309987 learned_addr learned my addr 172.21.15.26:0/961309987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.342+0000 7f990e800700 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d58662b0 0x55c2d5863250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f990d400700 1 -- 172.21.15.26:0/961309987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d58662b0 msgr2=0x55c2d5863250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f990d400700 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d58662b0 0x55c2d5863250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f990d400700 1 -- 172.21.15.26:0/961309987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 msgr2=0x55c2d58659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f990d400700 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58659a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f990d400700 1 -- 172.21.15.26:0/961309987 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d56fc880 con 0x55c2d585b560 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f990de00700 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:14.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f990d400700 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d5862d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0098e0 tx=0x7f98fc00fb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9caf707f1173bab server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f98efe00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T06:59:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.343+0000 7f98efe00700 1 -- 172.21.15.26:0/961309987 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc027040 con 0x55c2d585b560 2024-06-13T06:59:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f98efe00700 1 -- 172.21.15.26:0/961309987 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f98fc00fd50 con 0x55c2d585b560 2024-06-13T06:59:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f991fb36b80 1 -- 172.21.15.26:0/961309987 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98fc00bd10 con 0x55c2d585b560 2024-06-13T06:59:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f98efe00700 1 -- 172.21.15.26:0/961309987 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98fc005bc0 con 0x55c2d585b560 2024-06-13T06:59:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f991fb36b80 10 client.24997 fetch_fsmap learned FSMap version 26 2024-06-13T06:59:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f991fb36b80 1 -- 172.21.15.26:0/961309987 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2d5873630 con 0x55c2d585b560 2024-06-13T06:59:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f98efe00700 1 -- 172.21.15.26:0/961309987 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98fc00bd10 con 0x55c2d585b560 2024-06-13T06:59:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f98efe00700 1 -- 172.21.15.26:0/961309987 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f98fc02a070 con 0x55c2d585b560 2024-06-13T06:59:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f991fb36b80 10 client.24997 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:59:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.344+0000 7f991fb36b80 10 client.24997 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-13T06:59:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.345+0000 7f991fb36b80 20 client.24997 populate_metadata read hostname 'smithi026' 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.345+0000 7f991fb36b80 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x55c2d5874980 0x55c2d5876b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.345+0000 7f991fb36b80 4 client.24997 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.345+0000 7f991fb36b80 1 -- 172.21.15.26:0/961309987 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2d57bafb0 con 0x55c2d5874980 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.345+0000 7f990de00700 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x55c2d5874980 0x55c2d5876b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.345+0000 7f990de00700 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x55c2d5874980 0x55c2d5876b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c9d3e5c5c5e0d4d server_cookie=a9334a6627e86e3b in_seq=0 out_seq=0 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.345+0000 7f98efe00700 10 client.24997 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.346+0000 7f98efe00700 1 -- 172.21.15.26:0/961309987 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c2d57bafb0 con 0x55c2d5874980 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.346+0000 7f98efe00700 10 client.24997 handle_command_reply: tid=0 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 client.24997 shutdown 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 -- 172.21.15.26:0/961309987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 msgr2=0x55c2d5862d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d5862d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0098e0 tx=0x7f98fc00fb60 comp rx=0 tx=0).stop 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 -- 172.21.15.26:0/961309987 shutdown_connections 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x55c2d5874980 0x55c2d5876b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d58662b0 0x55c2d5863250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f98efe00700 0 client.24997 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T06:59:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 --2- 172.21.15.26:0/961309987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d585b560 0x55c2d5862d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 -- 172.21.15.26:0/961309987 >> 172.21.15.26:0/961309987 conn(0x55c2d5788120 msgr2=0x55c2d58566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 -- 172.21.15.26:0/961309987 shutdown_connections 2024-06-13T06:59:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 1 -- 172.21.15.26:0/961309987 wait complete. 2024-06-13T06:59:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.347+0000 7f991fb36b80 20 client.24997 trim_cache size 0 max 16384 2024-06-13T06:59:14.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.371+0000 7f991fb36b80 1 Processor -- start 2024-06-13T06:59:14.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.371+0000 7f991fb36b80 1 -- start start 2024-06-13T06:59:14.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.371+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d5875690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.371+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d585b560 0x55c2d58575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.371+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5865550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.371+0000 7f991fb36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5874e40 con 0x55c2d585b560 2024-06-13T06:59:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.371+0000 7f991fb36b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5875510 con 0x55c2d57d7df0 2024-06-13T06:59:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.371+0000 7f991fb36b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5875be0 con 0x55c2d58662b0 2024-06-13T06:59:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d5875690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d5875690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:48198/0 (socket says 172.21.15.26:48198) 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990de00700 1 -- 172.21.15.26:0/1885412687 learned_addr learned my addr 172.21.15.26:0/1885412687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990e800700 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5865550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990d400700 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d585b560 0x55c2d58575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990de00700 1 -- 172.21.15.26:0/1885412687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 msgr2=0x55c2d5865550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990de00700 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5865550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990de00700 1 -- 172.21.15.26:0/1885412687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d585b560 msgr2=0x55c2d58575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990de00700 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d585b560 0x55c2d58575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990de00700 1 -- 172.21.15.26:0/1885412687 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d5861280 con 0x55c2d57d7df0 2024-06-13T06:59:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990e800700 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5865550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.372+0000 7f990d400700 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d585b560 0x55c2d58575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f990de00700 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d5875690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9900098470 tx=0x7f9900097670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbea214920ba1049 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f98faa00700 1 -- 172.21.15.26:0/1885412687 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990009c1f0 con 0x55c2d57d7df0 2024-06-13T06:59:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f98faa00700 1 -- 172.21.15.26:0/1885412687 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990009baf0 con 0x55c2d57d7df0 2024-06-13T06:59:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f98faa00700 1 -- 172.21.15.26:0/1885412687 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900004260 con 0x55c2d57d7df0 2024-06-13T06:59:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f991fb36b80 1 -- 172.21.15.26:0/1885412687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 msgr2=0x55c2d5875690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d5875690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9900098470 tx=0x7f9900097670 comp rx=0 tx=0).stop 2024-06-13T06:59:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f991fb36b80 1 -- 172.21.15.26:0/1885412687 shutdown_connections 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d57d7df0 0x55c2d5875690 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5865550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.373+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1885412687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d585b560 0x55c2d58575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.374+0000 7f991fb36b80 1 -- 172.21.15.26:0/1885412687 >> 172.21.15.26:0/1885412687 conn(0x55c2d5788120 msgr2=0x55c2d58566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.374+0000 7f991fb36b80 1 -- 172.21.15.26:0/1885412687 shutdown_connections 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.374+0000 7f991fb36b80 1 -- 172.21.15.26:0/1885412687 wait complete. 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.374+0000 7f991fb36b80 1 -- start start 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.374+0000 7f991fb36b80 1 Processor -- start 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d585b560 0x55c2d58805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f991fb36b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5879500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f991fb36b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5876980 con 0x55c2d57d7df0 2024-06-13T06:59:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f991fb36b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c2d58762b0 con 0x55c2d585b560 2024-06-13T06:59:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f991fb36b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5877050 con 0x55c2d58662b0 2024-06-13T06:59:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f991fb36b80 10 client.0 _ll_register_callbacks cb 0x55c2d57476d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d585b560 0x55c2d58805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d585b560 0x55c2d58805e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:48202/0 (socket says 172.21.15.26:48202) 2024-06-13T06:59:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990d400700 1 -- 172.21.15.26:0/1067639758 learned_addr learned my addr 172.21.15.26:0/1067639758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:14.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990de00700 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990e800700 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5879500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990d400700 1 -- 172.21.15.26:0/1067639758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 msgr2=0x55c2d5879500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990d400700 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5879500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990d400700 1 -- 172.21.15.26:0/1067639758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 msgr2=0x55c2d58800a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990d400700 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58800a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.375+0000 7f990d400700 1 -- 172.21.15.26:0/1067639758 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d5879d40 con 0x55c2d585b560 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.376+0000 7f990d400700 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d585b560 0x55c2d58805e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f98fc000c00 tx=0x7f98fc00b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc66b72005691db1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.376+0000 7f98efe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.376+0000 7f98efe00700 1 -- 172.21.15.26:0/1067639758 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc027040 con 0x55c2d585b560 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.376+0000 7f991fb36b80 1 -- 172.21.15.26:0/1067639758 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f990009baf0 con 0x55c2d585b560 2024-06-13T06:59:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.376+0000 7f98efe00700 1 -- 172.21.15.26:0/1067639758 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f98fc00bc10 con 0x55c2d585b560 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.376+0000 7f98efe00700 1 -- 172.21.15.26:0/1067639758 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98fc045050 con 0x55c2d585b560 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.376+0000 7f991fb36b80 10 client.24971 fetch_fsmap learned FSMap version 26 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.376+0000 7f991fb36b80 1 -- 172.21.15.26:0/1067639758 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2d587e5f0 con 0x55c2d585b560 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f98efe00700 1 -- 172.21.15.26:0/1067639758 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990009baf0 con 0x55c2d585b560 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f98efe00700 1 -- 172.21.15.26:0/1067639758 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f98fc02a070 con 0x55c2d585b560 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f991fb36b80 10 client.24971 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f991fb36b80 10 client.24971 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f991fb36b80 20 client.24971 populate_metadata read hostname 'smithi026' 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x55c2d58e5500 0x55c2d58e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f991fb36b80 4 client.24971 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:59:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f991fb36b80 1 -- 172.21.15.26:0/1067639758 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "status"}) -- 0x55c2d57b5530 con 0x55c2d58e5500 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.377+0000 7f990de00700 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x55c2d58e5500 0x55c2d58e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f990de00700 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x55c2d58e5500 0x55c2d58e4210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d24a008927adaacb server_cookie=54188f77e0853d8c in_seq=0 out_seq=0 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f98efe00700 10 client.24971 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f98efe00700 1 -- 172.21.15.26:0/1067639758 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c2d57b5530 con 0x55c2d58e5500 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f98efe00700 10 client.24971 handle_command_reply: tid=0 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f991fb36b80 1 client.24971 shutdown 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f991fb36b80 1 -- 172.21.15.26:0/1067639758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d585b560 msgr2=0x55c2d58805e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d585b560 0x55c2d58805e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f98fc000c00 tx=0x7f98fc00b480 comp rx=0 tx=0).stop 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f991fb36b80 1 -- 172.21.15.26:0/1067639758 shutdown_connections 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.378+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x55c2d58e5500 0x55c2d58e4210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c2d58662b0 0x55c2d5879500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2d57d7df0 0x55c2d58800a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f991fb36b80 1 --2- 172.21.15.26:0/1067639758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2d585b560 0x55c2d58805e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f98efe00700 0 client.24971 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T06:59:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f991fb36b80 1 -- 172.21.15.26:0/1067639758 >> 172.21.15.26:0/1067639758 conn(0x55c2d5788120 msgr2=0x55c2d58711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f991fb36b80 1 -- 172.21.15.26:0/1067639758 shutdown_connections 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f991fb36b80 1 -- 172.21.15.26:0/1067639758 wait complete. 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f991fb36b80 20 client.24971 trim_cache size 0 max 16384 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24511, 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-06-13T06:59:14.384 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 166.887558619, 2024-06-13T06:59:14.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-06-13T06:59:14.385 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 99, 2024-06-13T06:59:14.385 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 99, 2024-06-13T06:59:14.385 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 289.85331506699998 2024-06-13T06:59:14.385 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T06:59:14.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f9915200700 1 -- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98f40a2ca0 msgr2=0x7f98f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.379+0000 7f9915200700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98f40a2ca0 0x7f98f40a5160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9908015b80 tx=0x7f9908009920 comp rx=0 tx=0).stop 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 -- 172.21.15.26:0/2570342812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99101046b0 msgr2=0x7f991019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99101046b0 0x7f991019ae50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f990000eb30 tx=0x7f990000ee40 comp rx=0 tx=0).stop 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 -- 172.21.15.26:0/2570342812 shutdown_connections 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98f40a2ca0 0x7f98f40a5160 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991010a6b0 0x7f991019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99101046b0 0x7f991019ae50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 --2- 172.21.15.26:0/2570342812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9910105070 0x7f991019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 -- 172.21.15.26:0/2570342812 >> 172.21.15.26:0/2570342812 conn(0x7f99101001a0 msgr2=0x7f9910100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 -- 172.21.15.26:0/2570342812 shutdown_connections 2024-06-13T06:59:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:14.380+0000 7f9915200700 1 -- 172.21.15.26:0/2570342812 wait complete. 2024-06-13T06:59:15.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.g status 2024-06-13T06:59:15.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:15 smithi026 ceph-mon[32041]: pgmap v472: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:15.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:15 smithi135 ceph-mon[36276]: pgmap v472: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:15.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:15 smithi136 ceph-mon[37229]: pgmap v472: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:17.739 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:17 smithi026 ceph-mon[32041]: pgmap v473: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:17.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:17 smithi135 ceph-mon[36276]: pgmap v473: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:17.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:17 smithi136 ceph-mon[37229]: pgmap v473: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 -- 172.21.15.26:0/254562341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f249410a690 msgr2=0x7f249410aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 --2- 172.21.15.26:0/254562341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f249410a690 0x7f249410aa70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2488009990 tx=0x7f2488009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 -- 172.21.15.26:0/254562341 shutdown_connections 2024-06-13T06:59:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 --2- 172.21.15.26:0/254562341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f249410a690 0x7f249410aa70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 --2- 172.21.15.26:0/254562341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494105050 0x7f2494111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 --2- 172.21.15.26:0/254562341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2494104690 0x7f2494104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 -- 172.21.15.26:0/254562341 >> 172.21.15.26:0/254562341 conn(0x7f24941001a0 msgr2=0x7f24941025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 -- 172.21.15.26:0/254562341 shutdown_connections 2024-06-13T06:59:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.341+0000 7f249c200700 1 -- 172.21.15.26:0/254562341 wait complete. 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.342+0000 7f249c200700 1 Processor -- start 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.342+0000 7f249c200700 1 -- start start 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f249c200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104690 0x7f249419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f249c200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2494105050 0x7f249419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f249c200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249410a690 0x7f249419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f249c200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24941149e0 con 0x7f249410a690 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f249c200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2494114860 con 0x7f2494105050 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f249c200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2494114b60 con 0x7f2494104690 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f2499800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104690 0x7f249419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f2498e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2494105050 0x7f249419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f2499800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104690 0x7f249419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53712/0 (socket says 172.21.15.26:53712) 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f2499800700 1 -- 172.21.15.26:0/239556970 learned_addr learned my addr 172.21.15.26:0/239556970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.343+0000 7f249a200700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249410a690 0x7f249419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2499800700 1 -- 172.21.15.26:0/239556970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2494105050 msgr2=0x7f249419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2499800700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2494105050 0x7f249419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2499800700 1 -- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249410a690 msgr2=0x7f249419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2499800700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249410a690 0x7f249419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2499800700 1 -- 172.21.15.26:0/239556970 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249419fe90 con 0x7f2494104690 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2498e00700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2494105050 0x7f249419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f249a200700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249410a690 0x7f249419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2499800700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104690 0x7f249419aed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2488000c00 tx=0x7f248800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2492000700 1 -- 172.21.15.26:0/239556970 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488027070 con 0x7f2494104690 2024-06-13T06:59:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2492000700 1 -- 172.21.15.26:0/239556970 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2488003d70 con 0x7f2494104690 2024-06-13T06:59:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f2492000700 1 -- 172.21.15.26:0/239556970 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248800f460 con 0x7f2494104690 2024-06-13T06:59:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.344+0000 7f249c200700 1 -- 172.21.15.26:0/239556970 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24941a0120 con 0x7f2494104690 2024-06-13T06:59:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.345+0000 7f249c200700 1 -- 172.21.15.26:0/239556970 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941a7a00 con 0x7f2494104690 2024-06-13T06:59:18.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.348+0000 7f2492000700 1 -- 172.21.15.26:0/239556970 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f248802a070 con 0x7f2494104690 2024-06-13T06:59:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.350+0000 7f2492000700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f247c0a2c20 0x7f247c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f24a6b65b80 1 Processor -- start 2024-06-13T06:59:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f2492000700 1 -- 172.21.15.26:0/239556970 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f24880d0170 con 0x7f2494104690 2024-06-13T06:59:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f24a6b65b80 1 -- start start 2024-06-13T06:59:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f2498e00700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f247c0a2c20 0x7f247c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36162f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba3615ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba3615d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba3606b9c0 con 0x55ba3615d1f0 2024-06-13T06:59:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba360aeab0 con 0x55ba36162b40 2024-06-13T06:59:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.351+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ba360698a0 con 0x55ba36163460 2024-06-13T06:59:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2499800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f249a200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba3615d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba3615ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba3615ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53724/0 (socket says 172.21.15.26:53724) 2024-06-13T06:59:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 -- 172.21.15.26:0/2847294624 learned_addr learned my addr 172.21.15.26:0/2847294624 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2499800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36162f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:48240/0 (socket says 172.21.15.26:48240) 2024-06-13T06:59:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f247c0a2c20 0x7f247c0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2494109ad0 tx=0x7f2480009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 -- 172.21.15.26:0/2847294624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 msgr2=0x55ba36162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 --2- 172.21.15.26:0/2847294624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36162f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 -- 172.21.15.26:0/2847294624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 msgr2=0x55ba3615d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 --2- 172.21.15.26:0/2847294624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba3615d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.352+0000 7f2498e00700 1 -- 172.21.15.26:0/2847294624 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba36056f40 con 0x55ba36163460 2024-06-13T06:59:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.353+0000 7f2498e00700 1 --2- 172.21.15.26:0/2847294624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba3615ccb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f24800042e0 tx=0x7f248001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d786fd2260c7e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:18.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.353+0000 7f2486a00700 1 -- 172.21.15.26:0/2847294624 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480029070 con 0x55ba36163460 2024-06-13T06:59:18.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.353+0000 7f2486a00700 1 -- 172.21.15.26:0/2847294624 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2480003c90 con 0x55ba36163460 2024-06-13T06:59:18.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.353+0000 7f2486a00700 1 -- 172.21.15.26:0/2847294624 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248001c660 con 0x55ba36163460 2024-06-13T06:59:18.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 -- 172.21.15.26:0/2847294624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 msgr2=0x55ba3615ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/2847294624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba3615ccb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f24800042e0 tx=0x7f248001c000 comp rx=0 tx=0).stop 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 -- 172.21.15.26:0/2847294624 shutdown_connections 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/2847294624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36162f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/2847294624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba3615d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/2847294624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba3615ccb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 -- 172.21.15.26:0/2847294624 >> 172.21.15.26:0/2847294624 conn(0x55ba36089120 msgr2=0x55ba36157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 -- 172.21.15.26:0/2847294624 shutdown_connections 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.354+0000 7f24a6b65b80 1 -- 172.21.15.26:0/2847294624 wait complete. 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.355+0000 7f24a6b65b80 1 -- start start 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.355+0000 7f24a6b65b80 1 Processor -- start 2024-06-13T06:59:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.355+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba36169190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.355+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36166490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba3604af10 con 0x55ba3615d1f0 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba360aaa70 con 0x55ba36162b40 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ba36169e40 con 0x55ba36163460 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2499800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba36169190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f24a6b65b80 10 client.0 _ll_register_callbacks cb 0x55ba36044330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2498e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36166490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2499800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba36169190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56040/0 (socket says 172.21.15.26:56040) 2024-06-13T06:59:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2499800700 1 -- 172.21.15.26:0/896845343 learned_addr learned my addr 172.21.15.26:0/896845343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f249a200700 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2498e00700 1 -- 172.21.15.26:0/896845343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 msgr2=0x55ba361669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2498e00700 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2498e00700 1 -- 172.21.15.26:0/896845343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 msgr2=0x55ba36169190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2498e00700 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba36169190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2498e00700 1 -- 172.21.15.26:0/896845343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba35ffd880 con 0x55ba36162b40 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.356+0000 7f2499800700 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba36169190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.357+0000 7f2498e00700 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36166490 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2480005920 tx=0x7f2480024930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f6e017f60910f3f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.357+0000 7f247b400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.357+0000 7f247b400700 1 -- 172.21.15.26:0/896845343 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480031020 con 0x55ba36162b40 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.357+0000 7f24a6b65b80 1 -- 172.21.15.26:0/896845343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2480003c90 con 0x55ba36162b40 2024-06-13T06:59:18.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.357+0000 7f247b400700 1 -- 172.21.15.26:0/896845343 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2480003e40 con 0x55ba36162b40 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.357+0000 7f247b400700 1 -- 172.21.15.26:0/896845343 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f248003c050 con 0x55ba36162b40 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.357+0000 7f24a6b65b80 10 client.24983 fetch_fsmap learned FSMap version 26 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.357+0000 7f24a6b65b80 1 -- 172.21.15.26:0/896845343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba361745b0 con 0x55ba36162b40 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.358+0000 7f247b400700 1 -- 172.21.15.26:0/896845343 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2480003c90 con 0x55ba36162b40 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.358+0000 7f247b400700 1 -- 172.21.15.26:0/896845343 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f248001a050 con 0x55ba36162b40 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.358+0000 7f24a6b65b80 10 client.24983 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.358+0000 7f24a6b65b80 10 client.24983 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.358+0000 7f24a6b65b80 20 client.24983 populate_metadata read hostname 'smithi026' 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.358+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ba36176920 0x55ba36177d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.358+0000 7f24a6b65b80 4 client.24983 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.358+0000 7f24a6b65b80 1 -- 172.21.15.26:0/896845343 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba360bbfb0 con 0x55ba36176920 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.359+0000 7f2499800700 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ba36176920 0x55ba36177d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.359+0000 7f2499800700 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ba36176920 0x55ba36177d90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66046977bf7bd26 server_cookie=e00d99beb99ea486 in_seq=0 out_seq=0 2024-06-13T06:59:18.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.359+0000 7f247b400700 10 client.24983 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T06:59:18.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f247b400700 1 -- 172.21.15.26:0/896845343 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ba360bbfb0 con 0x55ba36176920 2024-06-13T06:59:18.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f247b400700 10 client.24983 handle_command_reply: tid=0 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 client.24983 shutdown 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 -- 172.21.15.26:0/896845343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 msgr2=0x55ba36166490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36166490 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2480005920 tx=0x7f2480024930 comp rx=0 tx=0).stop 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 -- 172.21.15.26:0/896845343 shutdown_connections 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ba36176920 0x55ba36177d90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f247b400700 0 client.24983 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba3615d1f0 0x55ba36169190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/896845343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba36166490 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.360+0000 7f24a6b65b80 1 -- 172.21.15.26:0/896845343 >> 172.21.15.26:0/896845343 conn(0x55ba36089120 msgr2=0x55ba36151740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.361+0000 7f24a6b65b80 1 -- 172.21.15.26:0/896845343 shutdown_connections 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.361+0000 7f24a6b65b80 1 -- 172.21.15.26:0/896845343 wait complete. 2024-06-13T06:59:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.361+0000 7f24a6b65b80 20 client.24983 trim_cache size 0 max 16384 2024-06-13T06:59:18.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.384+0000 7f24a6b65b80 1 Processor -- start 2024-06-13T06:59:18.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.384+0000 7f24a6b65b80 1 -- start start 2024-06-13T06:59:18.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba3615d1f0 0x55ba3615ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36162b40 0x55ba361640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba361760d0 con 0x55ba36162b40 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba361767a0 con 0x55ba3615d1f0 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ba36176e70 con 0x55ba36163460 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f2498e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36162b40 0x55ba361640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f2499800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba3615d1f0 0x55ba3615ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f2498e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36162b40 0x55ba361640a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56048/0 (socket says 172.21.15.26:56048) 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f2498e00700 1 -- 172.21.15.26:0/3914858728 learned_addr learned my addr 172.21.15.26:0/3914858728 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.385+0000 7f249a200700 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2498e00700 1 -- 172.21.15.26:0/3914858728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 msgr2=0x55ba361653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2498e00700 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2498e00700 1 -- 172.21.15.26:0/3914858728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba3615d1f0 msgr2=0x55ba3615ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2498e00700 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba3615d1f0 0x55ba3615ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2498e00700 1 -- 172.21.15.26:0/3914858728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba361765c0 con 0x55ba36162b40 2024-06-13T06:59:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2499800700 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba3615d1f0 0x55ba3615ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f249a200700 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:18.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2498e00700 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36162b40 0x55ba361640a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2480011fa0 tx=0x7f2480011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb19eb2df8d9e61e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:18.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2486000700 1 -- 172.21.15.26:0/3914858728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480031070 con 0x55ba36162b40 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.386+0000 7f2486000700 1 -- 172.21.15.26:0/3914858728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2480024e50 con 0x55ba36162b40 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f2486000700 1 -- 172.21.15.26:0/3914858728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480029040 con 0x55ba36162b40 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 -- 172.21.15.26:0/3914858728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36162b40 msgr2=0x55ba361640a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36162b40 0x55ba361640a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2480011fa0 tx=0x7f2480011e20 comp rx=0 tx=0).stop 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 -- 172.21.15.26:0/3914858728 shutdown_connections 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba3615d1f0 0x55ba3615ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba36163460 0x55ba361653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/3914858728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36162b40 0x55ba361640a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 -- 172.21.15.26:0/3914858728 >> 172.21.15.26:0/3914858728 conn(0x55ba36089120 msgr2=0x55ba36163990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 -- 172.21.15.26:0/3914858728 shutdown_connections 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.387+0000 7f24a6b65b80 1 -- 172.21.15.26:0/3914858728 wait complete. 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.388+0000 7f24a6b65b80 1 -- start start 2024-06-13T06:59:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.388+0000 7f24a6b65b80 1 Processor -- start 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba3615d1f0 0x55ba36167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba361683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f24a6b65b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36163460 0x55ba36180200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba36177c10 con 0x55ba36163460 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba36177540 con 0x55ba36162b40 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f24a6b65b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ba361782e0 con 0x55ba3615d1f0 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f24a6b65b80 10 client.0 _ll_register_callbacks cb 0x55ba36044330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f2499800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba3615d1f0 0x55ba36167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f2499800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba3615d1f0 0x55ba36167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53734/0 (socket says 172.21.15.26:53734) 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f2499800700 1 -- 172.21.15.26:0/540439994 learned_addr learned my addr 172.21.15.26:0/540439994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f249a200700 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36163460 0x55ba36180200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.389+0000 7f2498e00700 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba361683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f2499800700 1 -- 172.21.15.26:0/540439994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 msgr2=0x55ba361683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f2499800700 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba361683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f2499800700 1 -- 172.21.15.26:0/540439994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36163460 msgr2=0x55ba36180200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f2499800700 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36163460 0x55ba36180200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f2499800700 1 -- 172.21.15.26:0/540439994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba361808a0 con 0x55ba3615d1f0 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f2499800700 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba3615d1f0 0x55ba36167ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2488098fe0 tx=0x7f2488022aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=376187fc6ac84098 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f247b400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f247b400700 1 -- 172.21.15.26:0/540439994 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24880b4020 con 0x55ba3615d1f0 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f24a6b65b80 1 -- 172.21.15.26:0/540439994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2480024e50 con 0x55ba3615d1f0 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.390+0000 7f247b400700 1 -- 172.21.15.26:0/540439994 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f248809a910 con 0x55ba3615d1f0 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f247b400700 1 -- 172.21.15.26:0/540439994 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f24880a51f0 con 0x55ba3615d1f0 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f24a6b65b80 10 client.25021 fetch_fsmap learned FSMap version 26 2024-06-13T06:59:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f24a6b65b80 1 -- 172.21.15.26:0/540439994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba36181b60 con 0x55ba3615d1f0 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f247b400700 1 -- 172.21.15.26:0/540439994 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2480024e50 con 0x55ba3615d1f0 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f247b400700 1 -- 172.21.15.26:0/540439994 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2488085c30 con 0x55ba3615d1f0 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f24a6b65b80 10 client.25021 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f24a6b65b80 10 client.25021 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f24a6b65b80 20 client.25021 populate_metadata read hostname 'smithi026' 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ba361ef0a0 0x55ba361eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f24a6b65b80 4 client.25021 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.391+0000 7f24a6b65b80 1 -- 172.21.15.26:0/540439994 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "status"}) -- 0x55ba360b6530 con 0x55ba361ef0a0 2024-06-13T06:59:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.392+0000 7f2498e00700 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ba361ef0a0 0x55ba361eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:18.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.392+0000 7f2498e00700 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ba361ef0a0 0x55ba361eddb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dfae3070cb3ddab server_cookie=e33ebc714a9406e5 in_seq=0 out_seq=0 2024-06-13T06:59:18.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.392+0000 7f247b400700 10 client.25021 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T06:59:18.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.392+0000 7f247b400700 1 -- 172.21.15.26:0/540439994 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ba360b6530 con 0x55ba361ef0a0 2024-06-13T06:59:18.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.392+0000 7f247b400700 10 client.25021 handle_command_reply: tid=0 2024-06-13T06:59:18.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.392+0000 7f24a6b65b80 1 client.25021 shutdown 2024-06-13T06:59:18.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 -- 172.21.15.26:0/540439994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba3615d1f0 msgr2=0x55ba36167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba3615d1f0 0x55ba36167ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2488098fe0 tx=0x7f2488022aa0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 -- 172.21.15.26:0/540439994 shutdown_connections 2024-06-13T06:59:18.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ba361ef0a0 0x55ba361eddb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba36163460 0x55ba36180200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ba3615d1f0 0x55ba36167ea0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 --2- 172.21.15.26:0/540439994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba36162b40 0x55ba361683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f247b400700 0 client.25021 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T06:59:18.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 -- 172.21.15.26:0/540439994 >> 172.21.15.26:0/540439994 conn(0x55ba36089120 msgr2=0x55ba36165d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:18.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 -- 172.21.15.26:0/540439994 shutdown_connections 2024-06-13T06:59:18.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 1 -- 172.21.15.26:0/540439994 wait complete. 2024-06-13T06:59:18.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.393+0000 7f24a6b65b80 20 client.25021 trim_cache size 0 max 16384 2024-06-13T06:59:18.400 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24517, 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 256.48605546099998, 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-06-13T06:59:18.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 99, 2024-06-13T06:59:18.402 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 99, 2024-06-13T06:59:18.402 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 289.14495416900002 2024-06-13T06:59:18.402 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T06:59:18.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 -- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f247c0a2c20 msgr2=0x7f247c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f247c0a2c20 0x7f247c0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2494109ad0 tx=0x7f2480009260 comp rx=0 tx=0).stop 2024-06-13T06:59:18.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 -- 172.21.15.26:0/239556970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104690 msgr2=0x7f249419aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:18.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104690 0x7f249419aed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2488000c00 tx=0x7f248800bab0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 -- 172.21.15.26:0/239556970 shutdown_connections 2024-06-13T06:59:18.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f247c0a2c20 0x7f247c0a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249410a690 0x7f249419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104690 0x7f249419aed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 --2- 172.21.15.26:0/239556970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2494105050 0x7f249419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:18.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 -- 172.21.15.26:0/239556970 >> 172.21.15.26:0/239556970 conn(0x7f24941001a0 msgr2=0x7f24941007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:18.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 -- 172.21.15.26:0/239556970 shutdown_connections 2024-06-13T06:59:18.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:18.394+0000 7f249c200700 1 -- 172.21.15.26:0/239556970 wait complete. 2024-06-13T06:59:19.411 INFO:tasks.cephadm:Setup complete, yielding 2024-06-13T06:59:19.411 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-13T06:59:19.421 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T06:59:19.422 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch status' 2024-06-13T06:59:19.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:19 smithi026 ceph-mon[32041]: pgmap v474: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:19.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:19 smithi135 ceph-mon[36276]: pgmap v474: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:19.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:19 smithi136 ceph-mon[37229]: pgmap v474: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:21 smithi026 ceph-mon[32041]: pgmap v475: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:21.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:21 smithi135 ceph-mon[36276]: pgmap v475: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:21.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:21 smithi136 ceph-mon[37229]: pgmap v475: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:22.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.472+0000 7f1f38e00700 1 -- 172.21.15.26:0/3589215304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 msgr2=0x7f1f34108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.472+0000 7f1f38e00700 1 --2- 172.21.15.26:0/3589215304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 0x7f1f34108f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009990 tx=0x7f1f24009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.472+0000 7f1f38e00700 1 -- 172.21.15.26:0/3589215304 shutdown_connections 2024-06-13T06:59:22.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.472+0000 7f1f38e00700 1 --2- 172.21.15.26:0/3589215304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f3410a6b0 0x7f1f34110f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.472+0000 7f1f38e00700 1 --2- 172.21.15.26:0/3589215304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 0x7f1f34108f30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.472+0000 7f1f38e00700 1 --2- 172.21.15.26:0/3589215304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f34109d10 0x7f1f3410a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.472+0000 7f1f38e00700 1 -- 172.21.15.26:0/3589215304 >> 172.21.15.26:0/3589215304 conn(0x7f1f34077ae0 msgr2=0x7f1f34079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.472+0000 7f1f38e00700 1 -- 172.21.15.26:0/3589215304 shutdown_connections 2024-06-13T06:59:22.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.473+0000 7f1f38e00700 1 -- 172.21.15.26:0/3589215304 wait complete. 2024-06-13T06:59:22.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.473+0000 7f1f38e00700 1 Processor -- start 2024-06-13T06:59:22.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f38e00700 1 -- start start 2024-06-13T06:59:22.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f38e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 0x7f1f3419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f38e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f34109d10 0x7f1f3419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f38e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f3410a6b0 0x7f1f341a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f38e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341146f0 con 0x7f1f34109d10 2024-06-13T06:59:22.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f38e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f34114570 con 0x7f1f3410a6b0 2024-06-13T06:59:22.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f38e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f34114870 con 0x7f1f34108b10 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f31400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f34109d10 0x7f1f3419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f31e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 0x7f1f3419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f31400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f34109d10 0x7f1f3419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36938/0 (socket says 172.21.15.26:36938) 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.474+0000 7f1f31400700 1 -- 172.21.15.26:0/298500799 learned_addr learned my addr 172.21.15.26:0/298500799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.475+0000 7f1f32800700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f3410a6b0 0x7f1f341a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.475+0000 7f1f31400700 1 -- 172.21.15.26:0/298500799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 msgr2=0x7f1f3419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.475+0000 7f1f31400700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 0x7f1f3419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.475+0000 7f1f31400700 1 -- 172.21.15.26:0/298500799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f3410a6b0 msgr2=0x7f1f341a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.475+0000 7f1f31400700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f3410a6b0 0x7f1f341a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.475+0000 7f1f31400700 1 -- 172.21.15.26:0/298500799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f341a6970 con 0x7f1f34109d10 2024-06-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.475+0000 7f1f31e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 0x7f1f3419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.475+0000 7f1f31400700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f34109d10 0x7f1f3419f260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000e9c0 tx=0x7f1f2000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.476+0000 7f1f2aa00700 1 -- 172.21.15.26:0/298500799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000cbd0 con 0x7f1f34109d10 2024-06-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.476+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f341a6c60 con 0x7f1f34109d10 2024-06-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.476+0000 7f1f2aa00700 1 -- 172.21.15.26:0/298500799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2000ee50 con 0x7f1f34109d10 2024-06-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.476+0000 7f1f2aa00700 1 -- 172.21.15.26:0/298500799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20005680 con 0x7f1f34109d10 2024-06-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.476+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f341a7150 con 0x7f1f34109d10 2024-06-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.477+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f34069180 con 0x7f1f34109d10 2024-06-13T06:59:22.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.480+0000 7f1f2aa00700 1 -- 172.21.15.26:0/298500799 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f20004750 con 0x7f1f34109d10 2024-06-13T06:59:22.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.481+0000 7f1f2aa00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1f180a2c20 0x7f1f180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.481+0000 7f1f2aa00700 1 -- 172.21.15.26:0/298500799 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1f200cf6e0 con 0x7f1f34109d10 2024-06-13T06:59:22.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.481+0000 7f1f31e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1f180a2c20 0x7f1f180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.482+0000 7f1f31e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1f180a2c20 0x7f1f180a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1f341a0200 tx=0x7f1f2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.486+0000 7f1f2aa00700 1 -- 172.21.15.26:0/298500799 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1f2009c1f0 con 0x7f1f34109d10 2024-06-13T06:59:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.735+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1f340628d0 con 0x7f1f180a2c20 2024-06-13T06:59:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.738+0000 7f1f2aa00700 1 -- 172.21.15.26:0/298500799 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1f340628d0 con 0x7f1f180a2c20 2024-06-13T06:59:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:Backend: cephadm 2024-06-13T06:59:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:Available: Yes 2024-06-13T06:59:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:Paused: No 2024-06-13T06:59:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1f180a2c20 msgr2=0x7f1f180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1f180a2c20 0x7f1f180a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1f341a0200 tx=0x7f1f2401a040 comp rx=0 tx=0).stop 2024-06-13T06:59:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f34109d10 msgr2=0x7f1f3419f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f34109d10 0x7f1f3419f260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000e9c0 tx=0x7f1f2000ecd0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 shutdown_connections 2024-06-13T06:59:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1f180a2c20 0x7f1f180a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f3410a6b0 0x7f1f341a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f34108b10 0x7f1f3419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 --2- 172.21.15.26:0/298500799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f34109d10 0x7f1f3419f260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 >> 172.21.15.26:0/298500799 conn(0x7f1f34077ae0 msgr2=0x7f1f3410d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.742+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 shutdown_connections 2024-06-13T06:59:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:22.743+0000 7f1f38e00700 1 -- 172.21.15.26:0/298500799 wait complete. 2024-06-13T06:59:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:23 smithi026 ceph-mon[32041]: pgmap v476: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:23 smithi026 ceph-mon[32041]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:23.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-13T06:59:23.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:23 smithi135 ceph-mon[36276]: pgmap v476: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:23.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:23 smithi135 ceph-mon[36276]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:23 smithi136 ceph-mon[37229]: pgmap v476: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:23.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:23 smithi136 ceph-mon[37229]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:25 smithi026 ceph-mon[32041]: pgmap v477: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:25 smithi135 ceph-mon[36276]: pgmap v477: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:25 smithi136 ceph-mon[37229]: pgmap v477: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 -- 172.21.15.26:0/2127919137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81036f0 msgr2=0x7f15c8103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 --2- 172.21.15.26:0/2127919137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81036f0 0x7f15c8103b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15b40099c0 tx=0x7f15b4009cd0 comp rx=0 tx=0).stop 2024-06-13T06:59:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 -- 172.21.15.26:0/2127919137 shutdown_connections 2024-06-13T06:59:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 --2- 172.21.15.26:0/2127919137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15c81024f0 0x7f15c8102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 --2- 172.21.15.26:0/2127919137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c8104090 0x7f15c810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 --2- 172.21.15.26:0/2127919137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81036f0 0x7f15c8103b50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 -- 172.21.15.26:0/2127919137 >> 172.21.15.26:0/2127919137 conn(0x7f15c80fda90 msgr2=0x7f15c80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 -- 172.21.15.26:0/2127919137 shutdown_connections 2024-06-13T06:59:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.709+0000 7f15cf000700 1 -- 172.21.15.26:0/2127919137 wait complete. 2024-06-13T06:59:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.710+0000 7f15cf000700 1 Processor -- start 2024-06-13T06:59:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15cf000700 1 -- start start 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15cf000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81024f0 0x7f15c819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15cf000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15c81036f0 0x7f15c819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15cf000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c8104090 0x7f15c81a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15cf000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8075d50 con 0x7f15c81024f0 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15cf000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8075bd0 con 0x7f15c81036f0 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15cf000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8075ed0 con 0x7f15c8104090 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15c7e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81024f0 0x7f15c819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15c7e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81024f0 0x7f15c819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36962/0 (socket says 172.21.15.26:36962) 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15c7e00700 1 -- 172.21.15.26:0/1158325927 learned_addr learned my addr 172.21.15.26:0/1158325927 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.711+0000 7f15c7400700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15c81036f0 0x7f15c819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.712+0000 7f15cd000700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c8104090 0x7f15c81a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.712+0000 7f15c7e00700 1 -- 172.21.15.26:0/1158325927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c8104090 msgr2=0x7f15c81a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.712+0000 7f15c7e00700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c8104090 0x7f15c81a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.712+0000 7f15c7e00700 1 -- 172.21.15.26:0/1158325927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15c81036f0 msgr2=0x7f15c819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.712+0000 7f15c7e00700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15c81036f0 0x7f15c819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.712+0000 7f15c7e00700 1 -- 172.21.15.26:0/1158325927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c81a2530 con 0x7f15c81024f0 2024-06-13T06:59:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.712+0000 7f15c7e00700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81024f0 0x7f15c819a8d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15b800d9b0 tx=0x7f15b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.713+0000 7f15c4c00700 1 -- 172.21.15.26:0/1158325927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8016070 con 0x7f15c81024f0 2024-06-13T06:59:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.713+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c81a2820 con 0x7f15c81024f0 2024-06-13T06:59:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.713+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c81a2db0 con 0x7f15c81024f0 2024-06-13T06:59:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.713+0000 7f15c4c00700 1 -- 172.21.15.26:0/1158325927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b800be30 con 0x7f15c81024f0 2024-06-13T06:59:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.713+0000 7f15c4c00700 1 -- 172.21.15.26:0/1158325927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b800f3c0 con 0x7f15c81024f0 2024-06-13T06:59:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.714+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a8005360 con 0x7f15c81024f0 2024-06-13T06:59:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.716+0000 7f15c4c00700 1 -- 172.21.15.26:0/1158325927 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15b8019070 con 0x7f15c81024f0 2024-06-13T06:59:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.717+0000 7f15c4c00700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15ac0a2ca0 0x7f15ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.717+0000 7f15c4c00700 1 -- 172.21.15.26:0/1158325927 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f15b80cee80 con 0x7f15c81024f0 2024-06-13T06:59:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.718+0000 7f15c7400700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15ac0a2ca0 0x7f15ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:26.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.718+0000 7f15c7400700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15ac0a2ca0 0x7f15ac0a5160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15b4000c00 tx=0x7f15b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.722+0000 7f15c4c00700 1 -- 172.21.15.26:0/1158325927 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f15b8020030 con 0x7f15c81024f0 2024-06-13T06:59:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:26.966+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f15a8000bf0 con 0x7f15ac0a2ca0 2024-06-13T06:59:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.008+0000 7f15c4c00700 1 -- 172.21.15.26:0/1158325927 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f15a8000bf0 con 0x7f15ac0a2ca0 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:mds.a smithi026 running (4m) 2m ago 4m 14.4M - 19.0.0-2946-g3d89accf b045ac545f82 80963fba2169 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:mds.b smithi135 running (4m) 2m ago 4m 19.5M - 19.0.0-2946-g3d89accf b045ac545f82 9e7a3d12d3b6 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:mds.c smithi136 running (5m) 2m ago 5m 15.9M - 19.0.0-2946-g3d89accf b045ac545f82 0a3f5b1f765a 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:mds.d smithi026 running (5m) 2m ago 5m 12.9M - 19.0.0-2946-g3d89accf b045ac545f82 7b56f2264536 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:mds.e smithi135 running (5m) 2m ago 5m 14.2M - 19.0.0-2946-g3d89accf b045ac545f82 35b31f9a25c9 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:mds.f smithi136 running (5m) 2m ago 5m 17.4M - 19.0.0-2946-g3d89accf b045ac545f82 39599f86b7a5 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:mds.g smithi026 running (4m) 2m ago 4m 24.7M - 19.0.0-2946-g3d89accf b045ac545f82 e8a84a9ef3c5 2024-06-13T06:59:27.008 INFO:teuthology.orchestra.run.smithi026.stdout:mds.h smithi135 running (5m) 2m ago 5m 15.5M - 19.0.0-2946-g3d89accf b045ac545f82 f27642e54dd8 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mds.i smithi136 running (5m) 2m ago 5m 17.6M - 19.0.0-2946-g3d89accf b045ac545f82 2de7eedbed6c 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mds.j smithi026 running (5m) 2m ago 5m 13.0M - 19.0.0-2946-g3d89accf b045ac545f82 d470254b4017 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mds.k smithi135 running (5m) 2m ago 5m 14.2M - 19.0.0-2946-g3d89accf b045ac545f82 2bd02c9f1caa 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mds.l smithi136 running (5m) 2m ago 5m 13.8M - 19.0.0-2946-g3d89accf b045ac545f82 f687d66e3dc9 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.x smithi026 *:9283,8765 running (14m) 2m ago 14m 517M - 19.0.0-2946-g3d89accf b045ac545f82 0d1fbd0ba318 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.y smithi135 *:8443,8765 running (11m) 2m ago 11m 455M - 19.0.0-2946-g3d89accf b045ac545f82 463b2b5d80ed 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.z smithi136 *:8443,8765 running (11m) 2m ago 11m 456M - 19.0.0-2946-g3d89accf b045ac545f82 e4d4b276fcb4 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mon.a smithi026 running (14m) 2m ago 14m 48.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 9dcd7ed34009 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mon.b smithi135 running (11m) 2m ago 11m 43.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 6bf89dff8b95 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:mon.c smithi136 running (12m) 2m ago 12m 42.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 6cc9ad3d952b 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 smithi026 running (11m) 2m ago 11m 52.3M 4273M 19.0.0-2946-g3d89accf b045ac545f82 78c35aa21b67 2024-06-13T06:59:27.009 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 smithi135 running (10m) 2m ago 10m 54.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 c6b241fd141c 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 smithi136 running (10m) 2m ago 10m 57.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 b700336d35d6 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 smithi026 running (9m) 2m ago 9m 52.9M 4273M 19.0.0-2946-g3d89accf b045ac545f82 d01020853db1 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 smithi135 running (9m) 2m ago 9m 51.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e8ed13ea0145 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 smithi136 running (8m) 2m ago 8m 47.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e4560e200a47 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 smithi026 running (8m) 2m ago 8m 48.9M 4273M 19.0.0-2946-g3d89accf b045ac545f82 1059948290d6 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 smithi135 running (7m) 2m ago 7m 44.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 c4f7b4f6e676 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 smithi136 running (7m) 2m ago 7m 46.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 9f0bcfbc7e18 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 smithi026 running (6m) 2m ago 6m 42.5M 4273M 19.0.0-2946-g3d89accf b045ac545f82 4f218e7c2e42 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 smithi135 running (6m) 2m ago 6m 53.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 03fbffabc1d8 2024-06-13T06:59:27.010 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 smithi136 running (5m) 2m ago 5m 49.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 90c99be6c202 2024-06-13T06:59:27.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15ac0a2ca0 msgr2=0x7f15ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15ac0a2ca0 0x7f15ac0a5160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15b4000c00 tx=0x7f15b4005fb0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81024f0 msgr2=0x7f15c819a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81024f0 0x7f15c819a8d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15b800d9b0 tx=0x7f15b800dcc0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 shutdown_connections 2024-06-13T06:59:27.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15ac0a2ca0 0x7f15ac0a5160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15c8104090 0x7f15c81a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15c81024f0 0x7f15c819a8d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 --2- 172.21.15.26:0/1158325927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15c81036f0 0x7f15c819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 >> 172.21.15.26:0/1158325927 conn(0x7f15c80fda90 msgr2=0x7f15c8106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 shutdown_connections 2024-06-13T06:59:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:27.011+0000 7f15cf000700 1 -- 172.21.15.26:0/1158325927 wait complete. 2024-06-13T06:59:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:27 smithi135 ceph-mon[36276]: pgmap v478: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:27 smithi135 ceph-mon[36276]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:27 smithi136 ceph-mon[37229]: pgmap v478: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:27.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:27 smithi136 ceph-mon[37229]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:28.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:27 smithi026 ceph-mon[32041]: pgmap v478: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:28.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:27 smithi026 ceph-mon[32041]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:28.095 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-13T06:59:28.573 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:29.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:29 smithi026 ceph-mon[32041]: pgmap v479: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:29.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:29 smithi135 ceph-mon[36276]: pgmap v479: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:29.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:29 smithi136 ceph-mon[37229]: pgmap v479: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 -- 172.21.15.26:0/2606162878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec076460 msgr2=0x7fdbec0748c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 --2- 172.21.15.26:0/2606162878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec076460 0x7fdbec0748c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009990 tx=0x7fdbdc009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 -- 172.21.15.26:0/2606162878 shutdown_connections 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 --2- 172.21.15.26:0/2606162878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbec10a310 0x7fdbec10c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 --2- 172.21.15.26:0/2606162878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec076460 0x7fdbec0748c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 --2- 172.21.15.26:0/2606162878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbec074e90 0x7fdbec075310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 -- 172.21.15.26:0/2606162878 >> 172.21.15.26:0/2606162878 conn(0x7fdbec0fd9d0 msgr2=0x7fdbec0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 -- 172.21.15.26:0/2606162878 shutdown_connections 2024-06-13T06:59:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.942+0000 7fdbebe00700 1 -- 172.21.15.26:0/2606162878 wait complete. 2024-06-13T06:59:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.943+0000 7fdbebe00700 1 Processor -- start 2024-06-13T06:59:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbebe00700 1 -- start start 2024-06-13T06:59:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbebe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec074e90 0x7fdbec19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbebe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbec076460 0x7fdbec19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbec10a310 0x7fdbec1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbebe00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec110240 con 0x7fdbec074e90 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbebe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1100c0 con 0x7fdbec10a310 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbebe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1103c0 con 0x7fdbec076460 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbeaa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec074e90 0x7fdbec19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbeaa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec074e90 0x7fdbec19a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36972/0 (socket says 172.21.15.26:36972) 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbeaa00700 1 -- 172.21.15.26:0/1927943391 learned_addr learned my addr 172.21.15.26:0/1927943391 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.944+0000 7fdbea000700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbec076460 0x7fdbec19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.945+0000 7fdbeb400700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbec10a310 0x7fdbec1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:30.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.945+0000 7fdbeaa00700 1 -- 172.21.15.26:0/1927943391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbec076460 msgr2=0x7fdbec19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.945+0000 7fdbeaa00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbec076460 0x7fdbec19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.945+0000 7fdbeaa00700 1 -- 172.21.15.26:0/1927943391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbec10a310 msgr2=0x7fdbec1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.945+0000 7fdbeaa00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbec10a310 0x7fdbec1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.945+0000 7fdbeaa00700 1 -- 172.21.15.26:0/1927943391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec1a2490 con 0x7fdbec074e90 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.945+0000 7fdbeaa00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec074e90 0x7fdbec19a880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005910 tx=0x7fdbdc00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.945+0000 7fdbe3400700 1 -- 172.21.15.26:0/1927943391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc026070 con 0x7fdbec074e90 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.946+0000 7fdbe3400700 1 -- 172.21.15.26:0/1927943391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc003d70 con 0x7fdbec074e90 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.946+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec1a2720 con 0x7fdbec074e90 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.946+0000 7fdbe3400700 1 -- 172.21.15.26:0/1927943391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc021410 con 0x7fdbec074e90 2024-06-13T06:59:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.946+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec1a2b70 con 0x7fdbec074e90 2024-06-13T06:59:30.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.947+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbec069180 con 0x7fdbec074e90 2024-06-13T06:59:30.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.949+0000 7fdbe3400700 1 -- 172.21.15.26:0/1927943391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdbdc029070 con 0x7fdbec074e90 2024-06-13T06:59:30.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.950+0000 7fdbe3400700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbd00a2c20 0x7fdbd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:30.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.950+0000 7fdbe3400700 1 -- 172.21.15.26:0/1927943391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdbdc0d03a0 con 0x7fdbec074e90 2024-06-13T06:59:30.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.950+0000 7fdbea000700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbd00a2c20 0x7fdbd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:30.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.951+0000 7fdbea000700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbd00a2c20 0x7fdbd00a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8007c60 tx=0x7fdbd80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:30.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:30.955+0000 7fdbe3400700 1 -- 172.21.15.26:0/1927943391 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdbdc05d850 con 0x7fdbec074e90 2024-06-13T06:59:31.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.188+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fdbec0628d0 con 0x7fdbd00a2c20 2024-06-13T06:59:31.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.205+0000 7fdbe3400700 1 -- 172.21.15.26:0/1927943391 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fdbec0628d0 con 0x7fdbd00a2c20 2024-06-13T06:59:31.206 INFO:teuthology.orchestra.run.smithi026.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-13T06:59:31.207 INFO:teuthology.orchestra.run.smithi026.stdout:mds.all 12/12 2m ago 5m smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-06-13T06:59:31.207 INFO:teuthology.orchestra.run.smithi026.stdout:mgr 3/3 2m ago 11m smithi026=x;smithi135=y;smithi136=z;count:3 2024-06-13T06:59:31.207 INFO:teuthology.orchestra.run.smithi026.stdout:mon 3/3 2m ago 12m smithi026:172.21.15.26=a;smithi135:172.21.15.135=b;smithi136:172.21.15.136=c;count:3 2024-06-13T06:59:31.207 INFO:teuthology.orchestra.run.smithi026.stdout:osd 12 2m ago - 2024-06-13T06:59:31.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.208+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbd00a2c20 msgr2=0x7fdbd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.208+0000 7fdbebe00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbd00a2c20 0x7fdbd00a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8007c60 tx=0x7fdbd80073d0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.208+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec074e90 msgr2=0x7fdbec19a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.208+0000 7fdbebe00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec074e90 0x7fdbec19a880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005910 tx=0x7fdbdc00bbb0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.209+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 shutdown_connections 2024-06-13T06:59:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.209+0000 7fdbebe00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbd00a2c20 0x7fdbd00a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.209+0000 7fdbebe00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbec10a310 0x7fdbec1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.209+0000 7fdbebe00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbec074e90 0x7fdbec19a880 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.209+0000 7fdbebe00700 1 --2- 172.21.15.26:0/1927943391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbec076460 0x7fdbec19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.209+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 >> 172.21.15.26:0/1927943391 conn(0x7fdbec0fd9d0 msgr2=0x7fdbec106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.209+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 shutdown_connections 2024-06-13T06:59:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:31.209+0000 7fdbebe00700 1 -- 172.21.15.26:0/1927943391 wait complete. 2024-06-13T06:59:31.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:31 smithi135 ceph-mon[36276]: pgmap v480: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:31.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:31 smithi136 ceph-mon[37229]: pgmap v480: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:31 smithi026 ceph-mon[32041]: pgmap v480: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:32.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-13T06:59:32.682 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:32 smithi135 ceph-mon[36276]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:32.836 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:32 smithi026 ceph-mon[32041]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:32.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:32 smithi136 ceph-mon[37229]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:33.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:33 smithi135 ceph-mon[36276]: pgmap v481: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:33.798 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:33 smithi026 ceph-mon[32041]: pgmap v481: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:33 smithi136 ceph-mon[37229]: pgmap v481: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 -- 172.21.15.26:0/4130825472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c48108b30 msgr2=0x7f9c48108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 --2- 172.21.15.26:0/4130825472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c48108b30 0x7f9c48108f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9c380099c0 tx=0x7f9c38009cd0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 -- 172.21.15.26:0/4130825472 shutdown_connections 2024-06-13T06:59:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 --2- 172.21.15.26:0/4130825472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c48108b30 0x7f9c48108f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 --2- 172.21.15.26:0/4130825472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c4810a6d0 0x7f9c48110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 --2- 172.21.15.26:0/4130825472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c48109d30 0x7f9c4810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 -- 172.21.15.26:0/4130825472 >> 172.21.15.26:0/4130825472 conn(0x7f9c48077b00 msgr2=0x7f9c48079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 -- 172.21.15.26:0/4130825472 shutdown_connections 2024-06-13T06:59:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.167+0000 7f9c4e400700 1 -- 172.21.15.26:0/4130825472 wait complete. 2024-06-13T06:59:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.169+0000 7f9c4e400700 1 Processor -- start 2024-06-13T06:59:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.169+0000 7f9c4e400700 1 -- start start 2024-06-13T06:59:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.169+0000 7f9c4e400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c48108b30 0x7f9c4819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.169+0000 7f9c4e400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c48109d30 0x7f9c4819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.169+0000 7f9c4e400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c4810a6d0 0x7f9c481a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c4e400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48114760 con 0x7f9c48109d30 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c4e400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c481145e0 con 0x7f9c4810a6d0 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c4e400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c481148e0 con 0x7f9c48108b30 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c46a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c48109d30 0x7f9c4819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c46a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c48109d30 0x7f9c4819f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57116/0 (socket says 172.21.15.26:57116) 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c46a00700 1 -- 172.21.15.26:0/3033906812 learned_addr learned my addr 172.21.15.26:0/3033906812 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c47400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c48108b30 0x7f9c4819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c47e00700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c4810a6d0 0x7f9c481a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c46a00700 1 -- 172.21.15.26:0/3033906812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c48108b30 msgr2=0x7f9c4819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c46a00700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c48108b30 0x7f9c4819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c46a00700 1 -- 172.21.15.26:0/3033906812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c4810a6d0 msgr2=0x7f9c481a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c46a00700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c4810a6d0 0x7f9c481a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.170+0000 7f9c46a00700 1 -- 172.21.15.26:0/3033906812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c481a69a0 con 0x7f9c48109d30 2024-06-13T06:59:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.171+0000 7f9c46a00700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c48109d30 0x7f9c4819f280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400e9f0 tx=0x7f9c3400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.171+0000 7f9c3fe00700 1 -- 172.21.15.26:0/3033906812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400cbd0 con 0x7f9c48109d30 2024-06-13T06:59:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.171+0000 7f9c3fe00700 1 -- 172.21.15.26:0/3033906812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c3400ee90 con 0x7f9c48109d30 2024-06-13T06:59:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.171+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c481a6c90 con 0x7f9c48109d30 2024-06-13T06:59:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.171+0000 7f9c3fe00700 1 -- 172.21.15.26:0/3033906812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34005680 con 0x7f9c48109d30 2024-06-13T06:59:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.171+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c481a7180 con 0x7f9c48109d30 2024-06-13T06:59:35.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.172+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c48069180 con 0x7f9c48109d30 2024-06-13T06:59:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.175+0000 7f9c3fe00700 1 -- 172.21.15.26:0/3033906812 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9c34004750 con 0x7f9c48109d30 2024-06-13T06:59:35.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.177+0000 7f9c3fe00700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c2c0a2c20 0x7f9c2c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.177+0000 7f9c3fe00700 1 -- 172.21.15.26:0/3033906812 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9c340cf810 con 0x7f9c48109d30 2024-06-13T06:59:35.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.177+0000 7f9c47400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c2c0a2c20 0x7f9c2c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.178+0000 7f9c47400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c2c0a2c20 0x7f9c2c0a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800b580 tx=0x7f9c38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:35.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.181+0000 7f9c3fe00700 1 -- 172.21.15.26:0/3033906812 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c3409c320 con 0x7f9c48109d30 2024-06-13T06:59:35.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.418+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9c480628d0 con 0x7f9c2c0a2c20 2024-06-13T06:59:35.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.422+0000 7f9c3fe00700 1 -- 172.21.15.26:0/3033906812 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9c480628d0 con 0x7f9c2c0a2c20 2024-06-13T06:59:35.422 INFO:teuthology.orchestra.run.smithi026.stdout:HOST ADDR LABELS STATUS 2024-06-13T06:59:35.422 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 172.21.15.26 2024-06-13T06:59:35.422 INFO:teuthology.orchestra.run.smithi026.stdout:smithi135 172.21.15.135 2024-06-13T06:59:35.422 INFO:teuthology.orchestra.run.smithi026.stdout:smithi136 172.21.15.136 2024-06-13T06:59:35.422 INFO:teuthology.orchestra.run.smithi026.stdout:3 hosts in cluster 2024-06-13T06:59:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.425+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c2c0a2c20 msgr2=0x7f9c2c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.425+0000 7f9c4e400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c2c0a2c20 0x7f9c2c0a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800b580 tx=0x7f9c38005fb0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.425+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c48109d30 msgr2=0x7f9c4819f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.425+0000 7f9c4e400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c48109d30 0x7f9c4819f280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400e9f0 tx=0x7f9c3400ed00 comp rx=0 tx=0).stop 2024-06-13T06:59:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.426+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 shutdown_connections 2024-06-13T06:59:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.426+0000 7f9c4e400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9c2c0a2c20 0x7f9c2c0a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.426+0000 7f9c4e400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c4810a6d0 0x7f9c481a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.426+0000 7f9c4e400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c48108b30 0x7f9c4819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.426+0000 7f9c4e400700 1 --2- 172.21.15.26:0/3033906812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c48109d30 0x7f9c4819f280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.426+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 >> 172.21.15.26:0/3033906812 conn(0x7f9c48077b00 msgr2=0x7f9c4810d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.426+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 shutdown_connections 2024-06-13T06:59:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:35.426+0000 7f9c4e400700 1 -- 172.21.15.26:0/3033906812 wait complete. 2024-06-13T06:59:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:35 smithi026 ceph-mon[32041]: pgmap v482: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:35.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:35 smithi136 ceph-mon[37229]: pgmap v482: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:36.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:35 smithi135 ceph-mon[36276]: pgmap v482: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:36.418 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-13T06:59:36.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:36 smithi026 ceph-mon[32041]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:36 smithi136 ceph-mon[37229]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:36.899 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:37.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:36 smithi135 ceph-mon[36276]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:37.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:37 smithi026 ceph-mon[32041]: pgmap v483: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:37 smithi136 ceph-mon[37229]: pgmap v483: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:37 smithi135 ceph-mon[36276]: pgmap v483: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:39.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 -- 172.21.15.26:0/2431203643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4078a50 msgr2=0x7f6dd40792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:39.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 --2- 172.21.15.26:0/2431203643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4078a50 0x7f6dd40792d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000b0a0 tx=0x7f6dd000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 -- 172.21.15.26:0/2431203643 shutdown_connections 2024-06-13T06:59:39.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 --2- 172.21.15.26:0/2431203643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4078a50 0x7f6dd40792d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 --2- 172.21.15.26:0/2431203643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dd4076e40 0x7f6dd4077260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 --2- 172.21.15.26:0/2431203643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd4078090 0x7f6dd4078510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 -- 172.21.15.26:0/2431203643 >> 172.21.15.26:0/2431203643 conn(0x7f6dd40ffbb0 msgr2=0x7f6dd4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:39.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 -- 172.21.15.26:0/2431203643 shutdown_connections 2024-06-13T06:59:39.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.529+0000 7f6ddb200700 1 -- 172.21.15.26:0/2431203643 wait complete. 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.530+0000 7f6ddb200700 1 Processor -- start 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.530+0000 7f6ddb200700 1 -- start start 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6ddb200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4076e40 0x7f6dd419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6ddb200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dd4078090 0x7f6dd419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6ddb200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd4078a50 0x7f6dd41a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6ddb200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4114680 con 0x7f6dd4076e40 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6ddb200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4114500 con 0x7f6dd4078a50 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6ddb200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4114800 con 0x7f6dd4078090 2024-06-13T06:59:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6dd9e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4076e40 0x7f6dd419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6dda800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd4078a50 0x7f6dd41a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6dd9e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4076e40 0x7f6dd419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57130/0 (socket says 172.21.15.26:57130) 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6dd9400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dd4078090 0x7f6dd419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.531+0000 7f6dd9e00700 1 -- 172.21.15.26:0/3177294178 learned_addr learned my addr 172.21.15.26:0/3177294178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dd9e00700 1 -- 172.21.15.26:0/3177294178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dd4078090 msgr2=0x7f6dd419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dd9e00700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dd4078090 0x7f6dd419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dd9e00700 1 -- 172.21.15.26:0/3177294178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd4078a50 msgr2=0x7f6dd41a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dd9e00700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd4078a50 0x7f6dd41a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dd9e00700 1 -- 172.21.15.26:0/3177294178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd41a67d0 con 0x7f6dd4076e40 2024-06-13T06:59:39.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dd9400700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dd4078090 0x7f6dd419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dd9e00700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4076e40 0x7f6dd419eb70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800e9d0 tx=0x7f6dc800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dcea00700 1 -- 172.21.15.26:0/3177294178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800cbd0 con 0x7f6dd4076e40 2024-06-13T06:59:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6dcea00700 1 -- 172.21.15.26:0/3177294178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc800ee70 con 0x7f6dd4076e40 2024-06-13T06:59:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.532+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd41a6ac0 con 0x7f6dd4076e40 2024-06-13T06:59:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.533+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd41a7000 con 0x7f6dd4076e40 2024-06-13T06:59:39.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.533+0000 7f6dcea00700 1 -- 172.21.15.26:0/3177294178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8018660 con 0x7f6dd4076e40 2024-06-13T06:59:39.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.534+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd4069180 con 0x7f6dd4076e40 2024-06-13T06:59:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.536+0000 7f6dcea00700 1 -- 172.21.15.26:0/3177294178 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6dc8010430 con 0x7f6dd4076e40 2024-06-13T06:59:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.537+0000 7f6dcea00700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6dbc0a2c20 0x7f6dbc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.538+0000 7f6dcea00700 1 -- 172.21.15.26:0/3177294178 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6dc80cf880 con 0x7f6dd4076e40 2024-06-13T06:59:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.538+0000 7f6dd9400700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6dbc0a2c20 0x7f6dbc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.539+0000 7f6dd9400700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6dbc0a2c20 0x7f6dbc0a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009680 tx=0x7f6dc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.540+0000 7f6dcea00700 1 -- 172.21.15.26:0/3177294178 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6dc809c390 con 0x7f6dd4076e40 2024-06-13T06:59:39.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:39 smithi026 ceph-mon[32041]: pgmap v484: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:39.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.784+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6dd40628d0 con 0x7f6dbc0a2c20 2024-06-13T06:59:39.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.789+0000 7f6dcea00700 1 -- 172.21.15.26:0/3177294178 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6dd40628d0 con 0x7f6dbc0a2c20 2024-06-13T06:59:39.788 INFO:teuthology.orchestra.run.smithi026.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-13T06:59:39.788 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 6m ago Has a FileSystem, LVM detected 2024-06-13T06:59:39.788 INFO:teuthology.orchestra.run.smithi026.stdout:smithi135 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401BA400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-13T06:59:39.789 INFO:teuthology.orchestra.run.smithi026.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 5m ago Has a FileSystem, LVM detected 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6dbc0a2c20 msgr2=0x7f6dbc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6dbc0a2c20 0x7f6dbc0a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009680 tx=0x7f6dc4009260 comp rx=0 tx=0).stop 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4076e40 msgr2=0x7f6dd419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4076e40 0x7f6dd419eb70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800e9d0 tx=0x7f6dc800ece0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 shutdown_connections 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6dbc0a2c20 0x7f6dbc0a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd4078a50 0x7f6dd41a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4076e40 0x7f6dd419eb70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 --2- 172.21.15.26:0/3177294178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dd4078090 0x7f6dd419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 >> 172.21.15.26:0/3177294178 conn(0x7f6dd40ffbb0 msgr2=0x7f6dd41016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 shutdown_connections 2024-06-13T06:59:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:39.792+0000 7f6ddb200700 1 -- 172.21.15.26:0/3177294178 wait complete. 2024-06-13T06:59:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:39 smithi136 ceph-mon[37229]: pgmap v484: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:39 smithi135 ceph-mon[36276]: pgmap v484: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:40.743 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-13T06:59:40.753 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T06:59:40.754 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-13T06:59:40.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:40 smithi026 ceph-mon[32041]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:40.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:40 smithi136 ceph-mon[37229]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:41.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:40 smithi135 ceph-mon[36276]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:59:41.213 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:41.712 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:41 smithi026 ceph-mon[32041]: pgmap v485: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:41 smithi136 ceph-mon[37229]: pgmap v485: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:42.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:41 smithi135 ceph-mon[36276]: pgmap v485: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:43.680 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:43 smithi026 ceph-mon[32041]: pgmap v486: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 -- 172.21.15.26:0/723959207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 msgr2=0x7f2a40103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 --2- 172.21.15.26:0/723959207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 0x7f2a40103ee0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2a340098f0 tx=0x7f2a34009c00 comp rx=0 tx=0).stop 2024-06-13T06:59:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 -- 172.21.15.26:0/723959207 shutdown_connections 2024-06-13T06:59:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 --2- 172.21.15.26:0/723959207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 0x7f2a40103ee0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 --2- 172.21.15.26:0/723959207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a40100ee0 0x7f2a4010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 --2- 172.21.15.26:0/723959207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a40100540 0x7f2a401009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 -- 172.21.15.26:0/723959207 >> 172.21.15.26:0/723959207 conn(0x7f2a400fc120 msgr2=0x7f2a400fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 -- 172.21.15.26:0/723959207 shutdown_connections 2024-06-13T06:59:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.724+0000 7f2a47c00700 1 -- 172.21.15.26:0/723959207 wait complete. 2024-06-13T06:59:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.725+0000 7f2a47c00700 1 Processor -- start 2024-06-13T06:59:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.725+0000 7f2a47c00700 1 -- start start 2024-06-13T06:59:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a47c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a40100540 0x7f2a4019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a47c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a40100ee0 0x7f2a4019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a47c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 0x7f2a401a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a47c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010e110 con 0x7f2a40100540 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a47c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010df90 con 0x7f2a40103ac0 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a47c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010e290 con 0x7f2a40100ee0 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a40100ee0 0x7f2a4019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a45c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 0x7f2a401a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a45c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 0x7f2a401a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:44506/0 (socket says 172.21.15.26:44506) 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a45c00700 1 -- 172.21.15.26:0/703652699 learned_addr learned my addr 172.21.15.26:0/703652699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a40100ee0 0x7f2a4019af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45814/0 (socket says 172.21.15.26:45814) 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.726+0000 7f2a45200700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a40100540 0x7f2a4019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a45c00700 1 -- 172.21.15.26:0/703652699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a40100ee0 msgr2=0x7f2a4019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a45c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a40100ee0 0x7f2a4019af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a45c00700 1 -- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a40100540 msgr2=0x7f2a4019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a45c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a40100540 0x7f2a4019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a45c00700 1 -- 172.21.15.26:0/703652699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a401a2620 con 0x7f2a40103ac0 2024-06-13T06:59:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a3fe00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a40100ee0 0x7f2a4019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a45c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 0x7f2a401a1f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c110 tx=0x7f2a3800c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a3d600700 1 -- 172.21.15.26:0/703652699 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380092e0 con 0x7f2a40103ac0 2024-06-13T06:59:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a401a28b0 con 0x7f2a40103ac0 2024-06-13T06:59:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.727+0000 7f2a3d600700 1 -- 172.21.15.26:0/703652699 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a38004bb0 con 0x7f2a40103ac0 2024-06-13T06:59:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.728+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a401a2df0 con 0x7f2a40103ac0 2024-06-13T06:59:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.728+0000 7f2a3d600700 1 -- 172.21.15.26:0/703652699 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38007ce0 con 0x7f2a40103ac0 2024-06-13T06:59:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.729+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a40069180 con 0x7f2a40103ac0 2024-06-13T06:59:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.730+0000 7f2a3d600700 1 -- 172.21.15.26:0/703652699 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a3801a020 con 0x7f2a40103ac0 2024-06-13T06:59:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.732+0000 7f2a3d600700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2a280a2c20 0x7f2a280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.732+0000 7f2a3d600700 1 -- 172.21.15.26:0/703652699 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2a380cef10 con 0x7f2a40103ac0 2024-06-13T06:59:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.732+0000 7f2a45200700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2a280a2c20 0x7f2a280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.733+0000 7f2a45200700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2a280a2c20 0x7f2a280a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2a340098c0 tx=0x7f2a34019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:43.737+0000 7f2a3d600700 1 -- 172.21.15.26:0/703652699 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a38010040 con 0x7f2a40103ac0 2024-06-13T06:59:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:43 smithi136 ceph-mon[37229]: pgmap v486: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:44.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.015+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2a401a3750 con 0x7f2a40103ac0 2024-06-13T06:59:44.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.016+0000 7f2a3d600700 1 -- 172.21.15.26:0/703652699 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3051 (secure 0 0 0) 0x7f2a3809ba20 con 0x7f2a40103ac0 2024-06-13T06:59:44.017 INFO:teuthology.orchestra.run.smithi026.stdout:e26 2024-06-13T06:59:44.017 INFO:teuthology.orchestra.run.smithi026.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-13T06:59:44.017 INFO:teuthology.orchestra.run.smithi026.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-13T06:59:44.017 INFO:teuthology.orchestra.run.smithi026.stdout:legacy client fscid: 1 2024-06-13T06:59:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:59:44.017 INFO:teuthology.orchestra.run.smithi026.stdout:Filesystem 'cephfs' (1) 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:fs_name cephfs 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 26 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-06-13T06:55:01.881338+0000 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2024-06-13T06:56:36.605518+0000 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:tableserver 0 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:root 0 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:session_timeout 300 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:session_autoclose 300 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:max_file_size 1099511627776 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:max_xattr_size 65536 2024-06-13T06:59:44.018 INFO:teuthology.orchestra.run.smithi026.stdout:required_client_features {} 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure 0 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure_osd_epoch 0 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:max_mds 3 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:in 0,1,2 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:up {0=24517,1=24511,2=24506} 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:failed 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:damaged 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:stopped 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:data_pools [3,4] 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:metadata_pool 2 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:inline_data disabled 2024-06-13T06:59:44.019 INFO:teuthology.orchestra.run.smithi026.stdout:balancer 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:bal_rank_mask -1 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:standby_count_wanted 0 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:qdb_cluster leader: 24517 members: 24511,24517,24506 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.g{0:24517} state up:active seq 10 addr [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c{0:24496} state up:standby-replay seq 1 addr [v2:172.21.15.136:6836/1512917649,v1:172.21.15.136:6837/1512917649] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.i{1:24511} state up:active seq 32 addr [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a{1:24494} state up:standby-replay seq 1 addr [v2:172.21.15.26:6840/3560957238,v1:172.21.15.26:6841/3560957238] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b{2:24506} state up:active seq 29 addr [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.f{2:24484} state up:standby-replay seq 1 addr [v2:172.21.15.136:6834/582335478,v1:172.21.15.136:6835/582335478] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:59:44.020 INFO:teuthology.orchestra.run.smithi026.stdout:Standby daemons: 2024-06-13T06:59:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:59:44.021 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.135:6832/2065578555,v1:172.21.15.135:6833/2065578555] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.021 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.l{-1:24448} state up:standby seq 1 addr [v2:172.21.15.136:6832/2047884215,v1:172.21.15.136:6833/2047884215] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.021 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.e{-1:24455} state up:standby seq 1 addr [v2:172.21.15.135:6834/1212410310,v1:172.21.15.135:6835/1212410310] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.021 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.j{-1:24463} state up:standby seq 1 addr [v2:172.21.15.26:6834/153334101,v1:172.21.15.26:6835/153334101] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.021 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.h{-1:24470} state up:standby seq 1 addr [v2:172.21.15.135:6836/363278001,v1:172.21.15.135:6837/363278001] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.021 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d{-1:24475} state up:standby seq 1 addr [v2:172.21.15.26:6836/69365853,v1:172.21.15.26:6837/69365853] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:59:44.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2a280a2c20 msgr2=0x7f2a280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2a280a2c20 0x7f2a280a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2a340098c0 tx=0x7f2a34019040 comp rx=0 tx=0).stop 2024-06-13T06:59:44.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 msgr2=0x7f2a401a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 0x7f2a401a1f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c110 tx=0x7f2a3800c420 comp rx=0 tx=0).stop 2024-06-13T06:59:44.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 shutdown_connections 2024-06-13T06:59:44.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2a280a2c20 0x7f2a280a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a40103ac0 0x7f2a401a1f80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a40100540 0x7f2a4019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 --2- 172.21.15.26:0/703652699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a40100ee0 0x7f2a4019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 >> 172.21.15.26:0/703652699 conn(0x7f2a400fc120 msgr2=0x7f2a40109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:44.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 shutdown_connections 2024-06-13T06:59:44.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:44.020+0000 7f2a47c00700 1 -- 172.21.15.26:0/703652699 wait complete. 2024-06-13T06:59:44.023 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-06-13T06:59:44.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:43 smithi135 ceph-mon[36276]: pgmap v486: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:44.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:44 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/703652699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-13T06:59:44.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:44 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/703652699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-13T06:59:44.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-13T06:59:45.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:44 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/703652699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-13T06:59:45.425 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:45 smithi136 ceph-mon[37229]: pgmap v487: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:45.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:45 smithi026 ceph-mon[32041]: pgmap v487: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:45 smithi135 ceph-mon[36276]: pgmap v487: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:47.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:47 smithi136 ceph-mon[37229]: pgmap v488: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.897+0000 7f9563e00700 1 -- 172.21.15.26:0/2473094007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c102f80 msgr2=0x7f955c105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.897+0000 7f9563e00700 1 --2- 172.21.15.26:0/2473094007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c102f80 0x7f955c105370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9550009990 tx=0x7f9550009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.897+0000 7f9563e00700 1 -- 172.21.15.26:0/2473094007 shutdown_connections 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.897+0000 7f9563e00700 1 --2- 172.21.15.26:0/2473094007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f955c1081e0 0x7f955c10a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.897+0000 7f9563e00700 1 --2- 172.21.15.26:0/2473094007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c102f80 0x7f955c105370 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.897+0000 7f9563e00700 1 --2- 172.21.15.26:0/2473094007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f955c1058b0 0x7f955c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.897+0000 7f9563e00700 1 -- 172.21.15.26:0/2473094007 >> 172.21.15.26:0/2473094007 conn(0x7f955c0fc990 msgr2=0x7f955c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.898+0000 7f9563e00700 1 -- 172.21.15.26:0/2473094007 shutdown_connections 2024-06-13T06:59:47.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.898+0000 7f9563e00700 1 -- 172.21.15.26:0/2473094007 wait complete. 2024-06-13T06:59:47.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:47 smithi026 ceph-mon[32041]: pgmap v488: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:47.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.899+0000 7f9563e00700 1 Processor -- start 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.899+0000 7f9563e00700 1 -- start start 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9563e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f955c102f80 0x7f955c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9563e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f955c1058b0 0x7f955c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9563e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c1081e0 0x7f955c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9563e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f955c10e100 con 0x7f955c1081e0 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9563e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f955c10df80 con 0x7f955c102f80 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9563e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f955c10e280 con 0x7f955c1058b0 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9560a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f955c1058b0 0x7f955c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9561400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f955c102f80 0x7f955c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9560a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f955c1058b0 0x7f955c19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45838/0 (socket says 172.21.15.26:45838) 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9560a00700 1 -- 172.21.15.26:0/3578099068 learned_addr learned my addr 172.21.15.26:0/3578099068 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:47.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.900+0000 7f9561e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c1081e0 0x7f955c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:47.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9561e00700 1 -- 172.21.15.26:0/3578099068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f955c1058b0 msgr2=0x7f955c19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:47.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9561e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f955c1058b0 0x7f955c19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:47.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9561e00700 1 -- 172.21.15.26:0/3578099068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f955c102f80 msgr2=0x7f955c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9561e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f955c102f80 0x7f955c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9561e00700 1 -- 172.21.15.26:0/3578099068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c1a25a0 con 0x7f955c1081e0 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9560a00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f955c1058b0 0x7f955c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9561400700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f955c102f80 0x7f955c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9561e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c1081e0 0x7f955c1a1f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f955400efc0 tx=0x7f955400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f955a000700 1 -- 172.21.15.26:0/3578099068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540092e0 con 0x7f955c1081e0 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f955a000700 1 -- 172.21.15.26:0/3578099068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9554004bb0 con 0x7f955c1081e0 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f955a000700 1 -- 172.21.15.26:0/3578099068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554004280 con 0x7f955c1081e0 2024-06-13T06:59:47.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c1a2830 con 0x7f955c1081e0 2024-06-13T06:59:47.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.901+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c1a2d70 con 0x7f955c1081e0 2024-06-13T06:59:47.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.903+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c069180 con 0x7f955c1081e0 2024-06-13T06:59:47.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.905+0000 7f955a000700 1 -- 172.21.15.26:0/3578099068 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f955401b030 con 0x7f955c1081e0 2024-06-13T06:59:47.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.906+0000 7f955a000700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f95440a2c20 0x7f95440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.906+0000 7f955a000700 1 -- 172.21.15.26:0/3578099068 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f95540cef30 con 0x7f955c1081e0 2024-06-13T06:59:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.907+0000 7f9561400700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f95440a2c20 0x7f95440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:47.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.908+0000 7f9561400700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f95440a2c20 0x7f95440a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f955000b7b0 tx=0x7f955000b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:47.911+0000 7f955a000700 1 -- 172.21.15.26:0/3578099068 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9554022020 con 0x7f955c1081e0 2024-06-13T06:59:48.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:47 smithi135 ceph-mon[36276]: pgmap v488: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:48.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.142+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f955c0630a0 con 0x7f955c1081e0 2024-06-13T06:59:48.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.143+0000 7f955a000700 1 -- 172.21.15.26:0/3578099068 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v99) ==== 56+0+5733 (secure 0 0 0) 0x7f9554010040 con 0x7f955c1081e0 2024-06-13T06:59:48.144 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 99 2024-06-13T06:59:48.144 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T06:59:48.144 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-06-13T06:45:08.671358+0000 2024-06-13T06:59:48.144 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2024-06-13T06:56:29.509131+0000 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:crush_version 27 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:full_ratio 0.95 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:backfillfull_ratio 0.9 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:nearfull_ratio 0.85 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:require_min_compat_client luminous 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:min_compat_client luminous 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:require_osd_release squid 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:stretch_mode_enabled false 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-13T06:59:48.145 INFO:teuthology.orchestra.run.smithi026.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 95 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:max_osd 12 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 up in weight 1 up_from 8 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6802/4252497348,v1:172.21.15.26:6803/4252497348] [v2:172.21.15.26:6804/4252497348,v1:172.21.15.26:6805/4252497348] exists,up 5a6b984b-52fb-4d69-9e00-ff59d760e29e 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 up in weight 1 up_from 13 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/2669544474,v1:172.21.15.135:6801/2669544474] [v2:172.21.15.135:6802/2669544474,v1:172.21.15.135:6803/2669544474] exists,up 270b9d62-0c93-48d4-b822-eb2e41ea3bb7 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/4076851589,v1:172.21.15.136:6801/4076851589] [v2:172.21.15.136:6802/4076851589,v1:172.21.15.136:6803/4076851589] exists,up 4f1bc547-0cb8-434f-9467-57da190ce910 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 up in weight 1 up_from 25 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6810/2922553278,v1:172.21.15.26:6811/2922553278] [v2:172.21.15.26:6812/2922553278,v1:172.21.15.26:6813/2922553278] exists,up 0a77d3db-9cde-4c90-86e2-9030466c4753 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/2020131902,v1:172.21.15.135:6809/2020131902] [v2:172.21.15.135:6810/2020131902,v1:172.21.15.135:6811/2020131902] exists,up 6cd55a9b-cc9a-4e67-b486-f1a420368bbc 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 up in weight 1 up_from 38 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/3459833212,v1:172.21.15.136:6809/3459833212] [v2:172.21.15.136:6810/3459833212,v1:172.21.15.136:6811/3459833212] exists,up 5ca38a30-516e-4466-bbdf-4873adf0779a 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 up in weight 1 up_from 46 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6818/2705956031,v1:172.21.15.26:6819/2705956031] [v2:172.21.15.26:6820/2705956031,v1:172.21.15.26:6821/2705956031] exists,up ad8f82b9-822e-4b98-8629-b0bd8723c4c1 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/2969819310,v1:172.21.15.135:6817/2969819310] [v2:172.21.15.135:6818/2969819310,v1:172.21.15.135:6819/2969819310] exists,up 9756cfaa-3ffb-42d6-9154-98f4d4aceb64 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 up in weight 1 up_from 59 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/2605237371,v1:172.21.15.136:6817/2605237371] [v2:172.21.15.136:6818/2605237371,v1:172.21.15.136:6819/2605237371] exists,up 9ef2c367-8eae-4f63-9ea8-5af0a4129fd9 2024-06-13T06:59:48.146 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 up in weight 1 up_from 67 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6826/1456557,v1:172.21.15.26:6827/1456557] [v2:172.21.15.26:6828/1456557,v1:172.21.15.26:6829/1456557] exists,up 47d5b8cf-ab62-44a6-b713-84d52661c442 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 up in weight 1 up_from 72 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6824/1309174884,v1:172.21.15.135:6825/1309174884] [v2:172.21.15.135:6826/1309174884,v1:172.21.15.135:6827/1309174884] exists,up aed06586-2329-4eb7-8d42-e8c06e20313f 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 up in weight 1 up_from 80 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/2902066544,v1:172.21.15.136:6825/2902066544] [v2:172.21.15.136:6826/2902066544,v1:172.21.15.136:6827/2902066544] exists,up ca783667-d631-4e49-bf3a-5baf754f95fa 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.5 [8,11] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.f [8,11] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.14 [2,4] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.1b [2,4] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.2 [3,9] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.11 [4,8] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.17 [4,6] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.1d [5,9] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.23 [4,8] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.24 [4,9] 2024-06-13T06:59:48.147 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.37 [5,7] 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 4.2 [9,1] 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 4.6 [9,11] 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 4.11 [5,8] 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 4.12 [9,10] 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3363905776 expires 2024-06-14T06:46:28.109797+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/3558618334 expires 2024-06-14T06:46:28.109797+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2900186702 expires 2024-06-14T06:46:28.109797+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/2776242437 expires 2024-06-14T06:45:51.299898+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/2776242437 expires 2024-06-14T06:45:51.299898+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/3558618334 expires 2024-06-14T06:46:28.109797+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/106059480 expires 2024-06-14T06:45:51.299898+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2539477633 expires 2024-06-14T06:45:51.299898+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3002519462 expires 2024-06-14T06:45:51.299898+0000 2024-06-13T06:59:48.148 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2082657826 expires 2024-06-14T06:46:28.109797+0000 2024-06-13T06:59:48.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f95440a2c20 msgr2=0x7f95440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:48.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f95440a2c20 0x7f95440a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f955000b7b0 tx=0x7f955000b880 comp rx=0 tx=0).stop 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c1081e0 msgr2=0x7f955c1a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c1081e0 0x7f955c1a1f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f955400efc0 tx=0x7f955400c3a0 comp rx=0 tx=0).stop 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 shutdown_connections 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f95440a2c20 0x7f95440a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f955c102f80 0x7f955c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f955c1081e0 0x7f955c1a1f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 --2- 172.21.15.26:0/3578099068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f955c1058b0 0x7f955c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 >> 172.21.15.26:0/3578099068 conn(0x7f955c0fc990 msgr2=0x7f955c102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.147+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 shutdown_connections 2024-06-13T06:59:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:48.148+0000 7f9563e00700 1 -- 172.21.15.26:0/3578099068 wait complete. 2024-06-13T06:59:48.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:48 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3578099068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-13T06:59:49.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:48 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3578099068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-13T06:59:49.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:48 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3578099068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-13T06:59:49.181 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-13T06:59:49.192 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:59:49.635 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:49.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:49 smithi136 ceph-mon[37229]: pgmap v489: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:49 smithi026 ceph-mon[32041]: pgmap v489: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:50.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:49 smithi135 ceph-mon[36276]: pgmap v489: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T06:59:51.869 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:51 smithi026 ceph-mon[32041]: pgmap v490: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:51.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:51 smithi136 ceph-mon[37229]: pgmap v490: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:52.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:51 smithi135 ceph-mon[36276]: pgmap v490: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:59:52.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 -- 172.21.15.26:0/3446509347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc102740 msgr2=0x7f59dc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:52.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 --2- 172.21.15.26:0/3446509347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc102740 0x7f59dc102b20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0099c0 tx=0x7f59cc009cd0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 -- 172.21.15.26:0/3446509347 shutdown_connections 2024-06-13T06:59:52.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 --2- 172.21.15.26:0/3446509347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc107840 0x7f59dc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 --2- 172.21.15.26:0/3446509347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc102740 0x7f59dc102b20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 --2- 172.21.15.26:0/3446509347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59dc1030f0 0x7f59dc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 -- 172.21.15.26:0/3446509347 >> 172.21.15.26:0/3446509347 conn(0x7f59dc0795f0 msgr2=0x7f59dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:52.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 -- 172.21.15.26:0/3446509347 shutdown_connections 2024-06-13T06:59:52.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.265+0000 7f59e3800700 1 -- 172.21.15.26:0/3446509347 wait complete. 2024-06-13T06:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.266+0000 7f59e3800700 1 Processor -- start 2024-06-13T06:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.266+0000 7f59e3800700 1 -- start start 2024-06-13T06:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e3800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59dc102740 0x7f59dc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e3800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc1030f0 0x7f59dc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e3800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc107840 0x7f59dc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e3800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc112780 con 0x7f59dc1030f0 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e3800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc112600 con 0x7f59dc107840 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e3800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc112900 con 0x7f59dc102740 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59d3e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc1030f0 0x7f59dc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e0e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59dc102740 0x7f59dc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e0e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59dc102740 0x7f59dc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45844/0 (socket says 172.21.15.26:45844) 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e0e00700 1 -- 172.21.15.26:0/2837497220 learned_addr learned my addr 172.21.15.26:0/2837497220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59d3e00700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc1030f0 0x7f59dc074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47754/0 (socket says 172.21.15.26:47754) 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.267+0000 7f59e1800700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc107840 0x7f59dc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59d3e00700 1 -- 172.21.15.26:0/2837497220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59dc102740 msgr2=0x7f59dc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59d3e00700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59dc102740 0x7f59dc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59d3e00700 1 -- 172.21.15.26:0/2837497220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc107840 msgr2=0x7f59dc075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59d3e00700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc107840 0x7f59dc075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59d3e00700 1 -- 172.21.15.26:0/2837497220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc1a7160 con 0x7f59dc1030f0 2024-06-13T06:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59e1800700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc107840 0x7f59dc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59d3e00700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc1030f0 0x7f59dc074cd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59c400e9f0 tx=0x7f59c400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59da000700 1 -- 172.21.15.26:0/2837497220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4004220 con 0x7f59dc1030f0 2024-06-13T06:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59da000700 1 -- 172.21.15.26:0/2837497220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59c4004d10 con 0x7f59dc1030f0 2024-06-13T06:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.268+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc1a7450 con 0x7f59dc1030f0 2024-06-13T06:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.269+0000 7f59da000700 1 -- 172.21.15.26:0/2837497220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4005020 con 0x7f59dc1030f0 2024-06-13T06:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.269+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc1a7940 con 0x7f59dc1030f0 2024-06-13T06:59:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.270+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc10aa80 con 0x7f59dc1030f0 2024-06-13T06:59:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.272+0000 7f59da000700 1 -- 172.21.15.26:0/2837497220 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f59c40053d0 con 0x7f59dc1030f0 2024-06-13T06:59:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.273+0000 7f59da000700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59c00a2c20 0x7f59c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.273+0000 7f59da000700 1 -- 172.21.15.26:0/2837497220 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f59c40cf5e0 con 0x7f59dc1030f0 2024-06-13T06:59:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.274+0000 7f59e0e00700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59c00a2c20 0x7f59c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.274+0000 7f59e0e00700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59c00a2c20 0x7f59c00a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009750 tx=0x7f59cc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.278+0000 7f59da000700 1 -- 172.21.15.26:0/2837497220 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f59c409c0f0 con 0x7f59dc1030f0 2024-06-13T06:59:52.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.560+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59dc069180 con 0x7f59dc1030f0 2024-06-13T06:59:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.562+0000 7f59da000700 1 -- 172.21.15.26:0/2837497220 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7f59c409b9f0 con 0x7f59dc1030f0 2024-06-13T06:59:52.562 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:59:52.562 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T06:56:36.605518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24511,24517,24506]},"id":1}]} 2024-06-13T06:59:52.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.565+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59c00a2c20 msgr2=0x7f59c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:52.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.565+0000 7f59e3800700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59c00a2c20 0x7f59c00a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009750 tx=0x7f59cc005960 comp rx=0 tx=0).stop 2024-06-13T06:59:52.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.565+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc1030f0 msgr2=0x7f59dc074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.565+0000 7f59e3800700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc1030f0 0x7f59dc074cd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59c400e9f0 tx=0x7f59c400ed00 comp rx=0 tx=0).stop 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.566+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 shutdown_connections 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.566+0000 7f59e3800700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f59c00a2c20 0x7f59c00a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.566+0000 7f59e3800700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc107840 0x7f59dc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.566+0000 7f59e3800700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59dc102740 0x7f59dc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.566+0000 7f59e3800700 1 --2- 172.21.15.26:0/2837497220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59dc1030f0 0x7f59dc074cd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.566+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 >> 172.21.15.26:0/2837497220 conn(0x7f59dc0795f0 msgr2=0x7f59dc108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.566+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 shutdown_connections 2024-06-13T06:59:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:52.566+0000 7f59e3800700 1 -- 172.21.15.26:0/2837497220 wait complete. 2024-06-13T06:59:52.567 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-06-13T06:59:52.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:52 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2837497220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:52.998 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:52 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2837497220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:53.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:52 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2837497220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:53.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:59:53.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:53 smithi136 ceph-mon[37229]: pgmap v491: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:54.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:53 smithi026 ceph-mon[32041]: pgmap v491: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:54.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:53 smithi135 ceph-mon[36276]: pgmap v491: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:59:54.060 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:55.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:55 smithi136 ceph-mon[37229]: pgmap v492: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:55.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:55 smithi026 ceph-mon[32041]: pgmap v492: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:56.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:55 smithi135 ceph-mon[36276]: pgmap v492: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:59:56.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.693+0000 7fd7eae00700 1 -- 172.21.15.26:0/130224325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e410eac0 msgr2=0x7fd7e410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.693+0000 7fd7eae00700 1 --2- 172.21.15.26:0/130224325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e410eac0 0x7fd7e410eea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009990 tx=0x7fd7d4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.694+0000 7fd7eae00700 1 -- 172.21.15.26:0/130224325 shutdown_connections 2024-06-13T06:59:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.694+0000 7fd7eae00700 1 --2- 172.21.15.26:0/130224325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e4109480 0x7fd7e4115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.694+0000 7fd7eae00700 1 --2- 172.21.15.26:0/130224325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e410eac0 0x7fd7e410eea0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.694+0000 7fd7eae00700 1 --2- 172.21.15.26:0/130224325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7e4108ac0 0x7fd7e4108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.694+0000 7fd7eae00700 1 -- 172.21.15.26:0/130224325 >> 172.21.15.26:0/130224325 conn(0x7fd7e4075c60 msgr2=0x7fd7e41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.694+0000 7fd7eae00700 1 -- 172.21.15.26:0/130224325 shutdown_connections 2024-06-13T06:59:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.694+0000 7fd7eae00700 1 -- 172.21.15.26:0/130224325 wait complete. 2024-06-13T06:59:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.695+0000 7fd7eae00700 1 Processor -- start 2024-06-13T06:59:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.695+0000 7fd7eae00700 1 -- start start 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.695+0000 7fd7eae00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e4108ac0 0x7fd7e419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.695+0000 7fd7eae00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e4109480 0x7fd7e419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.695+0000 7fd7eae00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7e410eac0 0x7fd7e41a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7eae00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4118e10 con 0x7fd7e4108ac0 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7eae00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4118c90 con 0x7fd7e4109480 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7eae00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4118f90 con 0x7fd7e410eac0 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e4108ac0 0x7fd7e419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e4108ac0 0x7fd7e419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49092/0 (socket says 172.21.15.26:49092) 2024-06-13T06:59:56.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3e00700 1 -- 172.21.15.26:0/1107293657 learned_addr learned my addr 172.21.15.26:0/1107293657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3400700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e4109480 0x7fd7e419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e8e00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7e410eac0 0x7fd7e41a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3e00700 1 -- 172.21.15.26:0/1107293657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7e410eac0 msgr2=0x7fd7e41a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3e00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7e410eac0 0x7fd7e41a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3e00700 1 -- 172.21.15.26:0/1107293657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e4109480 msgr2=0x7fd7e419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3e00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e4109480 0x7fd7e419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3e00700 1 -- 172.21.15.26:0/1107293657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e41a4270 con 0x7fd7e4108ac0 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.696+0000 7fd7e3400700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e4109480 0x7fd7e419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.697+0000 7fd7e3e00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e4108ac0 0x7fd7e419f260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400bae0 tx=0x7fd7d400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.697+0000 7fd7e0c00700 1 -- 172.21.15.26:0/1107293657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4026070 con 0x7fd7e4108ac0 2024-06-13T06:59:56.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.697+0000 7fd7e0c00700 1 -- 172.21.15.26:0/1107293657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d4003d70 con 0x7fd7e4108ac0 2024-06-13T06:59:56.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.697+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e41a4500 con 0x7fd7e4108ac0 2024-06-13T06:59:56.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.697+0000 7fd7e0c00700 1 -- 172.21.15.26:0/1107293657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4021410 con 0x7fd7e4108ac0 2024-06-13T06:59:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.697+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e41abde0 con 0x7fd7e4108ac0 2024-06-13T06:59:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.698+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e4111050 con 0x7fd7e4108ac0 2024-06-13T06:59:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.701+0000 7fd7e0c00700 1 -- 172.21.15.26:0/1107293657 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd7d4029070 con 0x7fd7e4108ac0 2024-06-13T06:59:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.702+0000 7fd7e0c00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd7c80a2ca0 0x7fd7c80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.702+0000 7fd7e0c00700 1 -- 172.21.15.26:0/1107293657 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd7d40d0680 con 0x7fd7e4108ac0 2024-06-13T06:59:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.702+0000 7fd7e3400700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd7c80a2ca0 0x7fd7c80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.703+0000 7fd7e3400700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd7c80a2ca0 0x7fd7c80a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4077b80 tx=0x7fd7d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:56.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.705+0000 7fd7e0c00700 1 -- 172.21.15.26:0/1107293657 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd7d4030080 con 0x7fd7e4108ac0 2024-06-13T06:59:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.982+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7e4069180 con 0x7fd7e4108ac0 2024-06-13T06:59:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.985+0000 7fd7e0c00700 1 -- 172.21.15.26:0/1107293657 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11869 (secure 0 0 0) 0x7fd7d4013070 con 0x7fd7e4108ac0 2024-06-13T06:59:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T06:59:56.985 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T06:56:36.605518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24511,24517,24506]},"id":1}]} 2024-06-13T06:59:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd7c80a2ca0 msgr2=0x7fd7c80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd7c80a2ca0 0x7fd7c80a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4077b80 tx=0x7fd7d0009260 comp rx=0 tx=0).stop 2024-06-13T06:59:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e4108ac0 msgr2=0x7fd7e419f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e4108ac0 0x7fd7e419f260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400bae0 tx=0x7fd7d400bb10 comp rx=0 tx=0).stop 2024-06-13T06:59:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 shutdown_connections 2024-06-13T06:59:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd7c80a2ca0 0x7fd7c80a5160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7e410eac0 0x7fd7e41a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7e4108ac0 0x7fd7e419f260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 --2- 172.21.15.26:0/1107293657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e4109480 0x7fd7e419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.989+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 >> 172.21.15.26:0/1107293657 conn(0x7fd7e4075c60 msgr2=0x7fd7e4076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.990+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 shutdown_connections 2024-06-13T06:59:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T06:59:56.990+0000 7fd7eae00700 1 -- 172.21.15.26:0/1107293657 wait complete. 2024-06-13T06:59:56.991 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-06-13T06:59:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:57 smithi136 ceph-mon[37229]: pgmap v493: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:57 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1107293657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:57.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:57 smithi026 ceph-mon[32041]: pgmap v493: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:57.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:57 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1107293657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:58.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:59:58.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:57 smithi135 ceph-mon[36276]: pgmap v493: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:59:58.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:57 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1107293657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 06:59:59 smithi026 ceph-mon[32041]: pgmap v494: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:00.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 06:59:59 smithi135 ceph-mon[36276]: pgmap v494: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:00.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 06:59:59 smithi136 ceph-mon[37229]: pgmap v494: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:00.742 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:00 smithi026 ceph-mon[32041]: overall HEALTH_OK 2024-06-13T07:00:01.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:00 smithi135 ceph-mon[36276]: overall HEALTH_OK 2024-06-13T07:00:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.093+0000 7faf4a000700 1 -- 172.21.15.26:0/3382729918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf4410a6b0 msgr2=0x7faf4410aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.093+0000 7faf4a000700 1 --2- 172.21.15.26:0/3382729918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf4410a6b0 0x7faf4410aa90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faf34009990 tx=0x7faf34009ca0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.093+0000 7faf4a000700 1 -- 172.21.15.26:0/3382729918 shutdown_connections 2024-06-13T07:00:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.093+0000 7faf4a000700 1 --2- 172.21.15.26:0/3382729918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf44105070 0x7faf44111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.093+0000 7faf4a000700 1 --2- 172.21.15.26:0/3382729918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf4410a6b0 0x7faf4410aa90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.093+0000 7faf4a000700 1 --2- 172.21.15.26:0/3382729918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf441046b0 0x7faf44104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.093+0000 7faf4a000700 1 -- 172.21.15.26:0/3382729918 >> 172.21.15.26:0/3382729918 conn(0x7faf441001a0 msgr2=0x7faf441025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.093+0000 7faf4a000700 1 -- 172.21.15.26:0/3382729918 shutdown_connections 2024-06-13T07:00:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.094+0000 7faf4a000700 1 -- 172.21.15.26:0/3382729918 wait complete. 2024-06-13T07:00:01.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.095+0000 7faf4a000700 1 Processor -- start 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.095+0000 7faf4a000700 1 -- start start 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf4a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf441046b0 0x7faf4419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf4a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf44105070 0x7faf4419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf4a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf4410a6b0 0x7faf4419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf4a000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf44114a00 con 0x7faf441046b0 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf4a000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf44114880 con 0x7faf44105070 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf4a000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf44114b80 con 0x7faf4410a6b0 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf43400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf441046b0 0x7faf4419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf43400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf441046b0 0x7faf4419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49120/0 (socket says 172.21.15.26:49120) 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf43400700 1 -- 172.21.15.26:0/1120781308 learned_addr learned my addr 172.21.15.26:0/1120781308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf42a00700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf44105070 0x7faf4419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:01.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.096+0000 7faf43e00700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf4410a6b0 0x7faf4419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:01.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.097+0000 7faf43400700 1 -- 172.21.15.26:0/1120781308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf4410a6b0 msgr2=0x7faf4419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:01.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.097+0000 7faf43400700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf4410a6b0 0x7faf4419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.097+0000 7faf43400700 1 -- 172.21.15.26:0/1120781308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf44105070 msgr2=0x7faf4419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.097+0000 7faf43400700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf44105070 0x7faf4419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.097+0000 7faf43400700 1 -- 172.21.15.26:0/1120781308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4419fe60 con 0x7faf441046b0 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.097+0000 7faf43400700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf441046b0 0x7faf4419ae50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faf3400bae0 tx=0x7faf3400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.097+0000 7faf3be00700 1 -- 172.21.15.26:0/1120781308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34026070 con 0x7faf441046b0 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.098+0000 7faf3be00700 1 -- 172.21.15.26:0/1120781308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf34003d70 con 0x7faf441046b0 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.098+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf441a00f0 con 0x7faf441046b0 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.098+0000 7faf3be00700 1 -- 172.21.15.26:0/1120781308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34021410 con 0x7faf441046b0 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.098+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf441a79d0 con 0x7faf441046b0 2024-06-13T07:00:01.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.099+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4410cc40 con 0x7faf441046b0 2024-06-13T07:00:01.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.101+0000 7faf3be00700 1 -- 172.21.15.26:0/1120781308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf34029070 con 0x7faf441046b0 2024-06-13T07:00:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.102+0000 7faf3be00700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf280a2ca0 0x7faf280a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.102+0000 7faf3be00700 1 -- 172.21.15.26:0/1120781308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faf340d03c0 con 0x7faf441046b0 2024-06-13T07:00:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.103+0000 7faf42a00700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf280a2ca0 0x7faf280a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.103+0000 7faf42a00700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf280a2ca0 0x7faf280a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf44102810 tx=0x7faf30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.107+0000 7faf3be00700 1 -- 172.21.15.26:0/1120781308 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faf34013070 con 0x7faf441046b0 2024-06-13T07:00:01.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:00 smithi136 ceph-mon[37229]: overall HEALTH_OK 2024-06-13T07:00:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.337+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf4419c1b0 con 0x7faf441046b0 2024-06-13T07:00:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.341+0000 7faf3be00700 1 -- 172.21.15.26:0/1120781308 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7faf3409cf60 con 0x7faf441046b0 2024-06-13T07:00:01.342 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:00:01.342 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T07:00:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.345+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf280a2ca0 msgr2=0x7faf280a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.345+0000 7faf4a000700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf280a2ca0 0x7faf280a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf44102810 tx=0x7faf30009260 comp rx=0 tx=0).stop 2024-06-13T07:00:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.345+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf441046b0 msgr2=0x7faf4419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.345+0000 7faf4a000700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf441046b0 0x7faf4419ae50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faf3400bae0 tx=0x7faf3400bb10 comp rx=0 tx=0).stop 2024-06-13T07:00:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.345+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 shutdown_connections 2024-06-13T07:00:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.346+0000 7faf4a000700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf280a2ca0 0x7faf280a5160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.346+0000 7faf4a000700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf4410a6b0 0x7faf4419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.346+0000 7faf4a000700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf441046b0 0x7faf4419ae50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.346+0000 7faf4a000700 1 --2- 172.21.15.26:0/1120781308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf44105070 0x7faf4419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.346+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 >> 172.21.15.26:0/1120781308 conn(0x7faf441001a0 msgr2=0x7faf441008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.346+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 shutdown_connections 2024-06-13T07:00:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:01.346+0000 7faf4a000700 1 -- 172.21.15.26:0/1120781308 wait complete. 2024-06-13T07:00:02.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:01 smithi026 ceph-mon[32041]: pgmap v495: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:02.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:01 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1120781308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:02.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:01 smithi135 ceph-mon[36276]: pgmap v495: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:02.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:01 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1120781308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:01 smithi136 ceph-mon[37229]: pgmap v495: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:01 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1120781308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:02.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-13T06:55:01.881338+0000', 'modified': '2024-06-13T06:56:36.605518+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24517, 'mds_1': 24511, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/582335478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 582335478}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 582335478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'a', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6841/3560957238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 3560957238}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 3560957238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6837/1512917649', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 1512917649}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 1512917649}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.135:6839/1905980530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 1905980530}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 1905980530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.136:6839/2531149732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 2531149732}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 2531149732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'g', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.26:6839/173910348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 173910348}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 173910348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24517, 'qdb_cluster': [24511, 24517, 24506]} 2024-06-13T07:00:02.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-13T07:00:02.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.b status 2024-06-13T07:00:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:03 smithi026 ceph-mon[32041]: pgmap v496: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:00:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:03 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:00:04.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:03 smithi135 ceph-mon[36276]: pgmap v496: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:00:04.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:03 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:00:04.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:03 smithi136 ceph-mon[37229]: pgmap v496: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:00:04.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:03 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:00:05.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.254+0000 7fb36e600700 1 -- 172.21.15.26:0/1349461993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 msgr2=0x7fb36806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.254+0000 7fb36e600700 1 --2- 172.21.15.26:0/1349461993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 0x7fb36806ab60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb358009990 tx=0x7fb358009ca0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.254+0000 7fb36e600700 1 -- 172.21.15.26:0/1349461993 shutdown_connections 2024-06-13T07:00:05.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.254+0000 7fb36e600700 1 --2- 172.21.15.26:0/1349461993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 0x7fb36806ab60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.254+0000 7fb36e600700 1 --2- 172.21.15.26:0/1349461993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb36810ede0 0x7fb3681111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.254+0000 7fb36e600700 1 --2- 172.21.15.26:0/1349461993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36806b0a0 0x7fb36806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.254+0000 7fb36e600700 1 -- 172.21.15.26:0/1349461993 >> 172.21.15.26:0/1349461993 conn(0x7fb368078340 msgr2=0x7fb368078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:05.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.255+0000 7fb36e600700 1 -- 172.21.15.26:0/1349461993 shutdown_connections 2024-06-13T07:00:05.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.255+0000 7fb36e600700 1 -- 172.21.15.26:0/1349461993 wait complete. 2024-06-13T07:00:05.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.256+0000 7fb36e600700 1 Processor -- start 2024-06-13T07:00:05.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.256+0000 7fb36e600700 1 -- start start 2024-06-13T07:00:05.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.256+0000 7fb36e600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36806b0a0 0x7fb36819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.256+0000 7fb36e600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 0x7fb36819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.256+0000 7fb36e600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb36810ede0 0x7fb36819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.256+0000 7fb36e600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb368114a10 con 0x7fb36806b0a0 2024-06-13T07:00:05.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.256+0000 7fb36e600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb368114890 con 0x7fb3681033a0 2024-06-13T07:00:05.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.256+0000 7fb36e600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb368114b90 con 0x7fb36810ede0 2024-06-13T07:00:05.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb367400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36806b0a0 0x7fb36819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 0x7fb36819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 0x7fb36819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56662/0 (socket says 172.21.15.26:56662) 2024-06-13T07:00:05.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 -- 172.21.15.26:0/3172015337 learned_addr learned my addr 172.21.15.26:0/3172015337 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb367400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36806b0a0 0x7fb36819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42908/0 (socket says 172.21.15.26:42908) 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb367e00700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb36810ede0 0x7fb36819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 -- 172.21.15.26:0/3172015337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb36810ede0 msgr2=0x7fb36819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb36810ede0 0x7fb36819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 -- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36806b0a0 msgr2=0x7fb36819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36806b0a0 0x7fb36819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 -- 172.21.15.26:0/3172015337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36819fed0 con 0x7fb3681033a0 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.257+0000 7fb366a00700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 0x7fb36819b400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb35400eb30 tx=0x7fb35400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.258+0000 7fb35fe00700 1 -- 172.21.15.26:0/3172015337 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35400cbd0 con 0x7fb3681033a0 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.258+0000 7fb35fe00700 1 -- 172.21.15.26:0/3172015337 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb354004500 con 0x7fb3681033a0 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.258+0000 7fb35fe00700 1 -- 172.21.15.26:0/3172015337 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354010430 con 0x7fb3681033a0 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.258+0000 7fb36e600700 1 -- 172.21.15.26:0/3172015337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3681a01c0 con 0x7fb3681033a0 2024-06-13T07:00:05.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.258+0000 7fb36e600700 1 -- 172.21.15.26:0/3172015337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3681a7a40 con 0x7fb3681033a0 2024-06-13T07:00:05.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.262+0000 7fb35fe00700 1 -- 172.21.15.26:0/3172015337 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb354003f00 con 0x7fb3681033a0 2024-06-13T07:00:05.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.263+0000 7fb378f85b80 1 Processor -- start 2024-06-13T07:00:05.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.263+0000 7fb378f85b80 1 -- start start 2024-06-13T07:00:05.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.263+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501dd6df0 0x565501e5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.263+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e5a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 0x565501e676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb378f85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565501d699c0 con 0x565501dd6df0 2024-06-13T07:00:05.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb378f85b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565501dacab0 con 0x565501e652b0 2024-06-13T07:00:05.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb378f85b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565501d678a0 con 0x565501e5a560 2024-06-13T07:00:05.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb367e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501dd6df0 0x565501e5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb367400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e5a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb367400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e5a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53294/0 (socket says 172.21.15.26:53294) 2024-06-13T07:00:05.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb367400700 1 -- 172.21.15.26:0/2618785913 learned_addr learned my addr 172.21.15.26:0/2618785913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb366a00700 1 --2- 172.21.15.26:0/2618785913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 0x565501e676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb35fe00700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb34c0a2ca0 0x7fb34c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb367400700 1 -- 172.21.15.26:0/2618785913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 msgr2=0x565501e676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb367400700 1 --2- 172.21.15.26:0/2618785913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 0x565501e676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb367400700 1 -- 172.21.15.26:0/2618785913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501dd6df0 msgr2=0x565501e5a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.264+0000 7fb367400700 1 --2- 172.21.15.26:0/2618785913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501dd6df0 0x565501e5a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.265+0000 7fb367400700 1 -- 172.21.15.26:0/2618785913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565501d54f40 con 0x565501e5a560 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.265+0000 7fb35fe00700 1 -- 172.21.15.26:0/3172015337 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb3540cf770 con 0x7fb3681033a0 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.265+0000 7fb367e00700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb34c0a2ca0 0x7fb34c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.265+0000 7fb367400700 1 --2- 172.21.15.26:0/2618785913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e5a9e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb3580096e0 tx=0x7fb358009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c5ad00c8adf9d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:05.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.265+0000 7fb35cc00700 1 -- 172.21.15.26:0/2618785913 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358027070 con 0x565501e5a560 2024-06-13T07:00:05.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.265+0000 7fb35cc00700 1 -- 172.21.15.26:0/2618785913 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35800bd10 con 0x565501e5a560 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.265+0000 7fb35cc00700 1 -- 172.21.15.26:0/2618785913 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358003bb0 con 0x565501e5a560 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.265+0000 7fb367e00700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb34c0a2ca0 0x7fb34c0a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x565501e57360 tx=0x7fb360009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.266+0000 7fb378f85b80 1 -- 172.21.15.26:0/2618785913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 msgr2=0x565501e5a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.266+0000 7fb378f85b80 1 --2- 172.21.15.26:0/2618785913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e5a9e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb3580096e0 tx=0x7fb358009710 comp rx=0 tx=0).stop 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.266+0000 7fb378f85b80 1 -- 172.21.15.26:0/2618785913 shutdown_connections 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.266+0000 7fb378f85b80 1 --2- 172.21.15.26:0/2618785913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 0x565501e676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.266+0000 7fb378f85b80 1 --2- 172.21.15.26:0/2618785913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501dd6df0 0x565501e5a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.266+0000 7fb378f85b80 1 --2- 172.21.15.26:0/2618785913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e5a9e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.266+0000 7fb378f85b80 1 -- 172.21.15.26:0/2618785913 >> 172.21.15.26:0/2618785913 conn(0x565501d87120 msgr2=0x565501e56460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.267+0000 7fb378f85b80 1 -- 172.21.15.26:0/2618785913 shutdown_connections 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.267+0000 7fb378f85b80 1 -- 172.21.15.26:0/2618785913 wait complete. 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.267+0000 7fb378f85b80 1 -- start start 2024-06-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.267+0000 7fb378f85b80 1 Processor -- start 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e64990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e652b0 0x565501e621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb378f85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565501d48f10 con 0x565501e5a560 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb378f85b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565501da8a70 con 0x565501dd6df0 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb378f85b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565501e67f10 con 0x565501e652b0 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb378f85b80 10 client.0 _ll_register_callbacks cb 0x565501d466d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb366a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb366a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e61c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42920/0 (socket says 172.21.15.26:42920) 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb367400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e64990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb366a00700 1 -- 172.21.15.26:0/564663652 learned_addr learned my addr 172.21.15.26:0/564663652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.268+0000 7fb367e00700 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e652b0 0x565501e621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb367400700 1 -- 172.21.15.26:0/564663652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e652b0 msgr2=0x565501e621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb367400700 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e652b0 0x565501e621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb367400700 1 -- 172.21.15.26:0/564663652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 msgr2=0x565501e61c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb367400700 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e61c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb367400700 1 -- 172.21.15.26:0/564663652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565501cfb880 con 0x565501dd6df0 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb366a00700 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb367400700 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e64990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb358000c00 tx=0x7fb358010850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff4113de1628802 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb351600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:00:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb351600700 1 -- 172.21.15.26:0/564663652 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35802f070 con 0x565501dd6df0 2024-06-13T07:00:05.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb351600700 1 -- 172.21.15.26:0/564663652 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb358010cc0 con 0x565501dd6df0 2024-06-13T07:00:05.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.269+0000 7fb378f85b80 1 -- 172.21.15.26:0/564663652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb35800bd10 con 0x565501dd6df0 2024-06-13T07:00:05.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.270+0000 7fb351600700 1 -- 172.21.15.26:0/564663652 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb358038050 con 0x565501dd6df0 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.270+0000 7fb378f85b80 10 client.25061 fetch_fsmap learned FSMap version 26 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.270+0000 7fb378f85b80 1 -- 172.21.15.26:0/564663652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565501e725e0 con 0x565501dd6df0 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.270+0000 7fb351600700 1 -- 172.21.15.26:0/564663652 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35800bd10 con 0x565501dd6df0 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.270+0000 7fb351600700 1 -- 172.21.15.26:0/564663652 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb358032020 con 0x565501dd6df0 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.271+0000 7fb378f85b80 10 client.25061 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.271+0000 7fb378f85b80 10 client.25061 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.271+0000 7fb378f85b80 20 client.25061 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.271+0000 7fb378f85b80 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x565501e73930 0x565501e75a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.271+0000 7fb378f85b80 4 client.25061 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.271+0000 7fb378f85b80 1 -- 172.21.15.26:0/564663652 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565501db9fb0 con 0x565501e73930 2024-06-13T07:00:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.271+0000 7fb366a00700 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x565501e73930 0x565501e75a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.272+0000 7fb366a00700 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x565501e73930 0x565501e75a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb8e87cf86b6343d server_cookie=7f2876ae766f649e in_seq=0 out_seq=0 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.272+0000 7fb351600700 10 client.25061 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb351600700 1 -- 172.21.15.26:0/564663652 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565501db9fb0 con 0x565501e73930 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb351600700 10 client.25061 handle_command_reply: tid=0 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 client.25061 shutdown 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 -- 172.21.15.26:0/564663652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 msgr2=0x565501e64990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e64990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb358000c00 tx=0x7fb358010850 comp rx=0 tx=0).stop 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 -- 172.21.15.26:0/564663652 shutdown_connections 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x565501e73930 0x565501e75a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e64990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e652b0 0x565501e621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 --2- 172.21.15.26:0/564663652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb351600700 0 client.25061 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T07:00:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.273+0000 7fb378f85b80 1 -- 172.21.15.26:0/564663652 >> 172.21.15.26:0/564663652 conn(0x565501d87120 msgr2=0x565501e556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.274+0000 7fb378f85b80 1 -- 172.21.15.26:0/564663652 shutdown_connections 2024-06-13T07:00:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.274+0000 7fb378f85b80 1 -- 172.21.15.26:0/564663652 wait complete. 2024-06-13T07:00:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.274+0000 7fb378f85b80 20 client.25061 trim_cache size 0 max 16384 2024-06-13T07:00:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.298+0000 7fb378f85b80 1 Processor -- start 2024-06-13T07:00:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.298+0000 7fb378f85b80 1 -- start start 2024-06-13T07:00:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.298+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.298+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.298+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e652b0 0x565501e64130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.298+0000 7fb378f85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565501e73dd0 con 0x565501e652b0 2024-06-13T07:00:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.298+0000 7fb378f85b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565501e744a0 con 0x565501dd6df0 2024-06-13T07:00:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.298+0000 7fb378f85b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565501e74b70 con 0x565501e5a560 2024-06-13T07:00:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e652b0 0x565501e64130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb366a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e652b0 0x565501e64130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42936/0 (socket says 172.21.15.26:42936) 2024-06-13T07:00:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367e00700 1 -- 172.21.15.26:0/311880977 learned_addr learned my addr 172.21.15.26:0/311880977 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367400700 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367e00700 1 -- 172.21.15.26:0/311880977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 msgr2=0x565501e666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367e00700 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367e00700 1 -- 172.21.15.26:0/311880977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 msgr2=0x565501e676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367e00700 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb367e00700 1 -- 172.21.15.26:0/311880977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565501e60200 con 0x565501e652b0 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.299+0000 7fb366a00700 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.300+0000 7fb367e00700 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e652b0 0x565501e64130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb360017760 tx=0x7fb360004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95737abf4b68b25 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.300+0000 7fb353e00700 1 -- 172.21.15.26:0/311880977 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360003c00 con 0x565501e652b0 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.300+0000 7fb353e00700 1 -- 172.21.15.26:0/311880977 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36001b030 con 0x565501e652b0 2024-06-13T07:00:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.300+0000 7fb353e00700 1 -- 172.21.15.26:0/311880977 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600237a0 con 0x565501e652b0 2024-06-13T07:00:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 -- 172.21.15.26:0/311880977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e652b0 msgr2=0x565501e64130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e652b0 0x565501e64130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb360017760 tx=0x7fb360004b40 comp rx=0 tx=0).stop 2024-06-13T07:00:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 -- 172.21.15.26:0/311880977 shutdown_connections 2024-06-13T07:00:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501dd6df0 0x565501e676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e652b0 0x565501e64130 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 --2- 172.21.15.26:0/311880977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501e5a560 0x565501e666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 -- 172.21.15.26:0/311880977 >> 172.21.15.26:0/311880977 conn(0x565501d87120 msgr2=0x565501e60580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 -- 172.21.15.26:0/311880977 shutdown_connections 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 -- 172.21.15.26:0/311880977 wait complete. 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.301+0000 7fb378f85b80 1 -- start start 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb378f85b80 1 Processor -- start 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501dd6df0 0x565501e5f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e5f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb378f85b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 0x565501e7ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb378f85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565501e75910 con 0x565501e5a560 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb378f85b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565501e75240 con 0x565501e652b0 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb378f85b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565501e75fe0 con 0x565501dd6df0 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb378f85b80 10 client.0 _ll_register_callbacks cb 0x565501d466d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.302+0000 7fb366a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e5f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb367400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501dd6df0 0x565501e5f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb366a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e5f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42950/0 (socket says 172.21.15.26:42950) 2024-06-13T07:00:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb366a00700 1 -- 172.21.15.26:0/3438929008 learned_addr learned my addr 172.21.15.26:0/3438929008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb367e00700 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 0x565501e7ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb366a00700 1 -- 172.21.15.26:0/3438929008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501dd6df0 msgr2=0x565501e5f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb366a00700 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501dd6df0 0x565501e5f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb366a00700 1 -- 172.21.15.26:0/3438929008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 msgr2=0x565501e7ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb366a00700 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 0x565501e7ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb366a00700 1 -- 172.21.15.26:0/3438929008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565501e7f6f0 con 0x565501e5a560 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb366a00700 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e5f810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb354097fe0 tx=0x7fb354097f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6ff28bb57d4589 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb351600700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb351600700 1 -- 172.21.15.26:0/3438929008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35409c280 con 0x565501e5a560 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.303+0000 7fb351600700 1 -- 172.21.15.26:0/3438929008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb35401fdd0 con 0x565501e5a560 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 1 -- 172.21.15.26:0/3438929008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36001b030 con 0x565501e5a560 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb351600700 1 -- 172.21.15.26:0/3438929008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb35409bb80 con 0x565501e5a560 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 10 client.14973 fetch_fsmap learned FSMap version 26 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 1 -- 172.21.15.26:0/3438929008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565501e643b0 con 0x565501e5a560 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb351600700 1 -- 172.21.15.26:0/3438929008 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36001b030 con 0x565501e5a560 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb351600700 1 -- 172.21.15.26:0/3438929008 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb3540d2ce0 con 0x565501e5a560 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 10 client.14973 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 10 client.14973 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T07:00:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 20 client.14973 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x565501ee4690 0x565501ee33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 4 client.14973 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.304+0000 7fb378f85b80 1 -- 172.21.15.26:0/3438929008 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "status"}) -- 0x565501db4530 con 0x565501ee4690 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.305+0000 7fb367400700 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x565501ee4690 0x565501ee33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.305+0000 7fb367400700 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x565501ee4690 0x565501ee33a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc164e118868f7cd server_cookie=3f04afb91d477c10 in_seq=0 out_seq=0 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.305+0000 7fb351600700 10 client.14973 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb351600700 1 -- 172.21.15.26:0/3438929008 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x565501db4530 con 0x565501ee4690 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb351600700 10 client.14973 handle_command_reply: tid=0 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 client.14973 shutdown 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 -- 172.21.15.26:0/3438929008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 msgr2=0x565501e5f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e5f810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb354097fe0 tx=0x7fb354097f50 comp rx=0 tx=0).stop 2024-06-13T07:00:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 -- 172.21.15.26:0/3438929008 shutdown_connections 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x565501ee4690 0x565501ee33a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565501e652b0 0x565501e7ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565501dd6df0 0x565501e5f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 --2- 172.21.15.26:0/3438929008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565501e5a560 0x565501e5f810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 -- 172.21.15.26:0/3438929008 >> 172.21.15.26:0/3438929008 conn(0x565501d87120 msgr2=0x565501e5bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb351600700 0 client.14973 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 -- 172.21.15.26:0/3438929008 shutdown_connections 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 1 -- 172.21.15.26:0/3438929008 wait complete. 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.306+0000 7fb378f85b80 20 client.14973 trim_cache size 0 max 16384 2024-06-13T07:00:05.309 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24506, 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 216.80401283500001, 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 99, 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 99, 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 327.904598018 2024-06-13T07:00:05.310 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:00:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 -- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb34c0a2ca0 msgr2=0x7fb34c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb34c0a2ca0 0x7fb34c0a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x565501e57360 tx=0x7fb360009040 comp rx=0 tx=0).stop 2024-06-13T07:00:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 -- 172.21.15.26:0/3172015337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 msgr2=0x7fb36819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 0x7fb36819b400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb35400eb30 tx=0x7fb35400ee40 comp rx=0 tx=0).stop 2024-06-13T07:00:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 -- 172.21.15.26:0/3172015337 shutdown_connections 2024-06-13T07:00:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb34c0a2ca0 0x7fb34c0a5160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb36810ede0 0x7fb36819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36806b0a0 0x7fb36819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 --2- 172.21.15.26:0/3172015337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3681033a0 0x7fb36819b400 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 -- 172.21.15.26:0/3172015337 >> 172.21.15.26:0/3172015337 conn(0x7fb368078340 msgr2=0x7fb368100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 -- 172.21.15.26:0/3172015337 shutdown_connections 2024-06-13T07:00:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:05.307+0000 7fb36e600700 1 -- 172.21.15.26:0/3172015337 wait complete. 2024-06-13T07:00:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:05 smithi136 ceph-mon[37229]: pgmap v497: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:00:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:00:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:00:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:00:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:05 smithi026 ceph-mon[32041]: pgmap v497: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:00:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:00:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:00:06.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:00:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:05 smithi135 ceph-mon[36276]: pgmap v497: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:00:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:00:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:00:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:00:06.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.i status 2024-06-13T07:00:06.809 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:07.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:07 smithi026 ceph-mon[32041]: pgmap v498: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:00:08.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:07 smithi135 ceph-mon[36276]: pgmap v498: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:00:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:07 smithi136 ceph-mon[37229]: pgmap v498: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 -- 172.21.15.26:0/646652350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 msgr2=0x7fe450115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 --2- 172.21.15.26:0/646652350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 0x7fe450115830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe44800b0a0 tx=0x7fe44800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 -- 172.21.15.26:0/646652350 shutdown_connections 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 --2- 172.21.15.26:0/646652350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe45010d8a0 0x7fe45010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 --2- 172.21.15.26:0/646652350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 0x7fe450115830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 --2- 172.21.15.26:0/646652350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4501089d0 0x7fe450108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 -- 172.21.15.26:0/646652350 >> 172.21.15.26:0/646652350 conn(0x7fe450078220 msgr2=0x7fe4501072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 -- 172.21.15.26:0/646652350 shutdown_connections 2024-06-13T07:00:09.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.384+0000 7fe455600700 1 -- 172.21.15.26:0/646652350 wait complete. 2024-06-13T07:00:09.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.386+0000 7fe455600700 1 Processor -- start 2024-06-13T07:00:09.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.386+0000 7fe455600700 1 -- start start 2024-06-13T07:00:09.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.386+0000 7fe455600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4501089d0 0x7fe45019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.386+0000 7fe455600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 0x7fe45019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.386+0000 7fe455600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe45010d8a0 0x7fe4501a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.386+0000 7fe455600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe450118d20 con 0x7fe450109390 2024-06-13T07:00:09.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.386+0000 7fe455600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe450118ba0 con 0x7fe4501089d0 2024-06-13T07:00:09.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.386+0000 7fe455600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe450118ea0 con 0x7fe45010d8a0 2024-06-13T07:00:09.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 0x7fe45019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4501089d0 0x7fe45019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4501089d0 0x7fe45019f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56732/0 (socket says 172.21.15.26:56732) 2024-06-13T07:00:09.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44fe00700 1 -- 172.21.15.26:0/3680512475 learned_addr learned my addr 172.21.15.26:0/3680512475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:09.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe454c00700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe45010d8a0 0x7fe4501a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44f400700 1 -- 172.21.15.26:0/3680512475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe45010d8a0 msgr2=0x7fe4501a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44f400700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe45010d8a0 0x7fe4501a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44f400700 1 -- 172.21.15.26:0/3680512475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4501089d0 msgr2=0x7fe45019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44f400700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4501089d0 0x7fe45019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44f400700 1 -- 172.21.15.26:0/3680512475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4501a4190 con 0x7fe450109390 2024-06-13T07:00:09.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe454c00700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe45010d8a0 0x7fe4501a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:09.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.387+0000 7fe44fe00700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4501089d0 0x7fe45019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:09.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.388+0000 7fe44f400700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 0x7fe45019f710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e9f0 tx=0x7fe43c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:09.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.388+0000 7fe44cc00700 1 -- 172.21.15.26:0/3680512475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c004220 con 0x7fe450109390 2024-06-13T07:00:09.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.388+0000 7fe44cc00700 1 -- 172.21.15.26:0/3680512475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43c004d10 con 0x7fe450109390 2024-06-13T07:00:09.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.388+0000 7fe455600700 1 -- 172.21.15.26:0/3680512475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501a4480 con 0x7fe450109390 2024-06-13T07:00:09.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.388+0000 7fe44cc00700 1 -- 172.21.15.26:0/3680512475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c005020 con 0x7fe450109390 2024-06-13T07:00:09.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.388+0000 7fe455600700 1 -- 172.21.15.26:0/3680512475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4501abd00 con 0x7fe450109390 2024-06-13T07:00:09.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.392+0000 7fe44cc00700 1 -- 172.21.15.26:0/3680512475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe43c004750 con 0x7fe450109390 2024-06-13T07:00:09.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.393+0000 7fe4615dcb80 1 Processor -- start 2024-06-13T07:00:09.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe4615dcb80 1 -- start start 2024-06-13T07:00:09.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe44cc00700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4340a2c20 0x7fe4340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x5606011023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe44cc00700 1 -- 172.21.15.26:0/3680512475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe43c0cf360 con 0x7fe450109390 2024-06-13T07:00:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 0x560601102d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 0x56060110d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56060100f9c0 con 0x56060107cdf0 2024-06-13T07:00:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560601052ab0 con 0x560601102910 2024-06-13T07:00:09.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.394+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56060100d8a0 con 0x56060110b520 2024-06-13T07:00:09.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x5606011023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 0x560601102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x5606011023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42970/0 (socket says 172.21.15.26:42970) 2024-06-13T07:00:09.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44fe00700 1 -- 172.21.15.26:0/2702649608 learned_addr learned my addr 172.21.15.26:0/2702649608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:09.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe454c00700 1 --2- 172.21.15.26:0/2702649608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 0x56060110d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe454c00700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4340a2c20 0x7fe4340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44fe00700 1 -- 172.21.15.26:0/2702649608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 msgr2=0x56060110d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2702649608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 0x56060110d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44fe00700 1 -- 172.21.15.26:0/2702649608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 msgr2=0x560601102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2702649608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 0x560601102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.395+0000 7fe44fe00700 1 -- 172.21.15.26:0/2702649608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560600ffaf40 con 0x56060107cdf0 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.396+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2702649608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x5606011023d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe440011650 tx=0x7fe440011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4092a45e3480e601 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.396+0000 7fe454c00700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4340a2c20 0x7fe4340a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe44800d010 tx=0x7fe448006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.396+0000 7fe445600700 1 -- 172.21.15.26:0/2702649608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44001e070 con 0x56060107cdf0 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.396+0000 7fe445600700 1 -- 172.21.15.26:0/2702649608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe440019b10 con 0x56060107cdf0 2024-06-13T07:00:09.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.396+0000 7fe445600700 1 -- 172.21.15.26:0/2702649608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44000ec10 con 0x56060107cdf0 2024-06-13T07:00:09.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.397+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2702649608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 msgr2=0x5606011023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.397+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2702649608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x5606011023d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe440011650 tx=0x7fe440011960 comp rx=0 tx=0).stop 2024-06-13T07:00:09.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.398+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2702649608 shutdown_connections 2024-06-13T07:00:09.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.398+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2702649608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 0x56060110d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.398+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2702649608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x5606011023d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.398+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2702649608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 0x560601102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.398+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2702649608 >> 172.21.15.26:0/2702649608 conn(0x56060102d120 msgr2=0x5606010fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:09.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.398+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2702649608 shutdown_connections 2024-06-13T07:00:09.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.398+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2702649608 wait complete. 2024-06-13T07:00:09.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 1 -- start start 2024-06-13T07:00:09.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 1 Processor -- start 2024-06-13T07:00:09.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x56060110acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560601102910 0x560601107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060110b520 0x560601108500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560600feef10 con 0x56060107cdf0 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56060104ea70 con 0x56060110b520 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56060110e190 con 0x560601102910 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.399+0000 7fe4615dcb80 10 client.0 _ll_register_callbacks cb 0x560600fec6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe454c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060110b520 0x560601108500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe44fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x56060110acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe44f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560601102910 0x560601107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe454c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060110b520 0x560601108500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56758/0 (socket says 172.21.15.26:56758) 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe454c00700 1 -- 172.21.15.26:0/2752446130 learned_addr learned my addr 172.21.15.26:0/2752446130 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe454c00700 1 -- 172.21.15.26:0/2752446130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560601102910 msgr2=0x560601107fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:09.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe454c00700 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560601102910 0x560601107fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe454c00700 1 -- 172.21.15.26:0/2752446130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 msgr2=0x56060110acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:09.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe454c00700 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x56060110acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.400+0000 7fe454c00700 1 -- 172.21.15.26:0/2752446130 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560600fa1880 con 0x56060110b520 2024-06-13T07:00:09.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.401+0000 7fe454c00700 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060110b520 0x560601108500 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe448015000 tx=0x7fe44800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1c4c17bb21c99e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:09.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.401+0000 7fe43a000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:00:09.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.401+0000 7fe43a000700 1 -- 172.21.15.26:0/2752446130 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448007df0 con 0x56060110b520 2024-06-13T07:00:09.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.401+0000 7fe43a000700 1 -- 172.21.15.26:0/2752446130 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe448004050 con 0x56060110b520 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.401+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2752446130 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe440019b10 con 0x56060110b520 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.402+0000 7fe43a000700 1 -- 172.21.15.26:0/2752446130 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe448039050 con 0x56060110b520 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.402+0000 7fe4615dcb80 10 client.25079 fetch_fsmap learned FSMap version 26 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.402+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2752446130 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5606011188e0 con 0x56060110b520 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.402+0000 7fe43a000700 1 -- 172.21.15.26:0/2752446130 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe440019b10 con 0x56060110b520 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.402+0000 7fe43a000700 1 -- 172.21.15.26:0/2752446130 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe448028030 con 0x56060110b520 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.403+0000 7fe4615dcb80 10 client.25079 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.403+0000 7fe4615dcb80 10 client.25079 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.403+0000 7fe4615dcb80 20 client.25079 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.403+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x56060111afa0 0x56060111c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.403+0000 7fe4615dcb80 4 client.25079 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:09.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.403+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2752446130 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56060105ffb0 con 0x56060111afa0 2024-06-13T07:00:09.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.403+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x56060111afa0 0x56060111c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.404+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x56060111afa0 0x56060111c410 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2efc8913cf258f47 server_cookie=58f8aeaff947f0bc in_seq=0 out_seq=0 2024-06-13T07:00:09.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.404+0000 7fe43a000700 10 client.25079 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T07:00:09.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe43a000700 1 -- 172.21.15.26:0/2752446130 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56060105ffb0 con 0x56060111afa0 2024-06-13T07:00:09.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe43a000700 10 client.25079 handle_command_reply: tid=0 2024-06-13T07:00:09.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe4615dcb80 1 client.25079 shutdown 2024-06-13T07:00:09.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2752446130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060110b520 msgr2=0x560601108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060110b520 0x560601108500 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe448015000 tx=0x7fe44800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2752446130 shutdown_connections 2024-06-13T07:00:09.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x56060111afa0 0x56060111c410 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060110b520 0x560601108500 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060107cdf0 0x56060110acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.405+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2752446130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560601102910 0x560601107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.406+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2752446130 >> 172.21.15.26:0/2752446130 conn(0x56060102d120 msgr2=0x5606010fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:09.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.406+0000 7fe43a000700 0 client.25079 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T07:00:09.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.406+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2752446130 shutdown_connections 2024-06-13T07:00:09.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.406+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2752446130 wait complete. 2024-06-13T07:00:09.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.406+0000 7fe4615dcb80 20 client.25079 trim_cache size 0 max 16384 2024-06-13T07:00:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.430+0000 7fe4615dcb80 1 Processor -- start 2024-06-13T07:00:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.430+0000 7fe4615dcb80 1 -- start start 2024-06-13T07:00:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060107cdf0 0x560601101180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 0x56060110d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060110b520 0x560601119da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56060111a750 con 0x56060110b520 2024-06-13T07:00:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56060111ae20 con 0x560601102910 2024-06-13T07:00:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56060111b4f0 con 0x56060107cdf0 2024-06-13T07:00:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe454c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060110b520 0x560601119da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe454c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060110b520 0x560601119da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42982/0 (socket says 172.21.15.26:42982) 2024-06-13T07:00:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe454c00700 1 -- 172.21.15.26:0/2030395915 learned_addr learned my addr 172.21.15.26:0/2030395915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060107cdf0 0x560601101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe44f400700 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 0x56060110d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe454c00700 1 -- 172.21.15.26:0/2030395915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060107cdf0 msgr2=0x560601101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe454c00700 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060107cdf0 0x560601101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe454c00700 1 -- 172.21.15.26:0/2030395915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 msgr2=0x56060110d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe454c00700 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 0x56060110d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.431+0000 7fe454c00700 1 -- 172.21.15.26:0/2030395915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560601106530 con 0x56060110b520 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.432+0000 7fe454c00700 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060110b520 0x560601119da0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe448009350 tx=0x7fe44800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fb09f3daab856a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.432+0000 7fe444c00700 1 -- 172.21.15.26:0/2030395915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44802b640 con 0x56060110b520 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.432+0000 7fe444c00700 1 -- 172.21.15.26:0/2030395915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448030070 con 0x56060110b520 2024-06-13T07:00:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.432+0000 7fe444c00700 1 -- 172.21.15.26:0/2030395915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448012b20 con 0x56060110b520 2024-06-13T07:00:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2030395915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060110b520 msgr2=0x560601119da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060110b520 0x560601119da0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe448009350 tx=0x7fe44800b700 comp rx=0 tx=0).stop 2024-06-13T07:00:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2030395915 shutdown_connections 2024-06-13T07:00:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56060110b520 0x560601119da0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060107cdf0 0x560601101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2030395915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560601102910 0x56060110d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2030395915 >> 172.21.15.26:0/2030395915 conn(0x56060102d120 msgr2=0x5606010fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2030395915 shutdown_connections 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.433+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2030395915 wait complete. 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 1 -- start start 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 1 Processor -- start 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060107cdf0 0x560601104650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560601102910 0x560601104b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 0x560601124880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56060111c290 con 0x560601102910 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56060111bbc0 con 0x56060107cdf0 2024-06-13T07:00:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56060111c960 con 0x56060110b520 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.434+0000 7fe4615dcb80 10 client.0 _ll_register_callbacks cb 0x560600fec6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560601102910 0x560601104b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060107cdf0 0x560601104650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560601102910 0x560601104b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42998/0 (socket says 172.21.15.26:42998) 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44f400700 1 -- 172.21.15.26:0/2727816050 learned_addr learned my addr 172.21.15.26:0/2727816050 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe454c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 0x560601124880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44fe00700 1 -- 172.21.15.26:0/2727816050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 msgr2=0x560601124880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 0x560601124880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44fe00700 1 -- 172.21.15.26:0/2727816050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560601102910 msgr2=0x560601104b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560601102910 0x560601104b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44fe00700 1 -- 172.21.15.26:0/2727816050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560601105450 con 0x56060107cdf0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.435+0000 7fe44f400700 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560601102910 0x560601104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.436+0000 7fe44fe00700 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060107cdf0 0x560601104650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe440011cb0 tx=0x7fe440009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=761cac5b507e6659 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.436+0000 7fe43a000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.436+0000 7fe43a000700 1 -- 172.21.15.26:0/2727816050 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44001e040 con 0x56060107cdf0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.436+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2727816050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe448030070 con 0x56060107cdf0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.436+0000 7fe43a000700 1 -- 172.21.15.26:0/2727816050 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe4400195d0 con 0x56060107cdf0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.436+0000 7fe43a000700 1 -- 172.21.15.26:0/2727816050 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe44003c050 con 0x56060107cdf0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.436+0000 7fe4615dcb80 10 client.25085 fetch_fsmap learned FSMap version 26 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.436+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2727816050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560601125eb0 con 0x56060107cdf0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe43a000700 1 -- 172.21.15.26:0/2727816050 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448030070 con 0x56060107cdf0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe43a000700 1 -- 172.21.15.26:0/2727816050 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe44000e7c0 con 0x56060107cdf0 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe4615dcb80 10 client.25085 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe4615dcb80 10 client.25085 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-13T07:00:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe4615dcb80 20 client.25085 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:09.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x56060118aa40 0x560601189750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:09.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe4615dcb80 4 client.25085 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T07:00:09.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2727816050 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "status"}) -- 0x56060105a530 con 0x56060118aa40 2024-06-13T07:00:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.437+0000 7fe44f400700 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x56060118aa40 0x560601189750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.438+0000 7fe44f400700 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x56060118aa40 0x560601189750 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ead393b825d6569 server_cookie=64a3db9370bba29d in_seq=0 out_seq=0 2024-06-13T07:00:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.438+0000 7fe43a000700 10 client.25085 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T07:00:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.438+0000 7fe43a000700 1 -- 172.21.15.26:0/2727816050 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56060105a530 con 0x56060118aa40 2024-06-13T07:00:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.438+0000 7fe43a000700 10 client.25085 handle_command_reply: tid=0 2024-06-13T07:00:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.438+0000 7fe4615dcb80 1 client.25085 shutdown 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.438+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2727816050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060107cdf0 msgr2=0x560601104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.438+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060107cdf0 0x560601104650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe440011cb0 tx=0x7fe440009350 comp rx=0 tx=0).stop 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2727816050 shutdown_connections 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x56060118aa40 0x560601189750 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56060107cdf0 0x560601104650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56060110b520 0x560601124880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 1 --2- 172.21.15.26:0/2727816050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560601102910 0x560601104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe43a000700 0 client.25085 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2727816050 >> 172.21.15.26:0/2727816050 conn(0x56060102d120 msgr2=0x5606010fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2727816050 shutdown_connections 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 1 -- 172.21.15.26:0/2727816050 wait complete. 2024-06-13T07:00:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.439+0000 7fe4615dcb80 20 client.25085 trim_cache size 0 max 16384 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24511, 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 221.947561471, 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-06-13T07:00:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 99, 2024-06-13T07:00:09.444 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 99, 2024-06-13T07:00:09.444 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 344.913317017 2024-06-13T07:00:09.444 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:00:09.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 -- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4340a2c20 msgr2=0x7fe4340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4340a2c20 0x7fe4340a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe44800d010 tx=0x7fe448006210 comp rx=0 tx=0).stop 2024-06-13T07:00:09.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 -- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 msgr2=0x7fe45019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:09.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 0x7fe45019f710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e9f0 tx=0x7fe43c00ed00 comp rx=0 tx=0).stop 2024-06-13T07:00:09.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 -- 172.21.15.26:0/3680512475 shutdown_connections 2024-06-13T07:00:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe4340a2c20 0x7fe4340a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4501089d0 0x7fe45019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe45010d8a0 0x7fe4501a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 --2- 172.21.15.26:0/3680512475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe450109390 0x7fe45019f710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 -- 172.21.15.26:0/3680512475 >> 172.21.15.26:0/3680512475 conn(0x7fe450078220 msgr2=0x7fe450078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 -- 172.21.15.26:0/3680512475 shutdown_connections 2024-06-13T07:00:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:09.440+0000 7fe455600700 1 -- 172.21.15.26:0/3680512475 wait complete. 2024-06-13T07:00:09.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:09 smithi026 ceph-mon[32041]: pgmap v499: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:10.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:09 smithi135 ceph-mon[36276]: pgmap v499: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:10.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:09 smithi136 ceph-mon[37229]: pgmap v499: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:10.364 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.g status 2024-06-13T07:00:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:11.984 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:11 smithi026 ceph-mon[32041]: pgmap v500: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:11 smithi135 ceph-mon[36276]: pgmap v500: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:12.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:11 smithi136 ceph-mon[37229]: pgmap v500: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 -- 172.21.15.26:0/1231106754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806ab30 msgr2=0x7f74d810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 --2- 172.21.15.26:0/1231106754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806ab30 0x7f74d810f320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f74d000b0a0 tx=0x7f74d000b3b0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 -- 172.21.15.26:0/1231106754 shutdown_connections 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 --2- 172.21.15.26:0/1231106754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806ab30 0x7f74d810f320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 --2- 172.21.15.26:0/1231106754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74d8108500 0x7f74d81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 --2- 172.21.15.26:0/1231106754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74d806a170 0x7f74d806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 -- 172.21.15.26:0/1231106754 >> 172.21.15.26:0/1231106754 conn(0x7f74d8078340 msgr2=0x7f74d8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 -- 172.21.15.26:0/1231106754 shutdown_connections 2024-06-13T07:00:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.419+0000 7f74dfc00700 1 -- 172.21.15.26:0/1231106754 wait complete. 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.420+0000 7f74dfc00700 1 Processor -- start 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.420+0000 7f74dfc00700 1 -- start start 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74dfc00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806a170 0x7f74d819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74dfc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74d806ab30 0x7f74d819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74dfc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74d8108500 0x7f74d819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74dfc00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81127d0 con 0x7f74d806a170 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74dfc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8112650 con 0x7f74d8108500 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74dfc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8112950 con 0x7f74d806ab30 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74dd200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806a170 0x7f74d819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74cfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74d806ab30 0x7f74d819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74cfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74d806ab30 0x7f74d819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54982/0 (socket says 172.21.15.26:54982) 2024-06-13T07:00:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74cfe00700 1 -- 172.21.15.26:0/2102428545 learned_addr learned my addr 172.21.15.26:0/2102428545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:13.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.421+0000 7f74ddc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74d8108500 0x7f74d819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.422+0000 7f74cfe00700 1 -- 172.21.15.26:0/2102428545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74d8108500 msgr2=0x7f74d819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.422+0000 7f74cfe00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74d8108500 0x7f74d819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.422+0000 7f74cfe00700 1 -- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806a170 msgr2=0x7f74d819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.422+0000 7f74cfe00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806a170 0x7f74d819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.422+0000 7f74cfe00700 1 -- 172.21.15.26:0/2102428545 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d819fe10 con 0x7f74d806ab30 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.422+0000 7f74dd200700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806a170 0x7f74d819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.422+0000 7f74cfe00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74d806ab30 0x7f74d819b340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f74c400d950 tx=0x7f74c400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.422+0000 7f74d6000700 1 -- 172.21.15.26:0/2102428545 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4016070 con 0x7f74d806ab30 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.423+0000 7f74dfc00700 1 -- 172.21.15.26:0/2102428545 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d81a0100 con 0x7f74d806ab30 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.423+0000 7f74d6000700 1 -- 172.21.15.26:0/2102428545 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c400fd70 con 0x7f74d806ab30 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.423+0000 7f74d6000700 1 -- 172.21.15.26:0/2102428545 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c400e610 con 0x7f74d806ab30 2024-06-13T07:00:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.423+0000 7f74dfc00700 1 -- 172.21.15.26:0/2102428545 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d81a7980 con 0x7f74d806ab30 2024-06-13T07:00:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.426+0000 7f74d6000700 1 -- 172.21.15.26:0/2102428545 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f74c4019070 con 0x7f74d806ab30 2024-06-13T07:00:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.427+0000 7f74ea4b0b80 1 Processor -- start 2024-06-13T07:00:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74ea4b0b80 1 -- start start 2024-06-13T07:00:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74d6000700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f74bc0a2ca0 0x7f74bc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74d6000700 1 -- 172.21.15.26:0/2102428545 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f74c40ceec0 con 0x7f74d806ab30 2024-06-13T07:00:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351db460 0x55a0351d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74ddc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f74bc0a2ca0 0x7f74bc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351d51f0 0x55a0351d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a0350e39c0 con 0x55a0351db460 2024-06-13T07:00:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a035126ab0 con 0x55a0351dab40 2024-06-13T07:00:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0350e18a0 con 0x55a0351d51f0 2024-06-13T07:00:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74dd200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74dd200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351daf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:40558/0 (socket says 172.21.15.26:40558) 2024-06-13T07:00:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74cfe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351db460 0x55a0351d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.428+0000 7f74dd200700 1 -- 172.21.15.26:0/1647945464 learned_addr learned my addr 172.21.15.26:0/1647945464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.429+0000 7f74ddc00700 1 --2- 172.21.15.26:0/1647945464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351d51f0 0x55a0351d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.429+0000 7f74dd200700 1 -- 172.21.15.26:0/1647945464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351d51f0 msgr2=0x55a0351d5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.429+0000 7f74dd200700 1 --2- 172.21.15.26:0/1647945464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351d51f0 0x55a0351d5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.429+0000 7f74dd200700 1 -- 172.21.15.26:0/1647945464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351db460 msgr2=0x55a0351d4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.429+0000 7f74dd200700 1 --2- 172.21.15.26:0/1647945464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351db460 0x55a0351d4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.429+0000 7f74dd200700 1 -- 172.21.15.26:0/1647945464 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0350cef40 con 0x55a0351dab40 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.429+0000 7f74ddc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f74bc0a2ca0 0x7f74bc0a5160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009e60 tx=0x7f74d0009440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.429+0000 7f74dd200700 1 --2- 172.21.15.26:0/1647945464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351daf20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74c80115c0 tx=0x7f74c80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8836a4be561b302f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.430+0000 7f74ce000700 1 -- 172.21.15.26:0/1647945464 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c801e070 con 0x55a0351dab40 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.430+0000 7f74ce000700 1 -- 172.21.15.26:0/1647945464 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8019b10 con 0x55a0351dab40 2024-06-13T07:00:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.430+0000 7f74ce000700 1 -- 172.21.15.26:0/1647945464 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c800ebe0 con 0x55a0351dab40 2024-06-13T07:00:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.430+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1647945464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 msgr2=0x55a0351daf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.430+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1647945464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351daf20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74c80115c0 tx=0x7f74c80118d0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.431+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1647945464 shutdown_connections 2024-06-13T07:00:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.431+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1647945464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351daf20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.431+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1647945464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351d51f0 0x55a0351d5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.431+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1647945464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351db460 0x55a0351d4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.431+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1647945464 >> 172.21.15.26:0/1647945464 conn(0x55a035101120 msgr2=0x55a0351cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.431+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1647945464 shutdown_connections 2024-06-13T07:00:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.431+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1647945464 wait complete. 2024-06-13T07:00:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.431+0000 7f74ea4b0b80 1 -- start start 2024-06-13T07:00:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74ea4b0b80 1 Processor -- start 2024-06-13T07:00:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a0350c2f10 con 0x55a0351d51f0 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a035122a70 con 0x55a0351dab40 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0351e1e40 con 0x55a0351db460 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74ea4b0b80 10 client.0 _ll_register_callbacks cb 0x55a0350bc330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.432+0000 7f74cfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74dd200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74cfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351de4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:40566/0 (socket says 172.21.15.26:40566) 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74ddc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74dd200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351e11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52244/0 (socket says 172.21.15.26:52244) 2024-06-13T07:00:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74dd200700 1 -- 172.21.15.26:0/1662910173 learned_addr learned my addr 172.21.15.26:0/1662910173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74ddc00700 1 -- 172.21.15.26:0/1662910173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 msgr2=0x55a0351de4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74ddc00700 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351de4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74ddc00700 1 -- 172.21.15.26:0/1662910173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 msgr2=0x55a0351e11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74ddc00700 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351e11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74ddc00700 1 -- 172.21.15.26:0/1662910173 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a035075880 con 0x55a0351db460 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74cfe00700 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.433+0000 7f74dd200700 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.434+0000 7f74ddc00700 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351de9e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009fd0 tx=0x7f74d000b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f0ed636b1fe90a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.434+0000 7f74c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.434+0000 7f74c2a00700 1 -- 172.21.15.26:0/1662910173 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d002d070 con 0x55a0351db460 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.434+0000 7f74c2a00700 1 -- 172.21.15.26:0/1662910173 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f74d0007970 con 0x55a0351db460 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.434+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1662910173 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c8019b10 con 0x55a0351db460 2024-06-13T07:00:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.434+0000 7f74c2a00700 1 -- 172.21.15.26:0/1662910173 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74d0039050 con 0x55a0351db460 2024-06-13T07:00:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.434+0000 7f74ea4b0b80 10 client.25129 fetch_fsmap learned FSMap version 26 2024-06-13T07:00:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.434+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1662910173 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0351ec5a0 con 0x55a0351db460 2024-06-13T07:00:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74c2a00700 1 -- 172.21.15.26:0/1662910173 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8019b10 con 0x55a0351db460 2024-06-13T07:00:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74c2a00700 1 -- 172.21.15.26:0/1662910173 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f74d0011070 con 0x55a0351db460 2024-06-13T07:00:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74ea4b0b80 10 client.25129 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74ea4b0b80 10 client.25129 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74ea4b0b80 20 client.25129 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0351eeb70 0x55a0351effe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74ea4b0b80 4 client.25129 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1662910173 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a035133fb0 con 0x55a0351eeb70 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.435+0000 7f74dd200700 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0351eeb70 0x55a0351effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.436+0000 7f74dd200700 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0351eeb70 0x55a0351effe0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d759bd912bf5adf2 server_cookie=97a0dddb260a5e6 in_seq=0 out_seq=0 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.436+0000 7f74c2a00700 10 client.25129 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74c2a00700 1 -- 172.21.15.26:0/1662910173 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a035133fb0 con 0x55a0351eeb70 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74c2a00700 10 client.25129 handle_command_reply: tid=0 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 client.25129 shutdown 2024-06-13T07:00:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1662910173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 msgr2=0x55a0351de9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351de9e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009fd0 tx=0x7f74d000b3b0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1662910173 shutdown_connections 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0351eeb70 0x55a0351effe0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351de9e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74c2a00700 0 client.25129 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/1662910173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351dab40 0x55a0351de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.437+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1662910173 >> 172.21.15.26:0/1662910173 conn(0x55a035101120 msgr2=0x55a0351c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.438+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1662910173 shutdown_connections 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.438+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/1662910173 wait complete. 2024-06-13T07:00:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.438+0000 7f74ea4b0b80 20 client.25129 trim_cache size 0 max 16384 2024-06-13T07:00:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.461+0000 7f74ea4b0b80 1 Processor -- start 2024-06-13T07:00:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.461+0000 7f74ea4b0b80 1 -- start start 2024-06-13T07:00:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351dab40 0x55a0351ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351db460 0x55a0351ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a0351ee320 con 0x55a0351d51f0 2024-06-13T07:00:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0351ee9f0 con 0x55a0351db460 2024-06-13T07:00:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0351ef0c0 con 0x55a0351dab40 2024-06-13T07:00:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74dd200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74dd200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351d4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52256/0 (socket says 172.21.15.26:52256) 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74dd200700 1 -- 172.21.15.26:0/3167669532 learned_addr learned my addr 172.21.15.26:0/3167669532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74cfe00700 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351dab40 0x55a0351ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.462+0000 7f74ddc00700 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351db460 0x55a0351ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cfe00700 1 -- 172.21.15.26:0/3167669532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351db460 msgr2=0x55a0351ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cfe00700 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351db460 0x55a0351ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cfe00700 1 -- 172.21.15.26:0/3167669532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 msgr2=0x55a0351d4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cfe00700 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351d4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cfe00700 1 -- 172.21.15.26:0/3167669532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0351dfa40 con 0x55a0351dab40 2024-06-13T07:00:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74dd200700 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351d4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cfe00700 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351dab40 0x55a0351ee4a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f74c40cc570 tx=0x7f74c40bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fda0635c634d687 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cd600700 1 -- 172.21.15.26:0/3167669532 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4014070 con 0x55a0351dab40 2024-06-13T07:00:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cd600700 1 -- 172.21.15.26:0/3167669532 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c40a3f40 con 0x55a0351dab40 2024-06-13T07:00:13.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.463+0000 7f74cd600700 1 -- 172.21.15.26:0/3167669532 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4020040 con 0x55a0351dab40 2024-06-13T07:00:13.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/3167669532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351dab40 msgr2=0x55a0351ee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351dab40 0x55a0351ee4a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f74c40cc570 tx=0x7f74c40bb040 comp rx=0 tx=0).stop 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/3167669532 shutdown_connections 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351db460 0x55a0351ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351d51f0 0x55a0351d4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/3167669532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351dab40 0x55a0351ee4a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/3167669532 >> 172.21.15.26:0/3167669532 conn(0x55a035101120 msgr2=0x55a0351db840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/3167669532 shutdown_connections 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.464+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/3167669532 wait complete. 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 1 -- start start 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 1 Processor -- start 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351d51f0 0x55a0351f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351dab40 0x55a0351f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a0351efe60 con 0x55a0351dab40 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0351ef790 con 0x55a0351d51f0 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0351f0530 con 0x55a0351db460 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.465+0000 7f74ea4b0b80 10 client.0 _ll_register_callbacks cb 0x55a0350bc330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74dd200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351d51f0 0x55a0351f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74dd200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351d51f0 0x55a0351f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:40570/0 (socket says 172.21.15.26:40570) 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74dd200700 1 -- 172.21.15.26:0/2720832266 learned_addr learned my addr 172.21.15.26:0/2720832266 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74cfe00700 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351dab40 0x55a0351f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74ddc00700 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74dd200700 1 -- 172.21.15.26:0/2720832266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 msgr2=0x55a0351f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74dd200700 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74dd200700 1 -- 172.21.15.26:0/2720832266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351dab40 msgr2=0x55a0351f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74dd200700 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351dab40 0x55a0351f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74dd200700 1 -- 172.21.15.26:0/2720832266 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0351f7a00 con 0x55a0351d51f0 2024-06-13T07:00:13.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.466+0000 7f74ddc00700 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.467+0000 7f74dd200700 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351d51f0 0x55a0351f31c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f74c8004690 tx=0x7f74c8005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b807aab29e93790e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.467+0000 7f74c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.467+0000 7f74c2a00700 1 -- 172.21.15.26:0/2720832266 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c801e040 con 0x55a0351d51f0 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.467+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/2720832266 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c40a3f40 con 0x55a0351d51f0 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.467+0000 7f74c2a00700 1 -- 172.21.15.26:0/2720832266 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f74c8019970 con 0x55a0351d51f0 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.467+0000 7f74c2a00700 1 -- 172.21.15.26:0/2720832266 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74c800e890 con 0x55a0351d51f0 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.467+0000 7f74ea4b0b80 10 client.25103 fetch_fsmap learned FSMap version 26 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.467+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/2720832266 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a035262f90 con 0x55a0351d51f0 2024-06-13T07:00:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74c2a00700 1 -- 172.21.15.26:0/2720832266 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c40a3f40 con 0x55a0351d51f0 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74c2a00700 1 -- 172.21.15.26:0/2720832266 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f74c80046e0 con 0x55a0351d51f0 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74ea4b0b80 10 client.25103 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74ea4b0b80 10 client.25103 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74ea4b0b80 20 client.25103 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0352679e0 0x55a0352666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74ea4b0b80 4 client.25103 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/2720832266 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "status"}) -- 0x55a03512e530 con 0x55a0352679e0 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.468+0000 7f74cfe00700 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0352679e0 0x55a0352666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.469+0000 7f74cfe00700 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0352679e0 0x55a0352666f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ed5f286e4fe569 server_cookie=3aa20cdedf6e94e7 in_seq=0 out_seq=0 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.469+0000 7f74c2a00700 10 client.25103 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.469+0000 7f74c2a00700 1 -- 172.21.15.26:0/2720832266 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a03512e530 con 0x55a0352679e0 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.469+0000 7f74c2a00700 10 client.25103 handle_command_reply: tid=0 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.469+0000 7f74ea4b0b80 1 client.25103 shutdown 2024-06-13T07:00:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.469+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/2720832266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351d51f0 msgr2=0x55a0351f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.469+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351d51f0 0x55a0351f31c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f74c8004690 tx=0x7f74c8005fa0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/2720832266 shutdown_connections 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0352679e0 0x55a0352666f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0351d51f0 0x55a0351f31c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0351db460 0x55a0351f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 1 --2- 172.21.15.26:0/2720832266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0351dab40 0x55a0351f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/2720832266 >> 172.21.15.26:0/2720832266 conn(0x55a035101120 msgr2=0x55a0351dfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74c2a00700 0 client.25103 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/2720832266 shutdown_connections 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 1 -- 172.21.15.26:0/2720832266 wait complete. 2024-06-13T07:00:13.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.470+0000 7f74ea4b0b80 20 client.25103 trim_cache size 0 max 16384 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "559474f0-2950-11ef-bc9c-c7b262605968", 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24517, 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 311.56282045299997, 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 26, 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 99, 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 99, 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 344.22172240399999 2024-06-13T07:00:13.473 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:00:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 -- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f74bc0a2ca0 msgr2=0x7f74bc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f74bc0a2ca0 0x7f74bc0a5160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009e60 tx=0x7f74d0009440 comp rx=0 tx=0).stop 2024-06-13T07:00:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 -- 172.21.15.26:0/2102428545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74d806ab30 msgr2=0x7f74d819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74d806ab30 0x7f74d819b340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f74c400d950 tx=0x7f74c400dc60 comp rx=0 tx=0).stop 2024-06-13T07:00:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 -- 172.21.15.26:0/2102428545 shutdown_connections 2024-06-13T07:00:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f74bc0a2ca0 0x7f74bc0a5160 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74d8108500 0x7f74d819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d806a170 0x7f74d819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 --2- 172.21.15.26:0/2102428545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74d806ab30 0x7f74d819b340 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 -- 172.21.15.26:0/2102428545 >> 172.21.15.26:0/2102428545 conn(0x7f74d8078340 msgr2=0x7f74d810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 -- 172.21.15.26:0/2102428545 shutdown_connections 2024-06-13T07:00:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:13.471+0000 7f74dfc00700 1 -- 172.21.15.26:0/2102428545 wait complete. 2024-06-13T07:00:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:13 smithi026 ceph-mon[32041]: pgmap v501: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:00:14.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:13 smithi135 ceph-mon[36276]: pgmap v501: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:00:14.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:13 smithi136 ceph-mon[37229]: pgmap v501: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:00:14.465 INFO:teuthology.run_tasks:Running task sequential... 2024-06-13T07:00:14.475 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-13T07:00:14.475 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-13T07:00:14.475 INFO:teuthology.task.print:Enabling mgr modules 2024-06-13T07:00:14.475 INFO:teuthology.run_tasks:Running task kclient... 2024-06-13T07:00:14.502 INFO:tasks.kclient:Mounting kernel clients... 2024-06-13T07:00:14.502 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-13T07:00:14.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:14.503 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-06-13T07:00:14.562 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-13T07:00:14.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:14.564 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T07:00:14.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:14.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-06-13T07:00:14.768 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-06-13T07:00:14.769 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-13T07:00:14.769 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-13T07:00:14.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs ls 2024-06-13T07:00:15.238 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:15.929 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:15 smithi136 ceph-mon[37229]: pgmap v502: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:00:15.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:15 smithi026 ceph-mon[32041]: pgmap v502: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:00:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:15 smithi135 ceph-mon[36276]: pgmap v502: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:00:17.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 -- 172.21.15.26:0/1970070892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa9c10a670 msgr2=0x7faa9c10aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 --2- 172.21.15.26:0/1970070892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa9c10a670 0x7faa9c10aa50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009920 tx=0x7faa8c009c30 comp rx=0 tx=0).stop 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 -- 172.21.15.26:0/1970070892 shutdown_connections 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 --2- 172.21.15.26:0/1970070892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa9c10a670 0x7faa9c10aa50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 --2- 172.21.15.26:0/1970070892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9c104f80 0x7faa9c111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 --2- 172.21.15.26:0/1970070892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c1045c0 0x7faa9c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 -- 172.21.15.26:0/1970070892 >> 172.21.15.26:0/1970070892 conn(0x7faa9c100070 msgr2=0x7faa9c102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 -- 172.21.15.26:0/1970070892 shutdown_connections 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.774+0000 7faaa0c00700 1 -- 172.21.15.26:0/1970070892 wait complete. 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.775+0000 7faaa0c00700 1 Processor -- start 2024-06-13T07:00:17.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.775+0000 7faaa0c00700 1 -- start start 2024-06-13T07:00:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faaa0c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c1045c0 0x7faa9c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faaa0c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9c104f80 0x7faa9c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faaa0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa9c10a670 0x7faa9c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faaa0c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c114930 con 0x7faa9c104f80 2024-06-13T07:00:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faaa0c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c1147b0 con 0x7faa9c10a670 2024-06-13T07:00:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faaa0c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c114ab0 con 0x7faa9c1045c0 2024-06-13T07:00:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faa92a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9c104f80 0x7faa9c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faa9b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c1045c0 0x7faa9c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faa9b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c1045c0 0x7faa9c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55030/0 (socket says 172.21.15.26:55030) 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faa92a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9c104f80 0x7faa9c19b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52276/0 (socket says 172.21.15.26:52276) 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faa9b400700 1 -- 172.21.15.26:0/3929681503 learned_addr learned my addr 172.21.15.26:0/3929681503 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.776+0000 7faa9be00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa9c10a670 0x7faa9c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa9b400700 1 -- 172.21.15.26:0/3929681503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa9c10a670 msgr2=0x7faa9c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa9b400700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa9c10a670 0x7faa9c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa9b400700 1 -- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9c104f80 msgr2=0x7faa9c19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa9b400700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9c104f80 0x7faa9c19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa9b400700 1 -- 172.21.15.26:0/3929681503 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c19fd90 con 0x7faa9c1045c0 2024-06-13T07:00:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa9b400700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c1045c0 0x7faa9c19ad80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa8c000c00 tx=0x7faa8c00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa98c00700 1 -- 172.21.15.26:0/3929681503 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c027070 con 0x7faa9c1045c0 2024-06-13T07:00:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa98c00700 1 -- 172.21.15.26:0/3929681503 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa8c005660 con 0x7faa9c1045c0 2024-06-13T07:00:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c1a0020 con 0x7faa9c1045c0 2024-06-13T07:00:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faa98c00700 1 -- 172.21.15.26:0/3929681503 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c004930 con 0x7faa9c1045c0 2024-06-13T07:00:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.777+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c1a7900 con 0x7faa9c1045c0 2024-06-13T07:00:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.779+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9c10cb00 con 0x7faa9c1045c0 2024-06-13T07:00:17.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.781+0000 7faa98c00700 1 -- 172.21.15.26:0/3929681503 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa8c02a070 con 0x7faa9c1045c0 2024-06-13T07:00:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.782+0000 7faa98c00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa800a2c20 0x7faa800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.782+0000 7faa98c00700 1 -- 172.21.15.26:0/3929681503 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faa8c0d02e0 con 0x7faa9c1045c0 2024-06-13T07:00:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.782+0000 7faa92a00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa800a2c20 0x7faa800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.783+0000 7faa92a00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa800a2c20 0x7faa800a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa9c109d50 tx=0x7faa84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:17.784+0000 7faa98c00700 1 -- 172.21.15.26:0/3929681503 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faa8c031020 con 0x7faa9c1045c0 2024-06-13T07:00:18.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:17 smithi026 ceph-mon[32041]: pgmap v503: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:00:18.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:17 smithi135 ceph-mon[36276]: pgmap v503: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:00:18.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.066+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faa9c105ce0 con 0x7faa9c1045c0 2024-06-13T07:00:18.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.067+0000 7faa98c00700 1 -- 172.21.15.26:0/3929681503 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7faa8c013030 con 0x7faa9c1045c0 2024-06-13T07:00:18.067 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa800a2c20 msgr2=0x7faa800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa800a2c20 0x7faa800a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa9c109d50 tx=0x7faa84009260 comp rx=0 tx=0).stop 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c1045c0 msgr2=0x7faa9c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c1045c0 0x7faa9c19ad80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa8c000c00 tx=0x7faa8c00fa70 comp rx=0 tx=0).stop 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 shutdown_connections 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faa800a2c20 0x7faa800a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa9c10a670 0x7faa9c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c1045c0 0x7faa9c19ad80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 --2- 172.21.15.26:0/3929681503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9c104f80 0x7faa9c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.070+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 >> 172.21.15.26:0/3929681503 conn(0x7faa9c100070 msgr2=0x7faa9c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.071+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 shutdown_connections 2024-06-13T07:00:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:18.071+0000 7faaa0c00700 1 -- 172.21.15.26:0/3929681503 wait complete. 2024-06-13T07:00:18.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:17 smithi136 ceph-mon[37229]: pgmap v503: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:00:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:18 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3929681503' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T07:00:19.013 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-13T07:00:19.013 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-13T07:00:19.013 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-06-13T07:00:19.013 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-13T07:00:19.013 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-13T07:00:19.013 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-13T07:00:19.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-13T07:00:19.013 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-13T07:00:19.014 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-13T07:00:19.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:19.014 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-06-13T07:00:19.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:18 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3929681503' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-06-13T07:00:19.076 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41714sec preferred_lft 41714sec 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-06-13T07:00:19.077 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-06-13T07:00:19.078 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-13T07:00:19.078 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T07:00:19.078 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T07:00:19.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-06-13T07:00:19.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-06-13T07:00:19.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-06-13T07:00:19.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-13T07:00:19.078 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T07:00:19.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:18 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3929681503' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T07:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T07:00:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:19.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:19.343 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-13T07:00:19.409 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-06-13T07:00:19.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:19.413 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-06-13T07:00:19.466 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-06-13T07:00:19.466 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-13T07:00:19.466 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-13T07:00:19.466 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-13T07:00:19.466 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-13T07:00:19.466 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T07:00:19.467 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T07:00:19.467 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-13T07:00:19.467 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-13T07:00:19.467 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-13T07:00:19.467 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T07:00:19.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T07:00:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:19.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:19.692 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-06-13T07:00:19.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:19.741 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-06-13T07:00:19.792 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T07:00:19.793 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T07:00:19.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-13T07:00:19.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-13T07:00:19.793 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T07:00:19.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T07:00:19.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:19 smithi026 ceph-mon[32041]: pgmap v504: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:19.989 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-13T07:00:19.989 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T07:00:19.989 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T07:00:19.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-13T07:00:19.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-13T07:00:19.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-13T07:00:19.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-13T07:00:19.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-13T07:00:19.990 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T07:00:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:19 smithi135 ceph-mon[36276]: pgmap v504: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:20.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T07:00:20.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:19 smithi136 ceph-mon[37229]: pgmap v504: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:00:20.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:20.280 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T07:00:20.281 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-06-13T07:00:20.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-06-13T07:00:20.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-06-13T07:00:20.281 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-06-13T07:00:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T07:00:20.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:20.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-13T07:00:20.489 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-13T07:00:20.532 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-13T07:00:20.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-13T07:00:20.532 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-13T07:00:20.625 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-13T07:00:20.625 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-13T07:00:20.626 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25 2024-06-13T07:00:20.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:20.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-06-13T07:00:20.755 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-06-13T07:00:20.756 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-06-13T07:00:21.468 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-06-13T07:00:21.468 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-06-13T07:00:21.468 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.135:3300,172.21.15.136:3300" 2024-06-13T07:00:21.468 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@559474f0-2950-11ef-bc9c-c7b262605968.cephfs=/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25 2024-06-13T07:00:21.468 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.26:3300/172.21.15.135:3300/172.21.15.136:3300" will pass to kernel 2024-06-13T07:00:21.469 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-13T07:00:21.469 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-06-13T07:00:21.469 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-13T07:00:21.469 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-06-13T07:00:21.469 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-13T07:00:21.473 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-13T07:00:21.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:00:21.474 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-13T07:00:21.556 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-13T07:00:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-13T07:00:21.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:21.673 INFO:teuthology.orchestra.run.smithi026.stdout:client25144 2024-06-13T07:00:21.673 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T07:00:21.673 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T07:00:21.719 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-13T07:00:21.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:21 smithi026 ceph-mon[32041]: pgmap v505: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-13T07:00:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:21.872 INFO:teuthology.orchestra.run.smithi026.stdout:client25144 2024-06-13T07:00:21.872 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-13T07:00:21.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-13T07:00:21.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:21.981 INFO:teuthology.orchestra.run.smithi026.stdout:client25144 2024-06-13T07:00:21.982 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/559474f0-2950-11ef-bc9c-c7b262605968.client25144/status') 2024-06-13T07:00:22.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:21 smithi135 ceph-mon[36276]: pgmap v505: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T07:00:22.119 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.25144 (3)172.21.15.26:0/3343695274 2024-06-13T07:00:22.119 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-06-13T07:00:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-06-13T07:00:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-06-13T07:00:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:71 bytes copied, 8.0779e-05 s, 879 kB/s 2024-06-13T07:00:22.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T07:00:22.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:21 smithi136 ceph-mon[37229]: pgmap v505: 129 pgs: 129 active+clean; 642 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T07:00:22.132 INFO:teuthology.run_tasks:Running task exec... 2024-06-13T07:00:22.144 INFO:teuthology.task.exec:Executing custom commands... 2024-06-13T07:00:22.144 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T07:00:22.144 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-13T07:00:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 -- 172.21.15.26:0/308970299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe10c1057e0 msgr2=0x7fe10c10c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 --2- 172.21.15.26:0/308970299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe10c1057e0 0x7fe10c10c070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe104009ca0 tx=0x7fe10402a8c0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 -- 172.21.15.26:0/308970299 shutdown_connections 2024-06-13T07:00:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 --2- 172.21.15.26:0/308970299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe10c104e20 0x7fe10c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 --2- 172.21.15.26:0/308970299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe10c1057e0 0x7fe10c10c070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 --2- 172.21.15.26:0/308970299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe10c103c30 0x7fe10c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 -- 172.21.15.26:0/308970299 >> 172.21.15.26:0/308970299 conn(0x7fe10c0ff1c0 msgr2=0x7fe10c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:22.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 -- 172.21.15.26:0/308970299 shutdown_connections 2024-06-13T07:00:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.356+0000 7fe113800640 1 -- 172.21.15.26:0/308970299 wait complete. 2024-06-13T07:00:22.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.357+0000 7fe113800640 1 Processor -- start 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.357+0000 7fe113800640 1 -- start start 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe113800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe10c103c30 0x7fe10c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe113800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe10c104e20 0x7fe10c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe113800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe10c1057e0 0x7fe10c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe113800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10f900 con 0x7fe10c1057e0 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe113800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10f780 con 0x7fe10c103c30 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe113800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10fa80 con 0x7fe10c104e20 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe110e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe10c103c30 0x7fe10c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe111800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe10c1057e0 0x7fe10c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe10be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe10c104e20 0x7fe10c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe110e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe10c103c30 0x7fe10c199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55636/0 (socket says 172.21.15.26:55636) 2024-06-13T07:00:22.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe110e00640 1 -- 172.21.15.26:0/1602272442 learned_addr learned my addr 172.21.15.26:0/1602272442 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe110e00640 1 -- 172.21.15.26:0/1602272442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe10c104e20 msgr2=0x7fe10c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe110e00640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe10c104e20 0x7fe10c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe110e00640 1 -- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe10c1057e0 msgr2=0x7fe10c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe110e00640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe10c1057e0 0x7fe10c1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.358+0000 7fe110e00640 1 -- 172.21.15.26:0/1602272442 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c1a1a50 con 0x7fe10c103c30 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.359+0000 7fe110e00640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe10c103c30 0x7fe10c199e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00e9c0 tx=0x7fe0fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.359+0000 7fe109600640 1 -- 172.21.15.26:0/1602272442 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00cd80 con 0x7fe10c103c30 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.359+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c1a1d40 con 0x7fe10c103c30 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.359+0000 7fe109600640 1 -- 172.21.15.26:0/1602272442 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0fc0035f0 con 0x7fe10c103c30 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.359+0000 7fe109600640 1 -- 172.21.15.26:0/1602272442 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc010690 con 0x7fe10c103c30 2024-06-13T07:00:22.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.359+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c1a21d0 con 0x7fe10c103c30 2024-06-13T07:00:22.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.360+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c10ae80 con 0x7fe10c103c30 2024-06-13T07:00:22.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.362+0000 7fe109600640 1 -- 172.21.15.26:0/1602272442 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe0fc0037a0 con 0x7fe10c103c30 2024-06-13T07:00:22.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.364+0000 7fe109600640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe0f00a2cb0 0x7fe0f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:22.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.364+0000 7fe109600640 1 -- 172.21.15.26:0/1602272442 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe0fc0cf780 con 0x7fe10c103c30 2024-06-13T07:00:22.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.364+0000 7fe10be00640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe0f00a2cb0 0x7fe0f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:22.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.365+0000 7fe10be00640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe0f00a2cb0 0x7fe0f00a5170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe10c19b340 tx=0x7fe0f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:22.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.368+0000 7fe109600640 1 -- 172.21.15.26:0/1602272442 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe0fc09c290 con 0x7fe10c103c30 2024-06-13T07:00:22.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.606+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fe10c02d2e0 con 0x7fe10c103c30 2024-06-13T07:00:22.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.714+0000 7fe109600640 1 -- 172.21.15.26:0/1602272442 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7fe10c02d2e0 con 0x7fe10c103c30 2024-06-13T07:00:22.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.717+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe0f00a2cb0 msgr2=0x7fe0f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:22.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.717+0000 7fe113800640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe0f00a2cb0 0x7fe0f00a5170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe10c19b340 tx=0x7fe0f8005f90 comp rx=0 tx=0).stop 2024-06-13T07:00:22.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.717+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe10c103c30 msgr2=0x7fe10c199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:22.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.717+0000 7fe113800640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe10c103c30 0x7fe10c199e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00e9c0 tx=0x7fe0fc00ee80 comp rx=0 tx=0).stop 2024-06-13T07:00:22.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.717+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 shutdown_connections 2024-06-13T07:00:22.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.717+0000 7fe113800640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe0f00a2cb0 0x7fe0f00a5170 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.718+0000 7fe113800640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe10c1057e0 0x7fe10c1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.718+0000 7fe113800640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe10c104e20 0x7fe10c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.718+0000 7fe113800640 1 --2- 172.21.15.26:0/1602272442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe10c103c30 0x7fe10c199e30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.718+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 >> 172.21.15.26:0/1602272442 conn(0x7fe10c0ff1c0 msgr2=0x7fe10c108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.718+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 shutdown_connections 2024-06-13T07:00:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.718+0000 7fe113800640 1 -- 172.21.15.26:0/1602272442 wait complete. 2024-06-13T07:00:22.738 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-13T07:00:22.751 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-13T07:00:22.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-13T07:00:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 -- 172.21.15.26:0/3475739608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4102300 msgr2=0x7f28f41046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 --2- 172.21.15.26:0/3475739608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4102300 0x7f28f41046f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009900 tx=0x7f28e402a3f0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 -- 172.21.15.26:0/3475739608 shutdown_connections 2024-06-13T07:00:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 --2- 172.21.15.26:0/3475739608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f4104c30 0x7f28f4107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 --2- 172.21.15.26:0/3475739608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28f4107560 0x7f28f4109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 --2- 172.21.15.26:0/3475739608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4102300 0x7f28f41046f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 -- 172.21.15.26:0/3475739608 >> 172.21.15.26:0/3475739608 conn(0x7f28f40fbc70 msgr2=0x7f28f40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 -- 172.21.15.26:0/3475739608 shutdown_connections 2024-06-13T07:00:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.973+0000 7f28f3200640 1 -- 172.21.15.26:0/3475739608 wait complete. 2024-06-13T07:00:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.974+0000 7f28f3200640 1 Processor -- start 2024-06-13T07:00:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.974+0000 7f28f3200640 1 -- start start 2024-06-13T07:00:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f3200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f4102300 0x7f28f4101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4104c30 0x7f28f41002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f3200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28f4107560 0x7f28f41007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28f410d510 con 0x7f28f4104c30 2024-06-13T07:00:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f3200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28f410d390 con 0x7f28f4102300 2024-06-13T07:00:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f3200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f28f410d690 con 0x7f28f4107560 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4104c30 0x7f28f41002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4104c30 0x7f28f41002b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38380/0 (socket says 172.21.15.26:38380) 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1400640 1 -- 172.21.15.26:0/171028252 learned_addr learned my addr 172.21.15.26:0/171028252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1e00640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f4102300 0x7f28f4101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f2800640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28f4107560 0x7f28f41007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1400640 1 -- 172.21.15.26:0/171028252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28f4107560 msgr2=0x7f28f41007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1400640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28f4107560 0x7f28f41007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1400640 1 -- 172.21.15.26:0/171028252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f4102300 msgr2=0x7f28f4101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1400640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f4102300 0x7f28f4101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1400640 1 -- 172.21.15.26:0/171028252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f4101080 con 0x7f28f4104c30 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.975+0000 7f28f1e00640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f4102300 0x7f28f4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.976+0000 7f28f1400640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4104c30 0x7f28f41002b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9c0 tx=0x7f28dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.976+0000 7f28eaa00640 1 -- 172.21.15.26:0/171028252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00cd80 con 0x7f28f4104c30 2024-06-13T07:00:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.976+0000 7f28eaa00640 1 -- 172.21.15.26:0/171028252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28dc005500 con 0x7f28f4104c30 2024-06-13T07:00:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.976+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f41a1c40 con 0x7f28f4104c30 2024-06-13T07:00:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.976+0000 7f28eaa00640 1 -- 172.21.15.26:0/171028252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc010690 con 0x7f28f4104c30 2024-06-13T07:00:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.976+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f41a2190 con 0x7f28f4104c30 2024-06-13T07:00:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.978+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f40fd760 con 0x7f28f4104c30 2024-06-13T07:00:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.980+0000 7f28eaa00640 1 -- 172.21.15.26:0/171028252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f28dc0046c0 con 0x7f28f4104c30 2024-06-13T07:00:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.981+0000 7f28eaa00640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28d80a2b90 0x7f28d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.981+0000 7f28eaa00640 1 -- 172.21.15.26:0/171028252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f28dc0cf870 con 0x7f28f4104c30 2024-06-13T07:00:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.981+0000 7f28f1e00640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28d80a2b90 0x7f28d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.982+0000 7f28f1e00640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28d80a2b90 0x7f28d80a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f28e4031440 tx=0x7f28e40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:22.986+0000 7f28eaa00640 1 -- 172.21.15.26:0/171028252 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f28dc09c380 con 0x7f28f4104c30 2024-06-13T07:00:23.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:22 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1602272442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T07:00:23.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:22 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T07:00:23.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:22 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1602272442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T07:00:23.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:22 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T07:00:23.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:22 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1602272442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T07:00:23.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:22 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T07:00:23.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.196+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28f40625d0 con 0x7f28f4104c30 2024-06-13T07:00:23.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.200+0000 7f28eaa00640 1 -- 172.21.15.26:0/171028252 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7f28dc09bc80 con 0x7f28f4104c30 2024-06-13T07:00:23.200 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T07:00:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28d80a2b90 msgr2=0x7f28d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28d80a2b90 0x7f28d80a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f28e4031440 tx=0x7f28e40315b0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4104c30 msgr2=0x7f28f41002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4104c30 0x7f28f41002b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9c0 tx=0x7f28dc00ee80 comp rx=0 tx=0).stop 2024-06-13T07:00:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 shutdown_connections 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f28d80a2b90 0x7f28d80a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28f4107560 0x7f28f41007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f4104c30 0x7f28f41002b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 --2- 172.21.15.26:0/171028252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f4102300 0x7f28f4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 >> 172.21.15.26:0/171028252 conn(0x7f28f40fbc70 msgr2=0x7f28f4102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 shutdown_connections 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.203+0000 7f28f3200640 1 -- 172.21.15.26:0/171028252 wait complete. 2024-06-13T07:00:23.224 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-13T06:49:18.857067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-13T06:54:52.893467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-13T06:54:57.814451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-13T06:55:10.534099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-13T07:00:23.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.447+0000 7fe752600640 1 -- 172.21.15.26:0/1605432170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 msgr2=0x7fe74c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.447+0000 7fe752600640 1 --2- 172.21.15.26:0/1605432170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 0x7fe74c101d50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe740009900 tx=0x7fe74002a400 comp rx=0 tx=0).stop 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.448+0000 7fe752600640 1 -- 172.21.15.26:0/1605432170 shutdown_connections 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.448+0000 7fe752600640 1 --2- 172.21.15.26:0/1605432170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c1034e0 0x7fe74c109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.448+0000 7fe752600640 1 --2- 172.21.15.26:0/1605432170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe74c102b20 0x7fe74c102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.448+0000 7fe752600640 1 --2- 172.21.15.26:0/1605432170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 0x7fe74c101d50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.448+0000 7fe752600640 1 -- 172.21.15.26:0/1605432170 >> 172.21.15.26:0/1605432170 conn(0x7fe74c0fcea0 msgr2=0x7fe74c0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.448+0000 7fe752600640 1 -- 172.21.15.26:0/1605432170 shutdown_connections 2024-06-13T07:00:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.448+0000 7fe752600640 1 -- 172.21.15.26:0/1605432170 wait complete. 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.449+0000 7fe752600640 1 Processor -- start 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.449+0000 7fe752600640 1 -- start start 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.449+0000 7fe752600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 0x7fe74c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.449+0000 7fe752600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c102b20 0x7fe74c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.449+0000 7fe752600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe74c1034e0 0x7fe74c1a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.449+0000 7fe752600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c10d560 con 0x7fe74c102b20 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.449+0000 7fe752600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c10d3e0 con 0x7fe74c101930 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.449+0000 7fe752600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c10d6e0 con 0x7fe74c1034e0 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe74be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c102b20 0x7fe74c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe751200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 0x7fe74c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe74be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c102b20 0x7fe74c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38402/0 (socket says 172.21.15.26:38402) 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe751200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 0x7fe74c199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55660/0 (socket says 172.21.15.26:55660) 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe74be00640 1 -- 172.21.15.26:0/1639523908 learned_addr learned my addr 172.21.15.26:0/1639523908 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe751c00640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe74c1034e0 0x7fe74c1a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe751200640 1 -- 172.21.15.26:0/1639523908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe74c1034e0 msgr2=0x7fe74c1a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe751200640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe74c1034e0 0x7fe74c1a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe751200640 1 -- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c102b20 msgr2=0x7fe74c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe751200640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c102b20 0x7fe74c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.450+0000 7fe751200640 1 -- 172.21.15.26:0/1639523908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c1a1a10 con 0x7fe74c101930 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.451+0000 7fe751200640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 0x7fe74c199df0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe740009230 tx=0x7fe740032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.451+0000 7fe749600640 1 -- 172.21.15.26:0/1639523908 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74003f030 con 0x7fe74c101930 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.451+0000 7fe749600640 1 -- 172.21.15.26:0/1639523908 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe740003ce0 con 0x7fe74c101930 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.451+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74c1a1ca0 con 0x7fe74c101930 2024-06-13T07:00:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.451+0000 7fe749600640 1 -- 172.21.15.26:0/1639523908 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740035070 con 0x7fe74c101930 2024-06-13T07:00:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.451+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74c1a2160 con 0x7fe74c101930 2024-06-13T07:00:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.452+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74c1a2330 con 0x7fe74c101930 2024-06-13T07:00:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.454+0000 7fe749600640 1 -- 172.21.15.26:0/1639523908 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe740042030 con 0x7fe74c101930 2024-06-13T07:00:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.456+0000 7fe749600640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe7340a2b90 0x7fe7340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.456+0000 7fe749600640 1 -- 172.21.15.26:0/1639523908 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe7400ef800 con 0x7fe74c101930 2024-06-13T07:00:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.456+0000 7fe74be00640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe7340a2b90 0x7fe7340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.457+0000 7fe74be00640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe7340a2b90 0x7fe7340a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe7380095f0 tx=0x7fe738005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.458+0000 7fe749600640 1 -- 172.21.15.26:0/1639523908 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe7400bbcc0 con 0x7fe74c101930 2024-06-13T07:00:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.663+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe74c1a2500 con 0x7fe74c101930 2024-06-13T07:00:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.664+0000 7fe749600640 1 -- 172.21.15.26:0/1639523908 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fe7400bffc0 con 0x7fe74c101930 2024-06-13T07:00:23.664 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-06-13T07:00:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.666+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe7340a2b90 msgr2=0x7fe7340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.666+0000 7fe752600640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe7340a2b90 0x7fe7340a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe7380095f0 tx=0x7fe738005f90 comp rx=0 tx=0).stop 2024-06-13T07:00:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 msgr2=0x7fe74c199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 0x7fe74c199df0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe740009230 tx=0x7fe740032ce0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 shutdown_connections 2024-06-13T07:00:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe7340a2b90 0x7fe7340a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe74c1034e0 0x7fe74c1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c102b20 0x7fe74c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 --2- 172.21.15.26:0/1639523908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c101930 0x7fe74c199df0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 >> 172.21.15.26:0/1639523908 conn(0x7fe74c0fcea0 msgr2=0x7fe74c073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 shutdown_connections 2024-06-13T07:00:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.667+0000 7fe752600640 1 -- 172.21.15.26:0/1639523908 wait complete. 2024-06-13T07:00:23.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-13T07:00:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.889+0000 7ff2c5a00640 1 -- 172.21.15.26:0/3515621017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 msgr2=0x7ff2c010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.889+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/3515621017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 0x7ff2c010da40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009860 tx=0x7ff2a802a340 comp rx=0 tx=0).stop 2024-06-13T07:00:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.890+0000 7ff2c5a00640 1 -- 172.21.15.26:0/3515621017 shutdown_connections 2024-06-13T07:00:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.890+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/3515621017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2c010df80 0x7ff2c0114810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.890+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/3515621017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 0x7ff2c010da40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.890+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/3515621017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c010c3d0 0x7ff2c010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.890+0000 7ff2c5a00640 1 -- 172.21.15.26:0/3515621017 >> 172.21.15.26:0/3515621017 conn(0x7ff2c0072d20 msgr2=0x7ff2c0073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.890+0000 7ff2c5a00640 1 -- 172.21.15.26:0/3515621017 shutdown_connections 2024-06-13T07:00:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.890+0000 7ff2c5a00640 1 -- 172.21.15.26:0/3515621017 wait complete. 2024-06-13T07:00:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.891+0000 7ff2c5a00640 1 Processor -- start 2024-06-13T07:00:23.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.891+0000 7ff2c5a00640 1 -- start start 2024-06-13T07:00:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2c5a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c010c3d0 0x7ff2c01a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2c5a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 0x7ff2c01a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2c5a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2c010df80 0x7ff2c01a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2c5a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0118060 con 0x7ff2c010c3d0 2024-06-13T07:00:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2c5a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0117ee0 con 0x7ff2c010df80 2024-06-13T07:00:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2c5a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01181e0 con 0x7ff2c010d5c0 2024-06-13T07:00:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2bea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c010c3d0 0x7ff2c01a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2bf400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2c010df80 0x7ff2c01a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2bf400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2c010df80 0x7ff2c01a9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55668/0 (socket says 172.21.15.26:55668) 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2bf400640 1 -- 172.21.15.26:0/4148585264 learned_addr learned my addr 172.21.15.26:0/4148585264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2be000640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 0x7ff2c01a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2bf400640 1 -- 172.21.15.26:0/4148585264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 msgr2=0x7ff2c01a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2bf400640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 0x7ff2c01a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2bf400640 1 -- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c010c3d0 msgr2=0x7ff2c01a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.892+0000 7ff2bf400640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c010c3d0 0x7ff2c01a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.893+0000 7ff2bf400640 1 -- 172.21.15.26:0/4148585264 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c01aa310 con 0x7ff2c010df80 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.893+0000 7ff2be000640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 0x7ff2c01a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.893+0000 7ff2bf400640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2c010df80 0x7ff2c01a9ca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800e970 tx=0x7ff2b800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.893+0000 7ff2b7400640 1 -- 172.21.15.26:0/4148585264 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800ccc0 con 0x7ff2c010df80 2024-06-13T07:00:23.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.893+0000 7ff2b7400640 1 -- 172.21.15.26:0/4148585264 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b8005500 con 0x7ff2c010df80 2024-06-13T07:00:23.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.893+0000 7ff2b7400640 1 -- 172.21.15.26:0/4148585264 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8010690 con 0x7ff2c010df80 2024-06-13T07:00:23.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.893+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c01aa600 con 0x7ff2c010df80 2024-06-13T07:00:23.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.893+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c01aaad0 con 0x7ff2c010df80 2024-06-13T07:00:23.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.895+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c01137c0 con 0x7ff2c010df80 2024-06-13T07:00:23.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.896+0000 7ff2b7400640 1 -- 172.21.15.26:0/4148585264 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2b8004e50 con 0x7ff2c010df80 2024-06-13T07:00:23.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.898+0000 7ff2b7400640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2a40a2b90 0x7ff2a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.898+0000 7ff2b7400640 1 -- 172.21.15.26:0/4148585264 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff2b80cf770 con 0x7ff2c010df80 2024-06-13T07:00:23.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.898+0000 7ff2bea00640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2a40a2b90 0x7ff2a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.899+0000 7ff2bea00640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2a40a2b90 0x7ff2a40a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2c010d420 tx=0x7ff2b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:23.902+0000 7ff2b7400640 1 -- 172.21.15.26:0/4148585264 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff2b809c280 con 0x7ff2c010df80 2024-06-13T07:00:24.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:23 smithi026 ceph-mon[32041]: pgmap v506: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T07:00:24.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:23 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-13T07:00:24.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:23 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:24.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:23 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/171028252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:24.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:23 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1639523908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:00:24.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:23 smithi135 ceph-mon[36276]: pgmap v506: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T07:00:24.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:23 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-13T07:00:24.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:23 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:24.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:23 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/171028252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:24.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:23 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1639523908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:00:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.112+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff2c00625a0 con 0x7ff2c010df80 2024-06-13T07:00:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.113+0000 7ff2b7400640 1 -- 172.21.15.26:0/4148585264 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7ff2b809bb80 con 0x7ff2c010df80 2024-06-13T07:00:24.112 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2a40a2b90 msgr2=0x7ff2a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2a40a2b90 0x7ff2a40a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2c010d420 tx=0x7ff2b0009500 comp rx=0 tx=0).stop 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2c010df80 msgr2=0x7ff2c01a9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2c010df80 0x7ff2c01a9ca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800e970 tx=0x7ff2b800ee30 comp rx=0 tx=0).stop 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 shutdown_connections 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2a40a2b90 0x7ff2a40a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2c010df80 0x7ff2c01a9ca0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2c010d5c0 0x7ff2c01a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 --2- 172.21.15.26:0/4148585264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c010c3d0 0x7ff2c01a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 >> 172.21.15.26:0/4148585264 conn(0x7ff2c0072d20 msgr2=0x7ff2c0110990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.116+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 shutdown_connections 2024-06-13T07:00:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.117+0000 7ff2c5a00640 1 -- 172.21.15.26:0/4148585264 wait complete. 2024-06-13T07:00:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:23 smithi136 ceph-mon[37229]: pgmap v506: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T07:00:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:23 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-13T07:00:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:23 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:23 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/171028252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:24.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:23 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1639523908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:00:24.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-13T07:00:24.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.347+0000 7f9db1800640 1 -- 172.21.15.26:0/317414385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102aa0 msgr2=0x7f9dac102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.347+0000 7f9db1800640 1 --2- 172.21.15.26:0/317414385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102aa0 0x7f9dac102f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009860 tx=0x7f9d9402a340 comp rx=0 tx=0).stop 2024-06-13T07:00:24.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.347+0000 7f9db1800640 1 -- 172.21.15.26:0/317414385 shutdown_connections 2024-06-13T07:00:24.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.347+0000 7f9db1800640 1 --2- 172.21.15.26:0/317414385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102aa0 0x7f9dac102f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.347+0000 7f9db1800640 1 --2- 172.21.15.26:0/317414385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dac103460 0x7f9dac109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.347+0000 7f9db1800640 1 --2- 172.21.15.26:0/317414385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dac1018b0 0x7f9dac101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.347+0000 7f9db1800640 1 -- 172.21.15.26:0/317414385 >> 172.21.15.26:0/317414385 conn(0x7f9dac0fce40 msgr2=0x7f9dac0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:24.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.347+0000 7f9db1800640 1 -- 172.21.15.26:0/317414385 shutdown_connections 2024-06-13T07:00:24.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.348+0000 7f9db1800640 1 -- 172.21.15.26:0/317414385 wait complete. 2024-06-13T07:00:24.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.349+0000 7f9db1800640 1 Processor -- start 2024-06-13T07:00:24.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.349+0000 7f9db1800640 1 -- start start 2024-06-13T07:00:24.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.349+0000 7f9db1800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac1018b0 0x7f9dac199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.349+0000 7f9db1800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dac102aa0 0x7f9dac19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.349+0000 7f9db1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dac103460 0x7f9dac1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.349+0000 7f9db1800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac10d510 con 0x7f9dac102aa0 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.349+0000 7f9db1800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac10d390 con 0x7f9dac1018b0 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.349+0000 7f9db1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac10d690 con 0x7f9dac103460 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac1018b0 0x7f9dac199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac1018b0 0x7f9dac199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55696/0 (socket says 172.21.15.26:55696) 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 -- 172.21.15.26:0/3917856571 learned_addr learned my addr 172.21.15.26:0/3917856571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9da3400640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dac102aa0 0x7f9dac19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9db0e00640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dac103460 0x7f9dac1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 -- 172.21.15.26:0/3917856571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dac103460 msgr2=0x7f9dac1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dac103460 0x7f9dac1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 -- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dac102aa0 msgr2=0x7f9dac19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dac102aa0 0x7f9dac19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 -- 172.21.15.26:0/3917856571 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac1a1980 con 0x7f9dac1018b0 2024-06-13T07:00:24.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9da3400640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dac102aa0 0x7f9dac19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.350+0000 7f9dabe00640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac1018b0 0x7f9dac199d60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00ea80 tx=0x7f9d9c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.351+0000 7f9da9600640 1 -- 172.21.15.26:0/3917856571 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00cd80 con 0x7f9dac1018b0 2024-06-13T07:00:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.351+0000 7f9da9600640 1 -- 172.21.15.26:0/3917856571 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c011950 con 0x7f9dac1018b0 2024-06-13T07:00:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.351+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac1a1c70 con 0x7f9dac1018b0 2024-06-13T07:00:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.351+0000 7f9da9600640 1 -- 172.21.15.26:0/3917856571 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c01aac0 con 0x7f9dac1018b0 2024-06-13T07:00:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.351+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1a2190 con 0x7f9dac1018b0 2024-06-13T07:00:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.352+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac1a23c0 con 0x7f9dac1018b0 2024-06-13T07:00:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.354+0000 7f9da9600640 1 -- 172.21.15.26:0/3917856571 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d9c011b00 con 0x7f9dac1018b0 2024-06-13T07:00:24.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.355+0000 7f9da9600640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9d900a2c10 0x7f9d900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:24.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.356+0000 7f9da9600640 1 -- 172.21.15.26:0/3917856571 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9d9c0cfae0 con 0x7f9dac1018b0 2024-06-13T07:00:24.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.356+0000 7f9da3400640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9d900a2c10 0x7f9d900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:24.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.356+0000 7f9da3400640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9d900a2c10 0x7f9d900a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d9402a930 tx=0x7f9d94030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.358+0000 7f9da9600640 1 -- 172.21.15.26:0/3917856571 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d9c016030 con 0x7f9dac1018b0 2024-06-13T07:00:24.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.563+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9dac1a2590 con 0x7f9dac1018b0 2024-06-13T07:00:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.564+0000 7f9da9600640 1 -- 172.21.15.26:0/3917856571 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f9d9c09c670 con 0x7f9dac1018b0 2024-06-13T07:00:24.564 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9d900a2c10 msgr2=0x7f9d900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9d900a2c10 0x7f9d900a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d9402a930 tx=0x7f9d94030570 comp rx=0 tx=0).stop 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac1018b0 msgr2=0x7f9dac199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac1018b0 0x7f9dac199d60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00ea80 tx=0x7f9d9c00ee80 comp rx=0 tx=0).stop 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 shutdown_connections 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9d900a2c10 0x7f9d900a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dac103460 0x7f9dac1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dac102aa0 0x7f9dac19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 --2- 172.21.15.26:0/3917856571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac1018b0 0x7f9dac199d60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 >> 172.21.15.26:0/3917856571 conn(0x7f9dac0fce40 msgr2=0x7f9dac073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.567+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 shutdown_connections 2024-06-13T07:00:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.568+0000 7f9db1800640 1 -- 172.21.15.26:0/3917856571 wait complete. 2024-06-13T07:00:24.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-13T07:00:24.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.805+0000 7fea40c00640 1 -- 172.21.15.26:0/1207566855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c0775a0 msgr2=0x7fea3c077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.805+0000 7fea40c00640 1 --2- 172.21.15.26:0/1207566855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c0775a0 0x7fea3c077a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea28009900 tx=0x7fea2802a410 comp rx=0 tx=0).stop 2024-06-13T07:00:24.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.806+0000 7fea40c00640 1 -- 172.21.15.26:0/1207566855 shutdown_connections 2024-06-13T07:00:24.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.806+0000 7fea40c00640 1 --2- 172.21.15.26:0/1207566855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea3c077f40 0x7fea3c0787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.806+0000 7fea40c00640 1 --2- 172.21.15.26:0/1207566855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c0775a0 0x7fea3c077a00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.806+0000 7fea40c00640 1 --2- 172.21.15.26:0/1207566855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea3c076360 0x7fea3c076780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.806+0000 7fea40c00640 1 -- 172.21.15.26:0/1207566855 >> 172.21.15.26:0/1207566855 conn(0x7fea3c0ff190 msgr2=0x7fea3c1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:24.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.806+0000 7fea40c00640 1 -- 172.21.15.26:0/1207566855 shutdown_connections 2024-06-13T07:00:24.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.806+0000 7fea40c00640 1 -- 172.21.15.26:0/1207566855 wait complete. 2024-06-13T07:00:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.807+0000 7fea40c00640 1 Processor -- start 2024-06-13T07:00:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.807+0000 7fea40c00640 1 -- start start 2024-06-13T07:00:24.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.807+0000 7fea40c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea3c076360 0x7fea3c19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.807+0000 7fea40c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c077f40 0x7fea3c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.807+0000 7fea40c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea3c1a3ae0 0x7fea3c1a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.807+0000 7fea40c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c113cd0 con 0x7fea3c077f40 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.807+0000 7fea40c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c113b50 con 0x7fea3c1a3ae0 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.807+0000 7fea40c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c113e50 con 0x7fea3c076360 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c077f40 0x7fea3c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c077f40 0x7fea3c19ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38456/0 (socket says 172.21.15.26:38456) 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39400640 1 -- 172.21.15.26:0/22948105 learned_addr learned my addr 172.21.15.26:0/22948105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:24.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39e00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea3c076360 0x7fea3c19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea3a800640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea3c1a3ae0 0x7fea3c1a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39400640 1 -- 172.21.15.26:0/22948105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea3c076360 msgr2=0x7fea3c19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39400640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea3c076360 0x7fea3c19e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39400640 1 -- 172.21.15.26:0/22948105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea3c1a3ae0 msgr2=0x7fea3c1a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39400640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea3c1a3ae0 0x7fea3c1a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39400640 1 -- 172.21.15.26:0/22948105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c1a6540 con 0x7fea3c077f40 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.808+0000 7fea39e00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea3c076360 0x7fea3c19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.809+0000 7fea39400640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c077f40 0x7fea3c19ea80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fea2802aa00 tx=0x7fea28032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.809+0000 7fea32a00640 1 -- 172.21.15.26:0/22948105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2803f030 con 0x7fea3c077f40 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.809+0000 7fea32a00640 1 -- 172.21.15.26:0/22948105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea28003ce0 con 0x7fea3c077f40 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.809+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c1a67d0 con 0x7fea3c077f40 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.809+0000 7fea32a00640 1 -- 172.21.15.26:0/22948105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28035070 con 0x7fea3c077f40 2024-06-13T07:00:24.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.809+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c1a6c70 con 0x7fea3c077f40 2024-06-13T07:00:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.810+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3c110570 con 0x7fea3c077f40 2024-06-13T07:00:24.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.812+0000 7fea32a00640 1 -- 172.21.15.26:0/22948105 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea28042030 con 0x7fea3c077f40 2024-06-13T07:00:24.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.814+0000 7fea32a00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fea200a2b90 0x7fea200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:24.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.814+0000 7fea32a00640 1 -- 172.21.15.26:0/22948105 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fea280ef570 con 0x7fea3c077f40 2024-06-13T07:00:24.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.814+0000 7fea39e00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fea200a2b90 0x7fea200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:24.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.815+0000 7fea39e00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fea200a2b90 0x7fea200a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009fc0 tx=0x7fea2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:24.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:24.818+0000 7fea32a00640 1 -- 172.21.15.26:0/22948105 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea280bba30 con 0x7fea3c077f40 2024-06-13T07:00:25.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:24 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/4148585264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:00:25.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:24 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3917856571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:00:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.024+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fea3c02d500 con 0x7fea3c077f40 2024-06-13T07:00:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.025+0000 7fea32a00640 1 -- 172.21.15.26:0/22948105 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) ==== 103+0+11 (secure 0 0 0) 0x7fea280bfd30 con 0x7fea3c077f40 2024-06-13T07:00:25.024 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-06-13T07:00:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fea200a2b90 msgr2=0x7fea200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fea200a2b90 0x7fea200a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009fc0 tx=0x7fea2c005f90 comp rx=0 tx=0).stop 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c077f40 msgr2=0x7fea3c19ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c077f40 0x7fea3c19ea80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fea2802aa00 tx=0x7fea28032ce0 comp rx=0 tx=0).stop 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 shutdown_connections 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fea200a2b90 0x7fea200a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea3c1a3ae0 0x7fea3c1a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c077f40 0x7fea3c19ea80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 --2- 172.21.15.26:0/22948105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea3c076360 0x7fea3c19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:25.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 >> 172.21.15.26:0/22948105 conn(0x7fea3c0ff190 msgr2=0x7fea3c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:25.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 shutdown_connections 2024-06-13T07:00:25.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:25.028+0000 7fea40c00640 1 -- 172.21.15.26:0/22948105 wait complete. 2024-06-13T07:00:25.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:24 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/4148585264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:00:25.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:24 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3917856571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:00:25.046 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-13T07:00:25.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:00:25.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:24 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/4148585264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:00:25.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:24 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3917856571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:00:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:25.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:25 smithi026 ceph-mon[32041]: pgmap v507: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T07:00:25.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:25 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/22948105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:00:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:25 smithi135 ceph-mon[36276]: pgmap v507: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T07:00:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:25 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/22948105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:00:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:25 smithi136 ceph-mon[37229]: pgmap v507: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T07:00:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:25 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/22948105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:00:27.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:27 smithi026 ceph-mon[32041]: pgmap v508: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T07:00:28.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:27 smithi135 ceph-mon[36276]: pgmap v508: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T07:00:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:27 smithi136 ceph-mon[37229]: pgmap v508: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T07:00:28.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3306820044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006b350 msgr2=0x7ff7c010e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3306820044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006b350 0x7ff7c010e9f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009990 tx=0x7ff7a8009ca0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3306820044 shutdown_connections 2024-06-13T07:00:28.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3306820044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7c010ef30 0x7ff7c0111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3306820044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7c006aa30 0x7ff7c006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3306820044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006b350 0x7ff7c010e9f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3306820044 >> 172.21.15.26:0/3306820044 conn(0x7ff7c0078300 msgr2=0x7ff7c0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:28.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3306820044 shutdown_connections 2024-06-13T07:00:28.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.256+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3306820044 wait complete. 2024-06-13T07:00:28.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.257+0000 7ff7c4c00700 1 Processor -- start 2024-06-13T07:00:28.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7c4c00700 1 -- start start 2024-06-13T07:00:28.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7c4c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006aa30 0x7ff7c019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:28.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7c4c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7c006b350 0x7ff7c019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:28.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7c4c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7c010ef30 0x7ff7c019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7c4c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01149e0 con 0x7ff7c006aa30 2024-06-13T07:00:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7c4c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0114860 con 0x7ff7c006b350 2024-06-13T07:00:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7c4c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0114b60 con 0x7ff7c010ef30 2024-06-13T07:00:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7bde00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006aa30 0x7ff7c019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.258+0000 7ff7bde00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006aa30 0x7ff7c019ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38488/0 (socket says 172.21.15.26:38488) 2024-06-13T07:00:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7bde00700 1 -- 172.21.15.26:0/3790722392 learned_addr learned my addr 172.21.15.26:0/3790722392 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7b5400700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7c006b350 0x7ff7c019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7be800700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7c010ef30 0x7ff7c019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7b5400700 1 -- 172.21.15.26:0/3790722392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7c010ef30 msgr2=0x7ff7c019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7b5400700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7c010ef30 0x7ff7c019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7b5400700 1 -- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006aa30 msgr2=0x7ff7c019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7b5400700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006aa30 0x7ff7c019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7b5400700 1 -- 172.21.15.26:0/3790722392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c019fe50 con 0x7ff7c006b350 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7bde00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006aa30 0x7ff7c019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.259+0000 7ff7b5400700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7c006b350 0x7ff7c019b3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80098e0 tx=0x7ff7a800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.260+0000 7ff7b7400700 1 -- 172.21.15.26:0/3790722392 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8027070 con 0x7ff7c006b350 2024-06-13T07:00:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.260+0000 7ff7b7400700 1 -- 172.21.15.26:0/3790722392 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a8003b60 con 0x7ff7c006b350 2024-06-13T07:00:28.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.260+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01a00e0 con 0x7ff7c006b350 2024-06-13T07:00:28.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.260+0000 7ff7b7400700 1 -- 172.21.15.26:0/3790722392 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8022d50 con 0x7ff7c006b350 2024-06-13T07:00:28.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.260+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01a79c0 con 0x7ff7c006b350 2024-06-13T07:00:28.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.261+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0069180 con 0x7ff7c006b350 2024-06-13T07:00:28.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.263+0000 7ff7b7400700 1 -- 172.21.15.26:0/3790722392 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff7a802a070 con 0x7ff7c006b350 2024-06-13T07:00:28.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.265+0000 7ff7b7400700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7a40a2c20 0x7ff7a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:28.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.265+0000 7ff7b7400700 1 -- 172.21.15.26:0/3790722392 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff7a80d0cb0 con 0x7ff7c006b350 2024-06-13T07:00:28.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.265+0000 7ff7bde00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7a40a2c20 0x7ff7a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:28.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.266+0000 7ff7bde00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7a40a2c20 0x7ff7a40a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0011550 tx=0x7ff7b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:28.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.267+0000 7ff7b7400700 1 -- 172.21.15.26:0/3790722392 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff7a8013070 con 0x7ff7c006b350 2024-06-13T07:00:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.546+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7c002dca0 con 0x7ff7c006b350 2024-06-13T07:00:28.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.547+0000 7ff7b7400700 1 -- 172.21.15.26:0/3790722392 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7ff7a809d970 con 0x7ff7c006b350 2024-06-13T07:00:28.547 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:00:28.547 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T07:00:22.609623+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24506,24511,24517]},"id":1}]} 2024-06-13T07:00:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.550+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7a40a2c20 msgr2=0x7ff7a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.550+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7a40a2c20 0x7ff7a40a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0011550 tx=0x7ff7b0009260 comp rx=0 tx=0).stop 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.550+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7c006b350 msgr2=0x7ff7c019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.550+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7c006b350 0x7ff7c019b3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80098e0 tx=0x7ff7a800b8a0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.551+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 shutdown_connections 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.551+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7a40a2c20 0x7ff7a40a50e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.551+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7c010ef30 0x7ff7c019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.551+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7c006aa30 0x7ff7c019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.551+0000 7ff7c4c00700 1 --2- 172.21.15.26:0/3790722392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7c006b350 0x7ff7c019b3d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.551+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 >> 172.21.15.26:0/3790722392 conn(0x7ff7c0078300 msgr2=0x7ff7c0110380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.551+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 shutdown_connections 2024-06-13T07:00:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:28.551+0000 7ff7c4c00700 1 -- 172.21.15.26:0/3790722392 wait complete. 2024-06-13T07:00:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2024-06-13T07:00:28.991 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:28 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3790722392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:29.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:28 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3790722392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:29.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:28 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3790722392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:29.583 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-13T07:00:29.583 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-13T07:00:29.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-13T07:00:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.814+0000 7fd797200640 1 -- 172.21.15.26:0/1504441725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7981055d0 msgr2=0x7fd79810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.814+0000 7fd797200640 1 --2- 172.21.15.26:0/1504441725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7981055d0 0x7fd79810be60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd790009ca0 tx=0x7fd79002a8b0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.815+0000 7fd797200640 1 -- 172.21.15.26:0/1504441725 shutdown_connections 2024-06-13T07:00:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.815+0000 7fd797200640 1 --2- 172.21.15.26:0/1504441725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7981055d0 0x7fd79810be60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.815+0000 7fd797200640 1 --2- 172.21.15.26:0/1504441725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd798104c10 0x7fd798105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.815+0000 7fd797200640 1 --2- 172.21.15.26:0/1504441725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd798103a20 0x7fd798103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.815+0000 7fd797200640 1 -- 172.21.15.26:0/1504441725 >> 172.21.15.26:0/1504441725 conn(0x7fd7980fefb0 msgr2=0x7fd7981013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.815+0000 7fd797200640 1 -- 172.21.15.26:0/1504441725 shutdown_connections 2024-06-13T07:00:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.815+0000 7fd797200640 1 -- 172.21.15.26:0/1504441725 wait complete. 2024-06-13T07:00:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.816+0000 7fd797200640 1 Processor -- start 2024-06-13T07:00:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.816+0000 7fd797200640 1 -- start start 2024-06-13T07:00:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.816+0000 7fd797200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd798103a20 0x7fd798199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.816+0000 7fd797200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd798104c10 0x7fd79819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.816+0000 7fd797200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7981055d0 0x7fd7981a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.816+0000 7fd797200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd79810f680 con 0x7fd798104c10 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.816+0000 7fd797200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd79810f500 con 0x7fd7981055d0 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.816+0000 7fd797200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd79810f800 con 0x7fd798103a20 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd798103a20 0x7fd798199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd796800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7981055d0 0x7fd7981a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd798103a20 0x7fd798199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33018/0 (socket says 172.21.15.26:33018) 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 -- 172.21.15.26:0/3835885451 learned_addr learned my addr 172.21.15.26:0/3835885451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd78d400640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd798104c10 0x7fd79819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 -- 172.21.15.26:0/3835885451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7981055d0 msgr2=0x7fd7981a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7981055d0 0x7fd7981a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 -- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd798104c10 msgr2=0x7fd79819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd798104c10 0x7fd79819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 -- 172.21.15.26:0/3835885451 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7981a18e0 con 0x7fd798103a20 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.817+0000 7fd795e00640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd798103a20 0x7fd798199ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd78800e9c0 tx=0x7fd78800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.818+0000 7fd78f400640 1 -- 172.21.15.26:0/3835885451 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78800cd80 con 0x7fd798103a20 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.818+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7981a1bd0 con 0x7fd798103a20 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.818+0000 7fd78f400640 1 -- 172.21.15.26:0/3835885451 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7880035f0 con 0x7fd798103a20 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.818+0000 7fd78f400640 1 -- 172.21.15.26:0/3835885451 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788010690 con 0x7fd798103a20 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.818+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7981a2170 con 0x7fd798103a20 2024-06-13T07:00:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.819+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79810ad40 con 0x7fd798103a20 2024-06-13T07:00:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.821+0000 7fd78f400640 1 -- 172.21.15.26:0/3835885451 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd7880037a0 con 0x7fd798103a20 2024-06-13T07:00:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.823+0000 7fd78f400640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.823+0000 7fd78f400640 1 -- 172.21.15.26:0/3835885451 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd7880cf7e0 con 0x7fd798103a20 2024-06-13T07:00:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.823+0000 7fd78d400640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.824+0000 7fd78d400640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd7800095f0 tx=0x7fd780005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:29.827+0000 7fd78f400640 1 -- 172.21.15.26:0/3835885451 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd78809c2f0 con 0x7fd798103a20 2024-06-13T07:00:30.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:29 smithi026 ceph-mon[32041]: pgmap v509: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T07:00:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:29 smithi135 ceph-mon[36276]: pgmap v509: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T07:00:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.035+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd798061de0 con 0x7fd77c0a2c10 2024-06-13T07:00:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.066+0000 7fd78f400640 1 -- 172.21.15.26:0/3835885451 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340499 (secure 0 0 0) 0x7fd798061de0 con 0x7fd77c0a2c10 2024-06-13T07:00:30.074 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:00:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-06-13T07:00:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd77c0a2c10 msgr2=0x7fd77c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd7800095f0 tx=0x7fd780005f90 comp rx=0 tx=0).stop 2024-06-13T07:00:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd798103a20 msgr2=0x7fd798199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd798103a20 0x7fd798199ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd78800e9c0 tx=0x7fd78800ee80 comp rx=0 tx=0).stop 2024-06-13T07:00:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 shutdown_connections 2024-06-13T07:00:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7981055d0 0x7fd7981a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd798104c10 0x7fd79819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 --2- 172.21.15.26:0/3835885451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd798103a20 0x7fd798199ca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 >> 172.21.15.26:0/3835885451 conn(0x7fd7980fefb0 msgr2=0x7fd798107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.072+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 shutdown_connections 2024-06-13T07:00:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.073+0000 7fd797200640 1 -- 172.21.15.26:0/3835885451 wait complete. 2024-06-13T07:00:30.094 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-06-13T07:00:28.423292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658259,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1555,"num_read_kb":895,"num_write":363,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182392,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402040,"kb_used_data":15448,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539784,"statfs":{"total":1151940427776,"available":1151528738816,"internally_reserved":0,"allocated":15818752,"data_stored":8802149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":612,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":37,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005884"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293186+0000","last_change":"2024-06-13T06:54:58.919669+0000","last_active":"2024-06-13T06:58:28.293186+0000","last_peered":"2024-06-13T06:58:28.293186+0000","last_clean":"2024-06-13T06:58:28.293186+0000","last_became_active":"2024-06-13T06:54:58.919334+0000","last_became_peered":"2024-06-13T06:54:58.919334+0000","last_unstale":"2024-06-13T06:58:28.293186+0000","last_undegraded":"2024-06-13T06:58:28.293186+0000","last_fullsized":"2024-06-13T06:58:28.293186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:45:40.831640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295249+0000","last_change":"2024-06-13T06:54:58.909508+0000","last_active":"2024-06-13T06:58:28.295249+0000","last_peered":"2024-06-13T06:58:28.295249+0000","last_clean":"2024-06-13T06:58:28.295249+0000","last_became_active":"2024-06-13T06:54:58.909221+0000","last_became_peered":"2024-06-13T06:54:58.909221+0000","last_unstale":"2024-06-13T06:58:28.295249+0000","last_undegraded":"2024-06-13T06:58:28.295249+0000","last_fullsized":"2024-06-13T06:58:28.295249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:17:07.476858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304063+0000","last_change":"2024-06-13T06:54:58.909283+0000","last_active":"2024-06-13T06:58:28.304063+0000","last_peered":"2024-06-13T06:58:28.304063+0000","last_clean":"2024-06-13T06:58:28.304063+0000","last_became_active":"2024-06-13T06:54:58.908912+0000","last_became_peered":"2024-06-13T06:54:58.908912+0000","last_unstale":"2024-06-13T06:58:28.304063+0000","last_undegraded":"2024-06-13T06:58:28.304063+0000","last_fullsized":"2024-06-13T06:58:28.304063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:48:26.937971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.264605+0000","last_change":"2024-06-13T06:54:58.902821+0000","last_active":"2024-06-13T06:58:28.264605+0000","last_peered":"2024-06-13T06:58:28.264605+0000","last_clean":"2024-06-13T06:58:28.264605+0000","last_became_active":"2024-06-13T06:54:58.902215+0000","last_became_peered":"2024-06-13T06:54:58.902215+0000","last_unstale":"2024-06-13T06:58:28.264605+0000","last_undegraded":"2024-06-13T06:58:28.264605+0000","last_fullsized":"2024-06-13T06:58:28.264605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:11:57.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296624+0000","last_change":"2024-06-13T06:54:58.897038+0000","last_active":"2024-06-13T06:58:28.296624+0000","last_peered":"2024-06-13T06:58:28.296624+0000","last_clean":"2024-06-13T06:58:28.296624+0000","last_became_active":"2024-06-13T06:54:58.896665+0000","last_became_peered":"2024-06-13T06:54:58.896665+0000","last_unstale":"2024-06-13T06:58:28.296624+0000","last_undegraded":"2024-06-13T06:58:28.296624+0000","last_fullsized":"2024-06-13T06:58:28.296624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:24:19.216932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.269970+0000","last_change":"2024-06-13T06:54:58.895501+0000","last_active":"2024-06-13T06:58:28.269970+0000","last_peered":"2024-06-13T06:58:28.269970+0000","last_clean":"2024-06-13T06:58:28.269970+0000","last_became_active":"2024-06-13T06:54:58.895144+0000","last_became_peered":"2024-06-13T06:54:58.895144+0000","last_unstale":"2024-06-13T06:58:28.269970+0000","last_undegraded":"2024-06-13T06:58:28.269970+0000","last_fullsized":"2024-06-13T06:58:28.269970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:50.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272336+0000","last_change":"2024-06-13T06:54:58.901097+0000","last_active":"2024-06-13T06:58:28.272336+0000","last_peered":"2024-06-13T06:58:28.272336+0000","last_clean":"2024-06-13T06:58:28.272336+0000","last_became_active":"2024-06-13T06:54:58.900727+0000","last_became_peered":"2024-06-13T06:54:58.900727+0000","last_unstale":"2024-06-13T06:58:28.272336+0000","last_undegraded":"2024-06-13T06:58:28.272336+0000","last_fullsized":"2024-06-13T06:58:28.272336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:09:38.349517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278488+0000","last_change":"2024-06-13T06:54:58.910740+0000","last_active":"2024-06-13T06:58:28.278488+0000","last_peered":"2024-06-13T06:58:28.278488+0000","last_clean":"2024-06-13T06:58:28.278488+0000","last_became_active":"2024-06-13T06:54:58.910351+0000","last_became_peered":"2024-06-13T06:54:58.910351+0000","last_unstale":"2024-06-13T06:58:28.278488+0000","last_undegraded":"2024-06-13T06:58:28.278488+0000","last_fullsized":"2024-06-13T06:58:28.278488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:45:12.854996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281927+0000","last_change":"2024-06-13T06:55:29.364516+0000","last_active":"2024-06-13T06:58:28.281927+0000","last_peered":"2024-06-13T06:58:28.281927+0000","last_clean":"2024-06-13T06:58:28.281927+0000","last_became_active":"2024-06-13T06:55:29.364079+0000","last_became_peered":"2024-06-13T06:55:29.364079+0000","last_unstale":"2024-06-13T06:58:28.281927+0000","last_undegraded":"2024-06-13T06:58:28.281927+0000","last_fullsized":"2024-06-13T06:58:28.281927+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:51:28.624359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267340+0000","last_change":"2024-06-13T06:54:58.903882+0000","last_active":"2024-06-13T06:58:28.267340+0000","last_peered":"2024-06-13T06:58:28.267340+0000","last_clean":"2024-06-13T06:58:28.267340+0000","last_became_active":"2024-06-13T06:54:58.903495+0000","last_became_peered":"2024-06-13T06:54:58.903495+0000","last_unstale":"2024-06-13T06:58:28.267340+0000","last_undegraded":"2024-06-13T06:58:28.267340+0000","last_fullsized":"2024-06-13T06:58:28.267340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:23.682993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281971+0000","last_change":"2024-06-13T06:54:58.900080+0000","last_active":"2024-06-13T06:58:28.281971+0000","last_peered":"2024-06-13T06:58:28.281971+0000","last_clean":"2024-06-13T06:58:28.281971+0000","last_became_active":"2024-06-13T06:54:58.899736+0000","last_became_peered":"2024-06-13T06:54:58.899736+0000","last_unstale":"2024-06-13T06:58:28.281971+0000","last_undegraded":"2024-06-13T06:58:28.281971+0000","last_fullsized":"2024-06-13T06:58:28.281971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:51:45.808821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.264389+0000","last_change":"2024-06-13T06:54:58.900238+0000","last_active":"2024-06-13T06:58:28.264389+0000","last_peered":"2024-06-13T06:58:28.264389+0000","last_clean":"2024-06-13T06:58:28.264389+0000","last_became_active":"2024-06-13T06:54:58.899872+0000","last_became_peered":"2024-06-13T06:54:58.899872+0000","last_unstale":"2024-06-13T06:58:28.264389+0000","last_undegraded":"2024-06-13T06:58:28.264389+0000","last_fullsized":"2024-06-13T06:58:28.264389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:12:40.780894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296863+0000","last_change":"2024-06-13T06:54:58.901981+0000","last_active":"2024-06-13T06:58:28.296863+0000","last_peered":"2024-06-13T06:58:28.296863+0000","last_clean":"2024-06-13T06:58:28.296863+0000","last_became_active":"2024-06-13T06:54:58.898704+0000","last_became_peered":"2024-06-13T06:54:58.898704+0000","last_unstale":"2024-06-13T06:58:28.296863+0000","last_undegraded":"2024-06-13T06:58:28.296863+0000","last_fullsized":"2024-06-13T06:58:28.296863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:03:56.658091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.299281+0000","last_change":"2024-06-13T06:54:58.909022+0000","last_active":"2024-06-13T06:58:28.299281+0000","last_peered":"2024-06-13T06:58:28.299281+0000","last_clean":"2024-06-13T06:58:28.299281+0000","last_became_active":"2024-06-13T06:54:58.907438+0000","last_became_peered":"2024-06-13T06:54:58.907438+0000","last_unstale":"2024-06-13T06:58:28.299281+0000","last_undegraded":"2024-06-13T06:58:28.299281+0000","last_fullsized":"2024-06-13T06:58:28.299281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:40:34.190510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305584+0000","last_change":"2024-06-13T06:54:58.897691+0000","last_active":"2024-06-13T06:58:28.305584+0000","last_peered":"2024-06-13T06:58:28.305584+0000","last_clean":"2024-06-13T06:58:28.305584+0000","last_became_active":"2024-06-13T06:54:58.897381+0000","last_became_peered":"2024-06-13T06:54:58.897381+0000","last_unstale":"2024-06-13T06:58:28.305584+0000","last_undegraded":"2024-06-13T06:58:28.305584+0000","last_fullsized":"2024-06-13T06:58:28.305584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:10.052052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295211+0000","last_change":"2024-06-13T06:54:58.907676+0000","last_active":"2024-06-13T06:58:28.295211+0000","last_peered":"2024-06-13T06:58:28.295211+0000","last_clean":"2024-06-13T06:58:28.295211+0000","last_became_active":"2024-06-13T06:54:58.907304+0000","last_became_peered":"2024-06-13T06:54:58.907304+0000","last_unstale":"2024-06-13T06:58:28.295211+0000","last_undegraded":"2024-06-13T06:58:28.295211+0000","last_fullsized":"2024-06-13T06:58:28.295211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:08:20.292516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293328+0000","last_change":"2024-06-13T06:54:58.920012+0000","last_active":"2024-06-13T06:58:28.293328+0000","last_peered":"2024-06-13T06:58:28.293328+0000","last_clean":"2024-06-13T06:58:28.293328+0000","last_became_active":"2024-06-13T06:54:58.919473+0000","last_became_peered":"2024-06-13T06:54:58.919473+0000","last_unstale":"2024-06-13T06:58:28.293328+0000","last_undegraded":"2024-06-13T06:58:28.293328+0000","last_fullsized":"2024-06-13T06:58:28.293328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:49:13.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.280156+0000","last_change":"2024-06-13T06:54:58.902576+0000","last_active":"2024-06-13T06:58:28.280156+0000","last_peered":"2024-06-13T06:58:28.280156+0000","last_clean":"2024-06-13T06:58:28.280156+0000","last_became_active":"2024-06-13T06:54:58.902205+0000","last_became_peered":"2024-06-13T06:54:58.902205+0000","last_unstale":"2024-06-13T06:58:28.280156+0000","last_undegraded":"2024-06-13T06:58:28.280156+0000","last_fullsized":"2024-06-13T06:58:28.280156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:13:24.107081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292347+0000","last_change":"2024-06-13T06:54:58.905392+0000","last_active":"2024-06-13T06:58:28.292347+0000","last_peered":"2024-06-13T06:58:28.292347+0000","last_clean":"2024-06-13T06:58:28.292347+0000","last_became_active":"2024-06-13T06:54:58.905017+0000","last_became_peered":"2024-06-13T06:54:58.905017+0000","last_unstale":"2024-06-13T06:58:28.292347+0000","last_undegraded":"2024-06-13T06:58:28.292347+0000","last_fullsized":"2024-06-13T06:58:28.292347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:05.931653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:37.654508+0000","last_change":"2024-06-13T06:55:12.021359+0000","last_active":"2024-06-13T06:57:37.654508+0000","last_peered":"2024-06-13T06:57:37.654508+0000","last_clean":"2024-06-13T06:57:37.654508+0000","last_became_active":"2024-06-13T06:55:12.020860+0000","last_became_peered":"2024-06-13T06:55:12.020860+0000","last_unstale":"2024-06-13T06:57:37.654508+0000","last_undegraded":"2024-06-13T06:57:37.654508+0000","last_fullsized":"2024-06-13T06:57:37.654508+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:05:00.833905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:00.551277+0000","last_change":"2024-06-13T06:54:54.339220+0000","last_active":"2024-06-13T06:57:00.551277+0000","last_peered":"2024-06-13T06:57:00.551277+0000","last_clean":"2024-06-13T06:57:00.551277+0000","last_became_active":"2024-06-13T06:54:54.338573+0000","last_became_peered":"2024-06-13T06:54:54.338573+0000","last_unstale":"2024-06-13T06:57:00.551277+0000","last_undegraded":"2024-06-13T06:57:00.551277+0000","last_fullsized":"2024-06-13T06:57:00.551277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:53:50.258097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.277556+0000","last_change":"2024-06-13T06:54:58.903920+0000","last_active":"2024-06-13T06:58:28.277556+0000","last_peered":"2024-06-13T06:58:28.277556+0000","last_clean":"2024-06-13T06:58:28.277556+0000","last_became_active":"2024-06-13T06:54:58.895364+0000","last_became_peered":"2024-06-13T06:54:58.895364+0000","last_unstale":"2024-06-13T06:58:28.277556+0000","last_undegraded":"2024-06-13T06:58:28.277556+0000","last_fullsized":"2024-06-13T06:58:28.277556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:04:35.109799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:38.288505+0000","last_change":"2024-06-13T06:55:12.017561+0000","last_active":"2024-06-13T06:57:38.288505+0000","last_peered":"2024-06-13T06:57:38.288505+0000","last_clean":"2024-06-13T06:57:38.288505+0000","last_became_active":"2024-06-13T06:55:12.017104+0000","last_became_peered":"2024-06-13T06:55:12.017104+0000","last_unstale":"2024-06-13T06:57:38.288505+0000","last_undegraded":"2024-06-13T06:57:38.288505+0000","last_fullsized":"2024-06-13T06:57:38.288505+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:08:22.570491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:56.075775+0000","last_change":"2024-06-13T06:54:54.631599+0000","last_active":"2024-06-13T06:56:56.075775+0000","last_peered":"2024-06-13T06:56:56.075775+0000","last_clean":"2024-06-13T06:56:56.075775+0000","last_became_active":"2024-06-13T06:54:54.631007+0000","last_became_peered":"2024-06-13T06:54:54.631007+0000","last_unstale":"2024-06-13T06:56:56.075775+0000","last_undegraded":"2024-06-13T06:56:56.075775+0000","last_fullsized":"2024-06-13T06:56:56.075775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:49:56.602192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304511+0000","last_change":"2024-06-13T06:54:58.917255+0000","last_active":"2024-06-13T06:58:28.304511+0000","last_peered":"2024-06-13T06:58:28.304511+0000","last_clean":"2024-06-13T06:58:28.304511+0000","last_became_active":"2024-06-13T06:54:58.916875+0000","last_became_peered":"2024-06-13T06:54:58.916875+0000","last_unstale":"2024-06-13T06:58:28.304511+0000","last_undegraded":"2024-06-13T06:58:28.304511+0000","last_fullsized":"2024-06-13T06:58:28.304511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:09:47.638802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:43.289081+0000","last_change":"2024-06-13T06:55:12.018209+0000","last_active":"2024-06-13T06:57:43.289081+0000","last_peered":"2024-06-13T06:57:43.289081+0000","last_clean":"2024-06-13T06:57:43.289081+0000","last_became_active":"2024-06-13T06:55:12.017038+0000","last_became_peered":"2024-06-13T06:55:12.017038+0000","last_unstale":"2024-06-13T06:57:43.289081+0000","last_undegraded":"2024-06-13T06:57:43.289081+0000","last_fullsized":"2024-06-13T06:57:43.289081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:36:26.476454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:25.059841+0000","last_change":"2024-06-13T06:54:54.328849+0000","last_active":"2024-06-13T06:57:25.059841+0000","last_peered":"2024-06-13T06:57:25.059841+0000","last_clean":"2024-06-13T06:57:25.059841+0000","last_became_active":"2024-06-13T06:54:54.328388+0000","last_became_peered":"2024-06-13T06:54:54.328388+0000","last_unstale":"2024-06-13T06:57:25.059841+0000","last_undegraded":"2024-06-13T06:57:25.059841+0000","last_fullsized":"2024-06-13T06:57:25.059841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:41.264403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.283009+0000","last_change":"2024-06-13T06:54:58.897588+0000","last_active":"2024-06-13T06:58:28.283009+0000","last_peered":"2024-06-13T06:58:28.283009+0000","last_clean":"2024-06-13T06:58:28.283009+0000","last_became_active":"2024-06-13T06:54:58.897236+0000","last_became_peered":"2024-06-13T06:54:58.897236+0000","last_unstale":"2024-06-13T06:58:28.283009+0000","last_undegraded":"2024-06-13T06:58:28.283009+0000","last_fullsized":"2024-06-13T06:58:28.283009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:20:38.460445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:43.289055+0000","last_change":"2024-06-13T06:55:12.014080+0000","last_active":"2024-06-13T06:57:43.289055+0000","last_peered":"2024-06-13T06:57:43.289055+0000","last_clean":"2024-06-13T06:57:43.289055+0000","last_became_active":"2024-06-13T06:55:12.013627+0000","last_became_peered":"2024-06-13T06:55:12.013627+0000","last_unstale":"2024-06-13T06:57:43.289055+0000","last_undegraded":"2024-06-13T06:57:43.289055+0000","last_fullsized":"2024-06-13T06:57:43.289055+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:29:54.081593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:25.059798+0000","last_change":"2024-06-13T06:54:54.328249+0000","last_active":"2024-06-13T06:57:25.059798+0000","last_peered":"2024-06-13T06:57:25.059798+0000","last_clean":"2024-06-13T06:57:25.059798+0000","last_became_active":"2024-06-13T06:54:54.327832+0000","last_became_peered":"2024-06-13T06:54:54.327832+0000","last_unstale":"2024-06-13T06:57:25.059798+0000","last_undegraded":"2024-06-13T06:57:25.059798+0000","last_fullsized":"2024-06-13T06:57:25.059798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:21:05.624075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281353+0000","last_change":"2024-06-13T06:55:29.383305+0000","last_active":"2024-06-13T06:58:28.281353+0000","last_peered":"2024-06-13T06:58:28.281353+0000","last_clean":"2024-06-13T06:58:28.281353+0000","last_became_active":"2024-06-13T06:55:29.381491+0000","last_became_peered":"2024-06-13T06:55:29.381491+0000","last_unstale":"2024-06-13T06:58:28.281353+0000","last_undegraded":"2024-06-13T06:58:28.281353+0000","last_fullsized":"2024-06-13T06:58:28.281353+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:52:26.437237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:42.655020+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:57:42.655020+0000","last_peered":"2024-06-13T06:57:42.655020+0000","last_clean":"2024-06-13T06:57:42.655020+0000","last_became_active":"2024-06-13T06:55:12.000759+0000","last_became_peered":"2024-06-13T06:55:12.000759+0000","last_unstale":"2024-06-13T06:57:42.655020+0000","last_undegraded":"2024-06-13T06:57:42.655020+0000","last_fullsized":"2024-06-13T06:57:42.655020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:11:35.805338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'8","reported_seq":299,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:25.040779+0000","last_change":"2024-06-13T06:55:29.367538+0000","last_active":"2024-06-13T07:00:25.040779+0000","last_peered":"2024-06-13T07:00:25.040779+0000","last_clean":"2024-06-13T07:00:25.040779+0000","last_became_active":"2024-06-13T06:55:29.367149+0000","last_became_peered":"2024-06-13T06:55:29.367149+0000","last_unstale":"2024-06-13T07:00:25.040779+0000","last_undegraded":"2024-06-13T07:00:25.040779+0000","last_fullsized":"2024-06-13T07:00:25.040779+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:30:55.632799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1459,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.269082+0000","last_change":"2024-06-13T06:54:58.896258+0000","last_active":"2024-06-13T06:58:28.269082+0000","last_peered":"2024-06-13T06:58:28.269082+0000","last_clean":"2024-06-13T06:58:28.269082+0000","last_became_active":"2024-06-13T06:54:58.895904+0000","last_became_peered":"2024-06-13T06:54:58.895904+0000","last_unstale":"2024-06-13T06:58:28.269082+0000","last_undegraded":"2024-06-13T06:58:28.269082+0000","last_fullsized":"2024-06-13T06:58:28.269082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:28:33.615338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:42.654996+0000","last_change":"2024-06-13T06:55:12.018499+0000","last_active":"2024-06-13T06:57:42.654996+0000","last_peered":"2024-06-13T06:57:42.654996+0000","last_clean":"2024-06-13T06:57:42.654996+0000","last_became_active":"2024-06-13T06:55:12.018073+0000","last_became_peered":"2024-06-13T06:55:12.018073+0000","last_unstale":"2024-06-13T06:57:42.654996+0000","last_undegraded":"2024-06-13T06:57:42.654996+0000","last_fullsized":"2024-06-13T06:57:42.654996+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:59:53.206259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:23.429829+0000","last_change":"2024-06-13T06:54:54.339482+0000","last_active":"2024-06-13T06:57:23.429829+0000","last_peered":"2024-06-13T06:57:23.429829+0000","last_clean":"2024-06-13T06:57:23.429829+0000","last_became_active":"2024-06-13T06:54:54.339066+0000","last_became_peered":"2024-06-13T06:54:54.339066+0000","last_unstale":"2024-06-13T06:57:23.429829+0000","last_undegraded":"2024-06-13T06:57:23.429829+0000","last_fullsized":"2024-06-13T06:57:23.429829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:31:04.136530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279253+0000","last_change":"2024-06-13T06:54:58.891154+0000","last_active":"2024-06-13T06:58:28.279253+0000","last_peered":"2024-06-13T06:58:28.279253+0000","last_clean":"2024-06-13T06:58:28.279253+0000","last_became_active":"2024-06-13T06:54:58.890810+0000","last_became_peered":"2024-06-13T06:54:58.890810+0000","last_unstale":"2024-06-13T06:58:28.279253+0000","last_undegraded":"2024-06-13T06:58:28.279253+0000","last_fullsized":"2024-06-13T06:58:28.279253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:14.618233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.836644+0000","last_change":"2024-06-13T06:55:12.025619+0000","last_active":"2024-06-13T06:57:19.836644+0000","last_peered":"2024-06-13T06:57:19.836644+0000","last_clean":"2024-06-13T06:57:19.836644+0000","last_became_active":"2024-06-13T06:55:12.025153+0000","last_became_peered":"2024-06-13T06:55:12.025153+0000","last_unstale":"2024-06-13T06:57:19.836644+0000","last_undegraded":"2024-06-13T06:57:19.836644+0000","last_fullsized":"2024-06-13T06:57:19.836644+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:12.124550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:22.395183+0000","last_change":"2024-06-13T06:54:54.341698+0000","last_active":"2024-06-13T06:57:22.395183+0000","last_peered":"2024-06-13T06:57:22.395183+0000","last_clean":"2024-06-13T06:57:22.395183+0000","last_became_active":"2024-06-13T06:54:54.341113+0000","last_became_peered":"2024-06-13T06:54:54.341113+0000","last_unstale":"2024-06-13T06:57:22.395183+0000","last_undegraded":"2024-06-13T06:57:22.395183+0000","last_fullsized":"2024-06-13T06:57:22.395183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:18:23.368705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279519+0000","last_change":"2024-06-13T06:54:58.913688+0000","last_active":"2024-06-13T06:58:28.279519+0000","last_peered":"2024-06-13T06:58:28.279519+0000","last_clean":"2024-06-13T06:58:28.279519+0000","last_became_active":"2024-06-13T06:54:58.910493+0000","last_became_peered":"2024-06-13T06:54:58.910493+0000","last_unstale":"2024-06-13T06:58:28.279519+0000","last_undegraded":"2024-06-13T06:58:28.279519+0000","last_fullsized":"2024-06-13T06:58:28.279519+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:29:18.081743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.540765+0000","last_change":"2024-06-13T06:55:12.016257+0000","last_active":"2024-06-13T06:57:12.540765+0000","last_peered":"2024-06-13T06:57:12.540765+0000","last_clean":"2024-06-13T06:57:12.540765+0000","last_became_active":"2024-06-13T06:55:12.015594+0000","last_became_peered":"2024-06-13T06:55:12.015594+0000","last_unstale":"2024-06-13T06:57:12.540765+0000","last_undegraded":"2024-06-13T06:57:12.540765+0000","last_fullsized":"2024-06-13T06:57:12.540765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:02:44.642075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542663+0000","last_change":"2024-06-13T06:54:54.330366+0000","last_active":"2024-06-13T06:57:21.542663+0000","last_peered":"2024-06-13T06:57:21.542663+0000","last_clean":"2024-06-13T06:57:21.542663+0000","last_became_active":"2024-06-13T06:54:54.330010+0000","last_became_peered":"2024-06-13T06:54:54.330010+0000","last_unstale":"2024-06-13T06:57:21.542663+0000","last_undegraded":"2024-06-13T06:57:21.542663+0000","last_fullsized":"2024-06-13T06:57:21.542663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:36:01.488766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276547+0000","last_change":"2024-06-13T06:54:58.918076+0000","last_active":"2024-06-13T06:58:28.276547+0000","last_peered":"2024-06-13T06:58:28.276547+0000","last_clean":"2024-06-13T06:58:28.276547+0000","last_became_active":"2024-06-13T06:54:58.917631+0000","last_became_peered":"2024-06-13T06:54:58.917631+0000","last_unstale":"2024-06-13T06:58:28.276547+0000","last_undegraded":"2024-06-13T06:58:28.276547+0000","last_fullsized":"2024-06-13T06:58:28.276547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:31:00.056837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:31.081713+0000","last_change":"2024-06-13T06:56:29.611761+0000","last_active":"2024-06-13T06:57:31.081713+0000","last_peered":"2024-06-13T06:57:31.081713+0000","last_clean":"2024-06-13T06:57:31.081713+0000","last_became_active":"2024-06-13T06:56:29.610385+0000","last_became_peered":"2024-06-13T06:56:29.610385+0000","last_unstale":"2024-06-13T06:57:31.081713+0000","last_undegraded":"2024-06-13T06:57:31.081713+0000","last_fullsized":"2024-06-13T06:57:31.081713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:26:10.678918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'5","reported_seq":282,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:22.033245+0000","last_change":"2024-06-13T06:54:54.335324+0000","last_active":"2024-06-13T07:00:22.033245+0000","last_peered":"2024-06-13T07:00:22.033245+0000","last_clean":"2024-06-13T07:00:22.033245+0000","last_became_active":"2024-06-13T06:54:54.334936+0000","last_became_peered":"2024-06-13T06:54:54.334936+0000","last_unstale":"2024-06-13T07:00:22.033245+0000","last_undegraded":"2024-06-13T07:00:22.033245+0000","last_fullsized":"2024-06-13T07:00:22.033245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:41:52.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":231,"num_read_kb":231,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281723+0000","last_change":"2024-06-13T06:54:58.902146+0000","last_active":"2024-06-13T06:58:28.281723+0000","last_peered":"2024-06-13T06:58:28.281723+0000","last_clean":"2024-06-13T06:58:28.281723+0000","last_became_active":"2024-06-13T06:54:58.901807+0000","last_became_peered":"2024-06-13T06:54:58.901807+0000","last_unstale":"2024-06-13T06:58:28.281723+0000","last_undegraded":"2024-06-13T06:58:28.281723+0000","last_fullsized":"2024-06-13T06:58:28.281723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:59:53.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:35.557156+0000","last_change":"2024-06-13T06:55:12.025418+0000","last_active":"2024-06-13T06:57:35.557156+0000","last_peered":"2024-06-13T06:57:35.557156+0000","last_clean":"2024-06-13T06:57:35.557156+0000","last_became_active":"2024-06-13T06:55:12.024859+0000","last_became_peered":"2024-06-13T06:55:12.024859+0000","last_unstale":"2024-06-13T06:57:35.557156+0000","last_undegraded":"2024-06-13T06:57:35.557156+0000","last_fullsized":"2024-06-13T06:57:35.557156+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:05:17.553169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.670600+0000","last_change":"2024-06-13T06:54:54.326011+0000","last_active":"2024-06-13T06:57:19.670600+0000","last_peered":"2024-06-13T06:57:19.670600+0000","last_clean":"2024-06-13T06:57:19.670600+0000","last_became_active":"2024-06-13T06:54:54.325664+0000","last_became_peered":"2024-06-13T06:54:54.325664+0000","last_unstale":"2024-06-13T06:57:19.670600+0000","last_undegraded":"2024-06-13T06:57:19.670600+0000","last_fullsized":"2024-06-13T06:57:19.670600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:32.973094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301872+0000","last_change":"2024-06-13T06:55:29.383612+0000","last_active":"2024-06-13T06:58:28.301872+0000","last_peered":"2024-06-13T06:58:28.301872+0000","last_clean":"2024-06-13T06:58:28.301872+0000","last_became_active":"2024-06-13T06:55:29.382174+0000","last_became_peered":"2024-06-13T06:55:29.382174+0000","last_unstale":"2024-06-13T06:58:28.301872+0000","last_undegraded":"2024-06-13T06:58:28.301872+0000","last_fullsized":"2024-06-13T06:58:28.301872+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:09:35.178209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:33.288040+0000","last_change":"2024-06-13T06:55:12.018654+0000","last_active":"2024-06-13T06:57:33.288040+0000","last_peered":"2024-06-13T06:57:33.288040+0000","last_clean":"2024-06-13T06:57:33.288040+0000","last_became_active":"2024-06-13T06:55:12.017219+0000","last_became_peered":"2024-06-13T06:55:12.017219+0000","last_unstale":"2024-06-13T06:57:33.288040+0000","last_undegraded":"2024-06-13T06:57:33.288040+0000","last_fullsized":"2024-06-13T06:57:33.288040+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:46:54.161912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542714+0000","last_change":"2024-06-13T06:54:54.632154+0000","last_active":"2024-06-13T06:57:21.542714+0000","last_peered":"2024-06-13T06:57:21.542714+0000","last_clean":"2024-06-13T06:57:21.542714+0000","last_became_active":"2024-06-13T06:54:54.631721+0000","last_became_peered":"2024-06-13T06:54:54.631721+0000","last_unstale":"2024-06-13T06:57:21.542714+0000","last_undegraded":"2024-06-13T06:57:21.542714+0000","last_fullsized":"2024-06-13T06:57:21.542714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:16:10.492144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278187+0000","last_change":"2024-06-13T06:54:58.898961+0000","last_active":"2024-06-13T06:58:28.278187+0000","last_peered":"2024-06-13T06:58:28.278187+0000","last_clean":"2024-06-13T06:58:28.278187+0000","last_became_active":"2024-06-13T06:54:58.898599+0000","last_became_peered":"2024-06-13T06:54:58.898599+0000","last_unstale":"2024-06-13T06:58:28.278187+0000","last_undegraded":"2024-06-13T06:58:28.278187+0000","last_fullsized":"2024-06-13T06:58:28.278187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:21:53.044205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:33.424302+0000","last_change":"2024-06-13T06:56:29.527242+0000","last_active":"2024-06-13T06:56:33.424302+0000","last_peered":"2024-06-13T06:56:33.424302+0000","last_clean":"2024-06-13T06:56:33.424302+0000","last_became_active":"2024-06-13T06:56:29.526816+0000","last_became_peered":"2024-06-13T06:56:29.526816+0000","last_unstale":"2024-06-13T06:56:33.424302+0000","last_undegraded":"2024-06-13T06:56:33.424302+0000","last_fullsized":"2024-06-13T06:56:33.424302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:45:17.770029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'38","reported_seq":572,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:22.934023+0000","last_change":"2024-06-13T06:55:29.376808+0000","last_active":"2024-06-13T07:00:22.934023+0000","last_peered":"2024-06-13T07:00:22.934023+0000","last_clean":"2024-06-13T07:00:22.934023+0000","last_became_active":"2024-06-13T06:55:29.367134+0000","last_became_peered":"2024-06-13T06:55:29.367134+0000","last_unstale":"2024-06-13T07:00:22.934023+0000","last_undegraded":"2024-06-13T07:00:22.934023+0000","last_fullsized":"2024-06-13T07:00:22.934023+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:19:08.784438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63210,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":472,"num_read_kb":299,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.265259+0000","last_change":"2024-06-13T06:54:58.897686+0000","last_active":"2024-06-13T06:58:28.265259+0000","last_peered":"2024-06-13T06:58:28.265259+0000","last_clean":"2024-06-13T06:58:28.265259+0000","last_became_active":"2024-06-13T06:54:58.897313+0000","last_became_peered":"2024-06-13T06:54:58.897313+0000","last_unstale":"2024-06-13T06:58:28.265259+0000","last_undegraded":"2024-06-13T06:58:28.265259+0000","last_fullsized":"2024-06-13T06:58:28.265259+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:54:49.558542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:36.544740+0000","last_change":"2024-06-13T06:55:12.013918+0000","last_active":"2024-06-13T06:57:36.544740+0000","last_peered":"2024-06-13T06:57:36.544740+0000","last_clean":"2024-06-13T06:57:36.544740+0000","last_became_active":"2024-06-13T06:55:12.013522+0000","last_became_peered":"2024-06-13T06:55:12.013522+0000","last_unstale":"2024-06-13T06:57:36.544740+0000","last_undegraded":"2024-06-13T06:57:36.544740+0000","last_fullsized":"2024-06-13T06:57:36.544740+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:51:46.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.652302+0000","last_change":"2024-06-13T06:54:54.328674+0000","last_active":"2024-06-13T06:57:17.652302+0000","last_peered":"2024-06-13T06:57:17.652302+0000","last_clean":"2024-06-13T06:57:17.652302+0000","last_became_active":"2024-06-13T06:54:54.326789+0000","last_became_peered":"2024-06-13T06:54:54.326789+0000","last_unstale":"2024-06-13T06:57:17.652302+0000","last_undegraded":"2024-06-13T06:57:17.652302+0000","last_fullsized":"2024-06-13T06:57:17.652302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:18.280514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293277+0000","last_change":"2024-06-13T06:54:58.918598+0000","last_active":"2024-06-13T06:58:28.293277+0000","last_peered":"2024-06-13T06:58:28.293277+0000","last_clean":"2024-06-13T06:58:28.293277+0000","last_became_active":"2024-06-13T06:54:58.918247+0000","last_became_peered":"2024-06-13T06:54:58.918247+0000","last_unstale":"2024-06-13T06:58:28.293277+0000","last_undegraded":"2024-06-13T06:58:28.293277+0000","last_fullsized":"2024-06-13T06:58:28.293277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:08:56.738806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:34.838358+0000","last_change":"2024-06-13T06:55:12.017074+0000","last_active":"2024-06-13T06:57:34.838358+0000","last_peered":"2024-06-13T06:57:34.838358+0000","last_clean":"2024-06-13T06:57:34.838358+0000","last_became_active":"2024-06-13T06:55:12.016675+0000","last_became_peered":"2024-06-13T06:55:12.016675+0000","last_unstale":"2024-06-13T06:57:34.838358+0000","last_undegraded":"2024-06-13T06:57:34.838358+0000","last_fullsized":"2024-06-13T06:57:34.838358+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:20:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:16.542158+0000","last_change":"2024-06-13T06:54:54.334994+0000","last_active":"2024-06-13T06:57:16.542158+0000","last_peered":"2024-06-13T06:57:16.542158+0000","last_clean":"2024-06-13T06:57:16.542158+0000","last_became_active":"2024-06-13T06:54:54.334647+0000","last_became_peered":"2024-06-13T06:54:54.334647+0000","last_unstale":"2024-06-13T06:57:16.542158+0000","last_undegraded":"2024-06-13T06:57:16.542158+0000","last_fullsized":"2024-06-13T06:57:16.542158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:59:15.317976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276010+0000","last_change":"2024-06-13T06:54:58.899107+0000","last_active":"2024-06-13T06:58:28.276010+0000","last_peered":"2024-06-13T06:58:28.276010+0000","last_clean":"2024-06-13T06:58:28.276010+0000","last_became_active":"2024-06-13T06:54:58.896034+0000","last_became_peered":"2024-06-13T06:54:58.896034+0000","last_unstale":"2024-06-13T06:58:28.276010+0000","last_undegraded":"2024-06-13T06:58:28.276010+0000","last_fullsized":"2024-06-13T06:58:28.276010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:00:01.132565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:40.558064+0000","last_change":"2024-06-13T06:55:12.025681+0000","last_active":"2024-06-13T06:57:40.558064+0000","last_peered":"2024-06-13T06:57:40.558064+0000","last_clean":"2024-06-13T06:57:40.558064+0000","last_became_active":"2024-06-13T06:55:12.025267+0000","last_became_peered":"2024-06-13T06:55:12.025267+0000","last_unstale":"2024-06-13T06:57:40.558064+0000","last_undegraded":"2024-06-13T06:57:40.558064+0000","last_fullsized":"2024-06-13T06:57:40.558064+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:58:23.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":287,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:27.044620+0000","last_change":"2024-06-13T06:54:54.325404+0000","last_active":"2024-06-13T07:00:27.044620+0000","last_peered":"2024-06-13T07:00:27.044620+0000","last_clean":"2024-06-13T07:00:27.044620+0000","last_became_active":"2024-06-13T06:54:54.325024+0000","last_became_peered":"2024-06-13T06:54:54.325024+0000","last_unstale":"2024-06-13T07:00:27.044620+0000","last_undegraded":"2024-06-13T07:00:27.044620+0000","last_fullsized":"2024-06-13T07:00:27.044620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:23.726350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272955+0000","last_change":"2024-06-13T06:54:58.894737+0000","last_active":"2024-06-13T06:58:28.272955+0000","last_peered":"2024-06-13T06:58:28.272955+0000","last_clean":"2024-06-13T06:58:28.272955+0000","last_became_active":"2024-06-13T06:54:58.894224+0000","last_became_peered":"2024-06-13T06:54:58.894224+0000","last_unstale":"2024-06-13T06:58:28.272955+0000","last_undegraded":"2024-06-13T06:58:28.272955+0000","last_fullsized":"2024-06-13T06:58:28.272955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:03.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:37.398002+0000","last_change":"2024-06-13T06:55:12.026587+0000","last_active":"2024-06-13T06:57:37.398002+0000","last_peered":"2024-06-13T06:57:37.398002+0000","last_clean":"2024-06-13T06:57:37.398002+0000","last_became_active":"2024-06-13T06:55:12.026114+0000","last_became_peered":"2024-06-13T06:55:12.026114+0000","last_unstale":"2024-06-13T06:57:37.398002+0000","last_undegraded":"2024-06-13T06:57:37.398002+0000","last_fullsized":"2024-06-13T06:57:37.398002+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:43:09.245641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:15.058741+0000","last_change":"2024-06-13T06:54:54.632069+0000","last_active":"2024-06-13T06:57:15.058741+0000","last_peered":"2024-06-13T06:57:15.058741+0000","last_clean":"2024-06-13T06:57:15.058741+0000","last_became_active":"2024-06-13T06:54:54.631692+0000","last_became_peered":"2024-06-13T06:54:54.631692+0000","last_unstale":"2024-06-13T06:57:15.058741+0000","last_undegraded":"2024-06-13T06:57:15.058741+0000","last_fullsized":"2024-06-13T06:57:15.058741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:07.109507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.265386+0000","last_change":"2024-06-13T06:55:29.366286+0000","last_active":"2024-06-13T06:58:28.265386+0000","last_peered":"2024-06-13T06:58:28.265386+0000","last_clean":"2024-06-13T06:58:28.265386+0000","last_became_active":"2024-06-13T06:55:29.365854+0000","last_became_peered":"2024-06-13T06:55:29.365854+0000","last_unstale":"2024-06-13T06:58:28.265386+0000","last_undegraded":"2024-06-13T06:58:28.265386+0000","last_fullsized":"2024-06-13T06:58:28.265386+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:04.209244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542690+0000","last_change":"2024-06-13T06:55:12.021621+0000","last_active":"2024-06-13T06:57:21.542690+0000","last_peered":"2024-06-13T06:57:21.542690+0000","last_clean":"2024-06-13T06:57:21.542690+0000","last_became_active":"2024-06-13T06:55:12.021150+0000","last_became_peered":"2024-06-13T06:55:12.021150+0000","last_unstale":"2024-06-13T06:57:21.542690+0000","last_undegraded":"2024-06-13T06:57:21.542690+0000","last_fullsized":"2024-06-13T06:57:21.542690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:48.318918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:44.839364+0000","last_change":"2024-06-13T06:55:29.369972+0000","last_active":"2024-06-13T06:57:44.839364+0000","last_peered":"2024-06-13T06:57:44.839364+0000","last_clean":"2024-06-13T06:57:44.839364+0000","last_became_active":"2024-06-13T06:55:29.366390+0000","last_became_peered":"2024-06-13T06:55:29.366390+0000","last_unstale":"2024-06-13T06:57:44.839364+0000","last_undegraded":"2024-06-13T06:57:44.839364+0000","last_fullsized":"2024-06-13T06:57:44.839364+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:18.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271886+0000","last_change":"2024-06-13T06:54:58.891309+0000","last_active":"2024-06-13T06:58:28.271886+0000","last_peered":"2024-06-13T06:58:28.271886+0000","last_clean":"2024-06-13T06:58:28.271886+0000","last_became_active":"2024-06-13T06:54:58.890972+0000","last_became_peered":"2024-06-13T06:54:58.890972+0000","last_unstale":"2024-06-13T06:58:28.271886+0000","last_undegraded":"2024-06-13T06:58:28.271886+0000","last_fullsized":"2024-06-13T06:58:28.271886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:41.240061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:20.059185+0000","last_change":"2024-06-13T06:56:29.610713+0000","last_active":"2024-06-13T06:57:20.059185+0000","last_peered":"2024-06-13T06:57:20.059185+0000","last_clean":"2024-06-13T06:57:20.059185+0000","last_became_active":"2024-06-13T06:56:29.610139+0000","last_became_peered":"2024-06-13T06:56:29.610139+0000","last_unstale":"2024-06-13T06:57:20.059185+0000","last_undegraded":"2024-06-13T06:57:20.059185+0000","last_fullsized":"2024-06-13T06:57:20.059185+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:14:36.601243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:00.551177+0000","last_change":"2024-06-13T06:54:54.633893+0000","last_active":"2024-06-13T06:57:00.551177+0000","last_peered":"2024-06-13T06:57:00.551177+0000","last_clean":"2024-06-13T06:57:00.551177+0000","last_became_active":"2024-06-13T06:54:54.632946+0000","last_became_peered":"2024-06-13T06:54:54.632946+0000","last_unstale":"2024-06-13T06:57:00.551177+0000","last_undegraded":"2024-06-13T06:57:00.551177+0000","last_fullsized":"2024-06-13T06:57:00.551177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:16:54.812477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279866+0000","last_change":"2024-06-13T06:54:58.907182+0000","last_active":"2024-06-13T06:58:28.279866+0000","last_peered":"2024-06-13T06:58:28.279866+0000","last_clean":"2024-06-13T06:58:28.279866+0000","last_became_active":"2024-06-13T06:54:58.906824+0000","last_became_peered":"2024-06-13T06:54:58.906824+0000","last_unstale":"2024-06-13T06:58:28.279866+0000","last_undegraded":"2024-06-13T06:58:28.279866+0000","last_fullsized":"2024-06-13T06:58:28.279866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:34:49.626991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.394303+0000","last_change":"2024-06-13T06:55:12.014803+0000","last_active":"2024-06-13T06:57:17.394303+0000","last_peered":"2024-06-13T06:57:17.394303+0000","last_clean":"2024-06-13T06:57:17.394303+0000","last_became_active":"2024-06-13T06:55:12.014398+0000","last_became_peered":"2024-06-13T06:55:12.014398+0000","last_unstale":"2024-06-13T06:57:17.394303+0000","last_undegraded":"2024-06-13T06:57:17.394303+0000","last_fullsized":"2024-06-13T06:57:17.394303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:07:44.960446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:59.668362+0000","last_change":"2024-06-13T06:54:54.339056+0000","last_active":"2024-06-13T06:56:59.668362+0000","last_peered":"2024-06-13T06:56:59.668362+0000","last_clean":"2024-06-13T06:56:59.668362+0000","last_became_active":"2024-06-13T06:54:54.338683+0000","last_became_peered":"2024-06-13T06:54:54.338683+0000","last_unstale":"2024-06-13T06:56:59.668362+0000","last_undegraded":"2024-06-13T06:56:59.668362+0000","last_fullsized":"2024-06-13T06:56:59.668362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:05:17.060471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.297043+0000","last_change":"2024-06-13T06:54:58.919843+0000","last_active":"2024-06-13T06:58:28.297043+0000","last_peered":"2024-06-13T06:58:28.297043+0000","last_clean":"2024-06-13T06:58:28.297043+0000","last_became_active":"2024-06-13T06:54:58.919405+0000","last_became_peered":"2024-06-13T06:54:58.919405+0000","last_unstale":"2024-06-13T06:58:28.297043+0000","last_undegraded":"2024-06-13T06:58:28.297043+0000","last_fullsized":"2024-06-13T06:58:28.297043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:00:20.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:22.395386+0000","last_change":"2024-06-13T06:55:12.097651+0000","last_active":"2024-06-13T06:57:22.395386+0000","last_peered":"2024-06-13T06:57:22.395386+0000","last_clean":"2024-06-13T06:57:22.395386+0000","last_became_active":"2024-06-13T06:55:12.095644+0000","last_became_peered":"2024-06-13T06:55:12.095644+0000","last_unstale":"2024-06-13T06:57:22.395386+0000","last_undegraded":"2024-06-13T06:57:22.395386+0000","last_fullsized":"2024-06-13T06:57:22.395386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:09:54.328428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.266581+0000","last_change":"2024-06-13T06:54:58.891187+0000","last_active":"2024-06-13T06:58:28.266581+0000","last_peered":"2024-06-13T06:58:28.266581+0000","last_clean":"2024-06-13T06:58:28.266581+0000","last_became_active":"2024-06-13T06:54:58.890848+0000","last_became_peered":"2024-06-13T06:54:58.890848+0000","last_unstale":"2024-06-13T06:58:28.266581+0000","last_undegraded":"2024-06-13T06:58:28.266581+0000","last_fullsized":"2024-06-13T06:58:28.266581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:13:31.709650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":285,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:26.041470+0000","last_change":"2024-06-13T06:54:54.631482+0000","last_active":"2024-06-13T07:00:26.041470+0000","last_peered":"2024-06-13T07:00:26.041470+0000","last_clean":"2024-06-13T07:00:26.041470+0000","last_became_active":"2024-06-13T06:54:54.630818+0000","last_became_peered":"2024-06-13T06:54:54.630818+0000","last_unstale":"2024-06-13T07:00:26.041470+0000","last_undegraded":"2024-06-13T07:00:26.041470+0000","last_fullsized":"2024-06-13T07:00:26.041470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:34:18.801727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.670568+0000","last_change":"2024-06-13T06:55:12.018856+0000","last_active":"2024-06-13T06:57:19.670568+0000","last_peered":"2024-06-13T06:57:19.670568+0000","last_clean":"2024-06-13T06:57:19.670568+0000","last_became_active":"2024-06-13T06:55:12.018488+0000","last_became_peered":"2024-06-13T06:55:12.018488+0000","last_unstale":"2024-06-13T06:57:19.670568+0000","last_undegraded":"2024-06-13T06:57:19.670568+0000","last_fullsized":"2024-06-13T06:57:19.670568+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:14.973197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":25,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281859+0000","last_change":"2024-06-13T06:55:29.377203+0000","last_active":"2024-06-13T06:58:28.281859+0000","last_peered":"2024-06-13T06:58:28.281859+0000","last_clean":"2024-06-13T06:58:28.281859+0000","last_became_active":"2024-06-13T06:55:29.375263+0000","last_became_peered":"2024-06-13T06:55:29.375263+0000","last_unstale":"2024-06-13T06:58:28.281859+0000","last_undegraded":"2024-06-13T06:58:28.281859+0000","last_fullsized":"2024-06-13T06:58:28.281859+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:06:07.935172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:03.284069+0000","last_change":"2024-06-13T06:54:54.335319+0000","last_active":"2024-06-13T06:57:03.284069+0000","last_peered":"2024-06-13T06:57:03.284069+0000","last_clean":"2024-06-13T06:57:03.284069+0000","last_became_active":"2024-06-13T06:54:54.334896+0000","last_became_peered":"2024-06-13T06:54:54.334896+0000","last_unstale":"2024-06-13T06:57:03.284069+0000","last_undegraded":"2024-06-13T06:57:03.284069+0000","last_fullsized":"2024-06-13T06:57:03.284069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:40.331535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:49.673756+0000","last_change":"2024-06-13T06:53:16.968830+0000","last_active":"2024-06-13T06:57:49.673756+0000","last_peered":"2024-06-13T06:57:49.673756+0000","last_clean":"2024-06-13T06:57:49.673756+0000","last_became_active":"2024-06-13T06:53:16.968361+0000","last_became_peered":"2024-06-13T06:53:16.968361+0000","last_unstale":"2024-06-13T06:57:49.673756+0000","last_undegraded":"2024-06-13T06:57:49.673756+0000","last_fullsized":"2024-06-13T06:57:49.673756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_clean_scrub_stamp":"2024-06-13T06:49:18.858540+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:32:26.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542736+0000","last_change":"2024-06-13T06:56:29.602124+0000","last_active":"2024-06-13T06:57:21.542736+0000","last_peered":"2024-06-13T06:57:21.542736+0000","last_clean":"2024-06-13T06:57:21.542736+0000","last_became_active":"2024-06-13T06:56:29.601650+0000","last_became_peered":"2024-06-13T06:56:29.601650+0000","last_unstale":"2024-06-13T06:57:21.542736+0000","last_undegraded":"2024-06-13T06:57:21.542736+0000","last_fullsized":"2024-06-13T06:57:21.542736+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:37:23.903614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.297076+0000","last_change":"2024-06-13T06:54:58.898930+0000","last_active":"2024-06-13T06:58:28.297076+0000","last_peered":"2024-06-13T06:58:28.297076+0000","last_clean":"2024-06-13T06:58:28.297076+0000","last_became_active":"2024-06-13T06:54:58.898384+0000","last_became_peered":"2024-06-13T06:54:58.898384+0000","last_unstale":"2024-06-13T06:58:28.297076+0000","last_undegraded":"2024-06-13T06:58:28.297076+0000","last_fullsized":"2024-06-13T06:58:28.297076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:06:02.210336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:54.667753+0000","last_change":"2024-06-13T06:54:54.328776+0000","last_active":"2024-06-13T06:56:54.667753+0000","last_peered":"2024-06-13T06:56:54.667753+0000","last_clean":"2024-06-13T06:56:54.667753+0000","last_became_active":"2024-06-13T06:54:54.327773+0000","last_became_peered":"2024-06-13T06:54:54.327773+0000","last_unstale":"2024-06-13T06:56:54.667753+0000","last_undegraded":"2024-06-13T06:56:54.667753+0000","last_fullsized":"2024-06-13T06:56:54.667753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:23:02.240774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.836611+0000","last_change":"2024-06-13T06:55:12.013862+0000","last_active":"2024-06-13T06:57:19.836611+0000","last_peered":"2024-06-13T06:57:19.836611+0000","last_clean":"2024-06-13T06:57:19.836611+0000","last_became_active":"2024-06-13T06:55:12.013389+0000","last_became_peered":"2024-06-13T06:55:12.013389+0000","last_unstale":"2024-06-13T06:57:19.836611+0000","last_undegraded":"2024-06-13T06:57:19.836611+0000","last_fullsized":"2024-06-13T06:57:19.836611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:49.701202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558954+0000","last_change":"2024-06-13T06:56:29.539862+0000","last_active":"2024-06-13T06:56:32.558954+0000","last_peered":"2024-06-13T06:56:32.558954+0000","last_clean":"2024-06-13T06:56:32.558954+0000","last_became_active":"2024-06-13T06:56:29.536109+0000","last_became_peered":"2024-06-13T06:56:29.536109+0000","last_unstale":"2024-06-13T06:56:32.558954+0000","last_undegraded":"2024-06-13T06:56:32.558954+0000","last_fullsized":"2024-06-13T06:56:32.558954+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:23.307850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.288243+0000","last_change":"2024-06-13T06:54:58.901773+0000","last_active":"2024-06-13T06:58:28.288243+0000","last_peered":"2024-06-13T06:58:28.288243+0000","last_clean":"2024-06-13T06:58:28.288243+0000","last_became_active":"2024-06-13T06:54:58.901168+0000","last_became_peered":"2024-06-13T06:54:58.901168+0000","last_unstale":"2024-06-13T06:58:28.288243+0000","last_undegraded":"2024-06-13T06:58:28.288243+0000","last_fullsized":"2024-06-13T06:58:28.288243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:57:15.848785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:11.078009+0000","last_change":"2024-06-13T06:55:12.095890+0000","last_active":"2024-06-13T06:57:11.078009+0000","last_peered":"2024-06-13T06:57:11.078009+0000","last_clean":"2024-06-13T06:57:11.078009+0000","last_became_active":"2024-06-13T06:55:12.095397+0000","last_became_peered":"2024-06-13T06:55:12.095397+0000","last_unstale":"2024-06-13T06:57:11.078009+0000","last_undegraded":"2024-06-13T06:57:11.078009+0000","last_fullsized":"2024-06-13T06:57:11.078009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:56:18.211036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'13","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:59:15.573034+0000","last_change":"2024-06-13T06:54:54.337946+0000","last_active":"2024-06-13T06:59:15.573034+0000","last_peered":"2024-06-13T06:59:15.573034+0000","last_clean":"2024-06-13T06:59:15.573034+0000","last_became_active":"2024-06-13T06:54:54.337028+0000","last_became_peered":"2024-06-13T06:54:54.337028+0000","last_unstale":"2024-06-13T06:59:15.573034+0000","last_undegraded":"2024-06-13T06:59:15.573034+0000","last_fullsized":"2024-06-13T06:59:15.573034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:08:52.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275687+0000","last_change":"2024-06-13T06:54:58.895110+0000","last_active":"2024-06-13T06:58:28.275687+0000","last_peered":"2024-06-13T06:58:28.275687+0000","last_clean":"2024-06-13T06:58:28.275687+0000","last_became_active":"2024-06-13T06:54:58.893481+0000","last_became_peered":"2024-06-13T06:54:58.893481+0000","last_unstale":"2024-06-13T06:58:28.275687+0000","last_undegraded":"2024-06-13T06:58:28.275687+0000","last_fullsized":"2024-06-13T06:58:28.275687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:54.932110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.652273+0000","last_change":"2024-06-13T06:55:11.997712+0000","last_active":"2024-06-13T06:57:17.652273+0000","last_peered":"2024-06-13T06:57:17.652273+0000","last_clean":"2024-06-13T06:57:17.652273+0000","last_became_active":"2024-06-13T06:55:11.997193+0000","last_became_peered":"2024-06-13T06:55:11.997193+0000","last_unstale":"2024-06-13T06:57:17.652273+0000","last_undegraded":"2024-06-13T06:57:17.652273+0000","last_fullsized":"2024-06-13T06:57:17.652273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:15.168070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:08.284826+0000","last_change":"2024-06-13T06:54:54.520121+0000","last_active":"2024-06-13T06:57:08.284826+0000","last_peered":"2024-06-13T06:57:08.284826+0000","last_clean":"2024-06-13T06:57:08.284826+0000","last_became_active":"2024-06-13T06:54:54.519723+0000","last_became_peered":"2024-06-13T06:54:54.519723+0000","last_unstale":"2024-06-13T06:57:08.284826+0000","last_undegraded":"2024-06-13T06:57:08.284826+0000","last_fullsized":"2024-06-13T06:57:08.284826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:19:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304745+0000","last_change":"2024-06-13T06:54:58.923626+0000","last_active":"2024-06-13T06:58:28.304745+0000","last_peered":"2024-06-13T06:58:28.304745+0000","last_clean":"2024-06-13T06:58:28.304745+0000","last_became_active":"2024-06-13T06:54:58.923192+0000","last_became_peered":"2024-06-13T06:54:58.923192+0000","last_unstale":"2024-06-13T06:58:28.304745+0000","last_undegraded":"2024-06-13T06:58:28.304745+0000","last_fullsized":"2024-06-13T06:58:28.304745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:26:13.231549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287371+0000","last_change":"2024-06-13T06:55:12.096244+0000","last_active":"2024-06-13T06:57:28.287371+0000","last_peered":"2024-06-13T06:57:28.287371+0000","last_clean":"2024-06-13T06:57:28.287371+0000","last_became_active":"2024-06-13T06:55:12.095758+0000","last_became_peered":"2024-06-13T06:55:12.095758+0000","last_unstale":"2024-06-13T06:57:28.287371+0000","last_undegraded":"2024-06-13T06:57:28.287371+0000","last_fullsized":"2024-06-13T06:57:28.287371+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:38.122282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:08.168621+0000","last_change":"2024-06-13T06:54:54.630488+0000","last_active":"2024-06-13T06:57:08.168621+0000","last_peered":"2024-06-13T06:57:08.168621+0000","last_clean":"2024-06-13T06:57:08.168621+0000","last_became_active":"2024-06-13T06:54:54.630224+0000","last_became_peered":"2024-06-13T06:54:54.630224+0000","last_unstale":"2024-06-13T06:57:08.168621+0000","last_undegraded":"2024-06-13T06:57:08.168621+0000","last_fullsized":"2024-06-13T06:57:08.168621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:33:19.107026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.263285+0000","last_change":"2024-06-13T06:54:58.894466+0000","last_active":"2024-06-13T06:58:28.263285+0000","last_peered":"2024-06-13T06:58:28.263285+0000","last_clean":"2024-06-13T06:58:28.263285+0000","last_became_active":"2024-06-13T06:54:58.894034+0000","last_became_peered":"2024-06-13T06:54:58.894034+0000","last_unstale":"2024-06-13T06:58:28.263285+0000","last_undegraded":"2024-06-13T06:58:28.263285+0000","last_fullsized":"2024-06-13T06:58:28.263285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:44.051565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:30.556536+0000","last_change":"2024-06-13T06:55:12.022078+0000","last_active":"2024-06-13T06:57:30.556536+0000","last_peered":"2024-06-13T06:57:30.556536+0000","last_clean":"2024-06-13T06:57:30.556536+0000","last_became_active":"2024-06-13T06:55:12.021652+0000","last_became_peered":"2024-06-13T06:55:12.021652+0000","last_unstale":"2024-06-13T06:57:30.556536+0000","last_undegraded":"2024-06-13T06:57:30.556536+0000","last_fullsized":"2024-06-13T06:57:30.556536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:25:57.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:06.077272+0000","last_change":"2024-06-13T06:54:54.336757+0000","last_active":"2024-06-13T06:57:06.077272+0000","last_peered":"2024-06-13T06:57:06.077272+0000","last_clean":"2024-06-13T06:57:06.077272+0000","last_became_active":"2024-06-13T06:54:54.336012+0000","last_became_peered":"2024-06-13T06:54:54.336012+0000","last_unstale":"2024-06-13T06:57:06.077272+0000","last_undegraded":"2024-06-13T06:57:06.077272+0000","last_fullsized":"2024-06-13T06:57:06.077272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:52:21.227464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292857+0000","last_change":"2024-06-13T06:54:58.919631+0000","last_active":"2024-06-13T06:58:28.292857+0000","last_peered":"2024-06-13T06:58:28.292857+0000","last_clean":"2024-06-13T06:58:28.292857+0000","last_became_active":"2024-06-13T06:54:58.919274+0000","last_became_peered":"2024-06-13T06:54:58.919274+0000","last_unstale":"2024-06-13T06:58:28.292857+0000","last_undegraded":"2024-06-13T06:58:28.292857+0000","last_fullsized":"2024-06-13T06:58:28.292857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:22.453797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287343+0000","last_change":"2024-06-13T06:55:12.017338+0000","last_active":"2024-06-13T06:57:28.287343+0000","last_peered":"2024-06-13T06:57:28.287343+0000","last_clean":"2024-06-13T06:57:28.287343+0000","last_became_active":"2024-06-13T06:55:12.016841+0000","last_became_peered":"2024-06-13T06:55:12.016841+0000","last_unstale":"2024-06-13T06:57:28.287343+0000","last_undegraded":"2024-06-13T06:57:28.287343+0000","last_fullsized":"2024-06-13T06:57:28.287343+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:40:13.525892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:07.651040+0000","last_change":"2024-06-13T06:54:54.328808+0000","last_active":"2024-06-13T06:57:07.651040+0000","last_peered":"2024-06-13T06:57:07.651040+0000","last_clean":"2024-06-13T06:57:07.651040+0000","last_became_active":"2024-06-13T06:54:54.326982+0000","last_became_peered":"2024-06-13T06:54:54.326982+0000","last_unstale":"2024-06-13T06:57:07.651040+0000","last_undegraded":"2024-06-13T06:57:07.651040+0000","last_fullsized":"2024-06-13T06:57:07.651040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:22:53.854275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.303609+0000","last_change":"2024-06-13T06:54:58.925728+0000","last_active":"2024-06-13T06:58:28.303609+0000","last_peered":"2024-06-13T06:58:28.303609+0000","last_clean":"2024-06-13T06:58:28.303609+0000","last_became_active":"2024-06-13T06:54:58.925407+0000","last_became_peered":"2024-06-13T06:54:58.925407+0000","last_unstale":"2024-06-13T06:58:28.303609+0000","last_undegraded":"2024-06-13T06:58:28.303609+0000","last_fullsized":"2024-06-13T06:58:28.303609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:38.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287363+0000","last_change":"2024-06-13T06:55:12.095733+0000","last_active":"2024-06-13T06:57:28.287363+0000","last_peered":"2024-06-13T06:57:28.287363+0000","last_clean":"2024-06-13T06:57:28.287363+0000","last_became_active":"2024-06-13T06:55:12.095064+0000","last_became_peered":"2024-06-13T06:55:12.095064+0000","last_unstale":"2024-06-13T06:57:28.287363+0000","last_undegraded":"2024-06-13T06:57:28.287363+0000","last_fullsized":"2024-06-13T06:57:28.287363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:26:05.486701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:10.058315+0000","last_change":"2024-06-13T06:54:54.331448+0000","last_active":"2024-06-13T06:57:10.058315+0000","last_peered":"2024-06-13T06:57:10.058315+0000","last_clean":"2024-06-13T06:57:10.058315+0000","last_became_active":"2024-06-13T06:54:54.330964+0000","last_became_peered":"2024-06-13T06:54:54.330964+0000","last_unstale":"2024-06-13T06:57:10.058315+0000","last_undegraded":"2024-06-13T06:57:10.058315+0000","last_fullsized":"2024-06-13T06:57:10.058315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:23:12.211412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282119+0000","last_change":"2024-06-13T06:54:58.899112+0000","last_active":"2024-06-13T06:58:28.282119+0000","last_peered":"2024-06-13T06:58:28.282119+0000","last_clean":"2024-06-13T06:58:28.282119+0000","last_became_active":"2024-06-13T06:54:58.898742+0000","last_became_peered":"2024-06-13T06:54:58.898742+0000","last_unstale":"2024-06-13T06:58:28.282119+0000","last_undegraded":"2024-06-13T06:58:28.282119+0000","last_fullsized":"2024-06-13T06:58:28.282119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:50:12.983287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:23.170398+0000","last_change":"2024-06-13T06:55:12.014149+0000","last_active":"2024-06-13T06:57:23.170398+0000","last_peered":"2024-06-13T06:57:23.170398+0000","last_clean":"2024-06-13T06:57:23.170398+0000","last_became_active":"2024-06-13T06:55:12.013454+0000","last_became_peered":"2024-06-13T06:55:12.013454+0000","last_unstale":"2024-06-13T06:57:23.170398+0000","last_undegraded":"2024-06-13T06:57:23.170398+0000","last_fullsized":"2024-06-13T06:57:23.170398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:13:39.368540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.651570+0000","last_change":"2024-06-13T06:54:54.633268+0000","last_active":"2024-06-13T06:57:12.651570+0000","last_peered":"2024-06-13T06:57:12.651570+0000","last_clean":"2024-06-13T06:57:12.651570+0000","last_became_active":"2024-06-13T06:54:54.632840+0000","last_became_peered":"2024-06-13T06:54:54.632840+0000","last_unstale":"2024-06-13T06:57:12.651570+0000","last_undegraded":"2024-06-13T06:57:12.651570+0000","last_fullsized":"2024-06-13T06:57:12.651570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:21.653727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281941+0000","last_change":"2024-06-13T06:54:58.893417+0000","last_active":"2024-06-13T06:58:28.281941+0000","last_peered":"2024-06-13T06:58:28.281941+0000","last_clean":"2024-06-13T06:58:28.281941+0000","last_became_active":"2024-06-13T06:54:58.892381+0000","last_became_peered":"2024-06-13T06:54:58.892381+0000","last_unstale":"2024-06-13T06:58:28.281941+0000","last_undegraded":"2024-06-13T06:58:28.281941+0000","last_fullsized":"2024-06-13T06:58:28.281941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:07.979654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:27.543545+0000","last_change":"2024-06-13T06:55:12.018336+0000","last_active":"2024-06-13T06:57:27.543545+0000","last_peered":"2024-06-13T06:57:27.543545+0000","last_clean":"2024-06-13T06:57:27.543545+0000","last_became_active":"2024-06-13T06:55:12.017861+0000","last_became_peered":"2024-06-13T06:55:12.017861+0000","last_unstale":"2024-06-13T06:57:27.543545+0000","last_undegraded":"2024-06-13T06:57:27.543545+0000","last_fullsized":"2024-06-13T06:57:27.543545+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:41.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:11.541680+0000","last_change":"2024-06-13T06:54:54.333036+0000","last_active":"2024-06-13T06:57:11.541680+0000","last_peered":"2024-06-13T06:57:11.541680+0000","last_clean":"2024-06-13T06:57:11.541680+0000","last_became_active":"2024-06-13T06:54:54.332642+0000","last_became_peered":"2024-06-13T06:54:54.332642+0000","last_unstale":"2024-06-13T06:57:11.541680+0000","last_undegraded":"2024-06-13T06:57:11.541680+0000","last_fullsized":"2024-06-13T06:57:11.541680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:06:06.809024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279872+0000","last_change":"2024-06-13T06:54:58.881894+0000","last_active":"2024-06-13T06:58:28.279872+0000","last_peered":"2024-06-13T06:58:28.279872+0000","last_clean":"2024-06-13T06:58:28.279872+0000","last_became_active":"2024-06-13T06:54:58.881591+0000","last_became_peered":"2024-06-13T06:54:58.881591+0000","last_unstale":"2024-06-13T06:58:28.279872+0000","last_undegraded":"2024-06-13T06:58:28.279872+0000","last_fullsized":"2024-06-13T06:58:28.279872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:49:00.944212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542642+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:57:21.542642+0000","last_peered":"2024-06-13T06:57:21.542642+0000","last_clean":"2024-06-13T06:57:21.542642+0000","last_became_active":"2024-06-13T06:55:12.000748+0000","last_became_peered":"2024-06-13T06:55:12.000748+0000","last_unstale":"2024-06-13T06:57:21.542642+0000","last_undegraded":"2024-06-13T06:57:21.542642+0000","last_fullsized":"2024-06-13T06:57:21.542642+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:22.035807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.393505+0000","last_change":"2024-06-13T06:54:54.342724+0000","last_active":"2024-06-13T06:57:12.393505+0000","last_peered":"2024-06-13T06:57:12.393505+0000","last_clean":"2024-06-13T06:57:12.393505+0000","last_became_active":"2024-06-13T06:54:54.342267+0000","last_became_peered":"2024-06-13T06:54:54.342267+0000","last_unstale":"2024-06-13T06:57:12.393505+0000","last_undegraded":"2024-06-13T06:57:12.393505+0000","last_fullsized":"2024-06-13T06:57:12.393505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:27:53.039202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271540+0000","last_change":"2024-06-13T06:54:58.892365+0000","last_active":"2024-06-13T06:58:28.271540+0000","last_peered":"2024-06-13T06:58:28.271540+0000","last_clean":"2024-06-13T06:58:28.271540+0000","last_became_active":"2024-06-13T06:54:58.889395+0000","last_became_peered":"2024-06-13T06:54:58.889395+0000","last_unstale":"2024-06-13T06:58:28.271540+0000","last_undegraded":"2024-06-13T06:58:28.271540+0000","last_fullsized":"2024-06-13T06:58:28.271540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:55.798293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272280+0000","last_change":"2024-06-13T06:54:58.888570+0000","last_active":"2024-06-13T06:58:28.272280+0000","last_peered":"2024-06-13T06:58:28.272280+0000","last_clean":"2024-06-13T06:58:28.272280+0000","last_became_active":"2024-06-13T06:54:58.888180+0000","last_became_peered":"2024-06-13T06:54:58.888180+0000","last_unstale":"2024-06-13T06:58:28.272280+0000","last_undegraded":"2024-06-13T06:58:28.272280+0000","last_fullsized":"2024-06-13T06:58:28.272280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:46:42.537790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271099+0000","last_change":"2024-06-13T06:54:58.893476+0000","last_active":"2024-06-13T06:58:28.271099+0000","last_peered":"2024-06-13T06:58:28.271099+0000","last_clean":"2024-06-13T06:58:28.271099+0000","last_became_active":"2024-06-13T06:54:58.893021+0000","last_became_peered":"2024-06-13T06:54:58.893021+0000","last_unstale":"2024-06-13T06:58:28.271099+0000","last_undegraded":"2024-06-13T06:58:28.271099+0000","last_fullsized":"2024-06-13T06:58:28.271099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:38.739070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301611+0000","last_change":"2024-06-13T06:54:58.911042+0000","last_active":"2024-06-13T06:58:28.301611+0000","last_peered":"2024-06-13T06:58:28.301611+0000","last_clean":"2024-06-13T06:58:28.301611+0000","last_became_active":"2024-06-13T06:54:58.910641+0000","last_became_peered":"2024-06-13T06:54:58.910641+0000","last_unstale":"2024-06-13T06:58:28.301611+0000","last_undegraded":"2024-06-13T06:58:28.301611+0000","last_fullsized":"2024-06-13T06:58:28.301611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:21:39.467223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305071+0000","last_change":"2024-06-13T06:55:29.382730+0000","last_active":"2024-06-13T06:58:28.305071+0000","last_peered":"2024-06-13T06:58:28.305071+0000","last_clean":"2024-06-13T06:58:28.305071+0000","last_became_active":"2024-06-13T06:55:29.382162+0000","last_became_peered":"2024-06-13T06:55:29.382162+0000","last_unstale":"2024-06-13T06:58:28.305071+0000","last_undegraded":"2024-06-13T06:58:28.305071+0000","last_fullsized":"2024-06-13T06:58:28.305071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:05.486414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.303120+0000","last_change":"2024-06-13T06:55:29.385380+0000","last_active":"2024-06-13T06:58:28.303120+0000","last_peered":"2024-06-13T06:58:28.303120+0000","last_clean":"2024-06-13T06:58:28.303120+0000","last_became_active":"2024-06-13T06:55:29.384855+0000","last_became_peered":"2024-06-13T06:55:29.384855+0000","last_unstale":"2024-06-13T06:58:28.303120+0000","last_undegraded":"2024-06-13T06:58:28.303120+0000","last_fullsized":"2024-06-13T06:58:28.303120+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:31:01.945632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271653+0000","last_change":"2024-06-13T06:54:58.889128+0000","last_active":"2024-06-13T06:58:28.271653+0000","last_peered":"2024-06-13T06:58:28.271653+0000","last_clean":"2024-06-13T06:58:28.271653+0000","last_became_active":"2024-06-13T06:54:58.888606+0000","last_became_peered":"2024-06-13T06:54:58.888606+0000","last_unstale":"2024-06-13T06:58:28.271653+0000","last_undegraded":"2024-06-13T06:58:28.271653+0000","last_fullsized":"2024-06-13T06:58:28.271653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:53:43.762968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276436+0000","last_change":"2024-06-13T06:54:58.896098+0000","last_active":"2024-06-13T06:58:28.276436+0000","last_peered":"2024-06-13T06:58:28.276436+0000","last_clean":"2024-06-13T06:58:28.276436+0000","last_became_active":"2024-06-13T06:54:58.895722+0000","last_became_peered":"2024-06-13T06:54:58.895722+0000","last_unstale":"2024-06-13T06:58:28.276436+0000","last_undegraded":"2024-06-13T06:58:28.276436+0000","last_fullsized":"2024-06-13T06:58:28.276436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:18:14.421363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279821+0000","last_change":"2024-06-13T06:54:58.905590+0000","last_active":"2024-06-13T06:58:28.279821+0000","last_peered":"2024-06-13T06:58:28.279821+0000","last_clean":"2024-06-13T06:58:28.279821+0000","last_became_active":"2024-06-13T06:54:58.905146+0000","last_became_peered":"2024-06-13T06:54:58.905146+0000","last_unstale":"2024-06-13T06:58:28.279821+0000","last_undegraded":"2024-06-13T06:58:28.279821+0000","last_fullsized":"2024-06-13T06:58:28.279821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:42:08.010906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278426+0000","last_change":"2024-06-13T06:54:58.898037+0000","last_active":"2024-06-13T06:58:28.278426+0000","last_peered":"2024-06-13T06:58:28.278426+0000","last_clean":"2024-06-13T06:58:28.278426+0000","last_became_active":"2024-06-13T06:54:58.897692+0000","last_became_peered":"2024-06-13T06:54:58.897692+0000","last_unstale":"2024-06-13T06:58:28.278426+0000","last_undegraded":"2024-06-13T06:58:28.278426+0000","last_fullsized":"2024-06-13T06:58:28.278426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:44:30.018192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275652+0000","last_change":"2024-06-13T06:54:58.903441+0000","last_active":"2024-06-13T06:58:28.275652+0000","last_peered":"2024-06-13T06:58:28.275652+0000","last_clean":"2024-06-13T06:58:28.275652+0000","last_became_active":"2024-06-13T06:54:58.893598+0000","last_became_peered":"2024-06-13T06:54:58.893598+0000","last_unstale":"2024-06-13T06:58:28.275652+0000","last_undegraded":"2024-06-13T06:58:28.275652+0000","last_fullsized":"2024-06-13T06:58:28.275652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:38:29.968619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282488+0000","last_change":"2024-06-13T06:54:58.918594+0000","last_active":"2024-06-13T06:58:28.282488+0000","last_peered":"2024-06-13T06:58:28.282488+0000","last_clean":"2024-06-13T06:58:28.282488+0000","last_became_active":"2024-06-13T06:54:58.918255+0000","last_became_peered":"2024-06-13T06:54:58.918255+0000","last_unstale":"2024-06-13T06:58:28.282488+0000","last_undegraded":"2024-06-13T06:58:28.282488+0000","last_fullsized":"2024-06-13T06:58:28.282488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:39:48.527707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305484+0000","last_change":"2024-06-13T06:54:58.926253+0000","last_active":"2024-06-13T06:58:28.305484+0000","last_peered":"2024-06-13T06:58:28.305484+0000","last_clean":"2024-06-13T06:58:28.305484+0000","last_became_active":"2024-06-13T06:54:58.924611+0000","last_became_peered":"2024-06-13T06:54:58.924611+0000","last_unstale":"2024-06-13T06:58:28.305484+0000","last_undegraded":"2024-06-13T06:58:28.305484+0000","last_fullsized":"2024-06-13T06:58:28.305484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:29:23.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267286+0000","last_change":"2024-06-13T06:54:58.890454+0000","last_active":"2024-06-13T06:58:28.267286+0000","last_peered":"2024-06-13T06:58:28.267286+0000","last_clean":"2024-06-13T06:58:28.267286+0000","last_became_active":"2024-06-13T06:54:58.890129+0000","last_became_peered":"2024-06-13T06:54:58.890129+0000","last_unstale":"2024-06-13T06:58:28.267286+0000","last_undegraded":"2024-06-13T06:58:28.267286+0000","last_fullsized":"2024-06-13T06:58:28.267286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:10.819536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67657,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1429,"num_read_kb":786,"num_write":180,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1656,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":213717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383764,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":570102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645400,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37124,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708028,"statfs":{"total":95995035648,"available":95957020672,"internally_reserved":0,"allocated":1830912,"data_stored":1220536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":574390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070565,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":570170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332203,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1166569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495728,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":565470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36516,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708636,"statfs":{"total":95995035648,"available":95957643264,"internally_reserved":0,"allocated":1208320,"data_stored":633963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":566858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":566108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182529,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":565664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920894,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1232896,"data_stored":640162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1162157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T07:00:30.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-13T07:00:30.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:29 smithi136 ceph-mon[37229]: pgmap v509: 129 pgs: 129 active+clean; 643 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T07:00:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 -- 172.21.15.26:0/2622955004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52f8102a00 msgr2=0x7f52f8102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 --2- 172.21.15.26:0/2622955004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52f8102a00 0x7f52f8102e80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009900 tx=0x7f52e402a410 comp rx=0 tx=0).stop 2024-06-13T07:00:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 -- 172.21.15.26:0/2622955004 shutdown_connections 2024-06-13T07:00:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 --2- 172.21.15.26:0/2622955004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52f8102a00 0x7f52f8102e80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 --2- 172.21.15.26:0/2622955004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f81033c0 0x7f52f8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 --2- 172.21.15.26:0/2622955004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f8101810 0x7f52f8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 -- 172.21.15.26:0/2622955004 >> 172.21.15.26:0/2622955004 conn(0x7f52f80fcd80 msgr2=0x7f52f80ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 -- 172.21.15.26:0/2622955004 shutdown_connections 2024-06-13T07:00:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.350+0000 7f52f7200640 1 -- 172.21.15.26:0/2622955004 wait complete. 2024-06-13T07:00:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.351+0000 7f52f7200640 1 Processor -- start 2024-06-13T07:00:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f7200640 1 -- start start 2024-06-13T07:00:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f8101810 0x7f52f8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f8102a00 0x7f52f819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f7200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52f81033c0 0x7f52f81a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52f810d490 con 0x7f52f8101810 2024-06-13T07:00:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f7200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52f810d310 con 0x7f52f81033c0 2024-06-13T07:00:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52f810d610 con 0x7f52f8102a00 2024-06-13T07:00:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f8101810 0x7f52f8199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f8101810 0x7f52f8199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38522/0 (socket says 172.21.15.26:38522) 2024-06-13T07:00:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f5e00640 1 -- 172.21.15.26:0/787831723 learned_addr learned my addr 172.21.15.26:0/787831723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f5400640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f8102a00 0x7f52f819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.352+0000 7f52f6800640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52f81033c0 0x7f52f81a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52f5e00640 1 -- 172.21.15.26:0/787831723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f8102a00 msgr2=0x7f52f819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52f5e00640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f8102a00 0x7f52f819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52f5e00640 1 -- 172.21.15.26:0/787831723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52f81033c0 msgr2=0x7f52f81a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52f5e00640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52f81033c0 0x7f52f81a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52f5e00640 1 -- 172.21.15.26:0/787831723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f81a1990 con 0x7f52f8101810 2024-06-13T07:00:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52f5e00640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f8101810 0x7f52f8199d70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f52e800e9c0 tx=0x7f52e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52eea00640 1 -- 172.21.15.26:0/787831723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e800cd80 con 0x7f52f8101810 2024-06-13T07:00:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52eea00640 1 -- 172.21.15.26:0/787831723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52e80035f0 con 0x7f52f8101810 2024-06-13T07:00:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f81a1c80 con 0x7f52f8101810 2024-06-13T07:00:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.353+0000 7f52eea00640 1 -- 172.21.15.26:0/787831723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8010690 con 0x7f52f8101810 2024-06-13T07:00:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.354+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f81a21a0 con 0x7f52f8101810 2024-06-13T07:00:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.355+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f81a2370 con 0x7f52f8101810 2024-06-13T07:00:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.357+0000 7f52eea00640 1 -- 172.21.15.26:0/787831723 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f52e80037a0 con 0x7f52f8101810 2024-06-13T07:00:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.359+0000 7f52eea00640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52dc0a2b90 0x7f52dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.360+0000 7f52eea00640 1 -- 172.21.15.26:0/787831723 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f52e80cf850 con 0x7f52f8101810 2024-06-13T07:00:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.360+0000 7f52f5400640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52dc0a2b90 0x7f52dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.360+0000 7f52f5400640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52dc0a2b90 0x7f52dc0a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f52e40314e0 tx=0x7f52e4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.365+0000 7f52eea00640 1 -- 172.21.15.26:0/787831723 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f52e809c360 con 0x7f52f8101810 2024-06-13T07:00:30.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.569+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52f81a2540 con 0x7f52dc0a2b90 2024-06-13T07:00:30.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.599+0000 7f52eea00640 1 -- 172.21.15.26:0/787831723 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340501 (secure 0 0 0) 0x7f52f81a2540 con 0x7f52dc0a2b90 2024-06-13T07:00:30.603 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:00:30.605 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-06-13T07:00:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52dc0a2b90 msgr2=0x7f52dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52dc0a2b90 0x7f52dc0a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f52e40314e0 tx=0x7f52e4000bc0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f8101810 msgr2=0x7f52f8199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f8101810 0x7f52f8199d70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f52e800e9c0 tx=0x7f52e800ee80 comp rx=0 tx=0).stop 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 shutdown_connections 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f52dc0a2b90 0x7f52dc0a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52f81033c0 0x7f52f81a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f8102a00 0x7f52f819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 --2- 172.21.15.26:0/787831723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f8101810 0x7f52f8199d70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 >> 172.21.15.26:0/787831723 conn(0x7f52f80fcd80 msgr2=0x7f52f8073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 shutdown_connections 2024-06-13T07:00:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.606+0000 7f52f7200640 1 -- 172.21.15.26:0/787831723 wait complete. 2024-06-13T07:00:30.629 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-06-13T07:00:30.424565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663632,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1571,"num_read_kb":911,"num_write":364,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182392,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402044,"kb_used_data":15452,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539780,"statfs":{"total":1151940427776,"available":1151528734720,"internally_reserved":0,"allocated":15822848,"data_stored":8807522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":48,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006016"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293186+0000","last_change":"2024-06-13T06:54:58.919669+0000","last_active":"2024-06-13T06:58:28.293186+0000","last_peered":"2024-06-13T06:58:28.293186+0000","last_clean":"2024-06-13T06:58:28.293186+0000","last_became_active":"2024-06-13T06:54:58.919334+0000","last_became_peered":"2024-06-13T06:54:58.919334+0000","last_unstale":"2024-06-13T06:58:28.293186+0000","last_undegraded":"2024-06-13T06:58:28.293186+0000","last_fullsized":"2024-06-13T06:58:28.293186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:45:40.831640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295249+0000","last_change":"2024-06-13T06:54:58.909508+0000","last_active":"2024-06-13T06:58:28.295249+0000","last_peered":"2024-06-13T06:58:28.295249+0000","last_clean":"2024-06-13T06:58:28.295249+0000","last_became_active":"2024-06-13T06:54:58.909221+0000","last_became_peered":"2024-06-13T06:54:58.909221+0000","last_unstale":"2024-06-13T06:58:28.295249+0000","last_undegraded":"2024-06-13T06:58:28.295249+0000","last_fullsized":"2024-06-13T06:58:28.295249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:17:07.476858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304063+0000","last_change":"2024-06-13T06:54:58.909283+0000","last_active":"2024-06-13T06:58:28.304063+0000","last_peered":"2024-06-13T06:58:28.304063+0000","last_clean":"2024-06-13T06:58:28.304063+0000","last_became_active":"2024-06-13T06:54:58.908912+0000","last_became_peered":"2024-06-13T06:54:58.908912+0000","last_unstale":"2024-06-13T06:58:28.304063+0000","last_undegraded":"2024-06-13T06:58:28.304063+0000","last_fullsized":"2024-06-13T06:58:28.304063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:48:26.937971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.264605+0000","last_change":"2024-06-13T06:54:58.902821+0000","last_active":"2024-06-13T06:58:28.264605+0000","last_peered":"2024-06-13T06:58:28.264605+0000","last_clean":"2024-06-13T06:58:28.264605+0000","last_became_active":"2024-06-13T06:54:58.902215+0000","last_became_peered":"2024-06-13T06:54:58.902215+0000","last_unstale":"2024-06-13T06:58:28.264605+0000","last_undegraded":"2024-06-13T06:58:28.264605+0000","last_fullsized":"2024-06-13T06:58:28.264605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:11:57.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296624+0000","last_change":"2024-06-13T06:54:58.897038+0000","last_active":"2024-06-13T06:58:28.296624+0000","last_peered":"2024-06-13T06:58:28.296624+0000","last_clean":"2024-06-13T06:58:28.296624+0000","last_became_active":"2024-06-13T06:54:58.896665+0000","last_became_peered":"2024-06-13T06:54:58.896665+0000","last_unstale":"2024-06-13T06:58:28.296624+0000","last_undegraded":"2024-06-13T06:58:28.296624+0000","last_fullsized":"2024-06-13T06:58:28.296624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:24:19.216932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.269970+0000","last_change":"2024-06-13T06:54:58.895501+0000","last_active":"2024-06-13T06:58:28.269970+0000","last_peered":"2024-06-13T06:58:28.269970+0000","last_clean":"2024-06-13T06:58:28.269970+0000","last_became_active":"2024-06-13T06:54:58.895144+0000","last_became_peered":"2024-06-13T06:54:58.895144+0000","last_unstale":"2024-06-13T06:58:28.269970+0000","last_undegraded":"2024-06-13T06:58:28.269970+0000","last_fullsized":"2024-06-13T06:58:28.269970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:50.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272336+0000","last_change":"2024-06-13T06:54:58.901097+0000","last_active":"2024-06-13T06:58:28.272336+0000","last_peered":"2024-06-13T06:58:28.272336+0000","last_clean":"2024-06-13T06:58:28.272336+0000","last_became_active":"2024-06-13T06:54:58.900727+0000","last_became_peered":"2024-06-13T06:54:58.900727+0000","last_unstale":"2024-06-13T06:58:28.272336+0000","last_undegraded":"2024-06-13T06:58:28.272336+0000","last_fullsized":"2024-06-13T06:58:28.272336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:09:38.349517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278488+0000","last_change":"2024-06-13T06:54:58.910740+0000","last_active":"2024-06-13T06:58:28.278488+0000","last_peered":"2024-06-13T06:58:28.278488+0000","last_clean":"2024-06-13T06:58:28.278488+0000","last_became_active":"2024-06-13T06:54:58.910351+0000","last_became_peered":"2024-06-13T06:54:58.910351+0000","last_unstale":"2024-06-13T06:58:28.278488+0000","last_undegraded":"2024-06-13T06:58:28.278488+0000","last_fullsized":"2024-06-13T06:58:28.278488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:45:12.854996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281927+0000","last_change":"2024-06-13T06:55:29.364516+0000","last_active":"2024-06-13T06:58:28.281927+0000","last_peered":"2024-06-13T06:58:28.281927+0000","last_clean":"2024-06-13T06:58:28.281927+0000","last_became_active":"2024-06-13T06:55:29.364079+0000","last_became_peered":"2024-06-13T06:55:29.364079+0000","last_unstale":"2024-06-13T06:58:28.281927+0000","last_undegraded":"2024-06-13T06:58:28.281927+0000","last_fullsized":"2024-06-13T06:58:28.281927+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:51:28.624359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267340+0000","last_change":"2024-06-13T06:54:58.903882+0000","last_active":"2024-06-13T06:58:28.267340+0000","last_peered":"2024-06-13T06:58:28.267340+0000","last_clean":"2024-06-13T06:58:28.267340+0000","last_became_active":"2024-06-13T06:54:58.903495+0000","last_became_peered":"2024-06-13T06:54:58.903495+0000","last_unstale":"2024-06-13T06:58:28.267340+0000","last_undegraded":"2024-06-13T06:58:28.267340+0000","last_fullsized":"2024-06-13T06:58:28.267340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:23.682993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281971+0000","last_change":"2024-06-13T06:54:58.900080+0000","last_active":"2024-06-13T06:58:28.281971+0000","last_peered":"2024-06-13T06:58:28.281971+0000","last_clean":"2024-06-13T06:58:28.281971+0000","last_became_active":"2024-06-13T06:54:58.899736+0000","last_became_peered":"2024-06-13T06:54:58.899736+0000","last_unstale":"2024-06-13T06:58:28.281971+0000","last_undegraded":"2024-06-13T06:58:28.281971+0000","last_fullsized":"2024-06-13T06:58:28.281971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:51:45.808821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.264389+0000","last_change":"2024-06-13T06:54:58.900238+0000","last_active":"2024-06-13T06:58:28.264389+0000","last_peered":"2024-06-13T06:58:28.264389+0000","last_clean":"2024-06-13T06:58:28.264389+0000","last_became_active":"2024-06-13T06:54:58.899872+0000","last_became_peered":"2024-06-13T06:54:58.899872+0000","last_unstale":"2024-06-13T06:58:28.264389+0000","last_undegraded":"2024-06-13T06:58:28.264389+0000","last_fullsized":"2024-06-13T06:58:28.264389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:12:40.780894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296863+0000","last_change":"2024-06-13T06:54:58.901981+0000","last_active":"2024-06-13T06:58:28.296863+0000","last_peered":"2024-06-13T06:58:28.296863+0000","last_clean":"2024-06-13T06:58:28.296863+0000","last_became_active":"2024-06-13T06:54:58.898704+0000","last_became_peered":"2024-06-13T06:54:58.898704+0000","last_unstale":"2024-06-13T06:58:28.296863+0000","last_undegraded":"2024-06-13T06:58:28.296863+0000","last_fullsized":"2024-06-13T06:58:28.296863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:03:56.658091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.299281+0000","last_change":"2024-06-13T06:54:58.909022+0000","last_active":"2024-06-13T06:58:28.299281+0000","last_peered":"2024-06-13T06:58:28.299281+0000","last_clean":"2024-06-13T06:58:28.299281+0000","last_became_active":"2024-06-13T06:54:58.907438+0000","last_became_peered":"2024-06-13T06:54:58.907438+0000","last_unstale":"2024-06-13T06:58:28.299281+0000","last_undegraded":"2024-06-13T06:58:28.299281+0000","last_fullsized":"2024-06-13T06:58:28.299281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:40:34.190510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305584+0000","last_change":"2024-06-13T06:54:58.897691+0000","last_active":"2024-06-13T06:58:28.305584+0000","last_peered":"2024-06-13T06:58:28.305584+0000","last_clean":"2024-06-13T06:58:28.305584+0000","last_became_active":"2024-06-13T06:54:58.897381+0000","last_became_peered":"2024-06-13T06:54:58.897381+0000","last_unstale":"2024-06-13T06:58:28.305584+0000","last_undegraded":"2024-06-13T06:58:28.305584+0000","last_fullsized":"2024-06-13T06:58:28.305584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:10.052052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295211+0000","last_change":"2024-06-13T06:54:58.907676+0000","last_active":"2024-06-13T06:58:28.295211+0000","last_peered":"2024-06-13T06:58:28.295211+0000","last_clean":"2024-06-13T06:58:28.295211+0000","last_became_active":"2024-06-13T06:54:58.907304+0000","last_became_peered":"2024-06-13T06:54:58.907304+0000","last_unstale":"2024-06-13T06:58:28.295211+0000","last_undegraded":"2024-06-13T06:58:28.295211+0000","last_fullsized":"2024-06-13T06:58:28.295211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:08:20.292516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293328+0000","last_change":"2024-06-13T06:54:58.920012+0000","last_active":"2024-06-13T06:58:28.293328+0000","last_peered":"2024-06-13T06:58:28.293328+0000","last_clean":"2024-06-13T06:58:28.293328+0000","last_became_active":"2024-06-13T06:54:58.919473+0000","last_became_peered":"2024-06-13T06:54:58.919473+0000","last_unstale":"2024-06-13T06:58:28.293328+0000","last_undegraded":"2024-06-13T06:58:28.293328+0000","last_fullsized":"2024-06-13T06:58:28.293328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:49:13.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.280156+0000","last_change":"2024-06-13T06:54:58.902576+0000","last_active":"2024-06-13T06:58:28.280156+0000","last_peered":"2024-06-13T06:58:28.280156+0000","last_clean":"2024-06-13T06:58:28.280156+0000","last_became_active":"2024-06-13T06:54:58.902205+0000","last_became_peered":"2024-06-13T06:54:58.902205+0000","last_unstale":"2024-06-13T06:58:28.280156+0000","last_undegraded":"2024-06-13T06:58:28.280156+0000","last_fullsized":"2024-06-13T06:58:28.280156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:13:24.107081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292347+0000","last_change":"2024-06-13T06:54:58.905392+0000","last_active":"2024-06-13T06:58:28.292347+0000","last_peered":"2024-06-13T06:58:28.292347+0000","last_clean":"2024-06-13T06:58:28.292347+0000","last_became_active":"2024-06-13T06:54:58.905017+0000","last_became_peered":"2024-06-13T06:54:58.905017+0000","last_unstale":"2024-06-13T06:58:28.292347+0000","last_undegraded":"2024-06-13T06:58:28.292347+0000","last_fullsized":"2024-06-13T06:58:28.292347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:05.931653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:37.654508+0000","last_change":"2024-06-13T06:55:12.021359+0000","last_active":"2024-06-13T06:57:37.654508+0000","last_peered":"2024-06-13T06:57:37.654508+0000","last_clean":"2024-06-13T06:57:37.654508+0000","last_became_active":"2024-06-13T06:55:12.020860+0000","last_became_peered":"2024-06-13T06:55:12.020860+0000","last_unstale":"2024-06-13T06:57:37.654508+0000","last_undegraded":"2024-06-13T06:57:37.654508+0000","last_fullsized":"2024-06-13T06:57:37.654508+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:05:00.833905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:00.551277+0000","last_change":"2024-06-13T06:54:54.339220+0000","last_active":"2024-06-13T06:57:00.551277+0000","last_peered":"2024-06-13T06:57:00.551277+0000","last_clean":"2024-06-13T06:57:00.551277+0000","last_became_active":"2024-06-13T06:54:54.338573+0000","last_became_peered":"2024-06-13T06:54:54.338573+0000","last_unstale":"2024-06-13T06:57:00.551277+0000","last_undegraded":"2024-06-13T06:57:00.551277+0000","last_fullsized":"2024-06-13T06:57:00.551277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:53:50.258097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.277556+0000","last_change":"2024-06-13T06:54:58.903920+0000","last_active":"2024-06-13T06:58:28.277556+0000","last_peered":"2024-06-13T06:58:28.277556+0000","last_clean":"2024-06-13T06:58:28.277556+0000","last_became_active":"2024-06-13T06:54:58.895364+0000","last_became_peered":"2024-06-13T06:54:58.895364+0000","last_unstale":"2024-06-13T06:58:28.277556+0000","last_undegraded":"2024-06-13T06:58:28.277556+0000","last_fullsized":"2024-06-13T06:58:28.277556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:04:35.109799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:38.288505+0000","last_change":"2024-06-13T06:55:12.017561+0000","last_active":"2024-06-13T06:57:38.288505+0000","last_peered":"2024-06-13T06:57:38.288505+0000","last_clean":"2024-06-13T06:57:38.288505+0000","last_became_active":"2024-06-13T06:55:12.017104+0000","last_became_peered":"2024-06-13T06:55:12.017104+0000","last_unstale":"2024-06-13T06:57:38.288505+0000","last_undegraded":"2024-06-13T06:57:38.288505+0000","last_fullsized":"2024-06-13T06:57:38.288505+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:08:22.570491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:56.075775+0000","last_change":"2024-06-13T06:54:54.631599+0000","last_active":"2024-06-13T06:56:56.075775+0000","last_peered":"2024-06-13T06:56:56.075775+0000","last_clean":"2024-06-13T06:56:56.075775+0000","last_became_active":"2024-06-13T06:54:54.631007+0000","last_became_peered":"2024-06-13T06:54:54.631007+0000","last_unstale":"2024-06-13T06:56:56.075775+0000","last_undegraded":"2024-06-13T06:56:56.075775+0000","last_fullsized":"2024-06-13T06:56:56.075775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:49:56.602192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304511+0000","last_change":"2024-06-13T06:54:58.917255+0000","last_active":"2024-06-13T06:58:28.304511+0000","last_peered":"2024-06-13T06:58:28.304511+0000","last_clean":"2024-06-13T06:58:28.304511+0000","last_became_active":"2024-06-13T06:54:58.916875+0000","last_became_peered":"2024-06-13T06:54:58.916875+0000","last_unstale":"2024-06-13T06:58:28.304511+0000","last_undegraded":"2024-06-13T06:58:28.304511+0000","last_fullsized":"2024-06-13T06:58:28.304511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:09:47.638802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:43.289081+0000","last_change":"2024-06-13T06:55:12.018209+0000","last_active":"2024-06-13T06:57:43.289081+0000","last_peered":"2024-06-13T06:57:43.289081+0000","last_clean":"2024-06-13T06:57:43.289081+0000","last_became_active":"2024-06-13T06:55:12.017038+0000","last_became_peered":"2024-06-13T06:55:12.017038+0000","last_unstale":"2024-06-13T06:57:43.289081+0000","last_undegraded":"2024-06-13T06:57:43.289081+0000","last_fullsized":"2024-06-13T06:57:43.289081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:36:26.476454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:25.059841+0000","last_change":"2024-06-13T06:54:54.328849+0000","last_active":"2024-06-13T06:57:25.059841+0000","last_peered":"2024-06-13T06:57:25.059841+0000","last_clean":"2024-06-13T06:57:25.059841+0000","last_became_active":"2024-06-13T06:54:54.328388+0000","last_became_peered":"2024-06-13T06:54:54.328388+0000","last_unstale":"2024-06-13T06:57:25.059841+0000","last_undegraded":"2024-06-13T06:57:25.059841+0000","last_fullsized":"2024-06-13T06:57:25.059841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:41.264403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.283009+0000","last_change":"2024-06-13T06:54:58.897588+0000","last_active":"2024-06-13T06:58:28.283009+0000","last_peered":"2024-06-13T06:58:28.283009+0000","last_clean":"2024-06-13T06:58:28.283009+0000","last_became_active":"2024-06-13T06:54:58.897236+0000","last_became_peered":"2024-06-13T06:54:58.897236+0000","last_unstale":"2024-06-13T06:58:28.283009+0000","last_undegraded":"2024-06-13T06:58:28.283009+0000","last_fullsized":"2024-06-13T06:58:28.283009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:20:38.460445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:43.289055+0000","last_change":"2024-06-13T06:55:12.014080+0000","last_active":"2024-06-13T06:57:43.289055+0000","last_peered":"2024-06-13T06:57:43.289055+0000","last_clean":"2024-06-13T06:57:43.289055+0000","last_became_active":"2024-06-13T06:55:12.013627+0000","last_became_peered":"2024-06-13T06:55:12.013627+0000","last_unstale":"2024-06-13T06:57:43.289055+0000","last_undegraded":"2024-06-13T06:57:43.289055+0000","last_fullsized":"2024-06-13T06:57:43.289055+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:29:54.081593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:25.059798+0000","last_change":"2024-06-13T06:54:54.328249+0000","last_active":"2024-06-13T06:57:25.059798+0000","last_peered":"2024-06-13T06:57:25.059798+0000","last_clean":"2024-06-13T06:57:25.059798+0000","last_became_active":"2024-06-13T06:54:54.327832+0000","last_became_peered":"2024-06-13T06:54:54.327832+0000","last_unstale":"2024-06-13T06:57:25.059798+0000","last_undegraded":"2024-06-13T06:57:25.059798+0000","last_fullsized":"2024-06-13T06:57:25.059798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:21:05.624075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281353+0000","last_change":"2024-06-13T06:55:29.383305+0000","last_active":"2024-06-13T06:58:28.281353+0000","last_peered":"2024-06-13T06:58:28.281353+0000","last_clean":"2024-06-13T06:58:28.281353+0000","last_became_active":"2024-06-13T06:55:29.381491+0000","last_became_peered":"2024-06-13T06:55:29.381491+0000","last_unstale":"2024-06-13T06:58:28.281353+0000","last_undegraded":"2024-06-13T06:58:28.281353+0000","last_fullsized":"2024-06-13T06:58:28.281353+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:52:26.437237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:42.655020+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:57:42.655020+0000","last_peered":"2024-06-13T06:57:42.655020+0000","last_clean":"2024-06-13T06:57:42.655020+0000","last_became_active":"2024-06-13T06:55:12.000759+0000","last_became_peered":"2024-06-13T06:55:12.000759+0000","last_unstale":"2024-06-13T06:57:42.655020+0000","last_undegraded":"2024-06-13T06:57:42.655020+0000","last_fullsized":"2024-06-13T06:57:42.655020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:11:35.805338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'8","reported_seq":299,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:25.040779+0000","last_change":"2024-06-13T06:55:29.367538+0000","last_active":"2024-06-13T07:00:25.040779+0000","last_peered":"2024-06-13T07:00:25.040779+0000","last_clean":"2024-06-13T07:00:25.040779+0000","last_became_active":"2024-06-13T06:55:29.367149+0000","last_became_peered":"2024-06-13T06:55:29.367149+0000","last_unstale":"2024-06-13T07:00:25.040779+0000","last_undegraded":"2024-06-13T07:00:25.040779+0000","last_fullsized":"2024-06-13T07:00:25.040779+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:30:55.632799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1459,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.269082+0000","last_change":"2024-06-13T06:54:58.896258+0000","last_active":"2024-06-13T06:58:28.269082+0000","last_peered":"2024-06-13T06:58:28.269082+0000","last_clean":"2024-06-13T06:58:28.269082+0000","last_became_active":"2024-06-13T06:54:58.895904+0000","last_became_peered":"2024-06-13T06:54:58.895904+0000","last_unstale":"2024-06-13T06:58:28.269082+0000","last_undegraded":"2024-06-13T06:58:28.269082+0000","last_fullsized":"2024-06-13T06:58:28.269082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:28:33.615338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:42.654996+0000","last_change":"2024-06-13T06:55:12.018499+0000","last_active":"2024-06-13T06:57:42.654996+0000","last_peered":"2024-06-13T06:57:42.654996+0000","last_clean":"2024-06-13T06:57:42.654996+0000","last_became_active":"2024-06-13T06:55:12.018073+0000","last_became_peered":"2024-06-13T06:55:12.018073+0000","last_unstale":"2024-06-13T06:57:42.654996+0000","last_undegraded":"2024-06-13T06:57:42.654996+0000","last_fullsized":"2024-06-13T06:57:42.654996+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:59:53.206259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:23.429829+0000","last_change":"2024-06-13T06:54:54.339482+0000","last_active":"2024-06-13T06:57:23.429829+0000","last_peered":"2024-06-13T06:57:23.429829+0000","last_clean":"2024-06-13T06:57:23.429829+0000","last_became_active":"2024-06-13T06:54:54.339066+0000","last_became_peered":"2024-06-13T06:54:54.339066+0000","last_unstale":"2024-06-13T06:57:23.429829+0000","last_undegraded":"2024-06-13T06:57:23.429829+0000","last_fullsized":"2024-06-13T06:57:23.429829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:31:04.136530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279253+0000","last_change":"2024-06-13T06:54:58.891154+0000","last_active":"2024-06-13T06:58:28.279253+0000","last_peered":"2024-06-13T06:58:28.279253+0000","last_clean":"2024-06-13T06:58:28.279253+0000","last_became_active":"2024-06-13T06:54:58.890810+0000","last_became_peered":"2024-06-13T06:54:58.890810+0000","last_unstale":"2024-06-13T06:58:28.279253+0000","last_undegraded":"2024-06-13T06:58:28.279253+0000","last_fullsized":"2024-06-13T06:58:28.279253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:14.618233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.836644+0000","last_change":"2024-06-13T06:55:12.025619+0000","last_active":"2024-06-13T06:57:19.836644+0000","last_peered":"2024-06-13T06:57:19.836644+0000","last_clean":"2024-06-13T06:57:19.836644+0000","last_became_active":"2024-06-13T06:55:12.025153+0000","last_became_peered":"2024-06-13T06:55:12.025153+0000","last_unstale":"2024-06-13T06:57:19.836644+0000","last_undegraded":"2024-06-13T06:57:19.836644+0000","last_fullsized":"2024-06-13T06:57:19.836644+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:12.124550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:22.395183+0000","last_change":"2024-06-13T06:54:54.341698+0000","last_active":"2024-06-13T06:57:22.395183+0000","last_peered":"2024-06-13T06:57:22.395183+0000","last_clean":"2024-06-13T06:57:22.395183+0000","last_became_active":"2024-06-13T06:54:54.341113+0000","last_became_peered":"2024-06-13T06:54:54.341113+0000","last_unstale":"2024-06-13T06:57:22.395183+0000","last_undegraded":"2024-06-13T06:57:22.395183+0000","last_fullsized":"2024-06-13T06:57:22.395183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:18:23.368705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279519+0000","last_change":"2024-06-13T06:54:58.913688+0000","last_active":"2024-06-13T06:58:28.279519+0000","last_peered":"2024-06-13T06:58:28.279519+0000","last_clean":"2024-06-13T06:58:28.279519+0000","last_became_active":"2024-06-13T06:54:58.910493+0000","last_became_peered":"2024-06-13T06:54:58.910493+0000","last_unstale":"2024-06-13T06:58:28.279519+0000","last_undegraded":"2024-06-13T06:58:28.279519+0000","last_fullsized":"2024-06-13T06:58:28.279519+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:29:18.081743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.540765+0000","last_change":"2024-06-13T06:55:12.016257+0000","last_active":"2024-06-13T06:57:12.540765+0000","last_peered":"2024-06-13T06:57:12.540765+0000","last_clean":"2024-06-13T06:57:12.540765+0000","last_became_active":"2024-06-13T06:55:12.015594+0000","last_became_peered":"2024-06-13T06:55:12.015594+0000","last_unstale":"2024-06-13T06:57:12.540765+0000","last_undegraded":"2024-06-13T06:57:12.540765+0000","last_fullsized":"2024-06-13T06:57:12.540765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:02:44.642075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542663+0000","last_change":"2024-06-13T06:54:54.330366+0000","last_active":"2024-06-13T06:57:21.542663+0000","last_peered":"2024-06-13T06:57:21.542663+0000","last_clean":"2024-06-13T06:57:21.542663+0000","last_became_active":"2024-06-13T06:54:54.330010+0000","last_became_peered":"2024-06-13T06:54:54.330010+0000","last_unstale":"2024-06-13T06:57:21.542663+0000","last_undegraded":"2024-06-13T06:57:21.542663+0000","last_fullsized":"2024-06-13T06:57:21.542663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:36:01.488766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276547+0000","last_change":"2024-06-13T06:54:58.918076+0000","last_active":"2024-06-13T06:58:28.276547+0000","last_peered":"2024-06-13T06:58:28.276547+0000","last_clean":"2024-06-13T06:58:28.276547+0000","last_became_active":"2024-06-13T06:54:58.917631+0000","last_became_peered":"2024-06-13T06:54:58.917631+0000","last_unstale":"2024-06-13T06:58:28.276547+0000","last_undegraded":"2024-06-13T06:58:28.276547+0000","last_fullsized":"2024-06-13T06:58:28.276547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:31:00.056837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:31.081713+0000","last_change":"2024-06-13T06:56:29.611761+0000","last_active":"2024-06-13T06:57:31.081713+0000","last_peered":"2024-06-13T06:57:31.081713+0000","last_clean":"2024-06-13T06:57:31.081713+0000","last_became_active":"2024-06-13T06:56:29.610385+0000","last_became_peered":"2024-06-13T06:56:29.610385+0000","last_unstale":"2024-06-13T06:57:31.081713+0000","last_undegraded":"2024-06-13T06:57:31.081713+0000","last_fullsized":"2024-06-13T06:57:31.081713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:26:10.678918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'5","reported_seq":287,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:27.044645+0000","last_change":"2024-06-13T06:54:54.335324+0000","last_active":"2024-06-13T07:00:27.044645+0000","last_peered":"2024-06-13T07:00:27.044645+0000","last_clean":"2024-06-13T07:00:27.044645+0000","last_became_active":"2024-06-13T06:54:54.334936+0000","last_became_peered":"2024-06-13T06:54:54.334936+0000","last_unstale":"2024-06-13T07:00:27.044645+0000","last_undegraded":"2024-06-13T07:00:27.044645+0000","last_fullsized":"2024-06-13T07:00:27.044645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:41:52.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281723+0000","last_change":"2024-06-13T06:54:58.902146+0000","last_active":"2024-06-13T06:58:28.281723+0000","last_peered":"2024-06-13T06:58:28.281723+0000","last_clean":"2024-06-13T06:58:28.281723+0000","last_became_active":"2024-06-13T06:54:58.901807+0000","last_became_peered":"2024-06-13T06:54:58.901807+0000","last_unstale":"2024-06-13T06:58:28.281723+0000","last_undegraded":"2024-06-13T06:58:28.281723+0000","last_fullsized":"2024-06-13T06:58:28.281723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:59:53.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:35.557156+0000","last_change":"2024-06-13T06:55:12.025418+0000","last_active":"2024-06-13T06:57:35.557156+0000","last_peered":"2024-06-13T06:57:35.557156+0000","last_clean":"2024-06-13T06:57:35.557156+0000","last_became_active":"2024-06-13T06:55:12.024859+0000","last_became_peered":"2024-06-13T06:55:12.024859+0000","last_unstale":"2024-06-13T06:57:35.557156+0000","last_undegraded":"2024-06-13T06:57:35.557156+0000","last_fullsized":"2024-06-13T06:57:35.557156+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:05:17.553169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.670600+0000","last_change":"2024-06-13T06:54:54.326011+0000","last_active":"2024-06-13T06:57:19.670600+0000","last_peered":"2024-06-13T06:57:19.670600+0000","last_clean":"2024-06-13T06:57:19.670600+0000","last_became_active":"2024-06-13T06:54:54.325664+0000","last_became_peered":"2024-06-13T06:54:54.325664+0000","last_unstale":"2024-06-13T06:57:19.670600+0000","last_undegraded":"2024-06-13T06:57:19.670600+0000","last_fullsized":"2024-06-13T06:57:19.670600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:32.973094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301872+0000","last_change":"2024-06-13T06:55:29.383612+0000","last_active":"2024-06-13T06:58:28.301872+0000","last_peered":"2024-06-13T06:58:28.301872+0000","last_clean":"2024-06-13T06:58:28.301872+0000","last_became_active":"2024-06-13T06:55:29.382174+0000","last_became_peered":"2024-06-13T06:55:29.382174+0000","last_unstale":"2024-06-13T06:58:28.301872+0000","last_undegraded":"2024-06-13T06:58:28.301872+0000","last_fullsized":"2024-06-13T06:58:28.301872+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:09:35.178209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:33.288040+0000","last_change":"2024-06-13T06:55:12.018654+0000","last_active":"2024-06-13T06:57:33.288040+0000","last_peered":"2024-06-13T06:57:33.288040+0000","last_clean":"2024-06-13T06:57:33.288040+0000","last_became_active":"2024-06-13T06:55:12.017219+0000","last_became_peered":"2024-06-13T06:55:12.017219+0000","last_unstale":"2024-06-13T06:57:33.288040+0000","last_undegraded":"2024-06-13T06:57:33.288040+0000","last_fullsized":"2024-06-13T06:57:33.288040+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:46:54.161912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542714+0000","last_change":"2024-06-13T06:54:54.632154+0000","last_active":"2024-06-13T06:57:21.542714+0000","last_peered":"2024-06-13T06:57:21.542714+0000","last_clean":"2024-06-13T06:57:21.542714+0000","last_became_active":"2024-06-13T06:54:54.631721+0000","last_became_peered":"2024-06-13T06:54:54.631721+0000","last_unstale":"2024-06-13T06:57:21.542714+0000","last_undegraded":"2024-06-13T06:57:21.542714+0000","last_fullsized":"2024-06-13T06:57:21.542714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:16:10.492144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278187+0000","last_change":"2024-06-13T06:54:58.898961+0000","last_active":"2024-06-13T06:58:28.278187+0000","last_peered":"2024-06-13T06:58:28.278187+0000","last_clean":"2024-06-13T06:58:28.278187+0000","last_became_active":"2024-06-13T06:54:58.898599+0000","last_became_peered":"2024-06-13T06:54:58.898599+0000","last_unstale":"2024-06-13T06:58:28.278187+0000","last_undegraded":"2024-06-13T06:58:28.278187+0000","last_fullsized":"2024-06-13T06:58:28.278187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:21:53.044205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:33.424302+0000","last_change":"2024-06-13T06:56:29.527242+0000","last_active":"2024-06-13T06:56:33.424302+0000","last_peered":"2024-06-13T06:56:33.424302+0000","last_clean":"2024-06-13T06:56:33.424302+0000","last_became_active":"2024-06-13T06:56:29.526816+0000","last_became_peered":"2024-06-13T06:56:29.526816+0000","last_unstale":"2024-06-13T06:56:33.424302+0000","last_undegraded":"2024-06-13T06:56:33.424302+0000","last_fullsized":"2024-06-13T06:56:33.424302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:45:17.770029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'39","reported_seq":584,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:27.942865+0000","last_change":"2024-06-13T06:55:29.376808+0000","last_active":"2024-06-13T07:00:27.942865+0000","last_peered":"2024-06-13T07:00:27.942865+0000","last_clean":"2024-06-13T07:00:27.942865+0000","last_became_active":"2024-06-13T06:55:29.367134+0000","last_became_peered":"2024-06-13T06:55:29.367134+0000","last_unstale":"2024-06-13T07:00:27.942865+0000","last_undegraded":"2024-06-13T07:00:27.942865+0000","last_fullsized":"2024-06-13T07:00:27.942865+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:19:08.784438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68583,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":483,"num_read_kb":310,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.265259+0000","last_change":"2024-06-13T06:54:58.897686+0000","last_active":"2024-06-13T06:58:28.265259+0000","last_peered":"2024-06-13T06:58:28.265259+0000","last_clean":"2024-06-13T06:58:28.265259+0000","last_became_active":"2024-06-13T06:54:58.897313+0000","last_became_peered":"2024-06-13T06:54:58.897313+0000","last_unstale":"2024-06-13T06:58:28.265259+0000","last_undegraded":"2024-06-13T06:58:28.265259+0000","last_fullsized":"2024-06-13T06:58:28.265259+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:54:49.558542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:36.544740+0000","last_change":"2024-06-13T06:55:12.013918+0000","last_active":"2024-06-13T06:57:36.544740+0000","last_peered":"2024-06-13T06:57:36.544740+0000","last_clean":"2024-06-13T06:57:36.544740+0000","last_became_active":"2024-06-13T06:55:12.013522+0000","last_became_peered":"2024-06-13T06:55:12.013522+0000","last_unstale":"2024-06-13T06:57:36.544740+0000","last_undegraded":"2024-06-13T06:57:36.544740+0000","last_fullsized":"2024-06-13T06:57:36.544740+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:51:46.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.652302+0000","last_change":"2024-06-13T06:54:54.328674+0000","last_active":"2024-06-13T06:57:17.652302+0000","last_peered":"2024-06-13T06:57:17.652302+0000","last_clean":"2024-06-13T06:57:17.652302+0000","last_became_active":"2024-06-13T06:54:54.326789+0000","last_became_peered":"2024-06-13T06:54:54.326789+0000","last_unstale":"2024-06-13T06:57:17.652302+0000","last_undegraded":"2024-06-13T06:57:17.652302+0000","last_fullsized":"2024-06-13T06:57:17.652302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:18.280514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293277+0000","last_change":"2024-06-13T06:54:58.918598+0000","last_active":"2024-06-13T06:58:28.293277+0000","last_peered":"2024-06-13T06:58:28.293277+0000","last_clean":"2024-06-13T06:58:28.293277+0000","last_became_active":"2024-06-13T06:54:58.918247+0000","last_became_peered":"2024-06-13T06:54:58.918247+0000","last_unstale":"2024-06-13T06:58:28.293277+0000","last_undegraded":"2024-06-13T06:58:28.293277+0000","last_fullsized":"2024-06-13T06:58:28.293277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:08:56.738806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:34.838358+0000","last_change":"2024-06-13T06:55:12.017074+0000","last_active":"2024-06-13T06:57:34.838358+0000","last_peered":"2024-06-13T06:57:34.838358+0000","last_clean":"2024-06-13T06:57:34.838358+0000","last_became_active":"2024-06-13T06:55:12.016675+0000","last_became_peered":"2024-06-13T06:55:12.016675+0000","last_unstale":"2024-06-13T06:57:34.838358+0000","last_undegraded":"2024-06-13T06:57:34.838358+0000","last_fullsized":"2024-06-13T06:57:34.838358+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:20:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:16.542158+0000","last_change":"2024-06-13T06:54:54.334994+0000","last_active":"2024-06-13T06:57:16.542158+0000","last_peered":"2024-06-13T06:57:16.542158+0000","last_clean":"2024-06-13T06:57:16.542158+0000","last_became_active":"2024-06-13T06:54:54.334647+0000","last_became_peered":"2024-06-13T06:54:54.334647+0000","last_unstale":"2024-06-13T06:57:16.542158+0000","last_undegraded":"2024-06-13T06:57:16.542158+0000","last_fullsized":"2024-06-13T06:57:16.542158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:59:15.317976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276010+0000","last_change":"2024-06-13T06:54:58.899107+0000","last_active":"2024-06-13T06:58:28.276010+0000","last_peered":"2024-06-13T06:58:28.276010+0000","last_clean":"2024-06-13T06:58:28.276010+0000","last_became_active":"2024-06-13T06:54:58.896034+0000","last_became_peered":"2024-06-13T06:54:58.896034+0000","last_unstale":"2024-06-13T06:58:28.276010+0000","last_undegraded":"2024-06-13T06:58:28.276010+0000","last_fullsized":"2024-06-13T06:58:28.276010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:00:01.132565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:40.558064+0000","last_change":"2024-06-13T06:55:12.025681+0000","last_active":"2024-06-13T06:57:40.558064+0000","last_peered":"2024-06-13T06:57:40.558064+0000","last_clean":"2024-06-13T06:57:40.558064+0000","last_became_active":"2024-06-13T06:55:12.025267+0000","last_became_peered":"2024-06-13T06:55:12.025267+0000","last_unstale":"2024-06-13T06:57:40.558064+0000","last_undegraded":"2024-06-13T06:57:40.558064+0000","last_fullsized":"2024-06-13T06:57:40.558064+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:58:23.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":287,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:27.044620+0000","last_change":"2024-06-13T06:54:54.325404+0000","last_active":"2024-06-13T07:00:27.044620+0000","last_peered":"2024-06-13T07:00:27.044620+0000","last_clean":"2024-06-13T07:00:27.044620+0000","last_became_active":"2024-06-13T06:54:54.325024+0000","last_became_peered":"2024-06-13T06:54:54.325024+0000","last_unstale":"2024-06-13T07:00:27.044620+0000","last_undegraded":"2024-06-13T07:00:27.044620+0000","last_fullsized":"2024-06-13T07:00:27.044620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:23.726350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272955+0000","last_change":"2024-06-13T06:54:58.894737+0000","last_active":"2024-06-13T06:58:28.272955+0000","last_peered":"2024-06-13T06:58:28.272955+0000","last_clean":"2024-06-13T06:58:28.272955+0000","last_became_active":"2024-06-13T06:54:58.894224+0000","last_became_peered":"2024-06-13T06:54:58.894224+0000","last_unstale":"2024-06-13T06:58:28.272955+0000","last_undegraded":"2024-06-13T06:58:28.272955+0000","last_fullsized":"2024-06-13T06:58:28.272955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:03.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:37.398002+0000","last_change":"2024-06-13T06:55:12.026587+0000","last_active":"2024-06-13T06:57:37.398002+0000","last_peered":"2024-06-13T06:57:37.398002+0000","last_clean":"2024-06-13T06:57:37.398002+0000","last_became_active":"2024-06-13T06:55:12.026114+0000","last_became_peered":"2024-06-13T06:55:12.026114+0000","last_unstale":"2024-06-13T06:57:37.398002+0000","last_undegraded":"2024-06-13T06:57:37.398002+0000","last_fullsized":"2024-06-13T06:57:37.398002+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:43:09.245641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:15.058741+0000","last_change":"2024-06-13T06:54:54.632069+0000","last_active":"2024-06-13T06:57:15.058741+0000","last_peered":"2024-06-13T06:57:15.058741+0000","last_clean":"2024-06-13T06:57:15.058741+0000","last_became_active":"2024-06-13T06:54:54.631692+0000","last_became_peered":"2024-06-13T06:54:54.631692+0000","last_unstale":"2024-06-13T06:57:15.058741+0000","last_undegraded":"2024-06-13T06:57:15.058741+0000","last_fullsized":"2024-06-13T06:57:15.058741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:07.109507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.265386+0000","last_change":"2024-06-13T06:55:29.366286+0000","last_active":"2024-06-13T06:58:28.265386+0000","last_peered":"2024-06-13T06:58:28.265386+0000","last_clean":"2024-06-13T06:58:28.265386+0000","last_became_active":"2024-06-13T06:55:29.365854+0000","last_became_peered":"2024-06-13T06:55:29.365854+0000","last_unstale":"2024-06-13T06:58:28.265386+0000","last_undegraded":"2024-06-13T06:58:28.265386+0000","last_fullsized":"2024-06-13T06:58:28.265386+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:04.209244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542690+0000","last_change":"2024-06-13T06:55:12.021621+0000","last_active":"2024-06-13T06:57:21.542690+0000","last_peered":"2024-06-13T06:57:21.542690+0000","last_clean":"2024-06-13T06:57:21.542690+0000","last_became_active":"2024-06-13T06:55:12.021150+0000","last_became_peered":"2024-06-13T06:55:12.021150+0000","last_unstale":"2024-06-13T06:57:21.542690+0000","last_undegraded":"2024-06-13T06:57:21.542690+0000","last_fullsized":"2024-06-13T06:57:21.542690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:48.318918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:44.839364+0000","last_change":"2024-06-13T06:55:29.369972+0000","last_active":"2024-06-13T06:57:44.839364+0000","last_peered":"2024-06-13T06:57:44.839364+0000","last_clean":"2024-06-13T06:57:44.839364+0000","last_became_active":"2024-06-13T06:55:29.366390+0000","last_became_peered":"2024-06-13T06:55:29.366390+0000","last_unstale":"2024-06-13T06:57:44.839364+0000","last_undegraded":"2024-06-13T06:57:44.839364+0000","last_fullsized":"2024-06-13T06:57:44.839364+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:18.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271886+0000","last_change":"2024-06-13T06:54:58.891309+0000","last_active":"2024-06-13T06:58:28.271886+0000","last_peered":"2024-06-13T06:58:28.271886+0000","last_clean":"2024-06-13T06:58:28.271886+0000","last_became_active":"2024-06-13T06:54:58.890972+0000","last_became_peered":"2024-06-13T06:54:58.890972+0000","last_unstale":"2024-06-13T06:58:28.271886+0000","last_undegraded":"2024-06-13T06:58:28.271886+0000","last_fullsized":"2024-06-13T06:58:28.271886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:41.240061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:20.059185+0000","last_change":"2024-06-13T06:56:29.610713+0000","last_active":"2024-06-13T06:57:20.059185+0000","last_peered":"2024-06-13T06:57:20.059185+0000","last_clean":"2024-06-13T06:57:20.059185+0000","last_became_active":"2024-06-13T06:56:29.610139+0000","last_became_peered":"2024-06-13T06:56:29.610139+0000","last_unstale":"2024-06-13T06:57:20.059185+0000","last_undegraded":"2024-06-13T06:57:20.059185+0000","last_fullsized":"2024-06-13T06:57:20.059185+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:14:36.601243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:00.551177+0000","last_change":"2024-06-13T06:54:54.633893+0000","last_active":"2024-06-13T06:57:00.551177+0000","last_peered":"2024-06-13T06:57:00.551177+0000","last_clean":"2024-06-13T06:57:00.551177+0000","last_became_active":"2024-06-13T06:54:54.632946+0000","last_became_peered":"2024-06-13T06:54:54.632946+0000","last_unstale":"2024-06-13T06:57:00.551177+0000","last_undegraded":"2024-06-13T06:57:00.551177+0000","last_fullsized":"2024-06-13T06:57:00.551177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:16:54.812477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279866+0000","last_change":"2024-06-13T06:54:58.907182+0000","last_active":"2024-06-13T06:58:28.279866+0000","last_peered":"2024-06-13T06:58:28.279866+0000","last_clean":"2024-06-13T06:58:28.279866+0000","last_became_active":"2024-06-13T06:54:58.906824+0000","last_became_peered":"2024-06-13T06:54:58.906824+0000","last_unstale":"2024-06-13T06:58:28.279866+0000","last_undegraded":"2024-06-13T06:58:28.279866+0000","last_fullsized":"2024-06-13T06:58:28.279866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:34:49.626991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.394303+0000","last_change":"2024-06-13T06:55:12.014803+0000","last_active":"2024-06-13T06:57:17.394303+0000","last_peered":"2024-06-13T06:57:17.394303+0000","last_clean":"2024-06-13T06:57:17.394303+0000","last_became_active":"2024-06-13T06:55:12.014398+0000","last_became_peered":"2024-06-13T06:55:12.014398+0000","last_unstale":"2024-06-13T06:57:17.394303+0000","last_undegraded":"2024-06-13T06:57:17.394303+0000","last_fullsized":"2024-06-13T06:57:17.394303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:07:44.960446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:59.668362+0000","last_change":"2024-06-13T06:54:54.339056+0000","last_active":"2024-06-13T06:56:59.668362+0000","last_peered":"2024-06-13T06:56:59.668362+0000","last_clean":"2024-06-13T06:56:59.668362+0000","last_became_active":"2024-06-13T06:54:54.338683+0000","last_became_peered":"2024-06-13T06:54:54.338683+0000","last_unstale":"2024-06-13T06:56:59.668362+0000","last_undegraded":"2024-06-13T06:56:59.668362+0000","last_fullsized":"2024-06-13T06:56:59.668362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:05:17.060471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.297043+0000","last_change":"2024-06-13T06:54:58.919843+0000","last_active":"2024-06-13T06:58:28.297043+0000","last_peered":"2024-06-13T06:58:28.297043+0000","last_clean":"2024-06-13T06:58:28.297043+0000","last_became_active":"2024-06-13T06:54:58.919405+0000","last_became_peered":"2024-06-13T06:54:58.919405+0000","last_unstale":"2024-06-13T06:58:28.297043+0000","last_undegraded":"2024-06-13T06:58:28.297043+0000","last_fullsized":"2024-06-13T06:58:28.297043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:00:20.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:22.395386+0000","last_change":"2024-06-13T06:55:12.097651+0000","last_active":"2024-06-13T06:57:22.395386+0000","last_peered":"2024-06-13T06:57:22.395386+0000","last_clean":"2024-06-13T06:57:22.395386+0000","last_became_active":"2024-06-13T06:55:12.095644+0000","last_became_peered":"2024-06-13T06:55:12.095644+0000","last_unstale":"2024-06-13T06:57:22.395386+0000","last_undegraded":"2024-06-13T06:57:22.395386+0000","last_fullsized":"2024-06-13T06:57:22.395386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:09:54.328428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.266581+0000","last_change":"2024-06-13T06:54:58.891187+0000","last_active":"2024-06-13T06:58:28.266581+0000","last_peered":"2024-06-13T06:58:28.266581+0000","last_clean":"2024-06-13T06:58:28.266581+0000","last_became_active":"2024-06-13T06:54:58.890848+0000","last_became_peered":"2024-06-13T06:54:58.890848+0000","last_unstale":"2024-06-13T06:58:28.266581+0000","last_undegraded":"2024-06-13T06:58:28.266581+0000","last_fullsized":"2024-06-13T06:58:28.266581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:13:31.709650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":285,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:26.041470+0000","last_change":"2024-06-13T06:54:54.631482+0000","last_active":"2024-06-13T07:00:26.041470+0000","last_peered":"2024-06-13T07:00:26.041470+0000","last_clean":"2024-06-13T07:00:26.041470+0000","last_became_active":"2024-06-13T06:54:54.630818+0000","last_became_peered":"2024-06-13T06:54:54.630818+0000","last_unstale":"2024-06-13T07:00:26.041470+0000","last_undegraded":"2024-06-13T07:00:26.041470+0000","last_fullsized":"2024-06-13T07:00:26.041470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:34:18.801727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.670568+0000","last_change":"2024-06-13T06:55:12.018856+0000","last_active":"2024-06-13T06:57:19.670568+0000","last_peered":"2024-06-13T06:57:19.670568+0000","last_clean":"2024-06-13T06:57:19.670568+0000","last_became_active":"2024-06-13T06:55:12.018488+0000","last_became_peered":"2024-06-13T06:55:12.018488+0000","last_unstale":"2024-06-13T06:57:19.670568+0000","last_undegraded":"2024-06-13T06:57:19.670568+0000","last_fullsized":"2024-06-13T06:57:19.670568+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:14.973197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":25,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281859+0000","last_change":"2024-06-13T06:55:29.377203+0000","last_active":"2024-06-13T06:58:28.281859+0000","last_peered":"2024-06-13T06:58:28.281859+0000","last_clean":"2024-06-13T06:58:28.281859+0000","last_became_active":"2024-06-13T06:55:29.375263+0000","last_became_peered":"2024-06-13T06:55:29.375263+0000","last_unstale":"2024-06-13T06:58:28.281859+0000","last_undegraded":"2024-06-13T06:58:28.281859+0000","last_fullsized":"2024-06-13T06:58:28.281859+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:06:07.935172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:03.284069+0000","last_change":"2024-06-13T06:54:54.335319+0000","last_active":"2024-06-13T06:57:03.284069+0000","last_peered":"2024-06-13T06:57:03.284069+0000","last_clean":"2024-06-13T06:57:03.284069+0000","last_became_active":"2024-06-13T06:54:54.334896+0000","last_became_peered":"2024-06-13T06:54:54.334896+0000","last_unstale":"2024-06-13T06:57:03.284069+0000","last_undegraded":"2024-06-13T06:57:03.284069+0000","last_fullsized":"2024-06-13T06:57:03.284069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:40.331535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:49.673756+0000","last_change":"2024-06-13T06:53:16.968830+0000","last_active":"2024-06-13T06:57:49.673756+0000","last_peered":"2024-06-13T06:57:49.673756+0000","last_clean":"2024-06-13T06:57:49.673756+0000","last_became_active":"2024-06-13T06:53:16.968361+0000","last_became_peered":"2024-06-13T06:53:16.968361+0000","last_unstale":"2024-06-13T06:57:49.673756+0000","last_undegraded":"2024-06-13T06:57:49.673756+0000","last_fullsized":"2024-06-13T06:57:49.673756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_clean_scrub_stamp":"2024-06-13T06:49:18.858540+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:32:26.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542736+0000","last_change":"2024-06-13T06:56:29.602124+0000","last_active":"2024-06-13T06:57:21.542736+0000","last_peered":"2024-06-13T06:57:21.542736+0000","last_clean":"2024-06-13T06:57:21.542736+0000","last_became_active":"2024-06-13T06:56:29.601650+0000","last_became_peered":"2024-06-13T06:56:29.601650+0000","last_unstale":"2024-06-13T06:57:21.542736+0000","last_undegraded":"2024-06-13T06:57:21.542736+0000","last_fullsized":"2024-06-13T06:57:21.542736+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:37:23.903614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.297076+0000","last_change":"2024-06-13T06:54:58.898930+0000","last_active":"2024-06-13T06:58:28.297076+0000","last_peered":"2024-06-13T06:58:28.297076+0000","last_clean":"2024-06-13T06:58:28.297076+0000","last_became_active":"2024-06-13T06:54:58.898384+0000","last_became_peered":"2024-06-13T06:54:58.898384+0000","last_unstale":"2024-06-13T06:58:28.297076+0000","last_undegraded":"2024-06-13T06:58:28.297076+0000","last_fullsized":"2024-06-13T06:58:28.297076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:06:02.210336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:54.667753+0000","last_change":"2024-06-13T06:54:54.328776+0000","last_active":"2024-06-13T06:56:54.667753+0000","last_peered":"2024-06-13T06:56:54.667753+0000","last_clean":"2024-06-13T06:56:54.667753+0000","last_became_active":"2024-06-13T06:54:54.327773+0000","last_became_peered":"2024-06-13T06:54:54.327773+0000","last_unstale":"2024-06-13T06:56:54.667753+0000","last_undegraded":"2024-06-13T06:56:54.667753+0000","last_fullsized":"2024-06-13T06:56:54.667753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:23:02.240774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.836611+0000","last_change":"2024-06-13T06:55:12.013862+0000","last_active":"2024-06-13T06:57:19.836611+0000","last_peered":"2024-06-13T06:57:19.836611+0000","last_clean":"2024-06-13T06:57:19.836611+0000","last_became_active":"2024-06-13T06:55:12.013389+0000","last_became_peered":"2024-06-13T06:55:12.013389+0000","last_unstale":"2024-06-13T06:57:19.836611+0000","last_undegraded":"2024-06-13T06:57:19.836611+0000","last_fullsized":"2024-06-13T06:57:19.836611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:49.701202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558954+0000","last_change":"2024-06-13T06:56:29.539862+0000","last_active":"2024-06-13T06:56:32.558954+0000","last_peered":"2024-06-13T06:56:32.558954+0000","last_clean":"2024-06-13T06:56:32.558954+0000","last_became_active":"2024-06-13T06:56:29.536109+0000","last_became_peered":"2024-06-13T06:56:29.536109+0000","last_unstale":"2024-06-13T06:56:32.558954+0000","last_undegraded":"2024-06-13T06:56:32.558954+0000","last_fullsized":"2024-06-13T06:56:32.558954+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:23.307850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.288243+0000","last_change":"2024-06-13T06:54:58.901773+0000","last_active":"2024-06-13T06:58:28.288243+0000","last_peered":"2024-06-13T06:58:28.288243+0000","last_clean":"2024-06-13T06:58:28.288243+0000","last_became_active":"2024-06-13T06:54:58.901168+0000","last_became_peered":"2024-06-13T06:54:58.901168+0000","last_unstale":"2024-06-13T06:58:28.288243+0000","last_undegraded":"2024-06-13T06:58:28.288243+0000","last_fullsized":"2024-06-13T06:58:28.288243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:57:15.848785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:11.078009+0000","last_change":"2024-06-13T06:55:12.095890+0000","last_active":"2024-06-13T06:57:11.078009+0000","last_peered":"2024-06-13T06:57:11.078009+0000","last_clean":"2024-06-13T06:57:11.078009+0000","last_became_active":"2024-06-13T06:55:12.095397+0000","last_became_peered":"2024-06-13T06:55:12.095397+0000","last_unstale":"2024-06-13T06:57:11.078009+0000","last_undegraded":"2024-06-13T06:57:11.078009+0000","last_fullsized":"2024-06-13T06:57:11.078009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:56:18.211036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'13","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:59:15.573034+0000","last_change":"2024-06-13T06:54:54.337946+0000","last_active":"2024-06-13T06:59:15.573034+0000","last_peered":"2024-06-13T06:59:15.573034+0000","last_clean":"2024-06-13T06:59:15.573034+0000","last_became_active":"2024-06-13T06:54:54.337028+0000","last_became_peered":"2024-06-13T06:54:54.337028+0000","last_unstale":"2024-06-13T06:59:15.573034+0000","last_undegraded":"2024-06-13T06:59:15.573034+0000","last_fullsized":"2024-06-13T06:59:15.573034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:08:52.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275687+0000","last_change":"2024-06-13T06:54:58.895110+0000","last_active":"2024-06-13T06:58:28.275687+0000","last_peered":"2024-06-13T06:58:28.275687+0000","last_clean":"2024-06-13T06:58:28.275687+0000","last_became_active":"2024-06-13T06:54:58.893481+0000","last_became_peered":"2024-06-13T06:54:58.893481+0000","last_unstale":"2024-06-13T06:58:28.275687+0000","last_undegraded":"2024-06-13T06:58:28.275687+0000","last_fullsized":"2024-06-13T06:58:28.275687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:54.932110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.652273+0000","last_change":"2024-06-13T06:55:11.997712+0000","last_active":"2024-06-13T06:57:17.652273+0000","last_peered":"2024-06-13T06:57:17.652273+0000","last_clean":"2024-06-13T06:57:17.652273+0000","last_became_active":"2024-06-13T06:55:11.997193+0000","last_became_peered":"2024-06-13T06:55:11.997193+0000","last_unstale":"2024-06-13T06:57:17.652273+0000","last_undegraded":"2024-06-13T06:57:17.652273+0000","last_fullsized":"2024-06-13T06:57:17.652273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:15.168070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:08.284826+0000","last_change":"2024-06-13T06:54:54.520121+0000","last_active":"2024-06-13T06:57:08.284826+0000","last_peered":"2024-06-13T06:57:08.284826+0000","last_clean":"2024-06-13T06:57:08.284826+0000","last_became_active":"2024-06-13T06:54:54.519723+0000","last_became_peered":"2024-06-13T06:54:54.519723+0000","last_unstale":"2024-06-13T06:57:08.284826+0000","last_undegraded":"2024-06-13T06:57:08.284826+0000","last_fullsized":"2024-06-13T06:57:08.284826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:19:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304745+0000","last_change":"2024-06-13T06:54:58.923626+0000","last_active":"2024-06-13T06:58:28.304745+0000","last_peered":"2024-06-13T06:58:28.304745+0000","last_clean":"2024-06-13T06:58:28.304745+0000","last_became_active":"2024-06-13T06:54:58.923192+0000","last_became_peered":"2024-06-13T06:54:58.923192+0000","last_unstale":"2024-06-13T06:58:28.304745+0000","last_undegraded":"2024-06-13T06:58:28.304745+0000","last_fullsized":"2024-06-13T06:58:28.304745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:26:13.231549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287371+0000","last_change":"2024-06-13T06:55:12.096244+0000","last_active":"2024-06-13T06:57:28.287371+0000","last_peered":"2024-06-13T06:57:28.287371+0000","last_clean":"2024-06-13T06:57:28.287371+0000","last_became_active":"2024-06-13T06:55:12.095758+0000","last_became_peered":"2024-06-13T06:55:12.095758+0000","last_unstale":"2024-06-13T06:57:28.287371+0000","last_undegraded":"2024-06-13T06:57:28.287371+0000","last_fullsized":"2024-06-13T06:57:28.287371+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:38.122282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:08.168621+0000","last_change":"2024-06-13T06:54:54.630488+0000","last_active":"2024-06-13T06:57:08.168621+0000","last_peered":"2024-06-13T06:57:08.168621+0000","last_clean":"2024-06-13T06:57:08.168621+0000","last_became_active":"2024-06-13T06:54:54.630224+0000","last_became_peered":"2024-06-13T06:54:54.630224+0000","last_unstale":"2024-06-13T06:57:08.168621+0000","last_undegraded":"2024-06-13T06:57:08.168621+0000","last_fullsized":"2024-06-13T06:57:08.168621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:33:19.107026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.263285+0000","last_change":"2024-06-13T06:54:58.894466+0000","last_active":"2024-06-13T06:58:28.263285+0000","last_peered":"2024-06-13T06:58:28.263285+0000","last_clean":"2024-06-13T06:58:28.263285+0000","last_became_active":"2024-06-13T06:54:58.894034+0000","last_became_peered":"2024-06-13T06:54:58.894034+0000","last_unstale":"2024-06-13T06:58:28.263285+0000","last_undegraded":"2024-06-13T06:58:28.263285+0000","last_fullsized":"2024-06-13T06:58:28.263285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:44.051565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:30.556536+0000","last_change":"2024-06-13T06:55:12.022078+0000","last_active":"2024-06-13T06:57:30.556536+0000","last_peered":"2024-06-13T06:57:30.556536+0000","last_clean":"2024-06-13T06:57:30.556536+0000","last_became_active":"2024-06-13T06:55:12.021652+0000","last_became_peered":"2024-06-13T06:55:12.021652+0000","last_unstale":"2024-06-13T06:57:30.556536+0000","last_undegraded":"2024-06-13T06:57:30.556536+0000","last_fullsized":"2024-06-13T06:57:30.556536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:25:57.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:06.077272+0000","last_change":"2024-06-13T06:54:54.336757+0000","last_active":"2024-06-13T06:57:06.077272+0000","last_peered":"2024-06-13T06:57:06.077272+0000","last_clean":"2024-06-13T06:57:06.077272+0000","last_became_active":"2024-06-13T06:54:54.336012+0000","last_became_peered":"2024-06-13T06:54:54.336012+0000","last_unstale":"2024-06-13T06:57:06.077272+0000","last_undegraded":"2024-06-13T06:57:06.077272+0000","last_fullsized":"2024-06-13T06:57:06.077272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:52:21.227464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292857+0000","last_change":"2024-06-13T06:54:58.919631+0000","last_active":"2024-06-13T06:58:28.292857+0000","last_peered":"2024-06-13T06:58:28.292857+0000","last_clean":"2024-06-13T06:58:28.292857+0000","last_became_active":"2024-06-13T06:54:58.919274+0000","last_became_peered":"2024-06-13T06:54:58.919274+0000","last_unstale":"2024-06-13T06:58:28.292857+0000","last_undegraded":"2024-06-13T06:58:28.292857+0000","last_fullsized":"2024-06-13T06:58:28.292857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:22.453797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287343+0000","last_change":"2024-06-13T06:55:12.017338+0000","last_active":"2024-06-13T06:57:28.287343+0000","last_peered":"2024-06-13T06:57:28.287343+0000","last_clean":"2024-06-13T06:57:28.287343+0000","last_became_active":"2024-06-13T06:55:12.016841+0000","last_became_peered":"2024-06-13T06:55:12.016841+0000","last_unstale":"2024-06-13T06:57:28.287343+0000","last_undegraded":"2024-06-13T06:57:28.287343+0000","last_fullsized":"2024-06-13T06:57:28.287343+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:40:13.525892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:07.651040+0000","last_change":"2024-06-13T06:54:54.328808+0000","last_active":"2024-06-13T06:57:07.651040+0000","last_peered":"2024-06-13T06:57:07.651040+0000","last_clean":"2024-06-13T06:57:07.651040+0000","last_became_active":"2024-06-13T06:54:54.326982+0000","last_became_peered":"2024-06-13T06:54:54.326982+0000","last_unstale":"2024-06-13T06:57:07.651040+0000","last_undegraded":"2024-06-13T06:57:07.651040+0000","last_fullsized":"2024-06-13T06:57:07.651040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:22:53.854275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.303609+0000","last_change":"2024-06-13T06:54:58.925728+0000","last_active":"2024-06-13T06:58:28.303609+0000","last_peered":"2024-06-13T06:58:28.303609+0000","last_clean":"2024-06-13T06:58:28.303609+0000","last_became_active":"2024-06-13T06:54:58.925407+0000","last_became_peered":"2024-06-13T06:54:58.925407+0000","last_unstale":"2024-06-13T06:58:28.303609+0000","last_undegraded":"2024-06-13T06:58:28.303609+0000","last_fullsized":"2024-06-13T06:58:28.303609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:38.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287363+0000","last_change":"2024-06-13T06:55:12.095733+0000","last_active":"2024-06-13T06:57:28.287363+0000","last_peered":"2024-06-13T06:57:28.287363+0000","last_clean":"2024-06-13T06:57:28.287363+0000","last_became_active":"2024-06-13T06:55:12.095064+0000","last_became_peered":"2024-06-13T06:55:12.095064+0000","last_unstale":"2024-06-13T06:57:28.287363+0000","last_undegraded":"2024-06-13T06:57:28.287363+0000","last_fullsized":"2024-06-13T06:57:28.287363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:26:05.486701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:10.058315+0000","last_change":"2024-06-13T06:54:54.331448+0000","last_active":"2024-06-13T06:57:10.058315+0000","last_peered":"2024-06-13T06:57:10.058315+0000","last_clean":"2024-06-13T06:57:10.058315+0000","last_became_active":"2024-06-13T06:54:54.330964+0000","last_became_peered":"2024-06-13T06:54:54.330964+0000","last_unstale":"2024-06-13T06:57:10.058315+0000","last_undegraded":"2024-06-13T06:57:10.058315+0000","last_fullsized":"2024-06-13T06:57:10.058315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:23:12.211412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282119+0000","last_change":"2024-06-13T06:54:58.899112+0000","last_active":"2024-06-13T06:58:28.282119+0000","last_peered":"2024-06-13T06:58:28.282119+0000","last_clean":"2024-06-13T06:58:28.282119+0000","last_became_active":"2024-06-13T06:54:58.898742+0000","last_became_peered":"2024-06-13T06:54:58.898742+0000","last_unstale":"2024-06-13T06:58:28.282119+0000","last_undegraded":"2024-06-13T06:58:28.282119+0000","last_fullsized":"2024-06-13T06:58:28.282119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:50:12.983287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:23.170398+0000","last_change":"2024-06-13T06:55:12.014149+0000","last_active":"2024-06-13T06:57:23.170398+0000","last_peered":"2024-06-13T06:57:23.170398+0000","last_clean":"2024-06-13T06:57:23.170398+0000","last_became_active":"2024-06-13T06:55:12.013454+0000","last_became_peered":"2024-06-13T06:55:12.013454+0000","last_unstale":"2024-06-13T06:57:23.170398+0000","last_undegraded":"2024-06-13T06:57:23.170398+0000","last_fullsized":"2024-06-13T06:57:23.170398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:13:39.368540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.651570+0000","last_change":"2024-06-13T06:54:54.633268+0000","last_active":"2024-06-13T06:57:12.651570+0000","last_peered":"2024-06-13T06:57:12.651570+0000","last_clean":"2024-06-13T06:57:12.651570+0000","last_became_active":"2024-06-13T06:54:54.632840+0000","last_became_peered":"2024-06-13T06:54:54.632840+0000","last_unstale":"2024-06-13T06:57:12.651570+0000","last_undegraded":"2024-06-13T06:57:12.651570+0000","last_fullsized":"2024-06-13T06:57:12.651570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:21.653727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281941+0000","last_change":"2024-06-13T06:54:58.893417+0000","last_active":"2024-06-13T06:58:28.281941+0000","last_peered":"2024-06-13T06:58:28.281941+0000","last_clean":"2024-06-13T06:58:28.281941+0000","last_became_active":"2024-06-13T06:54:58.892381+0000","last_became_peered":"2024-06-13T06:54:58.892381+0000","last_unstale":"2024-06-13T06:58:28.281941+0000","last_undegraded":"2024-06-13T06:58:28.281941+0000","last_fullsized":"2024-06-13T06:58:28.281941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:07.979654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:27.543545+0000","last_change":"2024-06-13T06:55:12.018336+0000","last_active":"2024-06-13T06:57:27.543545+0000","last_peered":"2024-06-13T06:57:27.543545+0000","last_clean":"2024-06-13T06:57:27.543545+0000","last_became_active":"2024-06-13T06:55:12.017861+0000","last_became_peered":"2024-06-13T06:55:12.017861+0000","last_unstale":"2024-06-13T06:57:27.543545+0000","last_undegraded":"2024-06-13T06:57:27.543545+0000","last_fullsized":"2024-06-13T06:57:27.543545+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:41.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:11.541680+0000","last_change":"2024-06-13T06:54:54.333036+0000","last_active":"2024-06-13T06:57:11.541680+0000","last_peered":"2024-06-13T06:57:11.541680+0000","last_clean":"2024-06-13T06:57:11.541680+0000","last_became_active":"2024-06-13T06:54:54.332642+0000","last_became_peered":"2024-06-13T06:54:54.332642+0000","last_unstale":"2024-06-13T06:57:11.541680+0000","last_undegraded":"2024-06-13T06:57:11.541680+0000","last_fullsized":"2024-06-13T06:57:11.541680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:06:06.809024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279872+0000","last_change":"2024-06-13T06:54:58.881894+0000","last_active":"2024-06-13T06:58:28.279872+0000","last_peered":"2024-06-13T06:58:28.279872+0000","last_clean":"2024-06-13T06:58:28.279872+0000","last_became_active":"2024-06-13T06:54:58.881591+0000","last_became_peered":"2024-06-13T06:54:58.881591+0000","last_unstale":"2024-06-13T06:58:28.279872+0000","last_undegraded":"2024-06-13T06:58:28.279872+0000","last_fullsized":"2024-06-13T06:58:28.279872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:49:00.944212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542642+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:57:21.542642+0000","last_peered":"2024-06-13T06:57:21.542642+0000","last_clean":"2024-06-13T06:57:21.542642+0000","last_became_active":"2024-06-13T06:55:12.000748+0000","last_became_peered":"2024-06-13T06:55:12.000748+0000","last_unstale":"2024-06-13T06:57:21.542642+0000","last_undegraded":"2024-06-13T06:57:21.542642+0000","last_fullsized":"2024-06-13T06:57:21.542642+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:22.035807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.393505+0000","last_change":"2024-06-13T06:54:54.342724+0000","last_active":"2024-06-13T06:57:12.393505+0000","last_peered":"2024-06-13T06:57:12.393505+0000","last_clean":"2024-06-13T06:57:12.393505+0000","last_became_active":"2024-06-13T06:54:54.342267+0000","last_became_peered":"2024-06-13T06:54:54.342267+0000","last_unstale":"2024-06-13T06:57:12.393505+0000","last_undegraded":"2024-06-13T06:57:12.393505+0000","last_fullsized":"2024-06-13T06:57:12.393505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:27:53.039202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271540+0000","last_change":"2024-06-13T06:54:58.892365+0000","last_active":"2024-06-13T06:58:28.271540+0000","last_peered":"2024-06-13T06:58:28.271540+0000","last_clean":"2024-06-13T06:58:28.271540+0000","last_became_active":"2024-06-13T06:54:58.889395+0000","last_became_peered":"2024-06-13T06:54:58.889395+0000","last_unstale":"2024-06-13T06:58:28.271540+0000","last_undegraded":"2024-06-13T06:58:28.271540+0000","last_fullsized":"2024-06-13T06:58:28.271540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:55.798293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272280+0000","last_change":"2024-06-13T06:54:58.888570+0000","last_active":"2024-06-13T06:58:28.272280+0000","last_peered":"2024-06-13T06:58:28.272280+0000","last_clean":"2024-06-13T06:58:28.272280+0000","last_became_active":"2024-06-13T06:54:58.888180+0000","last_became_peered":"2024-06-13T06:54:58.888180+0000","last_unstale":"2024-06-13T06:58:28.272280+0000","last_undegraded":"2024-06-13T06:58:28.272280+0000","last_fullsized":"2024-06-13T06:58:28.272280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:46:42.537790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271099+0000","last_change":"2024-06-13T06:54:58.893476+0000","last_active":"2024-06-13T06:58:28.271099+0000","last_peered":"2024-06-13T06:58:28.271099+0000","last_clean":"2024-06-13T06:58:28.271099+0000","last_became_active":"2024-06-13T06:54:58.893021+0000","last_became_peered":"2024-06-13T06:54:58.893021+0000","last_unstale":"2024-06-13T06:58:28.271099+0000","last_undegraded":"2024-06-13T06:58:28.271099+0000","last_fullsized":"2024-06-13T06:58:28.271099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:38.739070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301611+0000","last_change":"2024-06-13T06:54:58.911042+0000","last_active":"2024-06-13T06:58:28.301611+0000","last_peered":"2024-06-13T06:58:28.301611+0000","last_clean":"2024-06-13T06:58:28.301611+0000","last_became_active":"2024-06-13T06:54:58.910641+0000","last_became_peered":"2024-06-13T06:54:58.910641+0000","last_unstale":"2024-06-13T06:58:28.301611+0000","last_undegraded":"2024-06-13T06:58:28.301611+0000","last_fullsized":"2024-06-13T06:58:28.301611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:21:39.467223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305071+0000","last_change":"2024-06-13T06:55:29.382730+0000","last_active":"2024-06-13T06:58:28.305071+0000","last_peered":"2024-06-13T06:58:28.305071+0000","last_clean":"2024-06-13T06:58:28.305071+0000","last_became_active":"2024-06-13T06:55:29.382162+0000","last_became_peered":"2024-06-13T06:55:29.382162+0000","last_unstale":"2024-06-13T06:58:28.305071+0000","last_undegraded":"2024-06-13T06:58:28.305071+0000","last_fullsized":"2024-06-13T06:58:28.305071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:05.486414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.303120+0000","last_change":"2024-06-13T06:55:29.385380+0000","last_active":"2024-06-13T06:58:28.303120+0000","last_peered":"2024-06-13T06:58:28.303120+0000","last_clean":"2024-06-13T06:58:28.303120+0000","last_became_active":"2024-06-13T06:55:29.384855+0000","last_became_peered":"2024-06-13T06:55:29.384855+0000","last_unstale":"2024-06-13T06:58:28.303120+0000","last_undegraded":"2024-06-13T06:58:28.303120+0000","last_fullsized":"2024-06-13T06:58:28.303120+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:31:01.945632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271653+0000","last_change":"2024-06-13T06:54:58.889128+0000","last_active":"2024-06-13T06:58:28.271653+0000","last_peered":"2024-06-13T06:58:28.271653+0000","last_clean":"2024-06-13T06:58:28.271653+0000","last_became_active":"2024-06-13T06:54:58.888606+0000","last_became_peered":"2024-06-13T06:54:58.888606+0000","last_unstale":"2024-06-13T06:58:28.271653+0000","last_undegraded":"2024-06-13T06:58:28.271653+0000","last_fullsized":"2024-06-13T06:58:28.271653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:53:43.762968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276436+0000","last_change":"2024-06-13T06:54:58.896098+0000","last_active":"2024-06-13T06:58:28.276436+0000","last_peered":"2024-06-13T06:58:28.276436+0000","last_clean":"2024-06-13T06:58:28.276436+0000","last_became_active":"2024-06-13T06:54:58.895722+0000","last_became_peered":"2024-06-13T06:54:58.895722+0000","last_unstale":"2024-06-13T06:58:28.276436+0000","last_undegraded":"2024-06-13T06:58:28.276436+0000","last_fullsized":"2024-06-13T06:58:28.276436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:18:14.421363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279821+0000","last_change":"2024-06-13T06:54:58.905590+0000","last_active":"2024-06-13T06:58:28.279821+0000","last_peered":"2024-06-13T06:58:28.279821+0000","last_clean":"2024-06-13T06:58:28.279821+0000","last_became_active":"2024-06-13T06:54:58.905146+0000","last_became_peered":"2024-06-13T06:54:58.905146+0000","last_unstale":"2024-06-13T06:58:28.279821+0000","last_undegraded":"2024-06-13T06:58:28.279821+0000","last_fullsized":"2024-06-13T06:58:28.279821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:42:08.010906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278426+0000","last_change":"2024-06-13T06:54:58.898037+0000","last_active":"2024-06-13T06:58:28.278426+0000","last_peered":"2024-06-13T06:58:28.278426+0000","last_clean":"2024-06-13T06:58:28.278426+0000","last_became_active":"2024-06-13T06:54:58.897692+0000","last_became_peered":"2024-06-13T06:54:58.897692+0000","last_unstale":"2024-06-13T06:58:28.278426+0000","last_undegraded":"2024-06-13T06:58:28.278426+0000","last_fullsized":"2024-06-13T06:58:28.278426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:44:30.018192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275652+0000","last_change":"2024-06-13T06:54:58.903441+0000","last_active":"2024-06-13T06:58:28.275652+0000","last_peered":"2024-06-13T06:58:28.275652+0000","last_clean":"2024-06-13T06:58:28.275652+0000","last_became_active":"2024-06-13T06:54:58.893598+0000","last_became_peered":"2024-06-13T06:54:58.893598+0000","last_unstale":"2024-06-13T06:58:28.275652+0000","last_undegraded":"2024-06-13T06:58:28.275652+0000","last_fullsized":"2024-06-13T06:58:28.275652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:38:29.968619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282488+0000","last_change":"2024-06-13T06:54:58.918594+0000","last_active":"2024-06-13T06:58:28.282488+0000","last_peered":"2024-06-13T06:58:28.282488+0000","last_clean":"2024-06-13T06:58:28.282488+0000","last_became_active":"2024-06-13T06:54:58.918255+0000","last_became_peered":"2024-06-13T06:54:58.918255+0000","last_unstale":"2024-06-13T06:58:28.282488+0000","last_undegraded":"2024-06-13T06:58:28.282488+0000","last_fullsized":"2024-06-13T06:58:28.282488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:39:48.527707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305484+0000","last_change":"2024-06-13T06:54:58.926253+0000","last_active":"2024-06-13T06:58:28.305484+0000","last_peered":"2024-06-13T06:58:28.305484+0000","last_clean":"2024-06-13T06:58:28.305484+0000","last_became_active":"2024-06-13T06:54:58.924611+0000","last_became_peered":"2024-06-13T06:54:58.924611+0000","last_unstale":"2024-06-13T06:58:28.305484+0000","last_undegraded":"2024-06-13T06:58:28.305484+0000","last_fullsized":"2024-06-13T06:58:28.305484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:29:23.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267286+0000","last_change":"2024-06-13T06:54:58.890454+0000","last_active":"2024-06-13T06:58:28.267286+0000","last_peered":"2024-06-13T06:58:28.267286+0000","last_clean":"2024-06-13T06:58:28.267286+0000","last_became_active":"2024-06-13T06:54:58.890129+0000","last_became_peered":"2024-06-13T06:54:58.890129+0000","last_unstale":"2024-06-13T06:58:28.267286+0000","last_undegraded":"2024-06-13T06:58:28.267286+0000","last_fullsized":"2024-06-13T06:58:28.267286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:10.819536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73030,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1445,"num_read_kb":802,"num_write":181,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1656,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":219090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383765,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":570102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645401,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37128,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708024,"statfs":{"total":95995035648,"available":95957016576,"internally_reserved":0,"allocated":1835008,"data_stored":1225909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":574390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070566,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":570170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332204,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1166569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495728,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":565470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36516,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708636,"statfs":{"total":95995035648,"available":95957643264,"internally_reserved":0,"allocated":1208320,"data_stored":633963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":566858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":566108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182529,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":565664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920894,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1232896,"data_stored":640162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1162157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T07:00:30.634 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-13T07:00:30.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-13T07:00:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 -- 172.21.15.26:0/1797922565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 msgr2=0x7fa5b0108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 --2- 172.21.15.26:0/1797922565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 0x7fa5b0108420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009860 tx=0x7fa5a002a340 comp rx=0 tx=0).stop 2024-06-13T07:00:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 -- 172.21.15.26:0/1797922565 shutdown_connections 2024-06-13T07:00:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 --2- 172.21.15.26:0/1797922565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b0109bb0 0x7fa5b0110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 --2- 172.21.15.26:0/1797922565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b01091f0 0x7fa5b0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 --2- 172.21.15.26:0/1797922565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 0x7fa5b0108420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 -- 172.21.15.26:0/1797922565 >> 172.21.15.26:0/1797922565 conn(0x7fa5b0076fc0 msgr2=0x7fa5b00793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 -- 172.21.15.26:0/1797922565 shutdown_connections 2024-06-13T07:00:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.900+0000 7fa5b7600640 1 -- 172.21.15.26:0/1797922565 wait complete. 2024-06-13T07:00:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.901+0000 7fa5b7600640 1 Processor -- start 2024-06-13T07:00:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.901+0000 7fa5b7600640 1 -- start start 2024-06-13T07:00:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.901+0000 7fa5b7600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 0x7fa5b019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.901+0000 7fa5b7600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b01091f0 0x7fa5b019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.901+0000 7fa5b7600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b0109bb0 0x7fa5b01a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.901+0000 7fa5b7600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0113c90 con 0x7fa5b0108000 2024-06-13T07:00:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.901+0000 7fa5b7600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0113b10 con 0x7fa5b0109bb0 2024-06-13T07:00:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.901+0000 7fa5b7600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0113e10 con 0x7fa5b01091f0 2024-06-13T07:00:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 0x7fa5b019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 0x7fa5b019e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38534/0 (socket says 172.21.15.26:38534) 2024-06-13T07:00:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b4c00640 1 -- 172.21.15.26:0/2690668696 learned_addr learned my addr 172.21.15.26:0/2690668696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b5600640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b0109bb0 0x7fa5b01a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5a7e00640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b01091f0 0x7fa5b019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b4c00640 1 -- 172.21.15.26:0/2690668696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b01091f0 msgr2=0x7fa5b019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b4c00640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b01091f0 0x7fa5b019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b4c00640 1 -- 172.21.15.26:0/2690668696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b0109bb0 msgr2=0x7fa5b01a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b4c00640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b0109bb0 0x7fa5b01a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b4c00640 1 -- 172.21.15.26:0/2690668696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b01a5f50 con 0x7fa5b0108000 2024-06-13T07:00:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.902+0000 7fa5b5600640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b0109bb0 0x7fa5b01a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.903+0000 7fa5b4c00640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 0x7fa5b019e330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00305b0 tx=0x7fa5a0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.903+0000 7fa5ae000640 1 -- 172.21.15.26:0/2690668696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a003f020 con 0x7fa5b0108000 2024-06-13T07:00:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.903+0000 7fa5ae000640 1 -- 172.21.15.26:0/2690668696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0030850 con 0x7fa5b0108000 2024-06-13T07:00:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.903+0000 7fa5ae000640 1 -- 172.21.15.26:0/2690668696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0037040 con 0x7fa5b0108000 2024-06-13T07:00:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.903+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b01a61e0 con 0x7fa5b0108000 2024-06-13T07:00:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.903+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b01a6680 con 0x7fa5b0108000 2024-06-13T07:00:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.904+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b010f320 con 0x7fa5b0108000 2024-06-13T07:00:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.906+0000 7fa5ae000640 1 -- 172.21.15.26:0/2690668696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa5a0042020 con 0x7fa5b0108000 2024-06-13T07:00:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.908+0000 7fa5ae000640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5940a2c10 0x7fa5940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.908+0000 7fa5ae000640 1 -- 172.21.15.26:0/2690668696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa5a00ef730 con 0x7fa5b0108000 2024-06-13T07:00:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.908+0000 7fa5a7e00640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5940a2c10 0x7fa5940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.909+0000 7fa5a7e00640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5940a2c10 0x7fa5940a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa5b019f840 tx=0x7fa598005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:30.912+0000 7fa5ae000640 1 -- 172.21.15.26:0/2690668696 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5a0049020 con 0x7fa5b0108000 2024-06-13T07:00:31.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:30 smithi026 ceph-mon[32041]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:31.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:30 smithi135 ceph-mon[36276]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.116+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 --> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5b0061de0 con 0x7fa5940a2c10 2024-06-13T07:00:31.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:30 smithi136 ceph-mon[37229]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.131+0000 7fa5ae000640 1 -- 172.21.15.26:0/2690668696 <== mgr.14152 v2:172.21.15.26:6800/662324914 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340501 (secure 0 0 0) 0x7fa5b0061de0 con 0x7fa5940a2c10 2024-06-13T07:00:31.135 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:00:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-06-13T07:00:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5940a2c10 msgr2=0x7fa5940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5940a2c10 0x7fa5940a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa5b019f840 tx=0x7fa598005f90 comp rx=0 tx=0).stop 2024-06-13T07:00:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 msgr2=0x7fa5b019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 0x7fa5b019e330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00305b0 tx=0x7fa5a0033d90 comp rx=0 tx=0).stop 2024-06-13T07:00:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 shutdown_connections 2024-06-13T07:00:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa5940a2c10 0x7fa5940a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b0109bb0 0x7fa5b01a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5b01091f0 0x7fa5b019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:31.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 --2- 172.21.15.26:0/2690668696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b0108000 0x7fa5b019e330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:31.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.139+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 >> 172.21.15.26:0/2690668696 conn(0x7fa5b0076fc0 msgr2=0x7fa5b010c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:31.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.140+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 shutdown_connections 2024-06-13T07:00:31.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:31.140+0000 7fa5b7600640 1 -- 172.21.15.26:0/2690668696 wait complete. 2024-06-13T07:00:31.160 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-06-13T07:00:30.424565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663632,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1571,"num_read_kb":911,"num_write":364,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182392,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402044,"kb_used_data":15452,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539780,"statfs":{"total":1151940427776,"available":1151528734720,"internally_reserved":0,"allocated":15822848,"data_stored":8807522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":48,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006016"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293186+0000","last_change":"2024-06-13T06:54:58.919669+0000","last_active":"2024-06-13T06:58:28.293186+0000","last_peered":"2024-06-13T06:58:28.293186+0000","last_clean":"2024-06-13T06:58:28.293186+0000","last_became_active":"2024-06-13T06:54:58.919334+0000","last_became_peered":"2024-06-13T06:54:58.919334+0000","last_unstale":"2024-06-13T06:58:28.293186+0000","last_undegraded":"2024-06-13T06:58:28.293186+0000","last_fullsized":"2024-06-13T06:58:28.293186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:45:40.831640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295249+0000","last_change":"2024-06-13T06:54:58.909508+0000","last_active":"2024-06-13T06:58:28.295249+0000","last_peered":"2024-06-13T06:58:28.295249+0000","last_clean":"2024-06-13T06:58:28.295249+0000","last_became_active":"2024-06-13T06:54:58.909221+0000","last_became_peered":"2024-06-13T06:54:58.909221+0000","last_unstale":"2024-06-13T06:58:28.295249+0000","last_undegraded":"2024-06-13T06:58:28.295249+0000","last_fullsized":"2024-06-13T06:58:28.295249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:17:07.476858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304063+0000","last_change":"2024-06-13T06:54:58.909283+0000","last_active":"2024-06-13T06:58:28.304063+0000","last_peered":"2024-06-13T06:58:28.304063+0000","last_clean":"2024-06-13T06:58:28.304063+0000","last_became_active":"2024-06-13T06:54:58.908912+0000","last_became_peered":"2024-06-13T06:54:58.908912+0000","last_unstale":"2024-06-13T06:58:28.304063+0000","last_undegraded":"2024-06-13T06:58:28.304063+0000","last_fullsized":"2024-06-13T06:58:28.304063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:48:26.937971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.264605+0000","last_change":"2024-06-13T06:54:58.902821+0000","last_active":"2024-06-13T06:58:28.264605+0000","last_peered":"2024-06-13T06:58:28.264605+0000","last_clean":"2024-06-13T06:58:28.264605+0000","last_became_active":"2024-06-13T06:54:58.902215+0000","last_became_peered":"2024-06-13T06:54:58.902215+0000","last_unstale":"2024-06-13T06:58:28.264605+0000","last_undegraded":"2024-06-13T06:58:28.264605+0000","last_fullsized":"2024-06-13T06:58:28.264605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:11:57.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296624+0000","last_change":"2024-06-13T06:54:58.897038+0000","last_active":"2024-06-13T06:58:28.296624+0000","last_peered":"2024-06-13T06:58:28.296624+0000","last_clean":"2024-06-13T06:58:28.296624+0000","last_became_active":"2024-06-13T06:54:58.896665+0000","last_became_peered":"2024-06-13T06:54:58.896665+0000","last_unstale":"2024-06-13T06:58:28.296624+0000","last_undegraded":"2024-06-13T06:58:28.296624+0000","last_fullsized":"2024-06-13T06:58:28.296624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:24:19.216932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.269970+0000","last_change":"2024-06-13T06:54:58.895501+0000","last_active":"2024-06-13T06:58:28.269970+0000","last_peered":"2024-06-13T06:58:28.269970+0000","last_clean":"2024-06-13T06:58:28.269970+0000","last_became_active":"2024-06-13T06:54:58.895144+0000","last_became_peered":"2024-06-13T06:54:58.895144+0000","last_unstale":"2024-06-13T06:58:28.269970+0000","last_undegraded":"2024-06-13T06:58:28.269970+0000","last_fullsized":"2024-06-13T06:58:28.269970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:50.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272336+0000","last_change":"2024-06-13T06:54:58.901097+0000","last_active":"2024-06-13T06:58:28.272336+0000","last_peered":"2024-06-13T06:58:28.272336+0000","last_clean":"2024-06-13T06:58:28.272336+0000","last_became_active":"2024-06-13T06:54:58.900727+0000","last_became_peered":"2024-06-13T06:54:58.900727+0000","last_unstale":"2024-06-13T06:58:28.272336+0000","last_undegraded":"2024-06-13T06:58:28.272336+0000","last_fullsized":"2024-06-13T06:58:28.272336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:09:38.349517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278488+0000","last_change":"2024-06-13T06:54:58.910740+0000","last_active":"2024-06-13T06:58:28.278488+0000","last_peered":"2024-06-13T06:58:28.278488+0000","last_clean":"2024-06-13T06:58:28.278488+0000","last_became_active":"2024-06-13T06:54:58.910351+0000","last_became_peered":"2024-06-13T06:54:58.910351+0000","last_unstale":"2024-06-13T06:58:28.278488+0000","last_undegraded":"2024-06-13T06:58:28.278488+0000","last_fullsized":"2024-06-13T06:58:28.278488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:45:12.854996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281927+0000","last_change":"2024-06-13T06:55:29.364516+0000","last_active":"2024-06-13T06:58:28.281927+0000","last_peered":"2024-06-13T06:58:28.281927+0000","last_clean":"2024-06-13T06:58:28.281927+0000","last_became_active":"2024-06-13T06:55:29.364079+0000","last_became_peered":"2024-06-13T06:55:29.364079+0000","last_unstale":"2024-06-13T06:58:28.281927+0000","last_undegraded":"2024-06-13T06:58:28.281927+0000","last_fullsized":"2024-06-13T06:58:28.281927+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:51:28.624359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267340+0000","last_change":"2024-06-13T06:54:58.903882+0000","last_active":"2024-06-13T06:58:28.267340+0000","last_peered":"2024-06-13T06:58:28.267340+0000","last_clean":"2024-06-13T06:58:28.267340+0000","last_became_active":"2024-06-13T06:54:58.903495+0000","last_became_peered":"2024-06-13T06:54:58.903495+0000","last_unstale":"2024-06-13T06:58:28.267340+0000","last_undegraded":"2024-06-13T06:58:28.267340+0000","last_fullsized":"2024-06-13T06:58:28.267340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:50:23.682993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281971+0000","last_change":"2024-06-13T06:54:58.900080+0000","last_active":"2024-06-13T06:58:28.281971+0000","last_peered":"2024-06-13T06:58:28.281971+0000","last_clean":"2024-06-13T06:58:28.281971+0000","last_became_active":"2024-06-13T06:54:58.899736+0000","last_became_peered":"2024-06-13T06:54:58.899736+0000","last_unstale":"2024-06-13T06:58:28.281971+0000","last_undegraded":"2024-06-13T06:58:28.281971+0000","last_fullsized":"2024-06-13T06:58:28.281971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:51:45.808821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.264389+0000","last_change":"2024-06-13T06:54:58.900238+0000","last_active":"2024-06-13T06:58:28.264389+0000","last_peered":"2024-06-13T06:58:28.264389+0000","last_clean":"2024-06-13T06:58:28.264389+0000","last_became_active":"2024-06-13T06:54:58.899872+0000","last_became_peered":"2024-06-13T06:54:58.899872+0000","last_unstale":"2024-06-13T06:58:28.264389+0000","last_undegraded":"2024-06-13T06:58:28.264389+0000","last_fullsized":"2024-06-13T06:58:28.264389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:12:40.780894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.296863+0000","last_change":"2024-06-13T06:54:58.901981+0000","last_active":"2024-06-13T06:58:28.296863+0000","last_peered":"2024-06-13T06:58:28.296863+0000","last_clean":"2024-06-13T06:58:28.296863+0000","last_became_active":"2024-06-13T06:54:58.898704+0000","last_became_peered":"2024-06-13T06:54:58.898704+0000","last_unstale":"2024-06-13T06:58:28.296863+0000","last_undegraded":"2024-06-13T06:58:28.296863+0000","last_fullsized":"2024-06-13T06:58:28.296863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:03:56.658091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.299281+0000","last_change":"2024-06-13T06:54:58.909022+0000","last_active":"2024-06-13T06:58:28.299281+0000","last_peered":"2024-06-13T06:58:28.299281+0000","last_clean":"2024-06-13T06:58:28.299281+0000","last_became_active":"2024-06-13T06:54:58.907438+0000","last_became_peered":"2024-06-13T06:54:58.907438+0000","last_unstale":"2024-06-13T06:58:28.299281+0000","last_undegraded":"2024-06-13T06:58:28.299281+0000","last_fullsized":"2024-06-13T06:58:28.299281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:40:34.190510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305584+0000","last_change":"2024-06-13T06:54:58.897691+0000","last_active":"2024-06-13T06:58:28.305584+0000","last_peered":"2024-06-13T06:58:28.305584+0000","last_clean":"2024-06-13T06:58:28.305584+0000","last_became_active":"2024-06-13T06:54:58.897381+0000","last_became_peered":"2024-06-13T06:54:58.897381+0000","last_unstale":"2024-06-13T06:58:28.305584+0000","last_undegraded":"2024-06-13T06:58:28.305584+0000","last_fullsized":"2024-06-13T06:58:28.305584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:10.052052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.295211+0000","last_change":"2024-06-13T06:54:58.907676+0000","last_active":"2024-06-13T06:58:28.295211+0000","last_peered":"2024-06-13T06:58:28.295211+0000","last_clean":"2024-06-13T06:58:28.295211+0000","last_became_active":"2024-06-13T06:54:58.907304+0000","last_became_peered":"2024-06-13T06:54:58.907304+0000","last_unstale":"2024-06-13T06:58:28.295211+0000","last_undegraded":"2024-06-13T06:58:28.295211+0000","last_fullsized":"2024-06-13T06:58:28.295211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:08:20.292516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293328+0000","last_change":"2024-06-13T06:54:58.920012+0000","last_active":"2024-06-13T06:58:28.293328+0000","last_peered":"2024-06-13T06:58:28.293328+0000","last_clean":"2024-06-13T06:58:28.293328+0000","last_became_active":"2024-06-13T06:54:58.919473+0000","last_became_peered":"2024-06-13T06:54:58.919473+0000","last_unstale":"2024-06-13T06:58:28.293328+0000","last_undegraded":"2024-06-13T06:58:28.293328+0000","last_fullsized":"2024-06-13T06:58:28.293328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:49:13.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.280156+0000","last_change":"2024-06-13T06:54:58.902576+0000","last_active":"2024-06-13T06:58:28.280156+0000","last_peered":"2024-06-13T06:58:28.280156+0000","last_clean":"2024-06-13T06:58:28.280156+0000","last_became_active":"2024-06-13T06:54:58.902205+0000","last_became_peered":"2024-06-13T06:54:58.902205+0000","last_unstale":"2024-06-13T06:58:28.280156+0000","last_undegraded":"2024-06-13T06:58:28.280156+0000","last_fullsized":"2024-06-13T06:58:28.280156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:13:24.107081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292347+0000","last_change":"2024-06-13T06:54:58.905392+0000","last_active":"2024-06-13T06:58:28.292347+0000","last_peered":"2024-06-13T06:58:28.292347+0000","last_clean":"2024-06-13T06:58:28.292347+0000","last_became_active":"2024-06-13T06:54:58.905017+0000","last_became_peered":"2024-06-13T06:54:58.905017+0000","last_unstale":"2024-06-13T06:58:28.292347+0000","last_undegraded":"2024-06-13T06:58:28.292347+0000","last_fullsized":"2024-06-13T06:58:28.292347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:05.931653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:37.654508+0000","last_change":"2024-06-13T06:55:12.021359+0000","last_active":"2024-06-13T06:57:37.654508+0000","last_peered":"2024-06-13T06:57:37.654508+0000","last_clean":"2024-06-13T06:57:37.654508+0000","last_became_active":"2024-06-13T06:55:12.020860+0000","last_became_peered":"2024-06-13T06:55:12.020860+0000","last_unstale":"2024-06-13T06:57:37.654508+0000","last_undegraded":"2024-06-13T06:57:37.654508+0000","last_fullsized":"2024-06-13T06:57:37.654508+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:05:00.833905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:00.551277+0000","last_change":"2024-06-13T06:54:54.339220+0000","last_active":"2024-06-13T06:57:00.551277+0000","last_peered":"2024-06-13T06:57:00.551277+0000","last_clean":"2024-06-13T06:57:00.551277+0000","last_became_active":"2024-06-13T06:54:54.338573+0000","last_became_peered":"2024-06-13T06:54:54.338573+0000","last_unstale":"2024-06-13T06:57:00.551277+0000","last_undegraded":"2024-06-13T06:57:00.551277+0000","last_fullsized":"2024-06-13T06:57:00.551277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:53:50.258097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.277556+0000","last_change":"2024-06-13T06:54:58.903920+0000","last_active":"2024-06-13T06:58:28.277556+0000","last_peered":"2024-06-13T06:58:28.277556+0000","last_clean":"2024-06-13T06:58:28.277556+0000","last_became_active":"2024-06-13T06:54:58.895364+0000","last_became_peered":"2024-06-13T06:54:58.895364+0000","last_unstale":"2024-06-13T06:58:28.277556+0000","last_undegraded":"2024-06-13T06:58:28.277556+0000","last_fullsized":"2024-06-13T06:58:28.277556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:04:35.109799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:38.288505+0000","last_change":"2024-06-13T06:55:12.017561+0000","last_active":"2024-06-13T06:57:38.288505+0000","last_peered":"2024-06-13T06:57:38.288505+0000","last_clean":"2024-06-13T06:57:38.288505+0000","last_became_active":"2024-06-13T06:55:12.017104+0000","last_became_peered":"2024-06-13T06:55:12.017104+0000","last_unstale":"2024-06-13T06:57:38.288505+0000","last_undegraded":"2024-06-13T06:57:38.288505+0000","last_fullsized":"2024-06-13T06:57:38.288505+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:08:22.570491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:56.075775+0000","last_change":"2024-06-13T06:54:54.631599+0000","last_active":"2024-06-13T06:56:56.075775+0000","last_peered":"2024-06-13T06:56:56.075775+0000","last_clean":"2024-06-13T06:56:56.075775+0000","last_became_active":"2024-06-13T06:54:54.631007+0000","last_became_peered":"2024-06-13T06:54:54.631007+0000","last_unstale":"2024-06-13T06:56:56.075775+0000","last_undegraded":"2024-06-13T06:56:56.075775+0000","last_fullsized":"2024-06-13T06:56:56.075775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:49:56.602192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304511+0000","last_change":"2024-06-13T06:54:58.917255+0000","last_active":"2024-06-13T06:58:28.304511+0000","last_peered":"2024-06-13T06:58:28.304511+0000","last_clean":"2024-06-13T06:58:28.304511+0000","last_became_active":"2024-06-13T06:54:58.916875+0000","last_became_peered":"2024-06-13T06:54:58.916875+0000","last_unstale":"2024-06-13T06:58:28.304511+0000","last_undegraded":"2024-06-13T06:58:28.304511+0000","last_fullsized":"2024-06-13T06:58:28.304511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:09:47.638802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:43.289081+0000","last_change":"2024-06-13T06:55:12.018209+0000","last_active":"2024-06-13T06:57:43.289081+0000","last_peered":"2024-06-13T06:57:43.289081+0000","last_clean":"2024-06-13T06:57:43.289081+0000","last_became_active":"2024-06-13T06:55:12.017038+0000","last_became_peered":"2024-06-13T06:55:12.017038+0000","last_unstale":"2024-06-13T06:57:43.289081+0000","last_undegraded":"2024-06-13T06:57:43.289081+0000","last_fullsized":"2024-06-13T06:57:43.289081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:36:26.476454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:25.059841+0000","last_change":"2024-06-13T06:54:54.328849+0000","last_active":"2024-06-13T06:57:25.059841+0000","last_peered":"2024-06-13T06:57:25.059841+0000","last_clean":"2024-06-13T06:57:25.059841+0000","last_became_active":"2024-06-13T06:54:54.328388+0000","last_became_peered":"2024-06-13T06:54:54.328388+0000","last_unstale":"2024-06-13T06:57:25.059841+0000","last_undegraded":"2024-06-13T06:57:25.059841+0000","last_fullsized":"2024-06-13T06:57:25.059841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:41.264403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.283009+0000","last_change":"2024-06-13T06:54:58.897588+0000","last_active":"2024-06-13T06:58:28.283009+0000","last_peered":"2024-06-13T06:58:28.283009+0000","last_clean":"2024-06-13T06:58:28.283009+0000","last_became_active":"2024-06-13T06:54:58.897236+0000","last_became_peered":"2024-06-13T06:54:58.897236+0000","last_unstale":"2024-06-13T06:58:28.283009+0000","last_undegraded":"2024-06-13T06:58:28.283009+0000","last_fullsized":"2024-06-13T06:58:28.283009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:20:38.460445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:43.289055+0000","last_change":"2024-06-13T06:55:12.014080+0000","last_active":"2024-06-13T06:57:43.289055+0000","last_peered":"2024-06-13T06:57:43.289055+0000","last_clean":"2024-06-13T06:57:43.289055+0000","last_became_active":"2024-06-13T06:55:12.013627+0000","last_became_peered":"2024-06-13T06:55:12.013627+0000","last_unstale":"2024-06-13T06:57:43.289055+0000","last_undegraded":"2024-06-13T06:57:43.289055+0000","last_fullsized":"2024-06-13T06:57:43.289055+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:29:54.081593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:25.059798+0000","last_change":"2024-06-13T06:54:54.328249+0000","last_active":"2024-06-13T06:57:25.059798+0000","last_peered":"2024-06-13T06:57:25.059798+0000","last_clean":"2024-06-13T06:57:25.059798+0000","last_became_active":"2024-06-13T06:54:54.327832+0000","last_became_peered":"2024-06-13T06:54:54.327832+0000","last_unstale":"2024-06-13T06:57:25.059798+0000","last_undegraded":"2024-06-13T06:57:25.059798+0000","last_fullsized":"2024-06-13T06:57:25.059798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:21:05.624075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281353+0000","last_change":"2024-06-13T06:55:29.383305+0000","last_active":"2024-06-13T06:58:28.281353+0000","last_peered":"2024-06-13T06:58:28.281353+0000","last_clean":"2024-06-13T06:58:28.281353+0000","last_became_active":"2024-06-13T06:55:29.381491+0000","last_became_peered":"2024-06-13T06:55:29.381491+0000","last_unstale":"2024-06-13T06:58:28.281353+0000","last_undegraded":"2024-06-13T06:58:28.281353+0000","last_fullsized":"2024-06-13T06:58:28.281353+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:52:26.437237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:42.655020+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:57:42.655020+0000","last_peered":"2024-06-13T06:57:42.655020+0000","last_clean":"2024-06-13T06:57:42.655020+0000","last_became_active":"2024-06-13T06:55:12.000759+0000","last_became_peered":"2024-06-13T06:55:12.000759+0000","last_unstale":"2024-06-13T06:57:42.655020+0000","last_undegraded":"2024-06-13T06:57:42.655020+0000","last_fullsized":"2024-06-13T06:57:42.655020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:11:35.805338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'8","reported_seq":299,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:25.040779+0000","last_change":"2024-06-13T06:55:29.367538+0000","last_active":"2024-06-13T07:00:25.040779+0000","last_peered":"2024-06-13T07:00:25.040779+0000","last_clean":"2024-06-13T07:00:25.040779+0000","last_became_active":"2024-06-13T06:55:29.367149+0000","last_became_peered":"2024-06-13T06:55:29.367149+0000","last_unstale":"2024-06-13T07:00:25.040779+0000","last_undegraded":"2024-06-13T07:00:25.040779+0000","last_fullsized":"2024-06-13T07:00:25.040779+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:30:55.632799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1459,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.269082+0000","last_change":"2024-06-13T06:54:58.896258+0000","last_active":"2024-06-13T06:58:28.269082+0000","last_peered":"2024-06-13T06:58:28.269082+0000","last_clean":"2024-06-13T06:58:28.269082+0000","last_became_active":"2024-06-13T06:54:58.895904+0000","last_became_peered":"2024-06-13T06:54:58.895904+0000","last_unstale":"2024-06-13T06:58:28.269082+0000","last_undegraded":"2024-06-13T06:58:28.269082+0000","last_fullsized":"2024-06-13T06:58:28.269082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:28:33.615338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:42.654996+0000","last_change":"2024-06-13T06:55:12.018499+0000","last_active":"2024-06-13T06:57:42.654996+0000","last_peered":"2024-06-13T06:57:42.654996+0000","last_clean":"2024-06-13T06:57:42.654996+0000","last_became_active":"2024-06-13T06:55:12.018073+0000","last_became_peered":"2024-06-13T06:55:12.018073+0000","last_unstale":"2024-06-13T06:57:42.654996+0000","last_undegraded":"2024-06-13T06:57:42.654996+0000","last_fullsized":"2024-06-13T06:57:42.654996+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:59:53.206259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:23.429829+0000","last_change":"2024-06-13T06:54:54.339482+0000","last_active":"2024-06-13T06:57:23.429829+0000","last_peered":"2024-06-13T06:57:23.429829+0000","last_clean":"2024-06-13T06:57:23.429829+0000","last_became_active":"2024-06-13T06:54:54.339066+0000","last_became_peered":"2024-06-13T06:54:54.339066+0000","last_unstale":"2024-06-13T06:57:23.429829+0000","last_undegraded":"2024-06-13T06:57:23.429829+0000","last_fullsized":"2024-06-13T06:57:23.429829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:31:04.136530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279253+0000","last_change":"2024-06-13T06:54:58.891154+0000","last_active":"2024-06-13T06:58:28.279253+0000","last_peered":"2024-06-13T06:58:28.279253+0000","last_clean":"2024-06-13T06:58:28.279253+0000","last_became_active":"2024-06-13T06:54:58.890810+0000","last_became_peered":"2024-06-13T06:54:58.890810+0000","last_unstale":"2024-06-13T06:58:28.279253+0000","last_undegraded":"2024-06-13T06:58:28.279253+0000","last_fullsized":"2024-06-13T06:58:28.279253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:14.618233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.836644+0000","last_change":"2024-06-13T06:55:12.025619+0000","last_active":"2024-06-13T06:57:19.836644+0000","last_peered":"2024-06-13T06:57:19.836644+0000","last_clean":"2024-06-13T06:57:19.836644+0000","last_became_active":"2024-06-13T06:55:12.025153+0000","last_became_peered":"2024-06-13T06:55:12.025153+0000","last_unstale":"2024-06-13T06:57:19.836644+0000","last_undegraded":"2024-06-13T06:57:19.836644+0000","last_fullsized":"2024-06-13T06:57:19.836644+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:12.124550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:22.395183+0000","last_change":"2024-06-13T06:54:54.341698+0000","last_active":"2024-06-13T06:57:22.395183+0000","last_peered":"2024-06-13T06:57:22.395183+0000","last_clean":"2024-06-13T06:57:22.395183+0000","last_became_active":"2024-06-13T06:54:54.341113+0000","last_became_peered":"2024-06-13T06:54:54.341113+0000","last_unstale":"2024-06-13T06:57:22.395183+0000","last_undegraded":"2024-06-13T06:57:22.395183+0000","last_fullsized":"2024-06-13T06:57:22.395183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:18:23.368705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279519+0000","last_change":"2024-06-13T06:54:58.913688+0000","last_active":"2024-06-13T06:58:28.279519+0000","last_peered":"2024-06-13T06:58:28.279519+0000","last_clean":"2024-06-13T06:58:28.279519+0000","last_became_active":"2024-06-13T06:54:58.910493+0000","last_became_peered":"2024-06-13T06:54:58.910493+0000","last_unstale":"2024-06-13T06:58:28.279519+0000","last_undegraded":"2024-06-13T06:58:28.279519+0000","last_fullsized":"2024-06-13T06:58:28.279519+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:29:18.081743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.540765+0000","last_change":"2024-06-13T06:55:12.016257+0000","last_active":"2024-06-13T06:57:12.540765+0000","last_peered":"2024-06-13T06:57:12.540765+0000","last_clean":"2024-06-13T06:57:12.540765+0000","last_became_active":"2024-06-13T06:55:12.015594+0000","last_became_peered":"2024-06-13T06:55:12.015594+0000","last_unstale":"2024-06-13T06:57:12.540765+0000","last_undegraded":"2024-06-13T06:57:12.540765+0000","last_fullsized":"2024-06-13T06:57:12.540765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:02:44.642075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542663+0000","last_change":"2024-06-13T06:54:54.330366+0000","last_active":"2024-06-13T06:57:21.542663+0000","last_peered":"2024-06-13T06:57:21.542663+0000","last_clean":"2024-06-13T06:57:21.542663+0000","last_became_active":"2024-06-13T06:54:54.330010+0000","last_became_peered":"2024-06-13T06:54:54.330010+0000","last_unstale":"2024-06-13T06:57:21.542663+0000","last_undegraded":"2024-06-13T06:57:21.542663+0000","last_fullsized":"2024-06-13T06:57:21.542663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:36:01.488766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276547+0000","last_change":"2024-06-13T06:54:58.918076+0000","last_active":"2024-06-13T06:58:28.276547+0000","last_peered":"2024-06-13T06:58:28.276547+0000","last_clean":"2024-06-13T06:58:28.276547+0000","last_became_active":"2024-06-13T06:54:58.917631+0000","last_became_peered":"2024-06-13T06:54:58.917631+0000","last_unstale":"2024-06-13T06:58:28.276547+0000","last_undegraded":"2024-06-13T06:58:28.276547+0000","last_fullsized":"2024-06-13T06:58:28.276547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:31:00.056837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:31.081713+0000","last_change":"2024-06-13T06:56:29.611761+0000","last_active":"2024-06-13T06:57:31.081713+0000","last_peered":"2024-06-13T06:57:31.081713+0000","last_clean":"2024-06-13T06:57:31.081713+0000","last_became_active":"2024-06-13T06:56:29.610385+0000","last_became_peered":"2024-06-13T06:56:29.610385+0000","last_unstale":"2024-06-13T06:57:31.081713+0000","last_undegraded":"2024-06-13T06:57:31.081713+0000","last_fullsized":"2024-06-13T06:57:31.081713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:26:10.678918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'5","reported_seq":287,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:27.044645+0000","last_change":"2024-06-13T06:54:54.335324+0000","last_active":"2024-06-13T07:00:27.044645+0000","last_peered":"2024-06-13T07:00:27.044645+0000","last_clean":"2024-06-13T07:00:27.044645+0000","last_became_active":"2024-06-13T06:54:54.334936+0000","last_became_peered":"2024-06-13T06:54:54.334936+0000","last_unstale":"2024-06-13T07:00:27.044645+0000","last_undegraded":"2024-06-13T07:00:27.044645+0000","last_fullsized":"2024-06-13T07:00:27.044645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:41:52.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281723+0000","last_change":"2024-06-13T06:54:58.902146+0000","last_active":"2024-06-13T06:58:28.281723+0000","last_peered":"2024-06-13T06:58:28.281723+0000","last_clean":"2024-06-13T06:58:28.281723+0000","last_became_active":"2024-06-13T06:54:58.901807+0000","last_became_peered":"2024-06-13T06:54:58.901807+0000","last_unstale":"2024-06-13T06:58:28.281723+0000","last_undegraded":"2024-06-13T06:58:28.281723+0000","last_fullsized":"2024-06-13T06:58:28.281723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:59:53.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:35.557156+0000","last_change":"2024-06-13T06:55:12.025418+0000","last_active":"2024-06-13T06:57:35.557156+0000","last_peered":"2024-06-13T06:57:35.557156+0000","last_clean":"2024-06-13T06:57:35.557156+0000","last_became_active":"2024-06-13T06:55:12.024859+0000","last_became_peered":"2024-06-13T06:55:12.024859+0000","last_unstale":"2024-06-13T06:57:35.557156+0000","last_undegraded":"2024-06-13T06:57:35.557156+0000","last_fullsized":"2024-06-13T06:57:35.557156+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:05:17.553169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.670600+0000","last_change":"2024-06-13T06:54:54.326011+0000","last_active":"2024-06-13T06:57:19.670600+0000","last_peered":"2024-06-13T06:57:19.670600+0000","last_clean":"2024-06-13T06:57:19.670600+0000","last_became_active":"2024-06-13T06:54:54.325664+0000","last_became_peered":"2024-06-13T06:54:54.325664+0000","last_unstale":"2024-06-13T06:57:19.670600+0000","last_undegraded":"2024-06-13T06:57:19.670600+0000","last_fullsized":"2024-06-13T06:57:19.670600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:32.973094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301872+0000","last_change":"2024-06-13T06:55:29.383612+0000","last_active":"2024-06-13T06:58:28.301872+0000","last_peered":"2024-06-13T06:58:28.301872+0000","last_clean":"2024-06-13T06:58:28.301872+0000","last_became_active":"2024-06-13T06:55:29.382174+0000","last_became_peered":"2024-06-13T06:55:29.382174+0000","last_unstale":"2024-06-13T06:58:28.301872+0000","last_undegraded":"2024-06-13T06:58:28.301872+0000","last_fullsized":"2024-06-13T06:58:28.301872+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:09:35.178209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:33.288040+0000","last_change":"2024-06-13T06:55:12.018654+0000","last_active":"2024-06-13T06:57:33.288040+0000","last_peered":"2024-06-13T06:57:33.288040+0000","last_clean":"2024-06-13T06:57:33.288040+0000","last_became_active":"2024-06-13T06:55:12.017219+0000","last_became_peered":"2024-06-13T06:55:12.017219+0000","last_unstale":"2024-06-13T06:57:33.288040+0000","last_undegraded":"2024-06-13T06:57:33.288040+0000","last_fullsized":"2024-06-13T06:57:33.288040+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:46:54.161912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542714+0000","last_change":"2024-06-13T06:54:54.632154+0000","last_active":"2024-06-13T06:57:21.542714+0000","last_peered":"2024-06-13T06:57:21.542714+0000","last_clean":"2024-06-13T06:57:21.542714+0000","last_became_active":"2024-06-13T06:54:54.631721+0000","last_became_peered":"2024-06-13T06:54:54.631721+0000","last_unstale":"2024-06-13T06:57:21.542714+0000","last_undegraded":"2024-06-13T06:57:21.542714+0000","last_fullsized":"2024-06-13T06:57:21.542714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:16:10.492144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278187+0000","last_change":"2024-06-13T06:54:58.898961+0000","last_active":"2024-06-13T06:58:28.278187+0000","last_peered":"2024-06-13T06:58:28.278187+0000","last_clean":"2024-06-13T06:58:28.278187+0000","last_became_active":"2024-06-13T06:54:58.898599+0000","last_became_peered":"2024-06-13T06:54:58.898599+0000","last_unstale":"2024-06-13T06:58:28.278187+0000","last_undegraded":"2024-06-13T06:58:28.278187+0000","last_fullsized":"2024-06-13T06:58:28.278187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:21:53.044205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:33.424302+0000","last_change":"2024-06-13T06:56:29.527242+0000","last_active":"2024-06-13T06:56:33.424302+0000","last_peered":"2024-06-13T06:56:33.424302+0000","last_clean":"2024-06-13T06:56:33.424302+0000","last_became_active":"2024-06-13T06:56:29.526816+0000","last_became_peered":"2024-06-13T06:56:29.526816+0000","last_unstale":"2024-06-13T06:56:33.424302+0000","last_undegraded":"2024-06-13T06:56:33.424302+0000","last_fullsized":"2024-06-13T06:56:33.424302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:45:17.770029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'39","reported_seq":584,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:27.942865+0000","last_change":"2024-06-13T06:55:29.376808+0000","last_active":"2024-06-13T07:00:27.942865+0000","last_peered":"2024-06-13T07:00:27.942865+0000","last_clean":"2024-06-13T07:00:27.942865+0000","last_became_active":"2024-06-13T06:55:29.367134+0000","last_became_peered":"2024-06-13T06:55:29.367134+0000","last_unstale":"2024-06-13T07:00:27.942865+0000","last_undegraded":"2024-06-13T07:00:27.942865+0000","last_fullsized":"2024-06-13T07:00:27.942865+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:19:08.784438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68583,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":483,"num_read_kb":310,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.265259+0000","last_change":"2024-06-13T06:54:58.897686+0000","last_active":"2024-06-13T06:58:28.265259+0000","last_peered":"2024-06-13T06:58:28.265259+0000","last_clean":"2024-06-13T06:58:28.265259+0000","last_became_active":"2024-06-13T06:54:58.897313+0000","last_became_peered":"2024-06-13T06:54:58.897313+0000","last_unstale":"2024-06-13T06:58:28.265259+0000","last_undegraded":"2024-06-13T06:58:28.265259+0000","last_fullsized":"2024-06-13T06:58:28.265259+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:54:49.558542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:36.544740+0000","last_change":"2024-06-13T06:55:12.013918+0000","last_active":"2024-06-13T06:57:36.544740+0000","last_peered":"2024-06-13T06:57:36.544740+0000","last_clean":"2024-06-13T06:57:36.544740+0000","last_became_active":"2024-06-13T06:55:12.013522+0000","last_became_peered":"2024-06-13T06:55:12.013522+0000","last_unstale":"2024-06-13T06:57:36.544740+0000","last_undegraded":"2024-06-13T06:57:36.544740+0000","last_fullsized":"2024-06-13T06:57:36.544740+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:51:46.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.652302+0000","last_change":"2024-06-13T06:54:54.328674+0000","last_active":"2024-06-13T06:57:17.652302+0000","last_peered":"2024-06-13T06:57:17.652302+0000","last_clean":"2024-06-13T06:57:17.652302+0000","last_became_active":"2024-06-13T06:54:54.326789+0000","last_became_peered":"2024-06-13T06:54:54.326789+0000","last_unstale":"2024-06-13T06:57:17.652302+0000","last_undegraded":"2024-06-13T06:57:17.652302+0000","last_fullsized":"2024-06-13T06:57:17.652302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:18.280514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.293277+0000","last_change":"2024-06-13T06:54:58.918598+0000","last_active":"2024-06-13T06:58:28.293277+0000","last_peered":"2024-06-13T06:58:28.293277+0000","last_clean":"2024-06-13T06:58:28.293277+0000","last_became_active":"2024-06-13T06:54:58.918247+0000","last_became_peered":"2024-06-13T06:54:58.918247+0000","last_unstale":"2024-06-13T06:58:28.293277+0000","last_undegraded":"2024-06-13T06:58:28.293277+0000","last_fullsized":"2024-06-13T06:58:28.293277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:08:56.738806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:34.838358+0000","last_change":"2024-06-13T06:55:12.017074+0000","last_active":"2024-06-13T06:57:34.838358+0000","last_peered":"2024-06-13T06:57:34.838358+0000","last_clean":"2024-06-13T06:57:34.838358+0000","last_became_active":"2024-06-13T06:55:12.016675+0000","last_became_peered":"2024-06-13T06:55:12.016675+0000","last_unstale":"2024-06-13T06:57:34.838358+0000","last_undegraded":"2024-06-13T06:57:34.838358+0000","last_fullsized":"2024-06-13T06:57:34.838358+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:20:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:16.542158+0000","last_change":"2024-06-13T06:54:54.334994+0000","last_active":"2024-06-13T06:57:16.542158+0000","last_peered":"2024-06-13T06:57:16.542158+0000","last_clean":"2024-06-13T06:57:16.542158+0000","last_became_active":"2024-06-13T06:54:54.334647+0000","last_became_peered":"2024-06-13T06:54:54.334647+0000","last_unstale":"2024-06-13T06:57:16.542158+0000","last_undegraded":"2024-06-13T06:57:16.542158+0000","last_fullsized":"2024-06-13T06:57:16.542158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:59:15.317976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276010+0000","last_change":"2024-06-13T06:54:58.899107+0000","last_active":"2024-06-13T06:58:28.276010+0000","last_peered":"2024-06-13T06:58:28.276010+0000","last_clean":"2024-06-13T06:58:28.276010+0000","last_became_active":"2024-06-13T06:54:58.896034+0000","last_became_peered":"2024-06-13T06:54:58.896034+0000","last_unstale":"2024-06-13T06:58:28.276010+0000","last_undegraded":"2024-06-13T06:58:28.276010+0000","last_fullsized":"2024-06-13T06:58:28.276010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:00:01.132565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:40.558064+0000","last_change":"2024-06-13T06:55:12.025681+0000","last_active":"2024-06-13T06:57:40.558064+0000","last_peered":"2024-06-13T06:57:40.558064+0000","last_clean":"2024-06-13T06:57:40.558064+0000","last_became_active":"2024-06-13T06:55:12.025267+0000","last_became_peered":"2024-06-13T06:55:12.025267+0000","last_unstale":"2024-06-13T06:57:40.558064+0000","last_undegraded":"2024-06-13T06:57:40.558064+0000","last_fullsized":"2024-06-13T06:57:40.558064+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:58:23.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":287,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:27.044620+0000","last_change":"2024-06-13T06:54:54.325404+0000","last_active":"2024-06-13T07:00:27.044620+0000","last_peered":"2024-06-13T07:00:27.044620+0000","last_clean":"2024-06-13T07:00:27.044620+0000","last_became_active":"2024-06-13T06:54:54.325024+0000","last_became_peered":"2024-06-13T06:54:54.325024+0000","last_unstale":"2024-06-13T07:00:27.044620+0000","last_undegraded":"2024-06-13T07:00:27.044620+0000","last_fullsized":"2024-06-13T07:00:27.044620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:13:23.726350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272955+0000","last_change":"2024-06-13T06:54:58.894737+0000","last_active":"2024-06-13T06:58:28.272955+0000","last_peered":"2024-06-13T06:58:28.272955+0000","last_clean":"2024-06-13T06:58:28.272955+0000","last_became_active":"2024-06-13T06:54:58.894224+0000","last_became_peered":"2024-06-13T06:54:58.894224+0000","last_unstale":"2024-06-13T06:58:28.272955+0000","last_undegraded":"2024-06-13T06:58:28.272955+0000","last_fullsized":"2024-06-13T06:58:28.272955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:03.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:37.398002+0000","last_change":"2024-06-13T06:55:12.026587+0000","last_active":"2024-06-13T06:57:37.398002+0000","last_peered":"2024-06-13T06:57:37.398002+0000","last_clean":"2024-06-13T06:57:37.398002+0000","last_became_active":"2024-06-13T06:55:12.026114+0000","last_became_peered":"2024-06-13T06:55:12.026114+0000","last_unstale":"2024-06-13T06:57:37.398002+0000","last_undegraded":"2024-06-13T06:57:37.398002+0000","last_fullsized":"2024-06-13T06:57:37.398002+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:43:09.245641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:15.058741+0000","last_change":"2024-06-13T06:54:54.632069+0000","last_active":"2024-06-13T06:57:15.058741+0000","last_peered":"2024-06-13T06:57:15.058741+0000","last_clean":"2024-06-13T06:57:15.058741+0000","last_became_active":"2024-06-13T06:54:54.631692+0000","last_became_peered":"2024-06-13T06:54:54.631692+0000","last_unstale":"2024-06-13T06:57:15.058741+0000","last_undegraded":"2024-06-13T06:57:15.058741+0000","last_fullsized":"2024-06-13T06:57:15.058741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:07.109507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.265386+0000","last_change":"2024-06-13T06:55:29.366286+0000","last_active":"2024-06-13T06:58:28.265386+0000","last_peered":"2024-06-13T06:58:28.265386+0000","last_clean":"2024-06-13T06:58:28.265386+0000","last_became_active":"2024-06-13T06:55:29.365854+0000","last_became_peered":"2024-06-13T06:55:29.365854+0000","last_unstale":"2024-06-13T06:58:28.265386+0000","last_undegraded":"2024-06-13T06:58:28.265386+0000","last_fullsized":"2024-06-13T06:58:28.265386+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:04.209244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542690+0000","last_change":"2024-06-13T06:55:12.021621+0000","last_active":"2024-06-13T06:57:21.542690+0000","last_peered":"2024-06-13T06:57:21.542690+0000","last_clean":"2024-06-13T06:57:21.542690+0000","last_became_active":"2024-06-13T06:55:12.021150+0000","last_became_peered":"2024-06-13T06:55:12.021150+0000","last_unstale":"2024-06-13T06:57:21.542690+0000","last_undegraded":"2024-06-13T06:57:21.542690+0000","last_fullsized":"2024-06-13T06:57:21.542690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:48.318918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:44.839364+0000","last_change":"2024-06-13T06:55:29.369972+0000","last_active":"2024-06-13T06:57:44.839364+0000","last_peered":"2024-06-13T06:57:44.839364+0000","last_clean":"2024-06-13T06:57:44.839364+0000","last_became_active":"2024-06-13T06:55:29.366390+0000","last_became_peered":"2024-06-13T06:55:29.366390+0000","last_unstale":"2024-06-13T06:57:44.839364+0000","last_undegraded":"2024-06-13T06:57:44.839364+0000","last_fullsized":"2024-06-13T06:57:44.839364+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:57:18.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271886+0000","last_change":"2024-06-13T06:54:58.891309+0000","last_active":"2024-06-13T06:58:28.271886+0000","last_peered":"2024-06-13T06:58:28.271886+0000","last_clean":"2024-06-13T06:58:28.271886+0000","last_became_active":"2024-06-13T06:54:58.890972+0000","last_became_peered":"2024-06-13T06:54:58.890972+0000","last_unstale":"2024-06-13T06:58:28.271886+0000","last_undegraded":"2024-06-13T06:58:28.271886+0000","last_fullsized":"2024-06-13T06:58:28.271886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:41.240061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:20.059185+0000","last_change":"2024-06-13T06:56:29.610713+0000","last_active":"2024-06-13T06:57:20.059185+0000","last_peered":"2024-06-13T06:57:20.059185+0000","last_clean":"2024-06-13T06:57:20.059185+0000","last_became_active":"2024-06-13T06:56:29.610139+0000","last_became_peered":"2024-06-13T06:56:29.610139+0000","last_unstale":"2024-06-13T06:57:20.059185+0000","last_undegraded":"2024-06-13T06:57:20.059185+0000","last_fullsized":"2024-06-13T06:57:20.059185+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:14:36.601243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:00.551177+0000","last_change":"2024-06-13T06:54:54.633893+0000","last_active":"2024-06-13T06:57:00.551177+0000","last_peered":"2024-06-13T06:57:00.551177+0000","last_clean":"2024-06-13T06:57:00.551177+0000","last_became_active":"2024-06-13T06:54:54.632946+0000","last_became_peered":"2024-06-13T06:54:54.632946+0000","last_unstale":"2024-06-13T06:57:00.551177+0000","last_undegraded":"2024-06-13T06:57:00.551177+0000","last_fullsized":"2024-06-13T06:57:00.551177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:16:54.812477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279866+0000","last_change":"2024-06-13T06:54:58.907182+0000","last_active":"2024-06-13T06:58:28.279866+0000","last_peered":"2024-06-13T06:58:28.279866+0000","last_clean":"2024-06-13T06:58:28.279866+0000","last_became_active":"2024-06-13T06:54:58.906824+0000","last_became_peered":"2024-06-13T06:54:58.906824+0000","last_unstale":"2024-06-13T06:58:28.279866+0000","last_undegraded":"2024-06-13T06:58:28.279866+0000","last_fullsized":"2024-06-13T06:58:28.279866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:34:49.626991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.394303+0000","last_change":"2024-06-13T06:55:12.014803+0000","last_active":"2024-06-13T06:57:17.394303+0000","last_peered":"2024-06-13T06:57:17.394303+0000","last_clean":"2024-06-13T06:57:17.394303+0000","last_became_active":"2024-06-13T06:55:12.014398+0000","last_became_peered":"2024-06-13T06:55:12.014398+0000","last_unstale":"2024-06-13T06:57:17.394303+0000","last_undegraded":"2024-06-13T06:57:17.394303+0000","last_fullsized":"2024-06-13T06:57:17.394303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:07:44.960446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:59.668362+0000","last_change":"2024-06-13T06:54:54.339056+0000","last_active":"2024-06-13T06:56:59.668362+0000","last_peered":"2024-06-13T06:56:59.668362+0000","last_clean":"2024-06-13T06:56:59.668362+0000","last_became_active":"2024-06-13T06:54:54.338683+0000","last_became_peered":"2024-06-13T06:54:54.338683+0000","last_unstale":"2024-06-13T06:56:59.668362+0000","last_undegraded":"2024-06-13T06:56:59.668362+0000","last_fullsized":"2024-06-13T06:56:59.668362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:05:17.060471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.297043+0000","last_change":"2024-06-13T06:54:58.919843+0000","last_active":"2024-06-13T06:58:28.297043+0000","last_peered":"2024-06-13T06:58:28.297043+0000","last_clean":"2024-06-13T06:58:28.297043+0000","last_became_active":"2024-06-13T06:54:58.919405+0000","last_became_peered":"2024-06-13T06:54:58.919405+0000","last_unstale":"2024-06-13T06:58:28.297043+0000","last_undegraded":"2024-06-13T06:58:28.297043+0000","last_fullsized":"2024-06-13T06:58:28.297043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:00:20.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:22.395386+0000","last_change":"2024-06-13T06:55:12.097651+0000","last_active":"2024-06-13T06:57:22.395386+0000","last_peered":"2024-06-13T06:57:22.395386+0000","last_clean":"2024-06-13T06:57:22.395386+0000","last_became_active":"2024-06-13T06:55:12.095644+0000","last_became_peered":"2024-06-13T06:55:12.095644+0000","last_unstale":"2024-06-13T06:57:22.395386+0000","last_undegraded":"2024-06-13T06:57:22.395386+0000","last_fullsized":"2024-06-13T06:57:22.395386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:09:54.328428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.266581+0000","last_change":"2024-06-13T06:54:58.891187+0000","last_active":"2024-06-13T06:58:28.266581+0000","last_peered":"2024-06-13T06:58:28.266581+0000","last_clean":"2024-06-13T06:58:28.266581+0000","last_became_active":"2024-06-13T06:54:58.890848+0000","last_became_peered":"2024-06-13T06:54:58.890848+0000","last_unstale":"2024-06-13T06:58:28.266581+0000","last_undegraded":"2024-06-13T06:58:28.266581+0000","last_fullsized":"2024-06-13T06:58:28.266581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:13:31.709650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":285,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T07:00:26.041470+0000","last_change":"2024-06-13T06:54:54.631482+0000","last_active":"2024-06-13T07:00:26.041470+0000","last_peered":"2024-06-13T07:00:26.041470+0000","last_clean":"2024-06-13T07:00:26.041470+0000","last_became_active":"2024-06-13T06:54:54.630818+0000","last_became_peered":"2024-06-13T06:54:54.630818+0000","last_unstale":"2024-06-13T07:00:26.041470+0000","last_undegraded":"2024-06-13T07:00:26.041470+0000","last_fullsized":"2024-06-13T07:00:26.041470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:34:18.801727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.670568+0000","last_change":"2024-06-13T06:55:12.018856+0000","last_active":"2024-06-13T06:57:19.670568+0000","last_peered":"2024-06-13T06:57:19.670568+0000","last_clean":"2024-06-13T06:57:19.670568+0000","last_became_active":"2024-06-13T06:55:12.018488+0000","last_became_peered":"2024-06-13T06:55:12.018488+0000","last_unstale":"2024-06-13T06:57:19.670568+0000","last_undegraded":"2024-06-13T06:57:19.670568+0000","last_fullsized":"2024-06-13T06:57:19.670568+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:30:14.973197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":25,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281859+0000","last_change":"2024-06-13T06:55:29.377203+0000","last_active":"2024-06-13T06:58:28.281859+0000","last_peered":"2024-06-13T06:58:28.281859+0000","last_clean":"2024-06-13T06:58:28.281859+0000","last_became_active":"2024-06-13T06:55:29.375263+0000","last_became_peered":"2024-06-13T06:55:29.375263+0000","last_unstale":"2024-06-13T06:58:28.281859+0000","last_undegraded":"2024-06-13T06:58:28.281859+0000","last_fullsized":"2024-06-13T06:58:28.281859+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:06:07.935172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:03.284069+0000","last_change":"2024-06-13T06:54:54.335319+0000","last_active":"2024-06-13T06:57:03.284069+0000","last_peered":"2024-06-13T06:57:03.284069+0000","last_clean":"2024-06-13T06:57:03.284069+0000","last_became_active":"2024-06-13T06:54:54.334896+0000","last_became_peered":"2024-06-13T06:54:54.334896+0000","last_unstale":"2024-06-13T06:57:03.284069+0000","last_undegraded":"2024-06-13T06:57:03.284069+0000","last_fullsized":"2024-06-13T06:57:03.284069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:40.331535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:49.673756+0000","last_change":"2024-06-13T06:53:16.968830+0000","last_active":"2024-06-13T06:57:49.673756+0000","last_peered":"2024-06-13T06:57:49.673756+0000","last_clean":"2024-06-13T06:57:49.673756+0000","last_became_active":"2024-06-13T06:53:16.968361+0000","last_became_peered":"2024-06-13T06:53:16.968361+0000","last_unstale":"2024-06-13T06:57:49.673756+0000","last_undegraded":"2024-06-13T06:57:49.673756+0000","last_fullsized":"2024-06-13T06:57:49.673756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:49:18.858540+0000","last_clean_scrub_stamp":"2024-06-13T06:49:18.858540+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:32:26.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542736+0000","last_change":"2024-06-13T06:56:29.602124+0000","last_active":"2024-06-13T06:57:21.542736+0000","last_peered":"2024-06-13T06:57:21.542736+0000","last_clean":"2024-06-13T06:57:21.542736+0000","last_became_active":"2024-06-13T06:56:29.601650+0000","last_became_peered":"2024-06-13T06:56:29.601650+0000","last_unstale":"2024-06-13T06:57:21.542736+0000","last_undegraded":"2024-06-13T06:57:21.542736+0000","last_fullsized":"2024-06-13T06:57:21.542736+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:37:23.903614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.297076+0000","last_change":"2024-06-13T06:54:58.898930+0000","last_active":"2024-06-13T06:58:28.297076+0000","last_peered":"2024-06-13T06:58:28.297076+0000","last_clean":"2024-06-13T06:58:28.297076+0000","last_became_active":"2024-06-13T06:54:58.898384+0000","last_became_peered":"2024-06-13T06:54:58.898384+0000","last_unstale":"2024-06-13T06:58:28.297076+0000","last_undegraded":"2024-06-13T06:58:28.297076+0000","last_fullsized":"2024-06-13T06:58:28.297076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:06:02.210336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:54.667753+0000","last_change":"2024-06-13T06:54:54.328776+0000","last_active":"2024-06-13T06:56:54.667753+0000","last_peered":"2024-06-13T06:56:54.667753+0000","last_clean":"2024-06-13T06:56:54.667753+0000","last_became_active":"2024-06-13T06:54:54.327773+0000","last_became_peered":"2024-06-13T06:54:54.327773+0000","last_unstale":"2024-06-13T06:56:54.667753+0000","last_undegraded":"2024-06-13T06:56:54.667753+0000","last_fullsized":"2024-06-13T06:56:54.667753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:23:02.240774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:19.836611+0000","last_change":"2024-06-13T06:55:12.013862+0000","last_active":"2024-06-13T06:57:19.836611+0000","last_peered":"2024-06-13T06:57:19.836611+0000","last_clean":"2024-06-13T06:57:19.836611+0000","last_became_active":"2024-06-13T06:55:12.013389+0000","last_became_peered":"2024-06-13T06:55:12.013389+0000","last_unstale":"2024-06-13T06:57:19.836611+0000","last_undegraded":"2024-06-13T06:57:19.836611+0000","last_fullsized":"2024-06-13T06:57:19.836611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:49.701202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:56:32.558954+0000","last_change":"2024-06-13T06:56:29.539862+0000","last_active":"2024-06-13T06:56:32.558954+0000","last_peered":"2024-06-13T06:56:32.558954+0000","last_clean":"2024-06-13T06:56:32.558954+0000","last_became_active":"2024-06-13T06:56:29.536109+0000","last_became_peered":"2024-06-13T06:56:29.536109+0000","last_unstale":"2024-06-13T06:56:32.558954+0000","last_undegraded":"2024-06-13T06:56:32.558954+0000","last_fullsized":"2024-06-13T06:56:32.558954+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:31:23.307850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.288243+0000","last_change":"2024-06-13T06:54:58.901773+0000","last_active":"2024-06-13T06:58:28.288243+0000","last_peered":"2024-06-13T06:58:28.288243+0000","last_clean":"2024-06-13T06:58:28.288243+0000","last_became_active":"2024-06-13T06:54:58.901168+0000","last_became_peered":"2024-06-13T06:54:58.901168+0000","last_unstale":"2024-06-13T06:58:28.288243+0000","last_undegraded":"2024-06-13T06:58:28.288243+0000","last_fullsized":"2024-06-13T06:58:28.288243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:57:15.848785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:11.078009+0000","last_change":"2024-06-13T06:55:12.095890+0000","last_active":"2024-06-13T06:57:11.078009+0000","last_peered":"2024-06-13T06:57:11.078009+0000","last_clean":"2024-06-13T06:57:11.078009+0000","last_became_active":"2024-06-13T06:55:12.095397+0000","last_became_peered":"2024-06-13T06:55:12.095397+0000","last_unstale":"2024-06-13T06:57:11.078009+0000","last_undegraded":"2024-06-13T06:57:11.078009+0000","last_fullsized":"2024-06-13T06:57:11.078009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:56:18.211036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'13","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:59:15.573034+0000","last_change":"2024-06-13T06:54:54.337946+0000","last_active":"2024-06-13T06:59:15.573034+0000","last_peered":"2024-06-13T06:59:15.573034+0000","last_clean":"2024-06-13T06:59:15.573034+0000","last_became_active":"2024-06-13T06:54:54.337028+0000","last_became_peered":"2024-06-13T06:54:54.337028+0000","last_unstale":"2024-06-13T06:59:15.573034+0000","last_undegraded":"2024-06-13T06:59:15.573034+0000","last_fullsized":"2024-06-13T06:59:15.573034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:08:52.641595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275687+0000","last_change":"2024-06-13T06:54:58.895110+0000","last_active":"2024-06-13T06:58:28.275687+0000","last_peered":"2024-06-13T06:58:28.275687+0000","last_clean":"2024-06-13T06:58:28.275687+0000","last_became_active":"2024-06-13T06:54:58.893481+0000","last_became_peered":"2024-06-13T06:54:58.893481+0000","last_unstale":"2024-06-13T06:58:28.275687+0000","last_undegraded":"2024-06-13T06:58:28.275687+0000","last_fullsized":"2024-06-13T06:58:28.275687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:54.932110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:17.652273+0000","last_change":"2024-06-13T06:55:11.997712+0000","last_active":"2024-06-13T06:57:17.652273+0000","last_peered":"2024-06-13T06:57:17.652273+0000","last_clean":"2024-06-13T06:57:17.652273+0000","last_became_active":"2024-06-13T06:55:11.997193+0000","last_became_peered":"2024-06-13T06:55:11.997193+0000","last_unstale":"2024-06-13T06:57:17.652273+0000","last_undegraded":"2024-06-13T06:57:17.652273+0000","last_fullsized":"2024-06-13T06:57:17.652273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:05:15.168070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:08.284826+0000","last_change":"2024-06-13T06:54:54.520121+0000","last_active":"2024-06-13T06:57:08.284826+0000","last_peered":"2024-06-13T06:57:08.284826+0000","last_clean":"2024-06-13T06:57:08.284826+0000","last_became_active":"2024-06-13T06:54:54.519723+0000","last_became_peered":"2024-06-13T06:54:54.519723+0000","last_unstale":"2024-06-13T06:57:08.284826+0000","last_undegraded":"2024-06-13T06:57:08.284826+0000","last_fullsized":"2024-06-13T06:57:08.284826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:19:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.304745+0000","last_change":"2024-06-13T06:54:58.923626+0000","last_active":"2024-06-13T06:58:28.304745+0000","last_peered":"2024-06-13T06:58:28.304745+0000","last_clean":"2024-06-13T06:58:28.304745+0000","last_became_active":"2024-06-13T06:54:58.923192+0000","last_became_peered":"2024-06-13T06:54:58.923192+0000","last_unstale":"2024-06-13T06:58:28.304745+0000","last_undegraded":"2024-06-13T06:58:28.304745+0000","last_fullsized":"2024-06-13T06:58:28.304745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:26:13.231549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287371+0000","last_change":"2024-06-13T06:55:12.096244+0000","last_active":"2024-06-13T06:57:28.287371+0000","last_peered":"2024-06-13T06:57:28.287371+0000","last_clean":"2024-06-13T06:57:28.287371+0000","last_became_active":"2024-06-13T06:55:12.095758+0000","last_became_peered":"2024-06-13T06:55:12.095758+0000","last_unstale":"2024-06-13T06:57:28.287371+0000","last_undegraded":"2024-06-13T06:57:28.287371+0000","last_fullsized":"2024-06-13T06:57:28.287371+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:38.122282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:08.168621+0000","last_change":"2024-06-13T06:54:54.630488+0000","last_active":"2024-06-13T06:57:08.168621+0000","last_peered":"2024-06-13T06:57:08.168621+0000","last_clean":"2024-06-13T06:57:08.168621+0000","last_became_active":"2024-06-13T06:54:54.630224+0000","last_became_peered":"2024-06-13T06:54:54.630224+0000","last_unstale":"2024-06-13T06:57:08.168621+0000","last_undegraded":"2024-06-13T06:57:08.168621+0000","last_fullsized":"2024-06-13T06:57:08.168621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:33:19.107026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.263285+0000","last_change":"2024-06-13T06:54:58.894466+0000","last_active":"2024-06-13T06:58:28.263285+0000","last_peered":"2024-06-13T06:58:28.263285+0000","last_clean":"2024-06-13T06:58:28.263285+0000","last_became_active":"2024-06-13T06:54:58.894034+0000","last_became_peered":"2024-06-13T06:54:58.894034+0000","last_unstale":"2024-06-13T06:58:28.263285+0000","last_undegraded":"2024-06-13T06:58:28.263285+0000","last_fullsized":"2024-06-13T06:58:28.263285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:44.051565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:30.556536+0000","last_change":"2024-06-13T06:55:12.022078+0000","last_active":"2024-06-13T06:57:30.556536+0000","last_peered":"2024-06-13T06:57:30.556536+0000","last_clean":"2024-06-13T06:57:30.556536+0000","last_became_active":"2024-06-13T06:55:12.021652+0000","last_became_peered":"2024-06-13T06:55:12.021652+0000","last_unstale":"2024-06-13T06:57:30.556536+0000","last_undegraded":"2024-06-13T06:57:30.556536+0000","last_fullsized":"2024-06-13T06:57:30.556536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:25:57.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:06.077272+0000","last_change":"2024-06-13T06:54:54.336757+0000","last_active":"2024-06-13T06:57:06.077272+0000","last_peered":"2024-06-13T06:57:06.077272+0000","last_clean":"2024-06-13T06:57:06.077272+0000","last_became_active":"2024-06-13T06:54:54.336012+0000","last_became_peered":"2024-06-13T06:54:54.336012+0000","last_unstale":"2024-06-13T06:57:06.077272+0000","last_undegraded":"2024-06-13T06:57:06.077272+0000","last_fullsized":"2024-06-13T06:57:06.077272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:52:21.227464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.292857+0000","last_change":"2024-06-13T06:54:58.919631+0000","last_active":"2024-06-13T06:58:28.292857+0000","last_peered":"2024-06-13T06:58:28.292857+0000","last_clean":"2024-06-13T06:58:28.292857+0000","last_became_active":"2024-06-13T06:54:58.919274+0000","last_became_peered":"2024-06-13T06:54:58.919274+0000","last_unstale":"2024-06-13T06:58:28.292857+0000","last_undegraded":"2024-06-13T06:58:28.292857+0000","last_fullsized":"2024-06-13T06:58:28.292857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:22.453797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287343+0000","last_change":"2024-06-13T06:55:12.017338+0000","last_active":"2024-06-13T06:57:28.287343+0000","last_peered":"2024-06-13T06:57:28.287343+0000","last_clean":"2024-06-13T06:57:28.287343+0000","last_became_active":"2024-06-13T06:55:12.016841+0000","last_became_peered":"2024-06-13T06:55:12.016841+0000","last_unstale":"2024-06-13T06:57:28.287343+0000","last_undegraded":"2024-06-13T06:57:28.287343+0000","last_fullsized":"2024-06-13T06:57:28.287343+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:40:13.525892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:07.651040+0000","last_change":"2024-06-13T06:54:54.328808+0000","last_active":"2024-06-13T06:57:07.651040+0000","last_peered":"2024-06-13T06:57:07.651040+0000","last_clean":"2024-06-13T06:57:07.651040+0000","last_became_active":"2024-06-13T06:54:54.326982+0000","last_became_peered":"2024-06-13T06:54:54.326982+0000","last_unstale":"2024-06-13T06:57:07.651040+0000","last_undegraded":"2024-06-13T06:57:07.651040+0000","last_fullsized":"2024-06-13T06:57:07.651040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:22:53.854275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.303609+0000","last_change":"2024-06-13T06:54:58.925728+0000","last_active":"2024-06-13T06:58:28.303609+0000","last_peered":"2024-06-13T06:58:28.303609+0000","last_clean":"2024-06-13T06:58:28.303609+0000","last_became_active":"2024-06-13T06:54:58.925407+0000","last_became_peered":"2024-06-13T06:54:58.925407+0000","last_unstale":"2024-06-13T06:58:28.303609+0000","last_undegraded":"2024-06-13T06:58:28.303609+0000","last_fullsized":"2024-06-13T06:58:28.303609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:03:38.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:28.287363+0000","last_change":"2024-06-13T06:55:12.095733+0000","last_active":"2024-06-13T06:57:28.287363+0000","last_peered":"2024-06-13T06:57:28.287363+0000","last_clean":"2024-06-13T06:57:28.287363+0000","last_became_active":"2024-06-13T06:55:12.095064+0000","last_became_peered":"2024-06-13T06:55:12.095064+0000","last_unstale":"2024-06-13T06:57:28.287363+0000","last_undegraded":"2024-06-13T06:57:28.287363+0000","last_fullsized":"2024-06-13T06:57:28.287363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:26:05.486701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:10.058315+0000","last_change":"2024-06-13T06:54:54.331448+0000","last_active":"2024-06-13T06:57:10.058315+0000","last_peered":"2024-06-13T06:57:10.058315+0000","last_clean":"2024-06-13T06:57:10.058315+0000","last_became_active":"2024-06-13T06:54:54.330964+0000","last_became_peered":"2024-06-13T06:54:54.330964+0000","last_unstale":"2024-06-13T06:57:10.058315+0000","last_undegraded":"2024-06-13T06:57:10.058315+0000","last_fullsized":"2024-06-13T06:57:10.058315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:23:12.211412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282119+0000","last_change":"2024-06-13T06:54:58.899112+0000","last_active":"2024-06-13T06:58:28.282119+0000","last_peered":"2024-06-13T06:58:28.282119+0000","last_clean":"2024-06-13T06:58:28.282119+0000","last_became_active":"2024-06-13T06:54:58.898742+0000","last_became_peered":"2024-06-13T06:54:58.898742+0000","last_unstale":"2024-06-13T06:58:28.282119+0000","last_undegraded":"2024-06-13T06:58:28.282119+0000","last_fullsized":"2024-06-13T06:58:28.282119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:50:12.983287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:23.170398+0000","last_change":"2024-06-13T06:55:12.014149+0000","last_active":"2024-06-13T06:57:23.170398+0000","last_peered":"2024-06-13T06:57:23.170398+0000","last_clean":"2024-06-13T06:57:23.170398+0000","last_became_active":"2024-06-13T06:55:12.013454+0000","last_became_peered":"2024-06-13T06:55:12.013454+0000","last_unstale":"2024-06-13T06:57:23.170398+0000","last_undegraded":"2024-06-13T06:57:23.170398+0000","last_fullsized":"2024-06-13T06:57:23.170398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:13:39.368540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.651570+0000","last_change":"2024-06-13T06:54:54.633268+0000","last_active":"2024-06-13T06:57:12.651570+0000","last_peered":"2024-06-13T06:57:12.651570+0000","last_clean":"2024-06-13T06:57:12.651570+0000","last_became_active":"2024-06-13T06:54:54.632840+0000","last_became_peered":"2024-06-13T06:54:54.632840+0000","last_unstale":"2024-06-13T06:57:12.651570+0000","last_undegraded":"2024-06-13T06:57:12.651570+0000","last_fullsized":"2024-06-13T06:57:12.651570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:40:21.653727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.281941+0000","last_change":"2024-06-13T06:54:58.893417+0000","last_active":"2024-06-13T06:58:28.281941+0000","last_peered":"2024-06-13T06:58:28.281941+0000","last_clean":"2024-06-13T06:58:28.281941+0000","last_became_active":"2024-06-13T06:54:58.892381+0000","last_became_peered":"2024-06-13T06:54:58.892381+0000","last_unstale":"2024-06-13T06:58:28.281941+0000","last_undegraded":"2024-06-13T06:58:28.281941+0000","last_fullsized":"2024-06-13T06:58:28.281941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:07.979654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:27.543545+0000","last_change":"2024-06-13T06:55:12.018336+0000","last_active":"2024-06-13T06:57:27.543545+0000","last_peered":"2024-06-13T06:57:27.543545+0000","last_clean":"2024-06-13T06:57:27.543545+0000","last_became_active":"2024-06-13T06:55:12.017861+0000","last_became_peered":"2024-06-13T06:55:12.017861+0000","last_unstale":"2024-06-13T06:57:27.543545+0000","last_undegraded":"2024-06-13T06:57:27.543545+0000","last_fullsized":"2024-06-13T06:57:27.543545+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:47:41.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:11.541680+0000","last_change":"2024-06-13T06:54:54.333036+0000","last_active":"2024-06-13T06:57:11.541680+0000","last_peered":"2024-06-13T06:57:11.541680+0000","last_clean":"2024-06-13T06:57:11.541680+0000","last_became_active":"2024-06-13T06:54:54.332642+0000","last_became_peered":"2024-06-13T06:54:54.332642+0000","last_unstale":"2024-06-13T06:57:11.541680+0000","last_undegraded":"2024-06-13T06:57:11.541680+0000","last_fullsized":"2024-06-13T06:57:11.541680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:06:06.809024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279872+0000","last_change":"2024-06-13T06:54:58.881894+0000","last_active":"2024-06-13T06:58:28.279872+0000","last_peered":"2024-06-13T06:58:28.279872+0000","last_clean":"2024-06-13T06:58:28.279872+0000","last_became_active":"2024-06-13T06:54:58.881591+0000","last_became_peered":"2024-06-13T06:54:58.881591+0000","last_unstale":"2024-06-13T06:58:28.279872+0000","last_undegraded":"2024-06-13T06:58:28.279872+0000","last_fullsized":"2024-06-13T06:58:28.279872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:49:00.944212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:21.542642+0000","last_change":"2024-06-13T06:55:12.001180+0000","last_active":"2024-06-13T06:57:21.542642+0000","last_peered":"2024-06-13T06:57:21.542642+0000","last_clean":"2024-06-13T06:57:21.542642+0000","last_became_active":"2024-06-13T06:55:12.000748+0000","last_became_peered":"2024-06-13T06:55:12.000748+0000","last_unstale":"2024-06-13T06:57:21.542642+0000","last_undegraded":"2024-06-13T06:57:21.542642+0000","last_fullsized":"2024-06-13T06:57:21.542642+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:55:10.966106+0000","last_clean_scrub_stamp":"2024-06-13T06:55:10.966106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:22.035807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:57:12.393505+0000","last_change":"2024-06-13T06:54:54.342724+0000","last_active":"2024-06-13T06:57:12.393505+0000","last_peered":"2024-06-13T06:57:12.393505+0000","last_clean":"2024-06-13T06:57:12.393505+0000","last_became_active":"2024-06-13T06:54:54.342267+0000","last_became_peered":"2024-06-13T06:54:54.342267+0000","last_unstale":"2024-06-13T06:57:12.393505+0000","last_undegraded":"2024-06-13T06:57:12.393505+0000","last_fullsized":"2024-06-13T06:57:12.393505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:53.282098+0000","last_clean_scrub_stamp":"2024-06-13T06:54:53.282098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:27:53.039202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271540+0000","last_change":"2024-06-13T06:54:58.892365+0000","last_active":"2024-06-13T06:58:28.271540+0000","last_peered":"2024-06-13T06:58:28.271540+0000","last_clean":"2024-06-13T06:58:28.271540+0000","last_became_active":"2024-06-13T06:54:58.889395+0000","last_became_peered":"2024-06-13T06:54:58.889395+0000","last_unstale":"2024-06-13T06:58:28.271540+0000","last_undegraded":"2024-06-13T06:58:28.271540+0000","last_fullsized":"2024-06-13T06:58:28.271540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:55.798293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.272280+0000","last_change":"2024-06-13T06:54:58.888570+0000","last_active":"2024-06-13T06:58:28.272280+0000","last_peered":"2024-06-13T06:58:28.272280+0000","last_clean":"2024-06-13T06:58:28.272280+0000","last_became_active":"2024-06-13T06:54:58.888180+0000","last_became_peered":"2024-06-13T06:54:58.888180+0000","last_unstale":"2024-06-13T06:58:28.272280+0000","last_undegraded":"2024-06-13T06:58:28.272280+0000","last_fullsized":"2024-06-13T06:58:28.272280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:46:42.537790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271099+0000","last_change":"2024-06-13T06:54:58.893476+0000","last_active":"2024-06-13T06:58:28.271099+0000","last_peered":"2024-06-13T06:58:28.271099+0000","last_clean":"2024-06-13T06:58:28.271099+0000","last_became_active":"2024-06-13T06:54:58.893021+0000","last_became_peered":"2024-06-13T06:54:58.893021+0000","last_unstale":"2024-06-13T06:58:28.271099+0000","last_undegraded":"2024-06-13T06:58:28.271099+0000","last_fullsized":"2024-06-13T06:58:28.271099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:38.739070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.301611+0000","last_change":"2024-06-13T06:54:58.911042+0000","last_active":"2024-06-13T06:58:28.301611+0000","last_peered":"2024-06-13T06:58:28.301611+0000","last_clean":"2024-06-13T06:58:28.301611+0000","last_became_active":"2024-06-13T06:54:58.910641+0000","last_became_peered":"2024-06-13T06:54:58.910641+0000","last_unstale":"2024-06-13T06:58:28.301611+0000","last_undegraded":"2024-06-13T06:58:28.301611+0000","last_fullsized":"2024-06-13T06:58:28.301611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:21:39.467223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305071+0000","last_change":"2024-06-13T06:55:29.382730+0000","last_active":"2024-06-13T06:58:28.305071+0000","last_peered":"2024-06-13T06:58:28.305071+0000","last_clean":"2024-06-13T06:58:28.305071+0000","last_became_active":"2024-06-13T06:55:29.382162+0000","last_became_peered":"2024-06-13T06:55:29.382162+0000","last_unstale":"2024-06-13T06:58:28.305071+0000","last_undegraded":"2024-06-13T06:58:28.305071+0000","last_fullsized":"2024-06-13T06:58:28.305071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:05.486414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.303120+0000","last_change":"2024-06-13T06:55:29.385380+0000","last_active":"2024-06-13T06:58:28.303120+0000","last_peered":"2024-06-13T06:58:28.303120+0000","last_clean":"2024-06-13T06:58:28.303120+0000","last_became_active":"2024-06-13T06:55:29.384855+0000","last_became_peered":"2024-06-13T06:55:29.384855+0000","last_unstale":"2024-06-13T06:58:28.303120+0000","last_undegraded":"2024-06-13T06:58:28.303120+0000","last_fullsized":"2024-06-13T06:58:28.303120+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:31:01.945632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.271653+0000","last_change":"2024-06-13T06:54:58.889128+0000","last_active":"2024-06-13T06:58:28.271653+0000","last_peered":"2024-06-13T06:58:28.271653+0000","last_clean":"2024-06-13T06:58:28.271653+0000","last_became_active":"2024-06-13T06:54:58.888606+0000","last_became_peered":"2024-06-13T06:54:58.888606+0000","last_unstale":"2024-06-13T06:58:28.271653+0000","last_undegraded":"2024-06-13T06:58:28.271653+0000","last_fullsized":"2024-06-13T06:58:28.271653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:53:43.762968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.276436+0000","last_change":"2024-06-13T06:54:58.896098+0000","last_active":"2024-06-13T06:58:28.276436+0000","last_peered":"2024-06-13T06:58:28.276436+0000","last_clean":"2024-06-13T06:58:28.276436+0000","last_became_active":"2024-06-13T06:54:58.895722+0000","last_became_peered":"2024-06-13T06:54:58.895722+0000","last_unstale":"2024-06-13T06:58:28.276436+0000","last_undegraded":"2024-06-13T06:58:28.276436+0000","last_fullsized":"2024-06-13T06:58:28.276436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:18:14.421363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.279821+0000","last_change":"2024-06-13T06:54:58.905590+0000","last_active":"2024-06-13T06:58:28.279821+0000","last_peered":"2024-06-13T06:58:28.279821+0000","last_clean":"2024-06-13T06:58:28.279821+0000","last_became_active":"2024-06-13T06:54:58.905146+0000","last_became_peered":"2024-06-13T06:54:58.905146+0000","last_unstale":"2024-06-13T06:58:28.279821+0000","last_undegraded":"2024-06-13T06:58:28.279821+0000","last_fullsized":"2024-06-13T06:58:28.279821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:42:08.010906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.278426+0000","last_change":"2024-06-13T06:54:58.898037+0000","last_active":"2024-06-13T06:58:28.278426+0000","last_peered":"2024-06-13T06:58:28.278426+0000","last_clean":"2024-06-13T06:58:28.278426+0000","last_became_active":"2024-06-13T06:54:58.897692+0000","last_became_peered":"2024-06-13T06:54:58.897692+0000","last_unstale":"2024-06-13T06:58:28.278426+0000","last_undegraded":"2024-06-13T06:58:28.278426+0000","last_fullsized":"2024-06-13T06:58:28.278426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:44:30.018192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.275652+0000","last_change":"2024-06-13T06:54:58.903441+0000","last_active":"2024-06-13T06:58:28.275652+0000","last_peered":"2024-06-13T06:58:28.275652+0000","last_clean":"2024-06-13T06:58:28.275652+0000","last_became_active":"2024-06-13T06:54:58.893598+0000","last_became_peered":"2024-06-13T06:54:58.893598+0000","last_unstale":"2024-06-13T06:58:28.275652+0000","last_undegraded":"2024-06-13T06:58:28.275652+0000","last_fullsized":"2024-06-13T06:58:28.275652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:38:29.968619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.282488+0000","last_change":"2024-06-13T06:54:58.918594+0000","last_active":"2024-06-13T06:58:28.282488+0000","last_peered":"2024-06-13T06:58:28.282488+0000","last_clean":"2024-06-13T06:58:28.282488+0000","last_became_active":"2024-06-13T06:54:58.918255+0000","last_became_peered":"2024-06-13T06:54:58.918255+0000","last_unstale":"2024-06-13T06:58:28.282488+0000","last_undegraded":"2024-06-13T06:58:28.282488+0000","last_fullsized":"2024-06-13T06:58:28.282488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:39:48.527707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.305484+0000","last_change":"2024-06-13T06:54:58.926253+0000","last_active":"2024-06-13T06:58:28.305484+0000","last_peered":"2024-06-13T06:58:28.305484+0000","last_clean":"2024-06-13T06:58:28.305484+0000","last_became_active":"2024-06-13T06:54:58.924611+0000","last_became_peered":"2024-06-13T06:54:58.924611+0000","last_unstale":"2024-06-13T06:58:28.305484+0000","last_undegraded":"2024-06-13T06:58:28.305484+0000","last_fullsized":"2024-06-13T06:58:28.305484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:29:23.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-13T06:58:28.267286+0000","last_change":"2024-06-13T06:54:58.890454+0000","last_active":"2024-06-13T06:58:28.267286+0000","last_peered":"2024-06-13T06:58:28.267286+0000","last_clean":"2024-06-13T06:58:28.267286+0000","last_became_active":"2024-06-13T06:54:58.890129+0000","last_became_peered":"2024-06-13T06:54:58.890129+0000","last_unstale":"2024-06-13T06:58:28.267286+0000","last_undegraded":"2024-06-13T06:58:28.267286+0000","last_fullsized":"2024-06-13T06:58:28.267286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:54:57.864685+0000","last_clean_scrub_stamp":"2024-06-13T06:54:57.864685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:10.819536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73030,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1445,"num_read_kb":802,"num_write":181,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1656,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":219090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383765,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":570102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645401,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37128,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708024,"statfs":{"total":95995035648,"available":95957016576,"internally_reserved":0,"allocated":1835008,"data_stored":1225909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":574390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070566,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":570170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332204,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1166569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495728,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":565470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36516,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708636,"statfs":{"total":95995035648,"available":95957643264,"internally_reserved":0,"allocated":1208320,"data_stored":633963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":566858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":566108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182529,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":565664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920894,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1232896,"data_stored":640162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1162157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T07:00:31.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:00:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:32.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:31 smithi135 ceph-mon[36276]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:32.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:31 smithi135 ceph-mon[36276]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:32.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:31 smithi135 ceph-mon[36276]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:32.040 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:31 smithi026 ceph-mon[32041]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:32.040 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:31 smithi026 ceph-mon[32041]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:32.040 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:31 smithi026 ceph-mon[32041]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:32.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:31 smithi136 ceph-mon[37229]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:32.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:31 smithi136 ceph-mon[37229]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:32.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:31 smithi136 ceph-mon[37229]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T07:00:33.926 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:33 smithi026 ceph-mon[32041]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T07:00:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:33 smithi135 ceph-mon[36276]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T07:00:34.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:33 smithi136 ceph-mon[37229]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T07:00:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 -- 172.21.15.26:0/908565175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c108ac0 msgr2=0x7f583c108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 --2- 172.21.15.26:0/908565175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c108ac0 0x7f583c108f40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5828009990 tx=0x7f5828009ca0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 -- 172.21.15.26:0/908565175 shutdown_connections 2024-06-13T07:00:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 --2- 172.21.15.26:0/908565175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f583c109480 0x7f583c115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 --2- 172.21.15.26:0/908565175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f583c10eac0 0x7f583c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 --2- 172.21.15.26:0/908565175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c108ac0 0x7f583c108f40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 -- 172.21.15.26:0/908565175 >> 172.21.15.26:0/908565175 conn(0x7f583c075c60 msgr2=0x7f583c1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 -- 172.21.15.26:0/908565175 shutdown_connections 2024-06-13T07:00:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.283+0000 7f5843e00700 1 -- 172.21.15.26:0/908565175 wait complete. 2024-06-13T07:00:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.285+0000 7f5843e00700 1 Processor -- start 2024-06-13T07:00:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.285+0000 7f5843e00700 1 -- start start 2024-06-13T07:00:34.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.285+0000 7f5843e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f583c108ac0 0x7f583c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:34.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.285+0000 7f5843e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f583c109480 0x7f583c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:34.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.285+0000 7f5843e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c10eac0 0x7f583c1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.285+0000 7f5843e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f583c118ea0 con 0x7f583c10eac0 2024-06-13T07:00:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.285+0000 7f5843e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f583c118d20 con 0x7f583c108ac0 2024-06-13T07:00:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.285+0000 7f5843e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f583c119020 con 0x7f583c109480 2024-06-13T07:00:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f583c108ac0 0x7f583c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f583c108ac0 0x7f583c19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:33408/0 (socket says 172.21.15.26:33408) 2024-06-13T07:00:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5840a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f583c109480 0x7f583c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841400700 1 -- 172.21.15.26:0/552826321 learned_addr learned my addr 172.21.15.26:0/552826321 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841e00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c10eac0 0x7f583c1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841400700 1 -- 172.21.15.26:0/552826321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f583c109480 msgr2=0x7f583c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841400700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f583c109480 0x7f583c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841400700 1 -- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c10eac0 msgr2=0x7f583c1a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841400700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c10eac0 0x7f583c1a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5841400700 1 -- 172.21.15.26:0/552826321 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c1a42c0 con 0x7f583c108ac0 2024-06-13T07:00:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.286+0000 7f5840a00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f583c109480 0x7f583c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.287+0000 7f5841400700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f583c108ac0 0x7f583c19f2b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f583000eb30 tx=0x7f583000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.287+0000 7f583a000700 1 -- 172.21.15.26:0/552826321 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583000cbd0 con 0x7f583c108ac0 2024-06-13T07:00:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.287+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583c1a45b0 con 0x7f583c108ac0 2024-06-13T07:00:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.287+0000 7f583a000700 1 -- 172.21.15.26:0/552826321 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5830004500 con 0x7f583c108ac0 2024-06-13T07:00:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.287+0000 7f583a000700 1 -- 172.21.15.26:0/552826321 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830010430 con 0x7f583c108ac0 2024-06-13T07:00:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.287+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583c1abe30 con 0x7f583c108ac0 2024-06-13T07:00:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.288+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f583c069180 con 0x7f583c108ac0 2024-06-13T07:00:34.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.291+0000 7f583a000700 1 -- 172.21.15.26:0/552826321 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5830003f00 con 0x7f583c108ac0 2024-06-13T07:00:34.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.292+0000 7f583a000700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58240a2c20 0x7f58240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:34.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.292+0000 7f583a000700 1 -- 172.21.15.26:0/552826321 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f58300cf6b0 con 0x7f583c108ac0 2024-06-13T07:00:34.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.292+0000 7f5840a00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58240a2c20 0x7f58240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:34.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.293+0000 7f5840a00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58240a2c20 0x7f58240a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f583c077b80 tx=0x7f582800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:34.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.294+0000 7f583a000700 1 -- 172.21.15.26:0/552826321 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f583009c250 con 0x7f583c108ac0 2024-06-13T07:00:34.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.574+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f583c02dca0 con 0x7f583c108ac0 2024-06-13T07:00:34.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.576+0000 7f583a000700 1 -- 172.21.15.26:0/552826321 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f583009bb50 con 0x7f583c108ac0 2024-06-13T07:00:34.576 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:00:34.576 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T07:00:22.609623+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24506,24511,24517]},"id":1}]} 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58240a2c20 msgr2=0x7f58240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58240a2c20 0x7f58240a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f583c077b80 tx=0x7f582800b260 comp rx=0 tx=0).stop 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f583c108ac0 msgr2=0x7f583c19f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f583c108ac0 0x7f583c19f2b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f583000eb30 tx=0x7f583000ee40 comp rx=0 tx=0).stop 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 shutdown_connections 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58240a2c20 0x7f58240a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f583c108ac0 0x7f583c19f2b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c10eac0 0x7f583c1a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 --2- 172.21.15.26:0/552826321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f583c109480 0x7f583c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 >> 172.21.15.26:0/552826321 conn(0x7f583c075c60 msgr2=0x7f583c076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:34.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 shutdown_connections 2024-06-13T07:00:34.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:34.579+0000 7f5843e00700 1 -- 172.21.15.26:0/552826321 wait complete. 2024-06-13T07:00:34.581 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2024-06-13T07:00:34.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:34 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/552826321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:35.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:34 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/552826321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:35.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:34 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/552826321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:35.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T07:00:36.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:35 smithi026 ceph-mon[32041]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-06-13T07:00:36.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:35 smithi135 ceph-mon[36276]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-06-13T07:00:36.097 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:36.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:35 smithi136 ceph-mon[37229]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-06-13T07:00:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:37 smithi135 ceph-mon[36276]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 596 B/s wr, 5 op/s 2024-06-13T07:00:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:37 smithi136 ceph-mon[37229]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 596 B/s wr, 5 op/s 2024-06-13T07:00:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:37 smithi026 ceph-mon[32041]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 596 B/s wr, 5 op/s 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 -- 172.21.15.26:0/3418850267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 msgr2=0x7efc5406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 --2- 172.21.15.26:0/3418850267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 0x7efc5406a5d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009990 tx=0x7efc3c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 -- 172.21.15.26:0/3418850267 shutdown_connections 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 --2- 172.21.15.26:0/3418850267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc5406ab10 0x7efc5410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 --2- 172.21.15.26:0/3418850267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc541084e0 0x7efc541088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 --2- 172.21.15.26:0/3418850267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 0x7efc5406a5d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 -- 172.21.15.26:0/3418850267 >> 172.21.15.26:0/3418850267 conn(0x7efc540782c0 msgr2=0x7efc540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 -- 172.21.15.26:0/3418850267 shutdown_connections 2024-06-13T07:00:38.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.863+0000 7efc59000700 1 -- 172.21.15.26:0/3418850267 wait complete. 2024-06-13T07:00:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.864+0000 7efc59000700 1 Processor -- start 2024-06-13T07:00:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.864+0000 7efc59000700 1 -- start start 2024-06-13T07:00:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc59000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 0x7efc5419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc59000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5406ab10 0x7efc5419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc59000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541084e0 0x7efc5419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc59000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc541127b0 con 0x7efc5406ab10 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc59000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc54112630 con 0x7efc5406a170 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc59000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc54112930 con 0x7efc541084e0 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc49400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5406ab10 0x7efc5419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc49400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5406ab10 0x7efc5419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42872/0 (socket says 172.21.15.26:42872) 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc49400700 1 -- 172.21.15.26:0/2719346537 learned_addr learned my addr 172.21.15.26:0/2719346537 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc51e00700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 0x7efc5419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.865+0000 7efc52800700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541084e0 0x7efc5419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc51e00700 1 -- 172.21.15.26:0/2719346537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541084e0 msgr2=0x7efc5419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc51e00700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541084e0 0x7efc5419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc51e00700 1 -- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5406ab10 msgr2=0x7efc5419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc51e00700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5406ab10 0x7efc5419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc51e00700 1 -- 172.21.15.26:0/2719346537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5419fe10 con 0x7efc5406a170 2024-06-13T07:00:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc51e00700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 0x7efc5419ae50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc4400d950 tx=0x7efc4400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc4b400700 1 -- 172.21.15.26:0/2719346537 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44016070 con 0x7efc5406a170 2024-06-13T07:00:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc4b400700 1 -- 172.21.15.26:0/2719346537 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4400fd70 con 0x7efc5406a170 2024-06-13T07:00:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.866+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc541a0100 con 0x7efc5406a170 2024-06-13T07:00:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.867+0000 7efc4b400700 1 -- 172.21.15.26:0/2719346537 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4400e3b0 con 0x7efc5406a170 2024-06-13T07:00:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.867+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc541a7980 con 0x7efc5406a170 2024-06-13T07:00:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.868+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc54069180 con 0x7efc5406a170 2024-06-13T07:00:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.870+0000 7efc4b400700 1 -- 172.21.15.26:0/2719346537 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc44019070 con 0x7efc5406a170 2024-06-13T07:00:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.871+0000 7efc4b400700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc380a2ca0 0x7efc380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.871+0000 7efc4b400700 1 -- 172.21.15.26:0/2719346537 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7efc440ced70 con 0x7efc5406a170 2024-06-13T07:00:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.871+0000 7efc49400700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc380a2ca0 0x7efc380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.872+0000 7efc49400700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc380a2ca0 0x7efc380a5160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc540788c0 tx=0x7efc3c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:38.877+0000 7efc4b400700 1 -- 172.21.15.26:0/2719346537 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efc44010460 con 0x7efc5406a170 2024-06-13T07:00:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.109+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc5419c190 con 0x7efc5406a170 2024-06-13T07:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.112+0000 7efc4b400700 1 -- 172.21.15.26:0/2719346537 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7efc5419c190 con 0x7efc5406a170 2024-06-13T07:00:39.113 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:00:39.113 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T07:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.115+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc380a2ca0 msgr2=0x7efc380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.115+0000 7efc59000700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc380a2ca0 0x7efc380a5160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc540788c0 tx=0x7efc3c00b260 comp rx=0 tx=0).stop 2024-06-13T07:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.115+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 msgr2=0x7efc5419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.115+0000 7efc59000700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 0x7efc5419ae50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc4400d950 tx=0x7efc4400dc60 comp rx=0 tx=0).stop 2024-06-13T07:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.116+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 shutdown_connections 2024-06-13T07:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.116+0000 7efc59000700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc380a2ca0 0x7efc380a5160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.116+0000 7efc59000700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc5406a170 0x7efc5419ae50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.116+0000 7efc59000700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541084e0 0x7efc5419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.116+0000 7efc59000700 1 --2- 172.21.15.26:0/2719346537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5406ab10 0x7efc5419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.116+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 >> 172.21.15.26:0/2719346537 conn(0x7efc540782c0 msgr2=0x7efc5410d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.116+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 shutdown_connections 2024-06-13T07:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:39.116+0000 7efc59000700 1 -- 172.21.15.26:0/2719346537 wait complete. 2024-06-13T07:00:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:39 smithi135 ceph-mon[36276]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-13T07:00:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:39 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2719346537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:40.037 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:39 smithi026 ceph-mon[32041]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-13T07:00:40.037 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:39 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2719346537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:40.112 DEBUG:tasks.fwd_scrub:Yielding 2024-06-13T07:00:40.113 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-13T07:00:40.130 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T07:00:40.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-13T07:00:40.130 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-13T07:00:40.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:00:40.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:00:40.132 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:39 smithi136 ceph-mon[37229]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-13T07:00:40.132 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:39 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2719346537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:00:40.139 INFO:teuthology.run_tasks:Running task workunit... 2024-06-13T07:00:40.152 INFO:tasks.workunit:Pulling workunits from ref 1373963c202323287783ccb099bbae1ec8f14a5d 2024-06-13T07:00:40.153 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-13T07:00:40.153 INFO:tasks.workunit:timeout=3h 2024-06-13T07:00:40.153 INFO:tasks.workunit:cleanup=True 2024-06-13T07:00:40.153 DEBUG:teuthology.orchestra.run.smithi026:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-13T07:00:40.198 INFO:teuthology.orchestra.run.smithi026.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-13T07:00:40.198 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-13T07:00:40.198 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-13T07:00:40.198 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T07:00:40.198 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-13T07:00:40.198 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-06-13 06:56:54.140884448 +0000 2024-06-13T07:00:40.198 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-06-13 06:56:54.140884448 +0000 2024-06-13T07:00:40.199 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-06-13 07:00:21.549242809 +0000 2024-06-13T07:00:40.199 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-06-13 06:56:54.140884448 +0000 2024-06-13T07:00:40.199 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-13T07:00:40.199 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-13T07:00:40.319 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1373963c202323287783ccb099bbae1ec8f14a5d 2024-06-13T07:00:40.376 INFO:tasks.workunit.client.0.smithi026.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-13T07:00:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:42.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:41 smithi026 ceph-mon[32041]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-13T07:00:42.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:41 smithi136 ceph-mon[37229]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-13T07:00:42.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:41 smithi135 ceph-mon[36276]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-13T07:00:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3659626232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b4102e80 msgr2=0x7fa7b410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3659626232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b4102e80 0x7fa7b410f330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009f90 tx=0x7fa7ac00b330 comp rx=0 tx=0).stop 2024-06-13T07:00:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3659626232 shutdown_connections 2024-06-13T07:00:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3659626232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7b41084f0 0x7fa7b41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3659626232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b4102e80 0x7fa7b410f330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3659626232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7b41024e0 0x7fa7b4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3659626232 >> 172.21.15.26:0/3659626232 conn(0x7fa7b4079530 msgr2=0x7fa7b4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3659626232 shutdown_connections 2024-06-13T07:00:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.137+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3659626232 wait complete. 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.138+0000 7fa7b9a00700 1 Processor -- start 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.139+0000 7fa7b9a00700 1 -- start start 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.139+0000 7fa7b9a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b41024e0 0x7fa7b419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.139+0000 7fa7b9a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7b4102e80 0x7fa7b419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.139+0000 7fa7b9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7b41084f0 0x7fa7b419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.139+0000 7fa7b9a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4112740 con 0x7fa7b4102e80 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.139+0000 7fa7b9a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41125c0 con 0x7fa7b41084f0 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.139+0000 7fa7b9a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41128c0 con 0x7fa7b41024e0 2024-06-13T07:00:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.139+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b41024e0 0x7fa7b419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7aa000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7b4102e80 0x7fa7b419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b41024e0 0x7fa7b419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:32810/0 (socket says 172.21.15.26:32810) 2024-06-13T07:00:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7b41084f0 0x7fa7b419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b2a00700 1 -- 172.21.15.26:0/3982547436 learned_addr learned my addr 172.21.15.26:0/3982547436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b2a00700 1 -- 172.21.15.26:0/3982547436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7b41084f0 msgr2=0x7fa7b419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7b41084f0 0x7fa7b419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b2a00700 1 -- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7b4102e80 msgr2=0x7fa7b419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7b4102e80 0x7fa7b419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b2a00700 1 -- 172.21.15.26:0/3982547436 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b419fee0 con 0x7fa7b41024e0 2024-06-13T07:00:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.140+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b41024e0 0x7fa7b419aed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e9a0 tx=0x7fa7a400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.141+0000 7fa7abe00700 1 -- 172.21.15.26:0/3982547436 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400c9c0 con 0x7fa7b41024e0 2024-06-13T07:00:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.141+0000 7fa7abe00700 1 -- 172.21.15.26:0/3982547436 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a400cb60 con 0x7fa7b41024e0 2024-06-13T07:00:43.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.141+0000 7fa7abe00700 1 -- 172.21.15.26:0/3982547436 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4018660 con 0x7fa7b41024e0 2024-06-13T07:00:43.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.141+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3982547436 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b41a01d0 con 0x7fa7b41024e0 2024-06-13T07:00:43.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.141+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3982547436 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b41a7a50 con 0x7fa7b41024e0 2024-06-13T07:00:43.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.144+0000 7fa7abe00700 1 -- 172.21.15.26:0/3982547436 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7a400cd10 con 0x7fa7b41024e0 2024-06-13T07:00:43.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.146+0000 7fa7c42a4b80 1 Processor -- start 2024-06-13T07:00:43.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7c42a4b80 1 -- start start 2024-06-13T07:00:43.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7abe00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7980a2ca0 0x7fa7980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7abe00700 1 -- 172.21.15.26:0/3982547436 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa7a40cf510 con 0x7fa7b41024e0 2024-06-13T07:00:43.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f286ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7b3400700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7980a2ca0 0x7fa7980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f280d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f281700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56097f18fa90 con 0x56097f286c10 2024-06-13T07:00:43.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56097f1d2b80 con 0x56097f287530 2024-06-13T07:00:43.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.147+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56097f18d970 con 0x56097f2812c0 2024-06-13T07:00:43.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7aa000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f280d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f286ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7aa000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f280d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47760/0 (socket says 172.21.15.26:47760) 2024-06-13T07:00:43.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7aa000700 1 -- 172.21.15.26:0/1964777854 learned_addr learned my addr 172.21.15.26:0/1964777854 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:43.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b3400700 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f281700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b2a00700 1 -- 172.21.15.26:0/1964777854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 msgr2=0x56097f281700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:43.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f281700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b2a00700 1 -- 172.21.15.26:0/1964777854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 msgr2=0x56097f280d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f280d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b2a00700 1 -- 172.21.15.26:0/1964777854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097f17b010 con 0x56097f286c10 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7aa000700 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f280d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b3400700 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f281700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.148+0000 7fa7b3400700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7980a2ca0 0x7fa7980a5160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00b6c0 tx=0x7fa7ac003a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.149+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f286ff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c5b0 tx=0x7fa7a4009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc9c8999dfc93ca server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.149+0000 7fa7a3e00700 1 -- 172.21.15.26:0/1964777854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a409fcd0 con 0x56097f286c10 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.149+0000 7fa7a3e00700 1 -- 172.21.15.26:0/1964777854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a40a4590 con 0x56097f286c10 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.149+0000 7fa7a3e00700 1 -- 172.21.15.26:0/1964777854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40a4300 con 0x56097f286c10 2024-06-13T07:00:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1964777854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 msgr2=0x56097f286ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f286ff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c5b0 tx=0x7fa7a4009820 comp rx=0 tx=0).stop 2024-06-13T07:00:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1964777854 shutdown_connections 2024-06-13T07:00:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f281700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f286ff0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1964777854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f280d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1964777854 >> 172.21.15.26:0/1964777854 conn(0x56097f1ad1f0 msgr2=0x56097f27c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1964777854 shutdown_connections 2024-06-13T07:00:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.150+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1964777854 wait complete. 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.151+0000 7fa7c42a4b80 1 -- start start 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.151+0000 7fa7c42a4b80 1 Processor -- start 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.151+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f28d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.151+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f28a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.151+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f28aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56097f16efe0 con 0x56097f286c10 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56097f1ceb40 con 0x56097f287530 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56097f28de50 con 0x56097f2812c0 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7c42a4b80 10 client.0 _ll_register_callbacks cb 0x56097f168400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7aa000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f28a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7aa000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f28a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59586/0 (socket says 172.21.15.26:59586) 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7aa000700 1 -- 172.21.15.26:0/4036633656 learned_addr learned my addr 172.21.15.26:0/4036633656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7b3400700 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f28aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f28d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7aa000700 1 -- 172.21.15.26:0/4036633656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 msgr2=0x56097f28d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7aa000700 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f28d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7aa000700 1 -- 172.21.15.26:0/4036633656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 msgr2=0x56097f28aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7aa000700 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f28aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7aa000700 1 -- 172.21.15.26:0/4036633656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097f121950 con 0x56097f286c10 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.152+0000 7fa7b3400700 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f28aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.153+0000 7fa7aa000700 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f28a580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa79c012980 tx=0x7fa79c012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c7c0b3515d5781 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.153+0000 7fa7a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:00:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.153+0000 7fa7a0c00700 1 -- 172.21.15.26:0/4036633656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c004050 con 0x56097f286c10 2024-06-13T07:00:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.153+0000 7fa7a0c00700 1 -- 172.21.15.26:0/4036633656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa79c0041f0 con 0x56097f286c10 2024-06-13T07:00:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.153+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/4036633656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a40a4590 con 0x56097f286c10 2024-06-13T07:00:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7a0c00700 1 -- 172.21.15.26:0/4036633656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa79c0043f0 con 0x56097f286c10 2024-06-13T07:00:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7c42a4b80 10 client.15099 fetch_fsmap learned FSMap version 27 2024-06-13T07:00:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/4036633656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56097f298600 con 0x56097f286c10 2024-06-13T07:00:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7a0c00700 1 -- 172.21.15.26:0/4036633656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a40a4590 con 0x56097f286c10 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7a0c00700 1 -- 172.21.15.26:0/4036633656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa79c00fa80 con 0x56097f286c10 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7c42a4b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7c42a4b80 10 client.15099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7c42a4b80 20 client.15099 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56097f29ac40 0x56097f29d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7c42a4b80 4 client.15099 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.154+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/4036633656 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56097f1e0080 con 0x56097f29ac40 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.155+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56097f29ac40 0x56097f29d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.155+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56097f29ac40 0x56097f29d050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87afdf97b8437f04 server_cookie=84a8386d6d82abc5 in_seq=0 out_seq=0 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.155+0000 7fa7a0c00700 10 client.15099 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.156+0000 7fa7a0c00700 1 -- 172.21.15.26:0/4036633656 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56097f1e0080 con 0x56097f29ac40 2024-06-13T07:00:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.156+0000 7fa7a0c00700 10 client.15099 handle_command_reply: tid=0 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.156+0000 7fa7c42a4b80 1 client.15099 shutdown 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.156+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/4036633656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 msgr2=0x56097f28a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.156+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f28a580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa79c012980 tx=0x7fa79c012c90 comp rx=0 tx=0).stop 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.156+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/4036633656 shutdown_connections 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.156+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56097f29ac40 0x56097f29d050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.157+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f28aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.157+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f28d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.157+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/4036633656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f28a580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.157+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/4036633656 >> 172.21.15.26:0/4036633656 conn(0x56097f1ad1f0 msgr2=0x56097f275810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.157+0000 7fa7a0c00700 0 client.15099 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.157+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/4036633656 shutdown_connections 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.157+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/4036633656 wait complete. 2024-06-13T07:00:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.157+0000 7fa7c42a4b80 20 client.15099 trim_cache size 0 max 16384 2024-06-13T07:00:43.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.180+0000 7fa7c42a4b80 1 Processor -- start 2024-06-13T07:00:43.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.180+0000 7fa7c42a4b80 1 -- start start 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.180+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f2812c0 0x56097f280d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.180+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f286c10 0x56097f2881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f287530 0x56097f27c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41b2f90 con 0x56097f287530 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41b0820 con 0x56097f2812c0 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41ae670 con 0x56097f286c10 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f2812c0 0x56097f280d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f2812c0 0x56097f280d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47780/0 (socket says 172.21.15.26:47780) 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b2a00700 1 -- 172.21.15.26:0/3203326227 learned_addr learned my addr 172.21.15.26:0/3203326227 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b3400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f287530 0x56097f27c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7aa000700 1 --2- 172.21.15.26:0/3203326227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f286c10 0x56097f2881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b2a00700 1 -- 172.21.15.26:0/3203326227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f286c10 msgr2=0x56097f2881a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/3203326227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f286c10 0x56097f2881a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b2a00700 1 -- 172.21.15.26:0/3203326227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f287530 msgr2=0x56097f27c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/3203326227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f287530 0x56097f27c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.181+0000 7fa7b2a00700 1 -- 172.21.15.26:0/3203326227 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097f288b10 con 0x56097f2812c0 2024-06-13T07:00:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.182+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/3203326227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f2812c0 0x56097f280d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e970 tx=0x7fa7a4009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95286c274d09de76 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:43.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.182+0000 7fa7a3400700 1 -- 172.21.15.26:0/3203326227 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40b4070 con 0x56097f2812c0 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.182+0000 7fa7a3400700 1 -- 172.21.15.26:0/3203326227 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a40cf1c0 con 0x56097f2812c0 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.182+0000 7fa7a3400700 1 -- 172.21.15.26:0/3203326227 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4086310 con 0x56097f2812c0 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/3203326227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f2812c0 msgr2=0x56097f280d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/3203326227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f2812c0 0x56097f280d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e970 tx=0x7fa7a4009790 comp rx=0 tx=0).stop 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/3203326227 shutdown_connections 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/3203326227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f2812c0 0x56097f280d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/3203326227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f287530 0x56097f27c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/3203326227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f286c10 0x56097f2881a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/3203326227 >> 172.21.15.26:0/3203326227 conn(0x56097f1ad1f0 msgr2=0x56097f285690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:43.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/3203326227 shutdown_connections 2024-06-13T07:00:43.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.183+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/3203326227 wait complete. 2024-06-13T07:00:43.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 1 -- start start 2024-06-13T07:00:43.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 1 Processor -- start 2024-06-13T07:00:43.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f2a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f2a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f2a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41aece0 con 0x56097f286c10 2024-06-13T07:00:43.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41b0150 con 0x56097f287530 2024-06-13T07:00:43.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41af3b0 con 0x56097f2812c0 2024-06-13T07:00:43.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.184+0000 7fa7c42a4b80 10 client.0 _ll_register_callbacks cb 0x56097f168400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:43.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7aa000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f2a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7aa000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f2a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59606/0 (socket says 172.21.15.26:59606) 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7aa000700 1 -- 172.21.15.26:0/1067790534 learned_addr learned my addr 172.21.15.26:0/1067790534 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f2a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7b3400700 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f2a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7b2a00700 1 -- 172.21.15.26:0/1067790534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 msgr2=0x56097f2a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f2a30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7b2a00700 1 -- 172.21.15.26:0/1067790534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 msgr2=0x56097f2a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f2a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7b2a00700 1 -- 172.21.15.26:0/1067790534 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097f3049e0 con 0x56097f2812c0 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7b2a00700 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f2a2630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa7a409fe90 tx=0x7fa7a4009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=515bf416127d2a69 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.185+0000 7fa7a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7a0c00700 1 -- 172.21.15.26:0/1067790534 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40b4040 con 0x56097f2812c0 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7a0c00700 1 -- 172.21.15.26:0/1067790534 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa7a40a4310 con 0x56097f2812c0 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1067790534 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a40cf1c0 con 0x56097f2812c0 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7a0c00700 1 -- 172.21.15.26:0/1067790534 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7a4083050 con 0x56097f2812c0 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7c42a4b80 10 client.25231 fetch_fsmap learned FSMap version 27 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1067790534 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56097f305d00 con 0x56097f2812c0 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7a0c00700 1 -- 172.21.15.26:0/1067790534 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a40cf1c0 con 0x56097f2812c0 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7a0c00700 1 -- 172.21.15.26:0/1067790534 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa7a409ad70 con 0x56097f2812c0 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7c42a4b80 10 client.25231 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7c42a4b80 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7c42a4b80 20 client.25231 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.186+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56097f305f30 0x56097f308320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:43.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.187+0000 7fa7c42a4b80 4 client.25231 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:00:43.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.187+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1067790534 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56097f1da600 con 0x56097f305f30 2024-06-13T07:00:43.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.187+0000 7fa7aa000700 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56097f305f30 0x56097f308320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:43.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.187+0000 7fa7aa000700 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56097f305f30 0x56097f308320 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e13e6cb60db39d4 server_cookie=558749bf73d9ae2f in_seq=0 out_seq=0 2024-06-13T07:00:43.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.187+0000 7fa7a0c00700 10 client.25231 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.189+0000 7fa7a0c00700 1 -- 172.21.15.26:0/1067790534 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56097f1da600 con 0x56097f305f30 2024-06-13T07:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.189+0000 7fa7a0c00700 10 client.25231 handle_command_reply: tid=0 2024-06-13T07:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.189+0000 7fa7c42a4b80 1 client.25231 shutdown 2024-06-13T07:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.189+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1067790534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 msgr2=0x56097f2a2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.189+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f2a2630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa7a409fe90 tx=0x7fa7a4009820 comp rx=0 tx=0).stop 2024-06-13T07:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.189+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1067790534 shutdown_connections 2024-06-13T07:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56097f305f30 0x56097f308320 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097f287530 0x56097f2a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56097f2812c0 0x56097f2a2630 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7c42a4b80 1 --2- 172.21.15.26:0/1067790534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56097f286c10 0x56097f2a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7a0c00700 0 client.25231 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1067790534 >> 172.21.15.26:0/1067790534 conn(0x56097f1ad1f0 msgr2=0x56097f288020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1067790534 shutdown_connections 2024-06-13T07:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7c42a4b80 1 -- 172.21.15.26:0/1067790534 wait complete. 2024-06-13T07:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.190+0000 7fa7c42a4b80 20 client.25231 trim_cache size 0 max 16384 2024-06-13T07:00:43.193 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:00:43.193 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-06-13T07:00:43.193 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "8bbd8d2a-55bc-4af3-8a08-477a2a87f43b", 2024-06-13T07:00:43.193 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-06-13T07:00:43.193 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7980a2ca0 msgr2=0x7fa7980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7980a2ca0 0x7fa7980a5160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00b6c0 tx=0x7fa7ac003a00 comp rx=0 tx=0).stop 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3982547436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b41024e0 msgr2=0x7fa7b419aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b41024e0 0x7fa7b419aed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e9a0 tx=0x7fa7a400ecb0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3982547436 shutdown_connections 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7980a2ca0 0x7fa7980a5160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7b41084f0 0x7fa7b419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b41024e0 0x7fa7b419aed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 --2- 172.21.15.26:0/3982547436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7b4102e80 0x7fa7b419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.191+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3982547436 >> 172.21.15.26:0/3982547436 conn(0x7fa7b4079530 msgr2=0x7fa7b4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.192+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3982547436 shutdown_connections 2024-06-13T07:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:43.192+0000 7fa7b9a00700 1 -- 172.21.15.26:0/3982547436 wait complete. 2024-06-13T07:00:44.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:43 smithi026 ceph-mon[32041]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T07:00:44.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:43 smithi026 ceph-mon[32041]: scrub queued for path: / 2024-06-13T07:00:44.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:43 smithi026 ceph-mon[32041]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:44.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:43 smithi026 ceph-mon[32041]: scrub summary: active paths [/] 2024-06-13T07:00:44.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:43 smithi026 ceph-mon[32041]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:43 smithi136 ceph-mon[37229]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T07:00:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:43 smithi136 ceph-mon[37229]: scrub queued for path: / 2024-06-13T07:00:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:43 smithi136 ceph-mon[37229]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:43 smithi136 ceph-mon[37229]: scrub summary: active paths [/] 2024-06-13T07:00:44.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:43 smithi136 ceph-mon[37229]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:43 smithi135 ceph-mon[36276]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T07:00:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:43 smithi135 ceph-mon[36276]: scrub queued for path: / 2024-06-13T07:00:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:43 smithi135 ceph-mon[36276]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:43 smithi135 ceph-mon[36276]: scrub summary: active paths [/] 2024-06-13T07:00:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:43 smithi135 ceph-mon[36276]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:44.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:00:44.792 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:46.107 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:45 smithi135 ceph-mon[36276]: pgmap v517: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-06-13T07:00:46.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:45 smithi136 ceph-mon[37229]: pgmap v517: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-06-13T07:00:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:45 smithi026 ceph-mon[32041]: pgmap v517: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-06-13T07:00:47.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.316+0000 7f8535800700 1 -- 172.21.15.26:0/2462008761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853010a6b0 msgr2=0x7f853010aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.316+0000 7f8535800700 1 --2- 172.21.15.26:0/2462008761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853010a6b0 0x7f853010aa90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8520009990 tx=0x7f8520009ca0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.316+0000 7f8535800700 1 -- 172.21.15.26:0/2462008761 shutdown_connections 2024-06-13T07:00:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.316+0000 7f8535800700 1 --2- 172.21.15.26:0/2462008761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8530105070 0x7f8530111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.316+0000 7f8535800700 1 --2- 172.21.15.26:0/2462008761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853010a6b0 0x7f853010aa90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.316+0000 7f8535800700 1 --2- 172.21.15.26:0/2462008761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85301046b0 0x7f8530104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.316+0000 7f8535800700 1 -- 172.21.15.26:0/2462008761 >> 172.21.15.26:0/2462008761 conn(0x7f85301001a0 msgr2=0x7f85301025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.317+0000 7f8535800700 1 -- 172.21.15.26:0/2462008761 shutdown_connections 2024-06-13T07:00:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.317+0000 7f8535800700 1 -- 172.21.15.26:0/2462008761 wait complete. 2024-06-13T07:00:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.318+0000 7f8535800700 1 Processor -- start 2024-06-13T07:00:47.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.318+0000 7f8535800700 1 -- start start 2024-06-13T07:00:47.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.319+0000 7f8535800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85301046b0 0x7f853019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.319+0000 7f8535800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8530105070 0x7f853019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.319+0000 7f8535800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853010a6b0 0x7f853019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.319+0000 7f8535800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8530114a40 con 0x7f853010a6b0 2024-06-13T07:00:47.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.319+0000 7f8535800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85301148c0 con 0x7f85301046b0 2024-06-13T07:00:47.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.319+0000 7f8535800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8530114bc0 con 0x7f8530105070 2024-06-13T07:00:47.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.319+0000 7f852ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85301046b0 0x7f853019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f8527e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8530105070 0x7f853019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85301046b0 0x7f853019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47814/0 (socket says 172.21.15.26:47814) 2024-06-13T07:00:47.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852ea00700 1 -- 172.21.15.26:0/3320753469 learned_addr learned my addr 172.21.15.26:0/3320753469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:47.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852f400700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853010a6b0 0x7f853019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852ea00700 1 -- 172.21.15.26:0/3320753469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8530105070 msgr2=0x7f853019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852ea00700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8530105070 0x7f853019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852ea00700 1 -- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853010a6b0 msgr2=0x7f853019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852ea00700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853010a6b0 0x7f853019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852ea00700 1 -- 172.21.15.26:0/3320753469 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853019fe60 con 0x7f85301046b0 2024-06-13T07:00:47.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.320+0000 7f852ea00700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85301046b0 0x7f853019ae50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8520005910 tx=0x7f852000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:47.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.321+0000 7f8527400700 1 -- 172.21.15.26:0/3320753469 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520027070 con 0x7f85301046b0 2024-06-13T07:00:47.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.321+0000 7f8527400700 1 -- 172.21.15.26:0/3320753469 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8520004650 con 0x7f85301046b0 2024-06-13T07:00:47.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.321+0000 7f8535800700 1 -- 172.21.15.26:0/3320753469 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85301a00f0 con 0x7f85301046b0 2024-06-13T07:00:47.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.321+0000 7f8527400700 1 -- 172.21.15.26:0/3320753469 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852000f460 con 0x7f85301046b0 2024-06-13T07:00:47.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.321+0000 7f8535800700 1 -- 172.21.15.26:0/3320753469 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85301a79d0 con 0x7f85301046b0 2024-06-13T07:00:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.326+0000 7f8527400700 1 -- 172.21.15.26:0/3320753469 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f852002a070 con 0x7f85301046b0 2024-06-13T07:00:47.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8540093b80 1 Processor -- start 2024-06-13T07:00:47.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8540093b80 1 -- start start 2024-06-13T07:00:47.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8527400700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85140a2c20 0x7f85140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8540093b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c6361e40 0x5573c63e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8540093b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c63e55b0 0x5573c63e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8540093b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8527400700 1 -- 172.21.15.26:0/3320753469 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f85200d0360 con 0x7f85301046b0 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8540093b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5573c62f4a10 con 0x5573c6361e40 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8540093b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5573c6337b00 con 0x5573c63f0300 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.327+0000 7f8540093b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5573c62f28f0 con 0x5573c63e55b0 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f852f400700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85140a2c20 0x7f85140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f8527e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c63e55b0 0x5573c63e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f8527e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c63e55b0 0x5573c63e5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:32884/0 (socket says 172.21.15.26:32884) 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f852f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f8527e00700 1 -- 172.21.15.26:0/578918370 learned_addr learned my addr 172.21.15.26:0/578918370 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f852ea00700 1 --2- 172.21.15.26:0/578918370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c6361e40 0x5573c63e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f8527e00700 1 -- 172.21.15.26:0/578918370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 msgr2=0x5573c63f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f8527e00700 1 --2- 172.21.15.26:0/578918370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f8527e00700 1 -- 172.21.15.26:0/578918370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c6361e40 msgr2=0x5573c63e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f8527e00700 1 --2- 172.21.15.26:0/578918370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c6361e40 0x5573c63e5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.328+0000 7f8527e00700 1 -- 172.21.15.26:0/578918370 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c62dff90 con 0x5573c63e55b0 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.329+0000 7f8527e00700 1 --2- 172.21.15.26:0/578918370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c63e55b0 0x5573c63e5a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d20 tx=0x7f8518011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f354b426aec830a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.329+0000 7f852f400700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85140a2c20 0x7f85140a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8528009ec0 tx=0x7f85280134a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.329+0000 7f851fe00700 1 -- 172.21.15.26:0/578918370 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851801e070 con 0x5573c63e55b0 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.329+0000 7f851fe00700 1 -- 172.21.15.26:0/578918370 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8518019b10 con 0x5573c63e55b0 2024-06-13T07:00:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.329+0000 7f851fe00700 1 -- 172.21.15.26:0/578918370 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851800eb50 con 0x5573c63e55b0 2024-06-13T07:00:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 -- 172.21.15.26:0/578918370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c63e55b0 msgr2=0x5573c63e5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 --2- 172.21.15.26:0/578918370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c63e55b0 0x5573c63e5a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d20 tx=0x7f8518011720 comp rx=0 tx=0).stop 2024-06-13T07:00:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 -- 172.21.15.26:0/578918370 shutdown_connections 2024-06-13T07:00:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 --2- 172.21.15.26:0/578918370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 --2- 172.21.15.26:0/578918370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c6361e40 0x5573c63e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 --2- 172.21.15.26:0/578918370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c63e55b0 0x5573c63e5a30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 -- 172.21.15.26:0/578918370 >> 172.21.15.26:0/578918370 conn(0x5573c6312170 msgr2=0x5573c63e14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 -- 172.21.15.26:0/578918370 shutdown_connections 2024-06-13T07:00:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.330+0000 7f8540093b80 1 -- 172.21.15.26:0/578918370 wait complete. 2024-06-13T07:00:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.331+0000 7f8540093b80 1 -- start start 2024-06-13T07:00:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.331+0000 7f8540093b80 1 Processor -- start 2024-06-13T07:00:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8540093b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8540093b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8540093b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8540093b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5573c62d3f60 con 0x5573c63e55b0 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8540093b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5573c6333ac0 con 0x5573c63f0300 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8540093b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5573c63f2fc0 con 0x5573c6361e40 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8540093b80 10 client.0 _ll_register_callbacks cb 0x5573c62d1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8527e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8527e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63ecd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59628/0 (socket says 172.21.15.26:59628) 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f8527e00700 1 -- 172.21.15.26:0/4197990210 learned_addr learned my addr 172.21.15.26:0/4197990210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f852f400700 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.332+0000 7f852ea00700 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f852f400700 1 -- 172.21.15.26:0/4197990210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 msgr2=0x5573c63ef9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f852f400700 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63ef9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f852f400700 1 -- 172.21.15.26:0/4197990210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 msgr2=0x5573c63ecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f852f400700 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63ecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f852f400700 1 -- 172.21.15.26:0/4197990210 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c62868d0 con 0x5573c63f0300 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f852ea00700 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63ef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f852f400700 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63ed2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8528009fd0 tx=0x7f8528004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=670dd7985f54cbe3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f851cc00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:00:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f851cc00700 1 -- 172.21.15.26:0/4197990210 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852802b020 con 0x5573c63f0300 2024-06-13T07:00:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f851cc00700 1 -- 172.21.15.26:0/4197990210 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8528003a00 con 0x5573c63f0300 2024-06-13T07:00:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.333+0000 7f8540093b80 1 -- 172.21.15.26:0/4197990210 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8518019b10 con 0x5573c63f0300 2024-06-13T07:00:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.334+0000 7f851cc00700 1 -- 172.21.15.26:0/4197990210 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8528033050 con 0x5573c63f0300 2024-06-13T07:00:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.334+0000 7f8540093b80 10 client.25211 fetch_fsmap learned FSMap version 27 2024-06-13T07:00:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.334+0000 7f8540093b80 1 -- 172.21.15.26:0/4197990210 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573c63fd6e0 con 0x5573c63f0300 2024-06-13T07:00:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.334+0000 7f851cc00700 1 -- 172.21.15.26:0/4197990210 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8518019b10 con 0x5573c63f0300 2024-06-13T07:00:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.334+0000 7f851cc00700 1 -- 172.21.15.26:0/4197990210 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8528004020 con 0x5573c63f0300 2024-06-13T07:00:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.335+0000 7f8540093b80 10 client.25211 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:00:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.335+0000 7f8540093b80 10 client.25211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:00:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.335+0000 7f8540093b80 20 client.25211 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.335+0000 7f8540093b80 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5573c63ffd20 0x5573c6402180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.335+0000 7f8540093b80 4 client.25211 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.335+0000 7f8540093b80 1 -- 172.21.15.26:0/4197990210 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573c6345000 con 0x5573c63ffd20 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.335+0000 7f852ea00700 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5573c63ffd20 0x5573c6402180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.336+0000 7f852ea00700 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5573c63ffd20 0x5573c6402180 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce76c87af6f048d3 server_cookie=16075e5a2369ac43 in_seq=0 out_seq=0 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.336+0000 7f851cc00700 10 client.25211 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.336+0000 7f851cc00700 1 -- 172.21.15.26:0/4197990210 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5573c6345000 con 0x5573c63ffd20 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.336+0000 7f851cc00700 10 client.25211 handle_command_reply: tid=0 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 client.25211 shutdown 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 -- 172.21.15.26:0/4197990210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 msgr2=0x5573c63ed2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63ed2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8528009fd0 tx=0x7f8528004b40 comp rx=0 tx=0).stop 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 -- 172.21.15.26:0/4197990210 shutdown_connections 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5573c63ffd20 0x5573c6402180 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63ed2a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63ef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f851cc00700 0 client.25211 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 --2- 172.21.15.26:0/4197990210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 -- 172.21.15.26:0/4197990210 >> 172.21.15.26:0/4197990210 conn(0x5573c6312170 msgr2=0x5573c63e0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 -- 172.21.15.26:0/4197990210 shutdown_connections 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 1 -- 172.21.15.26:0/4197990210 wait complete. 2024-06-13T07:00:47.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.337+0000 7f8540093b80 20 client.25211 trim_cache size 0 max 16384 2024-06-13T07:00:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.360+0000 7f8540093b80 1 Processor -- start 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.360+0000 7f8540093b80 1 -- start start 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.360+0000 7f8540093b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.360+0000 7f8540093b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.360+0000 7f8540093b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63e8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.360+0000 7f8540093b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f85301b2e60 con 0x5573c63e55b0 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.360+0000 7f8540093b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85301b06f0 con 0x5573c63f0300 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.360+0000 7f8540093b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85301ae540 con 0x5573c6361e40 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f8527e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f8527e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63e7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59638/0 (socket says 172.21.15.26:59638) 2024-06-13T07:00:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f8527e00700 1 -- 172.21.15.26:0/2720282850 learned_addr learned my addr 172.21.15.26:0/2720282850 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f852ea00700 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f852f400700 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f8527e00700 1 -- 172.21.15.26:0/2720282850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 msgr2=0x5573c63f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f8527e00700 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f8527e00700 1 -- 172.21.15.26:0/2720282850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 msgr2=0x5573c63e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f8527e00700 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63e8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f8527e00700 1 -- 172.21.15.26:0/2720282850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c63e2080 con 0x5573c63e55b0 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.361+0000 7f852f400700 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.362+0000 7f8527e00700 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63e7fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f851800e680 tx=0x7f851800e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce74de377bb409 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.362+0000 7f851f400700 1 -- 172.21.15.26:0/2720282850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851801e040 con 0x5573c63e55b0 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.362+0000 7f851f400700 1 -- 172.21.15.26:0/2720282850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8518028070 con 0x5573c63e55b0 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.362+0000 7f851f400700 1 -- 172.21.15.26:0/2720282850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518003680 con 0x5573c63e55b0 2024-06-13T07:00:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 -- 172.21.15.26:0/2720282850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 msgr2=0x5573c63e7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63e7fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f851800e680 tx=0x7f851800e3b0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 -- 172.21.15.26:0/2720282850 shutdown_connections 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63f0300 0x5573c63e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 --2- 172.21.15.26:0/2720282850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63e55b0 0x5573c63e7fd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 -- 172.21.15.26:0/2720282850 >> 172.21.15.26:0/2720282850 conn(0x5573c6312170 msgr2=0x5573c63eee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 -- 172.21.15.26:0/2720282850 shutdown_connections 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 -- 172.21.15.26:0/2720282850 wait complete. 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.363+0000 7f8540093b80 1 -- start start 2024-06-13T07:00:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8540093b80 1 Processor -- start 2024-06-13T07:00:47.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8540093b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8540093b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63e55b0 0x5573c63f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8540093b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63f0300 0x5573c63f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8540093b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f85301aebb0 con 0x5573c63f0300 2024-06-13T07:00:47.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8540093b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85301b0020 con 0x5573c63e55b0 2024-06-13T07:00:47.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8540093b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85301af280 con 0x5573c6361e40 2024-06-13T07:00:47.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8540093b80 10 client.0 _ll_register_callbacks cb 0x5573c62d1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f852f400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63f0300 0x5573c63f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f852ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f852ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:32912/0 (socket says 172.21.15.26:32912) 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f852ea00700 1 -- 172.21.15.26:0/624983601 learned_addr learned my addr 172.21.15.26:0/624983601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.364+0000 7f8527e00700 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63e55b0 0x5573c63f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f852ea00700 1 -- 172.21.15.26:0/624983601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63e55b0 msgr2=0x5573c63f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f852ea00700 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63e55b0 0x5573c63f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f852ea00700 1 -- 172.21.15.26:0/624983601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63f0300 msgr2=0x5573c63f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:47.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f852ea00700 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63f0300 0x5573c63f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f852ea00700 1 -- 172.21.15.26:0/624983601 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c63f4030 con 0x5573c6361e40 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f852ea00700 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f8390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85200cda10 tx=0x7f852001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7c0f7473c548f61 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f851cc00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f851cc00700 1 -- 172.21.15.26:0/624983601 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85200ba060 con 0x5573c6361e40 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f8540093b80 1 -- 172.21.15.26:0/624983601 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8518028070 con 0x5573c6361e40 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.365+0000 7f851cc00700 1 -- 172.21.15.26:0/624983601 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f852009ab00 con 0x5573c6361e40 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f851cc00700 1 -- 172.21.15.26:0/624983601 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f852009cf00 con 0x5573c6361e40 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f8540093b80 10 client.25249 fetch_fsmap learned FSMap version 27 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f8540093b80 1 -- 172.21.15.26:0/624983601 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573c646a5a0 con 0x5573c6361e40 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f851cc00700 1 -- 172.21.15.26:0/624983601 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8518028070 con 0x5573c6361e40 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f851cc00700 1 -- 172.21.15.26:0/624983601 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8520022600 con 0x5573c6361e40 2024-06-13T07:00:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f8540093b80 10 client.25249 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f8540093b80 10 client.25249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f8540093b80 20 client.25249 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f8540093b80 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5573c646abb0 0x5573c646cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f8540093b80 4 client.25249 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.366+0000 7f8540093b80 1 -- 172.21.15.26:0/624983601 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573c633f580 con 0x5573c646abb0 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.367+0000 7f8527e00700 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5573c646abb0 0x5573c646cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.367+0000 7f8527e00700 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5573c646abb0 0x5573c646cfc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf7dc910b842a387 server_cookie=d2fa6bbc2187c13e in_seq=0 out_seq=0 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.367+0000 7f851cc00700 10 client.25249 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f851cc00700 1 -- 172.21.15.26:0/624983601 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5573c633f580 con 0x5573c646abb0 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f851cc00700 10 client.25249 handle_command_reply: tid=0 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 client.25249 shutdown 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 -- 172.21.15.26:0/624983601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 msgr2=0x5573c63f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f8390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85200cda10 tx=0x7f852001a040 comp rx=0 tx=0).stop 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 -- 172.21.15.26:0/624983601 shutdown_connections 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5573c646abb0 0x5573c646cfc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573c63f0300 0x5573c63f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573c6361e40 0x5573c63f8390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 --2- 172.21.15.26:0/624983601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c63e55b0 0x5573c63f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 -- 172.21.15.26:0/624983601 >> 172.21.15.26:0/624983601 conn(0x5573c6312170 msgr2=0x5573c63dfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f851cc00700 0 client.25249 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 -- 172.21.15.26:0/624983601 shutdown_connections 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 1 -- 172.21.15.26:0/624983601 wait complete. 2024-06-13T07:00:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.368+0000 7f8540093b80 20 client.25249 trim_cache size 0 max 16384 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout: "8bbd8d2a-55bc-4af3-8a08-477a2a87f43b": { 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "8bbd8d2a-55bc-4af3-8a08-477a2a87f43b", 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:00:47.373 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:00:47.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 -- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85140a2c20 msgr2=0x7f85140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85140a2c20 0x7f85140a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8528009ec0 tx=0x7f85280134a0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 -- 172.21.15.26:0/3320753469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85301046b0 msgr2=0x7f853019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:47.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85301046b0 0x7f853019ae50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8520005910 tx=0x7f852000bab0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 -- 172.21.15.26:0/3320753469 shutdown_connections 2024-06-13T07:00:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f85140a2c20 0x7f85140a50e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85301046b0 0x7f853019ae50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853010a6b0 0x7f853019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 --2- 172.21.15.26:0/3320753469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8530105070 0x7f853019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 -- 172.21.15.26:0/3320753469 >> 172.21.15.26:0/3320753469 conn(0x7f85301001a0 msgr2=0x7f85301006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 -- 172.21.15.26:0/3320753469 shutdown_connections 2024-06-13T07:00:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:00:47.369+0000 7f8535800700 1 -- 172.21.15.26:0/3320753469 wait complete. 2024-06-13T07:00:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:47 smithi136 ceph-mon[37229]: pgmap v518: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-06-13T07:00:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:47 smithi026 ceph-mon[32041]: pgmap v518: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-06-13T07:00:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:47 smithi135 ceph-mon[36276]: pgmap v518: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-06-13T07:00:48.523 INFO:tasks.cephfs.filesystem:scrub status for tag:8bbd8d2a-55bc-4af3-8a08-477a2a87f43b - {'path': '/', 'tag': '8bbd8d2a-55bc-4af3-8a08-477a2a87f43b', 'options': 'recursive,force'} 2024-06-13T07:00:50.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:49 smithi136 ceph-mon[37229]: pgmap v519: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:50.243 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:49 smithi026 ceph-mon[32041]: pgmap v519: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:49 smithi135 ceph-mon[36276]: pgmap v519: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:52.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:51 smithi136 ceph-mon[37229]: pgmap v520: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:51 smithi026 ceph-mon[32041]: pgmap v520: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:52.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:51 smithi135 ceph-mon[36276]: pgmap v520: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T07:00:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.387+0000 7f9e57200640 1 -- 172.21.15.26:0/1972064763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e58101840 msgr2=0x7f9e58101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.387+0000 7f9e57200640 1 --2- 172.21.15.26:0/1972064763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e58101840 0x7f9e58101c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009900 tx=0x7f9e4802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.387+0000 7f9e57200640 1 -- 172.21.15.26:0/1972064763 shutdown_connections 2024-06-13T07:00:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.387+0000 7f9e57200640 1 --2- 172.21.15.26:0/1972064763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e581033f0 0x7f9e58109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.387+0000 7f9e57200640 1 --2- 172.21.15.26:0/1972064763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e58102a30 0x7f9e58102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.387+0000 7f9e57200640 1 --2- 172.21.15.26:0/1972064763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e58101840 0x7f9e58101c60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.387+0000 7f9e57200640 1 -- 172.21.15.26:0/1972064763 >> 172.21.15.26:0/1972064763 conn(0x7f9e580fcdb0 msgr2=0x7f9e580ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.388+0000 7f9e57200640 1 -- 172.21.15.26:0/1972064763 shutdown_connections 2024-06-13T07:00:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.388+0000 7f9e57200640 1 -- 172.21.15.26:0/1972064763 wait complete. 2024-06-13T07:00:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.388+0000 7f9e57200640 1 Processor -- start 2024-06-13T07:00:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e57200640 1 -- start start 2024-06-13T07:00:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e57200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58101840 0x7f9e58075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e57200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e58102a30 0x7f9e58073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e57200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e581033f0 0x7f9e580742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e57200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d4c0 con 0x7f9e58102a30 2024-06-13T07:00:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e57200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d340 con 0x7f9e581033f0 2024-06-13T07:00:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e57200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d640 con 0x7f9e58101840 2024-06-13T07:00:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e58102a30 0x7f9e58073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e58102a30 0x7f9e58073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32924/0 (socket says 172.21.15.26:32924) 2024-06-13T07:00:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58101840 0x7f9e58075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e4fe00640 1 -- 172.21.15.26:0/3684780766 learned_addr learned my addr 172.21.15.26:0/3684780766 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.389+0000 7f9e56800640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e581033f0 0x7f9e580742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e4fe00640 1 -- 172.21.15.26:0/3684780766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58101840 msgr2=0x7f9e58075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58101840 0x7f9e58075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e4fe00640 1 -- 172.21.15.26:0/3684780766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e581033f0 msgr2=0x7f9e580742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e581033f0 0x7f9e580742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e4fe00640 1 -- 172.21.15.26:0/3684780766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58074b60 con 0x7f9e58102a30 2024-06-13T07:00:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e55e00640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58101840 0x7f9e58075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e58102a30 0x7f9e58073d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000ea60 tx=0x7f9e4000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e4ea00640 1 -- 172.21.15.26:0/3684780766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000cde0 con 0x7f9e58102a30 2024-06-13T07:00:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.390+0000 7f9e4ea00640 1 -- 172.21.15.26:0/3684780766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e400035f0 con 0x7f9e58102a30 2024-06-13T07:00:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.391+0000 7f9e4ea00640 1 -- 172.21.15.26:0/3684780766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40010670 con 0x7f9e58102a30 2024-06-13T07:00:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.391+0000 7f9e57200640 1 -- 172.21.15.26:0/3684780766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e581a1c60 con 0x7f9e58102a30 2024-06-13T07:00:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.391+0000 7f9e57200640 1 -- 172.21.15.26:0/3684780766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581a2160 con 0x7f9e58102a30 2024-06-13T07:00:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.394+0000 7f9e5fdb7740 1 Processor -- start 2024-06-13T07:00:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.394+0000 7f9e5fdb7740 1 -- start start 2024-06-13T07:00:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.394+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce0785fdc0 0x55ce078601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.394+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce078652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.394+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 0x55ce07867db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.394+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ce077c3420 con 0x55ce0785fdc0 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.394+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ce0776ac50 con 0x55ce07865920 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.394+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ce077bf3e0 con 0x55ce078607b0 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e56800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce0785fdc0 0x55ce078601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e4ea00640 1 -- 172.21.15.26:0/3684780766 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e400037a0 con 0x7f9e58102a30 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce078652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e56800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce0785fdc0 0x55ce078601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32934/0 (socket says 172.21.15.26:32934) 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce078652b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40192/0 (socket says 172.21.15.26:40192) 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e56800640 1 -- 172.21.15.26:0/856924817 learned_addr learned my addr 172.21.15.26:0/856924817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e55e00640 1 -- 172.21.15.26:0/856924817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 msgr2=0x55ce07867db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e55e00640 1 --2- 172.21.15.26:0/856924817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 0x55ce07867db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e55e00640 1 -- 172.21.15.26:0/856924817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce0785fdc0 msgr2=0x55ce078601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e55e00640 1 --2- 172.21.15.26:0/856924817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce0785fdc0 0x55ce078601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e55e00640 1 -- 172.21.15.26:0/856924817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce074d7210 con 0x55ce078607b0 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.395+0000 7f9e56800640 1 --2- 172.21.15.26:0/856924817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce0785fdc0 0x55ce078601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.396+0000 7f9e55e00640 1 --2- 172.21.15.26:0/856924817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce078652b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009230 tx=0x7f9e48005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a85859d8df86bd99 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.396+0000 7f9e47400640 1 -- 172.21.15.26:0/856924817 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48040070 con 0x55ce078607b0 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.396+0000 7f9e47400640 1 -- 172.21.15.26:0/856924817 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e480319b0 con 0x55ce078607b0 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.396+0000 7f9e47400640 1 -- 172.21.15.26:0/856924817 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48035040 con 0x55ce078607b0 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.396+0000 7f9e4ea00640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e3c0a2b80 0x7f9e3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e4ea00640 1 -- 172.21.15.26:0/3684780766 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9e400cf760 con 0x7f9e58102a30 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e56800640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e3c0a2b80 0x7f9e3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/856924817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 msgr2=0x55ce078652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/856924817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce078652b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009230 tx=0x7f9e48005de0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/856924817 shutdown_connections 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/856924817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 0x55ce07867db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/856924817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce078652b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/856924817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce0785fdc0 0x55ce078601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/856924817 >> 172.21.15.26:0/856924817 conn(0x55ce07827fb0 msgr2=0x55ce0785b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/856924817 shutdown_connections 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.397+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/856924817 wait complete. 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e56800640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e3c0a2b80 0x7f9e3c0a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009de0 tx=0x7f9e50009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 1 -- start start 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 1 Processor -- start 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce078607b0 0x55ce0786a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 0x55ce078716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ce07868870 con 0x55ce078607b0 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ce0777e0b0 con 0x55ce07865920 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ce078689f0 con 0x55ce0785fdc0 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.398+0000 7f9e5fdb7740 10 client.0 _ll_register_callbacks cb 0x55ce07769f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce078607b0 0x55ce0786a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce078607b0 0x55ce0786a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32940/0 (socket says 172.21.15.26:32940) 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e56800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 0x55ce078716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 -- 172.21.15.26:0/4059069794 learned_addr learned my addr 172.21.15.26:0/4059069794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 -- 172.21.15.26:0/4059069794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 msgr2=0x55ce0786a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 -- 172.21.15.26:0/4059069794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 msgr2=0x55ce078716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 0x55ce078716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 -- 172.21.15.26:0/4059069794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce075a3620 con 0x55ce078607b0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e55e00640 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e56800640 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 0x55ce078716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.399+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce078607b0 0x55ce0786a690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e40098030 tx=0x7f9e4009b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c1a4925933c05 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.400+0000 7f9e3be00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.400+0000 7f9e3be00640 1 -- 172.21.15.26:0/4059069794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4009ff80 con 0x55ce078607b0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.400+0000 7f9e3be00640 1 -- 172.21.15.26:0/4059069794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9e400cf470 con 0x55ce078607b0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.400+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/4059069794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e480319b0 con 0x55ce078607b0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.400+0000 7f9e3be00640 1 -- 172.21.15.26:0/4059069794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e400b9050 con 0x55ce078607b0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.400+0000 7f9e5fdb7740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.400+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/4059069794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce07874b00 con 0x55ce078607b0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.400+0000 7f9e3be00640 1 -- 172.21.15.26:0/4059069794 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e480319b0 con 0x55ce078607b0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.401+0000 7f9e3be00640 1 -- 172.21.15.26:0/4059069794 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9e4009b020 con 0x55ce078607b0 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.401+0000 7f9e5fdb7740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:00:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.401+0000 7f9e5fdb7740 10 client.15135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.401+0000 7f9e5fdb7740 20 client.15135 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.401+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce07876180 0x55ce07878570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.401+0000 7f9e5fdb7740 4 client.15135 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.401+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/4059069794 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce077d0b50 con 0x55ce07876180 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.401+0000 7f9e55e00640 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce07876180 0x55ce07878570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.402+0000 7f9e55e00640 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce07876180 0x55ce07878570 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad96be2cd0e6bf42 server_cookie=5d0c92545484eff8 in_seq=0 out_seq=0 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.402+0000 7f9e3be00640 10 client.15135 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e3be00640 1 -- 172.21.15.26:0/4059069794 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ce077d0b50 con 0x55ce07876180 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e3be00640 10 client.15135 handle_command_reply: tid=0 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 client.15135 shutdown 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/4059069794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce078607b0 msgr2=0x55ce0786a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce078607b0 0x55ce0786a690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e40098030 tx=0x7f9e4009b530 comp rx=0 tx=0).stop 2024-06-13T07:00:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/4059069794 shutdown_connections 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce07876180 0x55ce07878570 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce07865920 0x55ce078716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce078607b0 0x55ce0786a690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/4059069794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/4059069794 >> 172.21.15.26:0/4059069794 conn(0x55ce07827fb0 msgr2=0x55ce0785a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e3be00640 0 client.15135 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/4059069794 shutdown_connections 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/4059069794 wait complete. 2024-06-13T07:00:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.403+0000 7f9e5fdb7740 20 client.15135 trim_cache size 0 max 16384 2024-06-13T07:00:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.420+0000 7f9e5fdb7740 1 Processor -- start 2024-06-13T07:00:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e5fdb7740 1 -- start start 2024-06-13T07:00:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce0785fdc0 0x55ce0785ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce07871690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0787ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581af060 con 0x55ce07865920 2024-06-13T07:00:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581ac930 con 0x55ce0785fdc0 2024-06-13T07:00:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581aa780 con 0x55ce078607b0 2024-06-13T07:00:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e56800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0787ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e56800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0787ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32956/0 (socket says 172.21.15.26:32956) 2024-06-13T07:00:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e56800640 1 -- 172.21.15.26:0/3181483852 learned_addr learned my addr 172.21.15.26:0/3181483852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e55e00640 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce0785fdc0 0x55ce0785ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.421+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce07871690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e55e00640 1 -- 172.21.15.26:0/3181483852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 msgr2=0x55ce07871690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e55e00640 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce07871690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e55e00640 1 -- 172.21.15.26:0/3181483852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 msgr2=0x55ce0787ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e55e00640 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0787ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e55e00640 1 -- 172.21.15.26:0/3181483852 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce07855db0 con 0x55ce0785fdc0 2024-06-13T07:00:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e56800640 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0787ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce07871690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e55e00640 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce0785fdc0 0x55ce0785ee40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55ce0785c6f0 tx=0x7f9e4802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f48c85d49a8b66 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e46a00640 1 -- 172.21.15.26:0/3181483852 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48035040 con 0x55ce0785fdc0 2024-06-13T07:00:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e46a00640 1 -- 172.21.15.26:0/3181483852 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e48004760 con 0x55ce0785fdc0 2024-06-13T07:00:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.422+0000 7f9e46a00640 1 -- 172.21.15.26:0/3181483852 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48040040 con 0x55ce0785fdc0 2024-06-13T07:00:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/3181483852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce0785fdc0 msgr2=0x55ce0785ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce0785fdc0 0x55ce0785ee40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55ce0785c6f0 tx=0x7f9e4802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/3181483852 shutdown_connections 2024-06-13T07:00:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0787ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce078607b0 0x55ce07871690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/3181483852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce0785fdc0 0x55ce0785ee40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/3181483852 >> 172.21.15.26:0/3181483852 conn(0x55ce07827fb0 msgr2=0x55ce0785e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/3181483852 shutdown_connections 2024-06-13T07:00:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.423+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/3181483852 wait complete. 2024-06-13T07:00:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 1 -- start start 2024-06-13T07:00:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 1 Processor -- start 2024-06-13T07:00:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce078607b0 0x55ce0786ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0786fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581aadf0 con 0x55ce07865920 2024-06-13T07:00:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581ac260 con 0x55ce078607b0 2024-06-13T07:00:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581ab4c0 con 0x55ce0785fdc0 2024-06-13T07:00:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.424+0000 7f9e5fdb7740 10 client.0 _ll_register_callbacks cb 0x7f9e4000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0786fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0786fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32972/0 (socket says 172.21.15.26:32972) 2024-06-13T07:00:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 -- 172.21.15.26:0/2775481316 learned_addr learned my addr 172.21.15.26:0/2775481316 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:00:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e55e00640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce078607b0 0x55ce0786ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 -- 172.21.15.26:0/2775481316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 msgr2=0x55ce0786e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 -- 172.21.15.26:0/2775481316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce078607b0 msgr2=0x55ce0786ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce078607b0 0x55ce0786ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 -- 172.21.15.26:0/2775481316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce07870280 con 0x55ce07865920 2024-06-13T07:00:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e55e00640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e4fe00640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce078607b0 0x55ce0786ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e56800640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0786fc90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000fb30 tx=0x7f9e50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b8a271228222f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.425+0000 7f9e3be00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:00:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.426+0000 7f9e3be00640 1 -- 172.21.15.26:0/2775481316 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5002e060 con 0x55ce07865920 2024-06-13T07:00:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.426+0000 7f9e3be00640 1 -- 172.21.15.26:0/2775481316 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9e50010d40 con 0x55ce07865920 2024-06-13T07:00:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.426+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/2775481316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e48004760 con 0x55ce07865920 2024-06-13T07:00:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.426+0000 7f9e3be00640 1 -- 172.21.15.26:0/2775481316 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e50035050 con 0x55ce07865920 2024-06-13T07:00:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.426+0000 7f9e5fdb7740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-06-13T07:00:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.426+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/2775481316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce07871660 con 0x55ce07865920 2024-06-13T07:00:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.426+0000 7f9e3be00640 1 -- 172.21.15.26:0/2775481316 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e48004760 con 0x55ce07865920 2024-06-13T07:00:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.426+0000 7f9e3be00640 1 -- 172.21.15.26:0/2775481316 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9e50029400 con 0x55ce07865920 2024-06-13T07:00:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.427+0000 7f9e5fdb7740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:00:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.427+0000 7f9e5fdb7740 10 client.15141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:00:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.427+0000 7f9e5fdb7740 20 client.15141 populate_metadata read hostname 'smithi026' 2024-06-13T07:00:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.427+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce078e1330 0x55ce078e3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.427+0000 7f9e5fdb7740 4 client.15141 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:00:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.427+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/2775481316 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce077caea0 con 0x55ce078e1330 2024-06-13T07:00:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.427+0000 7f9e55e00640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce078e1330 0x55ce078e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.428+0000 7f9e55e00640 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce078e1330 0x55ce078e3750 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83db7c84ee2c1e98 server_cookie=6b1f775e912f92f1 in_seq=0 out_seq=0 2024-06-13T07:00:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.428+0000 7f9e3be00640 10 client.15141 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e3be00640 1 -- 172.21.15.26:0/2775481316 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ce077caea0 con 0x55ce078e1330 2024-06-13T07:00:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e3be00640 10 client.15141 handle_command_reply: tid=0 2024-06-13T07:00:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 client.15141 shutdown 2024-06-13T07:00:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/2775481316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 msgr2=0x55ce0786fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0786fc90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000fb30 tx=0x7f9e50004a40 comp rx=0 tx=0).stop 2024-06-13T07:00:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/2775481316 shutdown_connections 2024-06-13T07:00:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce078e1330 0x55ce078e3750 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce078607b0 0x55ce0786ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce07865920 0x55ce0786fc90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 --2- 172.21.15.26:0/2775481316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce0785fdc0 0x55ce0786e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e3be00640 0 client.15141 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:00:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.442+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/2775481316 >> 172.21.15.26:0/2775481316 conn(0x55ce07827fb0 msgr2=0x55ce0785a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.443+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/2775481316 shutdown_connections 2024-06-13T07:00:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.443+0000 7f9e5fdb7740 1 -- 172.21.15.26:0/2775481316 wait complete. 2024-06-13T07:00:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.443+0000 7f9e5fdb7740 20 client.15141 trim_cache size 0 max 16384 2024-06-13T07:00:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:00:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:00:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:00:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.443+0000 7f9e57200640 1 -- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e3c0a2b80 msgr2=0x7f9e3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.443+0000 7f9e57200640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e3c0a2b80 0x7f9e3c0a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009de0 tx=0x7f9e50009500 comp rx=0 tx=0).stop 2024-06-13T07:00:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.443+0000 7f9e57200640 1 -- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e58102a30 msgr2=0x7f9e58073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.443+0000 7f9e57200640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e58102a30 0x7f9e58073d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000ea60 tx=0x7f9e4000ef20 comp rx=0 tx=0).stop 2024-06-13T07:00:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.444+0000 7f9e57200640 1 -- 172.21.15.26:0/3684780766 shutdown_connections 2024-06-13T07:00:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.444+0000 7f9e57200640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e3c0a2b80 0x7f9e3c0a5040 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.444+0000 7f9e57200640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e581033f0 0x7f9e580742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.444+0000 7f9e57200640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e58102a30 0x7f9e58073d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.444+0000 7f9e57200640 1 --2- 172.21.15.26:0/3684780766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58101840 0x7f9e58075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.444+0000 7f9e57200640 1 -- 172.21.15.26:0/3684780766 >> 172.21.15.26:0/3684780766 conn(0x7f9e580fcdb0 msgr2=0x7f9e58105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.444+0000 7f9e57200640 1 -- 172.21.15.26:0/3684780766 shutdown_connections 2024-06-13T07:00:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:00:53.444+0000 7f9e57200640 1 -- 172.21.15.26:0/3684780766 wait complete. 2024-06-13T07:00:54.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:53 smithi136 ceph-mon[37229]: pgmap v521: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T07:00:54.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:53 smithi026 ceph-mon[32041]: pgmap v521: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T07:00:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:53 smithi135 ceph-mon[36276]: pgmap v521: 129 pgs: 129 active+clean; 654 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T07:00:55.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:54 smithi136 ceph-mon[37229]: scrub summary: idle 2024-06-13T07:00:55.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:54 smithi026 ceph-mon[32041]: scrub summary: idle 2024-06-13T07:00:55.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:54 smithi135 ceph-mon[36276]: scrub summary: idle 2024-06-13T07:00:56.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:55 smithi136 ceph-mon[37229]: pgmap v522: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-13T07:00:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:55 smithi026 ceph-mon[32041]: pgmap v522: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-13T07:00:56.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:55 smithi135 ceph-mon[36276]: pgmap v522: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-13T07:00:58.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:57 smithi136 ceph-mon[37229]: pgmap v523: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-06-13T07:00:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:57 smithi026 ceph-mon[32041]: pgmap v523: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-06-13T07:00:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:57 smithi135 ceph-mon[36276]: pgmap v523: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-06-13T07:01:00.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:00:59 smithi136 ceph-mon[37229]: pgmap v524: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-13T07:01:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:00:59 smithi026 ceph-mon[32041]: pgmap v524: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-13T07:01:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:00:59 smithi135 ceph-mon[36276]: pgmap v524: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-13T07:01:02.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:01 smithi135 ceph-mon[36276]: pgmap v525: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-13T07:01:02.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:01 smithi136 ceph-mon[37229]: pgmap v525: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-13T07:01:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:01 smithi026 ceph-mon[32041]: pgmap v525: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-13T07:01:04.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:03 smithi136 ceph-mon[37229]: pgmap v526: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-13T07:01:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:03 smithi026 ceph-mon[32041]: pgmap v526: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-13T07:01:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:03 smithi135 ceph-mon[36276]: pgmap v526: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-13T07:01:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:05 smithi136 ceph-mon[37229]: pgmap v527: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-13T07:01:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:01:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:05 smithi026 ceph-mon[32041]: pgmap v527: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-13T07:01:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:01:06.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:05 smithi135 ceph-mon[36276]: pgmap v527: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-13T07:01:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:01:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 -- 172.21.15.26:0/2188372050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 msgr2=0x7fe528107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 --2- 172.21.15.26:0/2188372050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 0x7fe528107020 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe514009900 tx=0x7fe51402a410 comp rx=0 tx=0).stop 2024-06-13T07:01:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 -- 172.21.15.26:0/2188372050 shutdown_connections 2024-06-13T07:01:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 --2- 172.21.15.26:0/2188372050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe528107560 0x7fe528109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 --2- 172.21.15.26:0/2188372050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 0x7fe528107020 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 --2- 172.21.15.26:0/2188372050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe528102300 0x7fe5281046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 -- 172.21.15.26:0/2188372050 >> 172.21.15.26:0/2188372050 conn(0x7fe5280fbc90 msgr2=0x7fe5280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 -- 172.21.15.26:0/2188372050 shutdown_connections 2024-06-13T07:01:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.598+0000 7fe52e600640 1 -- 172.21.15.26:0/2188372050 wait complete. 2024-06-13T07:01:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.599+0000 7fe52e600640 1 Processor -- start 2024-06-13T07:01:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.599+0000 7fe52e600640 1 -- start start 2024-06-13T07:01:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.599+0000 7fe52e600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe528102300 0x7fe528197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.599+0000 7fe52e600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 0x7fe5281980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.599+0000 7fe52e600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe528107560 0x7fe52819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe52e600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810d4e0 con 0x7fe528104c30 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe52e600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810d360 con 0x7fe528107560 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe52e600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810d660 con 0x7fe528102300 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe527e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 0x7fe5281980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe527e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 0x7fe5281980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56218/0 (socket says 172.21.15.26:56218) 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe52d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe528102300 0x7fe528197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe527e00640 1 -- 172.21.15.26:0/1312535185 learned_addr learned my addr 172.21.15.26:0/1312535185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe52dc00640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe528107560 0x7fe52819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe527e00640 1 -- 172.21.15.26:0/1312535185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe528102300 msgr2=0x7fe528197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe527e00640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe528102300 0x7fe528197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe527e00640 1 -- 172.21.15.26:0/1312535185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe528107560 msgr2=0x7fe52819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe527e00640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe528107560 0x7fe52819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe527e00640 1 -- 172.21.15.26:0/1312535185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52819f7d0 con 0x7fe528104c30 2024-06-13T07:01:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe52d200640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe528102300 0x7fe528197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.600+0000 7fe52dc00640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe528107560 0x7fe52819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.601+0000 7fe527e00640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 0x7fe5281980f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe514031440 tx=0x7fe514032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.601+0000 7fe525600640 1 -- 172.21.15.26:0/1312535185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51403f030 con 0x7fe528104c30 2024-06-13T07:01:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.601+0000 7fe525600640 1 -- 172.21.15.26:0/1312535185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe514003ce0 con 0x7fe528104c30 2024-06-13T07:01:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.601+0000 7fe52e600640 1 -- 172.21.15.26:0/1312535185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52819fa60 con 0x7fe528104c30 2024-06-13T07:01:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.601+0000 7fe525600640 1 -- 172.21.15.26:0/1312535185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514035070 con 0x7fe528104c30 2024-06-13T07:01:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.601+0000 7fe52e600640 1 -- 172.21.15.26:0/1312535185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52819ff50 con 0x7fe528104c30 2024-06-13T07:01:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.604+0000 7fe532a0e740 1 Processor -- start 2024-06-13T07:01:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe525600640 1 -- 172.21.15.26:0/1312535185 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe514042030 con 0x7fe528104c30 2024-06-13T07:01:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe532a0e740 1 -- start start 2024-06-13T07:01:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1cdc0 0x55f646b1d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1d7b0 0x55f646b222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 0x55f646b24db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe532a0e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f646a80420 con 0x55f646b22920 2024-06-13T07:01:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe532a0e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f646a27c50 con 0x55f646b1d7b0 2024-06-13T07:01:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe532a0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f646a7c3e0 con 0x55f646b1cdc0 2024-06-13T07:01:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe52d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1d7b0 0x55f646b222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe52dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1cdc0 0x55f646b1d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe527e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 0x55f646b24db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe52dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1cdc0 0x55f646b1d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44230/0 (socket says 172.21.15.26:44230) 2024-06-13T07:01:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe52d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1d7b0 0x55f646b222b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43210/0 (socket says 172.21.15.26:43210) 2024-06-13T07:01:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.605+0000 7fe52dc00640 1 -- 172.21.15.26:0/1686852458 learned_addr learned my addr 172.21.15.26:0/1686852458 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.606+0000 7fe52dc00640 1 -- 172.21.15.26:0/1686852458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1d7b0 msgr2=0x55f646b222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.606+0000 7fe52dc00640 1 --2- 172.21.15.26:0/1686852458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1d7b0 0x55f646b222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.606+0000 7fe52dc00640 1 -- 172.21.15.26:0/1686852458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 msgr2=0x55f646b24db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.606+0000 7fe52dc00640 1 --2- 172.21.15.26:0/1686852458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 0x55f646b24db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.606+0000 7fe52dc00640 1 -- 172.21.15.26:0/1686852458 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f646794210 con 0x55f646b1cdc0 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.606+0000 7fe52d200640 1 --2- 172.21.15.26:0/1686852458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1d7b0 0x55f646b222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.606+0000 7fe52dc00640 1 --2- 172.21.15.26:0/1686852458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1cdc0 0x55f646b1d1e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe520011810 tx=0x7fe520011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e33d1926738bb3db server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.607+0000 7fe525600640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe5100a2b80 0x7fe5100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.607+0000 7fe51a000640 1 -- 172.21.15.26:0/1686852458 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520011310 con 0x55f646b1cdc0 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.607+0000 7fe51a000640 1 -- 172.21.15.26:0/1686852458 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52000f9a0 con 0x55f646b1cdc0 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.607+0000 7fe51a000640 1 -- 172.21.15.26:0/1686852458 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520005720 con 0x55f646b1cdc0 2024-06-13T07:01:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.607+0000 7fe525600640 1 -- 172.21.15.26:0/1312535185 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe5140ef8b0 con 0x7fe528104c30 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.607+0000 7fe52d200640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe5100a2b80 0x7fe5100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.607+0000 7fe532a0e740 1 -- 172.21.15.26:0/1686852458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1cdc0 msgr2=0x55f646b1d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.607+0000 7fe532a0e740 1 --2- 172.21.15.26:0/1686852458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1cdc0 0x55f646b1d1e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe520011810 tx=0x7fe520011cd0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 -- 172.21.15.26:0/1686852458 shutdown_connections 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 --2- 172.21.15.26:0/1686852458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1d7b0 0x55f646b222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 --2- 172.21.15.26:0/1686852458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 0x55f646b24db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 --2- 172.21.15.26:0/1686852458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1cdc0 0x55f646b1d1e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 -- 172.21.15.26:0/1686852458 >> 172.21.15.26:0/1686852458 conn(0x55f646ae4fb0 msgr2=0x55f646b18840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 -- 172.21.15.26:0/1686852458 shutdown_connections 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe52d200640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe5100a2b80 0x7fe5100a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009fd0 tx=0x7fe51c015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 -- 172.21.15.26:0/1686852458 wait complete. 2024-06-13T07:01:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 -- start start 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.608+0000 7fe532a0e740 1 Processor -- start 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b27150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b27690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b2e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe532a0e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f646b258d0 con 0x55f646b1d7b0 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe532a0e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f646a3b0b0 con 0x55f646b1cdc0 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe532a0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f646b25a50 con 0x55f646b22920 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe532a0e740 10 client.0 _ll_register_callbacks cb 0x55f646a26f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe52d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b27150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe52dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b2e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe52d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b27150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43216/0 (socket says 172.21.15.26:43216) 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe52d200640 1 -- 172.21.15.26:0/2102555395 learned_addr learned my addr 172.21.15.26:0/2102555395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.609+0000 7fe527e00640 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b27690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe52d200640 1 -- 172.21.15.26:0/2102555395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 msgr2=0x55f646b2e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe52d200640 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b2e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe52d200640 1 -- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 msgr2=0x55f646b27690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe52d200640 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b27690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe52d200640 1 -- 172.21.15.26:0/2102555395 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f646860620 con 0x55f646b1cdc0 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe52dc00640 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b2e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe527e00640 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b27690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe52d200640 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b27150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0042f0 tx=0x7fe51c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15f6395f5888cc49 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe50ea00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe50ea00640 1 -- 172.21.15.26:0/2102555395 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c02e060 con 0x55f646b1cdc0 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.610+0000 7fe532a0e740 1 -- 172.21.15.26:0/2102555395 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe52000f9a0 con 0x55f646b1cdc0 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.611+0000 7fe50ea00640 1 -- 172.21.15.26:0/2102555395 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe51c029420 con 0x55f646b1cdc0 2024-06-13T07:01:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.611+0000 7fe50ea00640 1 -- 172.21.15.26:0/2102555395 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe51c035050 con 0x55f646b1cdc0 2024-06-13T07:01:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.611+0000 7fe532a0e740 10 client.25244 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.611+0000 7fe532a0e740 1 -- 172.21.15.26:0/2102555395 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f646b31b00 con 0x55f646b1cdc0 2024-06-13T07:01:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.611+0000 7fe50ea00640 1 -- 172.21.15.26:0/2102555395 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52000f9a0 con 0x55f646b1cdc0 2024-06-13T07:01:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.611+0000 7fe50ea00640 1 -- 172.21.15.26:0/2102555395 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe51c0283c0 con 0x55f646b1cdc0 2024-06-13T07:01:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.612+0000 7fe532a0e740 10 client.25244 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.612+0000 7fe532a0e740 10 client.25244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.612+0000 7fe532a0e740 20 client.25244 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.612+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f646b33180 0x55f646b35590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.612+0000 7fe532a0e740 4 client.25244 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.612+0000 7fe532a0e740 1 -- 172.21.15.26:0/2102555395 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f646a8db50 con 0x55f646b33180 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.612+0000 7fe527e00640 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f646b33180 0x55f646b35590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.613+0000 7fe527e00640 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f646b33180 0x55f646b35590 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b705d31ce2d9f61 server_cookie=b929ae615a4b17ce in_seq=0 out_seq=0 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.613+0000 7fe50ea00640 10 client.25244 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.613+0000 7fe50ea00640 1 -- 172.21.15.26:0/2102555395 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f646a8db50 con 0x55f646b33180 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.613+0000 7fe50ea00640 10 client.25244 handle_command_reply: tid=0 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.613+0000 7fe532a0e740 1 client.25244 shutdown 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 -- 172.21.15.26:0/2102555395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 msgr2=0x55f646b27150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b27150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0042f0 tx=0x7fe51c004730 comp rx=0 tx=0).stop 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 -- 172.21.15.26:0/2102555395 shutdown_connections 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f646b33180 0x55f646b35590 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b2e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b27690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2102555395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b27150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 -- 172.21.15.26:0/2102555395 >> 172.21.15.26:0/2102555395 conn(0x55f646ae4fb0 msgr2=0x55f646b179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe50ea00640 0 client.25244 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 -- 172.21.15.26:0/2102555395 shutdown_connections 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 1 -- 172.21.15.26:0/2102555395 wait complete. 2024-06-13T07:01:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.614+0000 7fe532a0e740 20 client.25244 trim_cache size 0 max 16384 2024-06-13T07:01:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe532a0e740 1 Processor -- start 2024-06-13T07:01:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe532a0e740 1 -- start start 2024-06-13T07:01:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b2e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b26d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe532a0e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281aad10 con 0x55f646b1d7b0 2024-06-13T07:01:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe532a0e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281a85e0 con 0x55f646b1cdc0 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe532a0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281a6430 con 0x55f646b22920 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe52dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b26d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe527e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b2e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe52dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b26d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44250/0 (socket says 172.21.15.26:44250) 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe52dc00640 1 -- 172.21.15.26:0/2396220525 learned_addr learned my addr 172.21.15.26:0/2396220525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.638+0000 7fe52d200640 1 --2- 172.21.15.26:0/2396220525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe52dc00640 1 -- 172.21.15.26:0/2396220525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 msgr2=0x55f646b189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe52dc00640 1 --2- 172.21.15.26:0/2396220525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe52dc00640 1 -- 172.21.15.26:0/2396220525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 msgr2=0x55f646b2e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe52dc00640 1 --2- 172.21.15.26:0/2396220525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b2e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe52dc00640 1 -- 172.21.15.26:0/2396220525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f646b1d1a0 con 0x55f646b22920 2024-06-13T07:01:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe52dc00640 1 --2- 172.21.15.26:0/2396220525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b26d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe520011810 tx=0x7fe520005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37bd71528dc0b5d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe519600640 1 -- 172.21.15.26:0/2396220525 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520011110 con 0x55f646b22920 2024-06-13T07:01:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe519600640 1 -- 172.21.15.26:0/2396220525 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52001f040 con 0x55f646b22920 2024-06-13T07:01:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.639+0000 7fe519600640 1 -- 172.21.15.26:0/2396220525 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520005710 con 0x55f646b22920 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 -- 172.21.15.26:0/2396220525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 msgr2=0x55f646b26d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2396220525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b26d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe520011810 tx=0x7fe520005bd0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 -- 172.21.15.26:0/2396220525 shutdown_connections 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2396220525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b22920 0x55f646b26d60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2396220525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b1d7b0 0x55f646b2e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 --2- 172.21.15.26:0/2396220525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 -- 172.21.15.26:0/2396220525 >> 172.21.15.26:0/2396220525 conn(0x55f646ae4fb0 msgr2=0x55f646b182c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 -- 172.21.15.26:0/2396220525 shutdown_connections 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.640+0000 7fe532a0e740 1 -- 172.21.15.26:0/2396220525 wait complete. 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.641+0000 7fe532a0e740 1 -- start start 2024-06-13T07:01:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.641+0000 7fe532a0e740 1 Processor -- start 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.641+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.641+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1d7b0 0x55f646b2b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.641+0000 7fe532a0e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 0x55f646b9be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.641+0000 7fe532a0e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281a6aa0 con 0x55f646b22920 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe532a0e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281a7f10 con 0x55f646b1cdc0 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe532a0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281a7170 con 0x55f646b1d7b0 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe532a0e740 10 client.0 _ll_register_callbacks cb 0x7fe51c022040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe527e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1d7b0 0x55f646b2b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe527e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1d7b0 0x55f646b2b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44260/0 (socket says 172.21.15.26:44260) 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe527e00640 1 -- 172.21.15.26:0/664312778 learned_addr learned my addr 172.21.15.26:0/664312778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe52d200640 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe52dc00640 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 0x55f646b9be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe52d200640 1 -- 172.21.15.26:0/664312778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1d7b0 msgr2=0x55f646b2b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe52d200640 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1d7b0 0x55f646b2b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe52d200640 1 -- 172.21.15.26:0/664312778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 msgr2=0x55f646b9be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe52d200640 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 0x55f646b9be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe52d200640 1 -- 172.21.15.26:0/664312778 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f646b2bef0 con 0x55f646b1cdc0 2024-06-13T07:01:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.642+0000 7fe527e00640 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1d7b0 0x55f646b2b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.643+0000 7fe52d200640 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b2b370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0287b0 tx=0x7fe51c0285f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7904cc35efe46dfd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.643+0000 7fe50ea00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.643+0000 7fe50ea00640 1 -- 172.21.15.26:0/664312778 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c025040 con 0x55f646b1cdc0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.643+0000 7fe532a0e740 1 -- 172.21.15.26:0/664312778 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe52001f040 con 0x55f646b1cdc0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.643+0000 7fe50ea00640 1 -- 172.21.15.26:0/664312778 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe51c004530 con 0x55f646b1cdc0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.643+0000 7fe50ea00640 1 -- 172.21.15.26:0/664312778 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe51c04a050 con 0x55f646b1cdc0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.643+0000 7fe532a0e740 10 client.25250 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.643+0000 7fe532a0e740 1 -- 172.21.15.26:0/664312778 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f646b9d480 con 0x55f646b1cdc0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe50ea00640 1 -- 172.21.15.26:0/664312778 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52001f040 con 0x55f646b1cdc0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe50ea00640 1 -- 172.21.15.26:0/664312778 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe51c0298f0 con 0x55f646b1cdc0 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe532a0e740 10 client.25250 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe532a0e740 10 client.25250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe532a0e740 20 client.25250 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe532a0e740 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f646b9de30 0x55f646ba0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe532a0e740 4 client.25250 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe532a0e740 1 -- 172.21.15.26:0/664312778 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f646a87ea0 con 0x55f646b9de30 2024-06-13T07:01:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.644+0000 7fe527e00640 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f646b9de30 0x55f646ba0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.645+0000 7fe527e00640 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f646b9de30 0x55f646ba0250 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=858c894804e1db58 server_cookie=3e0d91bb617fd5a6 in_seq=0 out_seq=0 2024-06-13T07:01:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.645+0000 7fe50ea00640 10 client.25250 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.652+0000 7fe50ea00640 1 -- 172.21.15.26:0/664312778 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f646a87ea0 con 0x55f646b9de30 2024-06-13T07:01:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.652+0000 7fe50ea00640 10 client.25250 handle_command_reply: tid=0 2024-06-13T07:01:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.652+0000 7fe532a0e740 1 client.25250 shutdown 2024-06-13T07:01:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.652+0000 7fe532a0e740 1 -- 172.21.15.26:0/664312778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 msgr2=0x55f646b2b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.652+0000 7fe532a0e740 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b2b370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0287b0 tx=0x7fe51c0285f0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 1 -- 172.21.15.26:0/664312778 shutdown_connections 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f646b9de30 0x55f646ba0250 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f646b22920 0x55f646b9be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f646b1d7b0 0x55f646b2b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe50ea00640 0 client.25250 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 1 --2- 172.21.15.26:0/664312778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f646b1cdc0 0x55f646b2b370 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 1 -- 172.21.15.26:0/664312778 >> 172.21.15.26:0/664312778 conn(0x55f646ae4fb0 msgr2=0x55f646b26d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 1 -- 172.21.15.26:0/664312778 shutdown_connections 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 1 -- 172.21.15.26:0/664312778 wait complete. 2024-06-13T07:01:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.653+0000 7fe532a0e740 20 client.25250 trim_cache size 0 max 16384 2024-06-13T07:01:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:01:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:01:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 -- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe5100a2b80 msgr2=0x7fe5100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe5100a2b80 0x7fe5100a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009fd0 tx=0x7fe51c015040 comp rx=0 tx=0).stop 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 -- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 msgr2=0x7fe5281980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 0x7fe5281980f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe514031440 tx=0x7fe514032ce0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 -- 172.21.15.26:0/1312535185 shutdown_connections 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe5100a2b80 0x7fe5100a5040 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe528107560 0x7fe52819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528104c30 0x7fe5281980f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 --2- 172.21.15.26:0/1312535185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe528102300 0x7fe528197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 -- 172.21.15.26:0/1312535185 >> 172.21.15.26:0/1312535185 conn(0x7fe5280fbc90 msgr2=0x7fe528102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 -- 172.21.15.26:0/1312535185 shutdown_connections 2024-06-13T07:01:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:06.654+0000 7fe52e600640 1 -- 172.21.15.26:0/1312535185 wait complete. 2024-06-13T07:01:07.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:06 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:01:07.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:06 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:01:07.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:06 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:01:07.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:06 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:01:07.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:06 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:01:07.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:06 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:01:07.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:06 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:01:07.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:06 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:01:07.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:06 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:01:08.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:07 smithi136 ceph-mon[37229]: pgmap v528: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-13T07:01:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:07 smithi026 ceph-mon[32041]: pgmap v528: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-13T07:01:08.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:07 smithi135 ceph-mon[36276]: pgmap v528: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-13T07:01:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:10 smithi026 ceph-mon[32041]: pgmap v529: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-13T07:01:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:10 smithi135 ceph-mon[36276]: pgmap v529: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-13T07:01:10.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:10 smithi136 ceph-mon[37229]: pgmap v529: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-13T07:01:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:12 smithi026 ceph-mon[32041]: pgmap v530: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-13T07:01:12.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:12 smithi135 ceph-mon[36276]: pgmap v530: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-13T07:01:12.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:12 smithi136 ceph-mon[37229]: pgmap v530: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-13T07:01:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:14 smithi026 ceph-mon[32041]: pgmap v531: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:14.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:14 smithi135 ceph-mon[36276]: pgmap v531: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:14 smithi136 ceph-mon[37229]: pgmap v531: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:15.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:15 smithi026 ceph-mon[32041]: pgmap v532: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:15.528 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:15 smithi136 ceph-mon[37229]: pgmap v532: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:15.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:15 smithi135 ceph-mon[36276]: pgmap v532: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:17.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:17 smithi026 ceph-mon[32041]: pgmap v533: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-13T07:01:17.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:17 smithi135 ceph-mon[36276]: pgmap v533: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-13T07:01:17.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:17 smithi136 ceph-mon[37229]: pgmap v533: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-13T07:01:18.524 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:01:19.022 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:19.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:19 smithi135 ceph-mon[36276]: pgmap v534: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.842+0000 7f5c26600640 1 -- 172.21.15.26:0/3844847452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 msgr2=0x7f5c20100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.842+0000 7f5c26600640 1 --2- 172.21.15.26:0/3844847452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 0x7f5c20100eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009900 tx=0x7f5c0c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.843+0000 7f5c26600640 1 -- 172.21.15.26:0/3844847452 shutdown_connections 2024-06-13T07:01:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.843+0000 7f5c26600640 1 --2- 172.21.15.26:0/3844847452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c201013f0 0x7f5c20101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.843+0000 7f5c26600640 1 --2- 172.21.15.26:0/3844847452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 0x7f5c20100eb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.843+0000 7f5c26600640 1 --2- 172.21.15.26:0/3844847452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c201000f0 0x7f5c20100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.843+0000 7f5c26600640 1 -- 172.21.15.26:0/3844847452 >> 172.21.15.26:0/3844847452 conn(0x7f5c200fbc90 msgr2=0x7f5c200fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.844+0000 7f5c26600640 1 -- 172.21.15.26:0/3844847452 shutdown_connections 2024-06-13T07:01:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.844+0000 7f5c26600640 1 -- 172.21.15.26:0/3844847452 wait complete. 2024-06-13T07:01:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.845+0000 7f5c26600640 1 Processor -- start 2024-06-13T07:01:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.845+0000 7f5c26600640 1 -- start start 2024-06-13T07:01:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.845+0000 7f5c26600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c201000f0 0x7f5c20110c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c26600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 0x7f5c201161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c26600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c201013f0 0x7f5c20111140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c26600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201037b0 con 0x7f5c20100a50 2024-06-13T07:01:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c26600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20103630 con 0x7f5c201013f0 2024-06-13T07:01:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c26600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20103930 con 0x7f5c201000f0 2024-06-13T07:01:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 0x7f5c201161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 0x7f5c201161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50758/0 (socket says 172.21.15.26:50758) 2024-06-13T07:01:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c25c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c201013f0 0x7f5c20111140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c1fe00640 1 -- 172.21.15.26:0/468257690 learned_addr learned my addr 172.21.15.26:0/468257690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c25200640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c201000f0 0x7f5c20110c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c25c00640 1 -- 172.21.15.26:0/468257690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c201000f0 msgr2=0x7f5c20110c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c25c00640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c201000f0 0x7f5c20110c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c25c00640 1 -- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 msgr2=0x7f5c201161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c25c00640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 0x7f5c201161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.846+0000 7f5c25c00640 1 -- 172.21.15.26:0/468257690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c201117e0 con 0x7f5c201013f0 2024-06-13T07:01:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.847+0000 7f5c25c00640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c201013f0 0x7f5c20111140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009fd0 tx=0x7f5c1800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.847+0000 7f5c1d600640 1 -- 172.21.15.26:0/468257690 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800ce20 con 0x7f5c201013f0 2024-06-13T07:01:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.847+0000 7f5c1d600640 1 -- 172.21.15.26:0/468257690 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c18005020 con 0x7f5c201013f0 2024-06-13T07:01:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.847+0000 7f5c1d600640 1 -- 172.21.15.26:0/468257690 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18010720 con 0x7f5c201013f0 2024-06-13T07:01:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.847+0000 7f5c26600640 1 -- 172.21.15.26:0/468257690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c201ae890 con 0x7f5c201013f0 2024-06-13T07:01:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.848+0000 7f5c26600640 1 -- 172.21.15.26:0/468257690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c201aed60 con 0x7f5c201013f0 2024-06-13T07:01:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.852+0000 7f5c2aa80740 1 Processor -- start 2024-06-13T07:01:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.853+0000 7f5c1d600640 1 -- 172.21.15.26:0/468257690 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c18005540 con 0x7f5c201013f0 2024-06-13T07:01:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.853+0000 7f5c2aa80740 1 -- start start 2024-06-13T07:01:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.853+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.853+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56437d013420 con 0x56437d0b5920 2024-06-13T07:01:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56437cfbac50 con 0x56437d0b07b0 2024-06-13T07:01:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56437d00f3e0 con 0x56437d0afdc0 2024-06-13T07:01:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c25200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c25200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59678/0 (socket says 172.21.15.26:59678) 2024-06-13T07:01:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c25200640 1 -- 172.21.15.26:0/837828929 learned_addr learned my addr 172.21.15.26:0/837828929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c25200640 1 -- 172.21.15.26:0/837828929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 msgr2=0x56437d0b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c25200640 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c25200640 1 -- 172.21.15.26:0/837828929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 msgr2=0x56437d0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c25200640 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c25200640 1 -- 172.21.15.26:0/837828929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56437cd27210 con 0x56437d0b07b0 2024-06-13T07:01:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.854+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.855+0000 7f5c25200640 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0b52b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009cf0 tx=0x7f5c14011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa42d6876b35fc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.855+0000 7f5c12000640 1 -- 172.21.15.26:0/837828929 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1401e070 con 0x56437d0b07b0 2024-06-13T07:01:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.855+0000 7f5c12000640 1 -- 172.21.15.26:0/837828929 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c14019b10 con 0x56437d0b07b0 2024-06-13T07:01:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.855+0000 7f5c12000640 1 -- 172.21.15.26:0/837828929 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400ea60 con 0x56437d0b07b0 2024-06-13T07:01:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.855+0000 7f5c1d600640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5c080a2c00 0x7f5c080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c1d600640 1 -- 172.21.15.26:0/468257690 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5c180cfa20 con 0x7f5c201013f0 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5c080a2c00 0x7f5c080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c2aa80740 1 -- 172.21.15.26:0/837828929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 msgr2=0x56437d0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0b52b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009cf0 tx=0x7f5c14011810 comp rx=0 tx=0).stop 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c2aa80740 1 -- 172.21.15.26:0/837828929 shutdown_connections 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0b52b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/837828929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c2aa80740 1 -- 172.21.15.26:0/837828929 >> 172.21.15.26:0/837828929 conn(0x56437d077fb0 msgr2=0x56437d0ab840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.856+0000 7f5c2aa80740 1 -- 172.21.15.26:0/837828929 shutdown_connections 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.857+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5c080a2c00 0x7f5c080a50c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5c20112110 tx=0x7f5c0c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.857+0000 7f5c2aa80740 1 -- 172.21.15.26:0/837828929 wait complete. 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.858+0000 7f5c2aa80740 1 -- start start 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.858+0000 7f5c2aa80740 1 Processor -- start 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0ba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56437d0b88d0 con 0x56437d0b5920 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56437cfce0b0 con 0x56437d0b07b0 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56437d0b8a50 con 0x56437d0afdc0 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c2aa80740 10 client.0 _ll_register_callbacks cb 0x56437cfb9f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c25c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0ba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59692/0 (socket says 172.21.15.26:59692) 2024-06-13T07:01:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c1fe00640 1 -- 172.21.15.26:0/3435231019 learned_addr learned my addr 172.21.15.26:0/3435231019 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.859+0000 7f5c25200640 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.860+0000 7f5c25200640 1 -- 172.21.15.26:0/3435231019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 msgr2=0x56437d0ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.860+0000 7f5c25200640 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.860+0000 7f5c25200640 1 -- 172.21.15.26:0/3435231019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 msgr2=0x56437d0c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.860+0000 7f5c25200640 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0c16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.860+0000 7f5c25200640 1 -- 172.21.15.26:0/3435231019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56437cdf3620 con 0x56437d0afdc0 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.860+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.860+0000 7f5c25200640 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0ba160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x56437d0ac6f0 tx=0x7f5c14009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e4d14d773f30fff server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.860+0000 7f5c06a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.861+0000 7f5c06a00640 1 -- 172.21.15.26:0/3435231019 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14027070 con 0x56437d0afdc0 2024-06-13T07:01:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.861+0000 7f5c06a00640 1 -- 172.21.15.26:0/3435231019 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5c140198c0 con 0x56437d0afdc0 2024-06-13T07:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.861+0000 7f5c2aa80740 1 -- 172.21.15.26:0/3435231019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c14019b10 con 0x56437d0afdc0 2024-06-13T07:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.861+0000 7f5c06a00640 1 -- 172.21.15.26:0/3435231019 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c14033050 con 0x56437d0afdc0 2024-06-13T07:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.861+0000 7f5c2aa80740 10 client.25297 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.861+0000 7f5c2aa80740 1 -- 172.21.15.26:0/3435231019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56437d0c49c0 con 0x56437d0afdc0 2024-06-13T07:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.862+0000 7f5c06a00640 1 -- 172.21.15.26:0/3435231019 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c14019b10 con 0x56437d0afdc0 2024-06-13T07:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.862+0000 7f5c06a00640 1 -- 172.21.15.26:0/3435231019 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5c140046c0 con 0x56437d0afdc0 2024-06-13T07:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.862+0000 7f5c2aa80740 10 client.25297 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.862+0000 7f5c2aa80740 10 client.25297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.862+0000 7f5c2aa80740 20 client.25297 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.862+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56437d0c5dc0 0x56437d0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.862+0000 7f5c2aa80740 4 client.25297 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.862+0000 7f5c2aa80740 1 -- 172.21.15.26:0/3435231019 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56437d020b50 con 0x56437d0c5dc0 2024-06-13T07:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.863+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56437d0c5dc0 0x56437d0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.864+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56437d0c5dc0 0x56437d0c81b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f713f196a10dd5 server_cookie=840de743c95eb702 in_seq=0 out_seq=0 2024-06-13T07:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.864+0000 7f5c06a00640 10 client.25297 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.865+0000 7f5c06a00640 1 -- 172.21.15.26:0/3435231019 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56437d020b50 con 0x56437d0c5dc0 2024-06-13T07:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.865+0000 7f5c06a00640 10 client.25297 handle_command_reply: tid=0 2024-06-13T07:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.865+0000 7f5c2aa80740 1 client.25297 shutdown 2024-06-13T07:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 -- 172.21.15.26:0/3435231019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 msgr2=0x56437d0ba160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0ba160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x56437d0ac6f0 tx=0x7f5c14009910 comp rx=0 tx=0).stop 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 -- 172.21.15.26:0/3435231019 shutdown_connections 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56437d0c5dc0 0x56437d0c81b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b5920 0x56437d0c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/3435231019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0afdc0 0x56437d0ba160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 -- 172.21.15.26:0/3435231019 >> 172.21.15.26:0/3435231019 conn(0x56437d077fb0 msgr2=0x56437d0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c06a00640 0 client.25297 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 -- 172.21.15.26:0/3435231019 shutdown_connections 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 1 -- 172.21.15.26:0/3435231019 wait complete. 2024-06-13T07:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.866+0000 7f5c2aa80740 20 client.25297 trim_cache size 0 max 16384 2024-06-13T07:01:19.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:19 smithi136 ceph-mon[37229]: pgmap v534: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.898+0000 7f5c2aa80740 1 Processor -- start 2024-06-13T07:01:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.898+0000 7f5c2aa80740 1 -- start start 2024-06-13T07:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.899+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0afdc0 0x56437d0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.899+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.899+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.899+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201bbcb0 con 0x56437d0afdc0 2024-06-13T07:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.899+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b9580 con 0x56437d0b07b0 2024-06-13T07:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.899+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b73d0 con 0x56437d0b5920 2024-06-13T07:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.899+0000 7f5c25c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c25200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0afdc0 0x56437d0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c25c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45934/0 (socket says 172.21.15.26:45934) 2024-06-13T07:01:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c25c00640 1 -- 172.21.15.26:0/4039469867 learned_addr learned my addr 172.21.15.26:0/4039469867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c25c00640 1 -- 172.21.15.26:0/4039469867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 msgr2=0x56437d0c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c25c00640 1 --2- 172.21.15.26:0/4039469867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c25c00640 1 -- 172.21.15.26:0/4039469867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0afdc0 msgr2=0x56437d0aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c25c00640 1 --2- 172.21.15.26:0/4039469867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0afdc0 0x56437d0aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.900+0000 7f5c25c00640 1 -- 172.21.15.26:0/4039469867 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56437d0c8830 con 0x56437d0b5920 2024-06-13T07:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.901+0000 7f5c25c00640 1 --2- 172.21.15.26:0/4039469867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0b9780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5c18000f80 tx=0x7f5c180ccfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1698c99c3755b592 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.901+0000 7f5c11600640 1 -- 172.21.15.26:0/4039469867 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1809a130 con 0x56437d0b5920 2024-06-13T07:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.901+0000 7f5c11600640 1 -- 172.21.15.26:0/4039469867 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c180051d0 con 0x56437d0b5920 2024-06-13T07:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.901+0000 7f5c11600640 1 -- 172.21.15.26:0/4039469867 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1809b4e0 con 0x56437d0b5920 2024-06-13T07:01:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 -- 172.21.15.26:0/4039469867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 msgr2=0x56437d0b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/4039469867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0b9780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5c18000f80 tx=0x7f5c180ccfa0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 -- 172.21.15.26:0/4039469867 shutdown_connections 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/4039469867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0b07b0 0x56437d0c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/4039469867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0b9780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/4039469867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0afdc0 0x56437d0aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 -- 172.21.15.26:0/4039469867 >> 172.21.15.26:0/4039469867 conn(0x56437d077fb0 msgr2=0x56437d0ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 -- 172.21.15.26:0/4039469867 shutdown_connections 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.902+0000 7f5c2aa80740 1 -- 172.21.15.26:0/4039469867 wait complete. 2024-06-13T07:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.903+0000 7f5c2aa80740 1 -- start start 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.903+0000 7f5c2aa80740 1 Processor -- start 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0afdc0 0x56437d0be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b07b0 0x56437d0beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c2aa80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b7a40 con 0x56437d0b07b0 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b8eb0 con 0x56437d0afdc0 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c2aa80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b8110 con 0x56437d0b5920 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c2aa80740 10 client.0 _ll_register_callbacks cb 0x7f5c1401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b07b0 0x56437d0beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b07b0 0x56437d0beb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50808/0 (socket says 172.21.15.26:50808) 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c1fe00640 1 -- 172.21.15.26:0/2185243253 learned_addr learned my addr 172.21.15.26:0/2185243253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c25c00640 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.904+0000 7f5c25200640 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0afdc0 0x56437d0be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c1fe00640 1 -- 172.21.15.26:0/2185243253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 msgr2=0x56437d0bfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0bfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c1fe00640 1 -- 172.21.15.26:0/2185243253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0afdc0 msgr2=0x56437d0be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0afdc0 0x56437d0be620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c1fe00640 1 -- 172.21.15.26:0/2185243253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56437d0c0160 con 0x56437d0b07b0 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c25200640 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0afdc0 0x56437d0be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c1fe00640 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b07b0 0x56437d0beb60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c000c00 tx=0x7f5c0c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706bc21423781157 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c06a00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.905+0000 7f5c06a00640 1 -- 172.21.15.26:0/2185243253 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0035f0 con 0x56437d0b07b0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.906+0000 7f5c2aa80740 1 -- 172.21.15.26:0/2185243253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c180051d0 con 0x56437d0b07b0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.906+0000 7f5c06a00640 1 -- 172.21.15.26:0/2185243253 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5c0c003790 con 0x56437d0b07b0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.906+0000 7f5c06a00640 1 -- 172.21.15.26:0/2185243253 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c0c064050 con 0x56437d0b07b0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c2aa80740 10 client.15174 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c2aa80740 1 -- 172.21.15.26:0/2185243253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56437d0c1510 con 0x56437d0b07b0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c06a00640 1 -- 172.21.15.26:0/2185243253 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c180051d0 con 0x56437d0b07b0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c06a00640 1 -- 172.21.15.26:0/2185243253 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5c0c044050 con 0x56437d0b07b0 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c2aa80740 10 client.15174 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c2aa80740 10 client.15174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c2aa80740 20 client.15174 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56437d130e00 0x56437d133220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c2aa80740 4 client.15174 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.907+0000 7f5c2aa80740 1 -- 172.21.15.26:0/2185243253 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56437d01aea0 con 0x56437d130e00 2024-06-13T07:01:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.908+0000 7f5c25200640 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56437d130e00 0x56437d133220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.909+0000 7f5c06a00640 10 client.15174 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.909+0000 7f5c25200640 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56437d130e00 0x56437d133220 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d3db7ec490dcb5 server_cookie=5eb5aad556f009f6 in_seq=0 out_seq=0 2024-06-13T07:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c06a00640 1 -- 172.21.15.26:0/2185243253 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56437d01aea0 con 0x56437d130e00 2024-06-13T07:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c06a00640 10 client.15174 handle_command_reply: tid=0 2024-06-13T07:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 client.15174 shutdown 2024-06-13T07:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 -- 172.21.15.26:0/2185243253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b07b0 msgr2=0x56437d0beb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b07b0 0x56437d0beb60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c000c00 tx=0x7f5c0c005e50 comp rx=0 tx=0).stop 2024-06-13T07:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 -- 172.21.15.26:0/2185243253 shutdown_connections 2024-06-13T07:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56437d130e00 0x56437d133220 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56437d0b5920 0x56437d0bfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56437d0b07b0 0x56437d0beb60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 --2- 172.21.15.26:0/2185243253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56437d0afdc0 0x56437d0be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 -- 172.21.15.26:0/2185243253 >> 172.21.15.26:0/2185243253 conn(0x56437d077fb0 msgr2=0x56437d0aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c06a00640 0 client.15174 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 -- 172.21.15.26:0/2185243253 shutdown_connections 2024-06-13T07:01:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.919+0000 7f5c2aa80740 1 -- 172.21.15.26:0/2185243253 wait complete. 2024-06-13T07:01:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.920+0000 7f5c2aa80740 20 client.15174 trim_cache size 0 max 16384 2024-06-13T07:01:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:01:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:01:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:01:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.920+0000 7f5c26600640 1 -- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5c080a2c00 msgr2=0x7f5c080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.920+0000 7f5c26600640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5c080a2c00 0x7f5c080a50c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5c20112110 tx=0x7f5c0c0315b0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 -- 172.21.15.26:0/468257690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c201013f0 msgr2=0x7f5c20111140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c201013f0 0x7f5c20111140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009fd0 tx=0x7f5c1800ef10 comp rx=0 tx=0).stop 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 -- 172.21.15.26:0/468257690 shutdown_connections 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5c080a2c00 0x7f5c080a50c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c201013f0 0x7f5c20111140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c20100a50 0x7f5c201161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 --2- 172.21.15.26:0/468257690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c201000f0 0x7f5c20110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 -- 172.21.15.26:0/468257690 >> 172.21.15.26:0/468257690 conn(0x7f5c200fbc90 msgr2=0x7f5c200fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 -- 172.21.15.26:0/468257690 shutdown_connections 2024-06-13T07:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:19.921+0000 7f5c26600640 1 -- 172.21.15.26:0/468257690 wait complete. 2024-06-13T07:01:20.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:19 smithi026 ceph-mon[32041]: pgmap v534: 129 pgs: 129 active+clean; 662 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-13T07:01:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.492+0000 7f0e97e00700 1 -- 172.21.15.26:0/3121001660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 msgr2=0x7f0e98104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.492+0000 7f0e97e00700 1 --2- 172.21.15.26:0/3121001660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 0x7f0e98104a40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e840099c0 tx=0x7f0e84009cd0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.492+0000 7f0e97e00700 1 -- 172.21.15.26:0/3121001660 shutdown_connections 2024-06-13T07:01:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.492+0000 7f0e97e00700 1 --2- 172.21.15.26:0/3121001660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e9810a670 0x7f0e9810aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.492+0000 7f0e97e00700 1 --2- 172.21.15.26:0/3121001660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e98104f80 0x7f0e98111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.492+0000 7f0e97e00700 1 --2- 172.21.15.26:0/3121001660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 0x7f0e98104a40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.492+0000 7f0e97e00700 1 -- 172.21.15.26:0/3121001660 >> 172.21.15.26:0/3121001660 conn(0x7f0e98100070 msgr2=0x7f0e98102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.494+0000 7f0e97e00700 1 -- 172.21.15.26:0/3121001660 shutdown_connections 2024-06-13T07:01:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.494+0000 7f0e97e00700 1 -- 172.21.15.26:0/3121001660 wait complete. 2024-06-13T07:01:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.495+0000 7f0e97e00700 1 Processor -- start 2024-06-13T07:01:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.495+0000 7f0e97e00700 1 -- start start 2024-06-13T07:01:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.496+0000 7f0e97e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 0x7f0e981111c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.496+0000 7f0e97e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e98104f80 0x7f0e9810e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.496+0000 7f0e97e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9810a670 0x7f0e9810e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.496+0000 7f0e97e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98114970 con 0x7f0e981045c0 2024-06-13T07:01:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.496+0000 7f0e97e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e981147f0 con 0x7f0e98104f80 2024-06-13T07:01:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.496+0000 7f0e97e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98114af0 con 0x7f0e9810a670 2024-06-13T07:01:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.497+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 0x7f0e981111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.497+0000 7f0e96000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e98104f80 0x7f0e9810e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.497+0000 7f0e97400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9810a670 0x7f0e9810e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.497+0000 7f0e96000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e98104f80 0x7f0e9810e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59726/0 (socket says 172.21.15.26:59726) 2024-06-13T07:01:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.497+0000 7f0e96000700 1 -- 172.21.15.26:0/1259061703 learned_addr learned my addr 172.21.15.26:0/1259061703 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.497+0000 7f0e96000700 1 -- 172.21.15.26:0/1259061703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9810a670 msgr2=0x7f0e9810e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.497+0000 7f0e96000700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9810a670 0x7f0e9810e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.498+0000 7f0e96000700 1 -- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 msgr2=0x7f0e981111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.498+0000 7f0e96000700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 0x7f0e981111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.498+0000 7f0e96000700 1 -- 172.21.15.26:0/1259061703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9810f010 con 0x7f0e98104f80 2024-06-13T07:01:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.498+0000 7f0e96a00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 0x7f0e981111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.498+0000 7f0e97400700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9810a670 0x7f0e9810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.498+0000 7f0e96000700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e98104f80 0x7f0e9810e2c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e84005b40 tx=0x7f0e840038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.499+0000 7f0e8f400700 1 -- 172.21.15.26:0/1259061703 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84026070 con 0x7f0e98104f80 2024-06-13T07:01:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.499+0000 7f0e8f400700 1 -- 172.21.15.26:0/1259061703 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e84004050 con 0x7f0e98104f80 2024-06-13T07:01:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.499+0000 7f0e97e00700 1 -- 172.21.15.26:0/1259061703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e981ab650 con 0x7f0e98104f80 2024-06-13T07:01:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.499+0000 7f0e8f400700 1 -- 172.21.15.26:0/1259061703 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400fda0 con 0x7f0e98104f80 2024-06-13T07:01:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.499+0000 7f0e97e00700 1 -- 172.21.15.26:0/1259061703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e981abb00 con 0x7f0e98104f80 2024-06-13T07:01:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.504+0000 7f0e8f400700 1 -- 172.21.15.26:0/1259061703 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e84029070 con 0x7f0e98104f80 2024-06-13T07:01:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.505+0000 7f0e8f400700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0e7c0a2ca0 0x7f0e7c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.505+0000 7f0e8f400700 1 -- 172.21.15.26:0/1259061703 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0e840cffc0 con 0x7f0e98104f80 2024-06-13T07:01:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.506+0000 7f0ea80d3b80 1 Processor -- start 2024-06-13T07:01:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.506+0000 7f0e96a00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0e7c0a2ca0 0x7f0e7c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.506+0000 7f0ea80d3b80 1 -- start start 2024-06-13T07:01:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4bd60 0x562894b4c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4c680 0x562894b56ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b59920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0e96000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4c680 0x562894b56ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562894a5ba10 con 0x562894b57530 2024-06-13T07:01:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562894a9eb00 con 0x562894b4bd60 2024-06-13T07:01:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562894a598f0 con 0x562894b4c680 2024-06-13T07:01:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0e96000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4c680 0x562894b56ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45960/0 (socket says 172.21.15.26:45960) 2024-06-13T07:01:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.507+0000 7f0e96000700 1 -- 172.21.15.26:0/184638712 learned_addr learned my addr 172.21.15.26:0/184638712 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.508+0000 7f0e97400700 1 --2- 172.21.15.26:0/184638712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b59920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.508+0000 7f0e96000700 1 -- 172.21.15.26:0/184638712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4bd60 msgr2=0x562894b4c140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:01:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.508+0000 7f0e96000700 1 --2- 172.21.15.26:0/184638712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4bd60 0x562894b4c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.508+0000 7f0e96000700 1 -- 172.21.15.26:0/184638712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 msgr2=0x562894b59920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.508+0000 7f0e96000700 1 --2- 172.21.15.26:0/184638712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b59920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.508+0000 7f0e96000700 1 -- 172.21.15.26:0/184638712 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562894a46f90 con 0x562894b4c680 2024-06-13T07:01:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.509+0000 7f0e96000700 1 --2- 172.21.15.26:0/184638712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4c680 0x562894b56ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e8409c3f0 tx=0x7f0e8409bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=843b77d0a782a6f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.509+0000 7f0e83e00700 1 -- 172.21.15.26:0/184638712 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e840b5070 con 0x562894b4c680 2024-06-13T07:01:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.509+0000 7f0e83e00700 1 -- 172.21.15.26:0/184638712 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8409c460 con 0x562894b4c680 2024-06-13T07:01:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.509+0000 7f0e83e00700 1 -- 172.21.15.26:0/184638712 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e840a4e10 con 0x562894b4c680 2024-06-13T07:01:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.510+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/184638712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4c680 msgr2=0x562894b56ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.510+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/184638712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4c680 0x562894b56ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e8409c3f0 tx=0x7f0e8409bef0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.510+0000 7f0e96a00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0e7c0a2ca0 0x7f0e7c0a5160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005950 tx=0x7f0e8800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.511+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/184638712 shutdown_connections 2024-06-13T07:01:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.511+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/184638712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4bd60 0x562894b4c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.511+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/184638712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b59920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.511+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/184638712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4c680 0x562894b56ff0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.511+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/184638712 >> 172.21.15.26:0/184638712 conn(0x562894a79170 msgr2=0x562894b48280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.511+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/184638712 shutdown_connections 2024-06-13T07:01:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.512+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/184638712 wait complete. 2024-06-13T07:01:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.512+0000 7f0ea80d3b80 1 -- start start 2024-06-13T07:01:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.513+0000 7f0ea80d3b80 1 Processor -- start 2024-06-13T07:01:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.513+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b56cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.513+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4c680 0x562894b53fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.514+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b57530 0x562894b54510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.514+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562894a3af60 con 0x562894b4c680 2024-06-13T07:01:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.514+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562894a9aac0 con 0x562894b57530 2024-06-13T07:01:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.514+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562894a598f0 con 0x562894b4bd60 2024-06-13T07:01:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.514+0000 7f0ea80d3b80 10 client.0 _ll_register_callbacks cb 0x562894a34380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.514+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b56cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.514+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b56cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45964/0 (socket says 172.21.15.26:45964) 2024-06-13T07:01:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.514+0000 7f0e96a00700 1 -- 172.21.15.26:0/1539050003 learned_addr learned my addr 172.21.15.26:0/1539050003 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.515+0000 7f0e97400700 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b57530 0x562894b54510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.515+0000 7f0e97400700 1 -- 172.21.15.26:0/1539050003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 msgr2=0x562894b56cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.515+0000 7f0e97400700 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b56cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.515+0000 7f0e97400700 1 -- 172.21.15.26:0/1539050003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4c680 msgr2=0x562894b53fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.515+0000 7f0e97400700 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4c680 0x562894b53fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.515+0000 7f0e97400700 1 -- 172.21.15.26:0/1539050003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628949ed8d0 con 0x562894b57530 2024-06-13T07:01:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.516+0000 7f0e97400700 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b57530 0x562894b54510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e900078c0 tx=0x7f0e90007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f0a17562319efa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.516+0000 7f0e80c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.516+0000 7f0e80c00700 1 -- 172.21.15.26:0/1539050003 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90019020 con 0x562894b57530 2024-06-13T07:01:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.516+0000 7f0e80c00700 1 -- 172.21.15.26:0/1539050003 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0e90010860 con 0x562894b57530 2024-06-13T07:01:21.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.517+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1539050003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e8409c460 con 0x562894b57530 2024-06-13T07:01:21.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0e80c00700 1 -- 172.21.15.26:0/1539050003 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e90010a60 con 0x562894b57530 2024-06-13T07:01:21.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0ea80d3b80 10 client.25274 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:21.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1539050003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562894b754f0 con 0x562894b57530 2024-06-13T07:01:21.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0e80c00700 1 -- 172.21.15.26:0/1539050003 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8409c460 con 0x562894b57530 2024-06-13T07:01:21.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0e80c00700 1 -- 172.21.15.26:0/1539050003 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0e90026020 con 0x562894b57530 2024-06-13T07:01:21.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0ea80d3b80 10 client.25274 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:21.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0ea80d3b80 10 client.25274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0ea80d3b80 20 client.25274 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562894b77bd0 0x562894b79fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0ea80d3b80 4 client.25274 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.518+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1539050003 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562894aac000 con 0x562894b77bd0 2024-06-13T07:01:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.520+0000 7f0e96a00700 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562894b77bd0 0x562894b79fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.521+0000 7f0e96a00700 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562894b77bd0 0x562894b79fc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0293908a455d7e server_cookie=8e634267f3aec7a9 in_seq=0 out_seq=0 2024-06-13T07:01:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.521+0000 7f0e80c00700 10 client.25274 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:21.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.523+0000 7f0e80c00700 1 -- 172.21.15.26:0/1539050003 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562894aac000 con 0x562894b77bd0 2024-06-13T07:01:21.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.523+0000 7f0e80c00700 10 client.25274 handle_command_reply: tid=0 2024-06-13T07:01:21.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.523+0000 7f0ea80d3b80 1 client.25274 shutdown 2024-06-13T07:01:21.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1539050003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b57530 msgr2=0x562894b54510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b57530 0x562894b54510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e900078c0 tx=0x7f0e90007c80 comp rx=0 tx=0).stop 2024-06-13T07:01:21.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1539050003 shutdown_connections 2024-06-13T07:01:21.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562894b77bd0 0x562894b79fc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0e80c00700 0 client.25274 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b57530 0x562894b54510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b56cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1539050003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4c680 0x562894b53fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1539050003 >> 172.21.15.26:0/1539050003 conn(0x562894a79170 msgr2=0x562894b47e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.524+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1539050003 shutdown_connections 2024-06-13T07:01:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.525+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1539050003 wait complete. 2024-06-13T07:01:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.525+0000 7f0ea80d3b80 20 client.25274 trim_cache size 0 max 16384 2024-06-13T07:01:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.548+0000 7f0ea80d3b80 1 Processor -- start 2024-06-13T07:01:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.548+0000 7f0ea80d3b80 1 -- start start 2024-06-13T07:01:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.548+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4bd60 0x562894b52c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.548+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.548+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b57530 0x562894b7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.548+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78011500 con 0x562894b4bd60 2024-06-13T07:01:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.549+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7800ed90 con 0x562894b4c680 2024-06-13T07:01:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.549+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7800cbe0 con 0x562894b57530 2024-06-13T07:01:21.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.549+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4bd60 0x562894b52c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.549+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4bd60 0x562894b52c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50848/0 (socket says 172.21.15.26:50848) 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.549+0000 7f0e96a00700 1 -- 172.21.15.26:0/1793210341 learned_addr learned my addr 172.21.15.26:0/1793210341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.550+0000 7f0e96000700 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.550+0000 7f0e96a00700 1 -- 172.21.15.26:0/1793210341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b57530 msgr2=0x562894b7f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.550+0000 7f0e97400700 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b57530 0x562894b7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.550+0000 7f0e96a00700 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b57530 0x562894b7f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.550+0000 7f0e96a00700 1 -- 172.21.15.26:0/1793210341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 msgr2=0x562894b54560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.550+0000 7f0e96a00700 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b54560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.550+0000 7f0e96a00700 1 -- 172.21.15.26:0/1793210341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562894b7a640 con 0x562894b4bd60 2024-06-13T07:01:21.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.550+0000 7f0e96000700 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b54560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.551+0000 7f0e96a00700 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4bd60 0x562894b52c50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x562894b7a870 tx=0x7f0e88012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e23cfbee9b3dfa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.551+0000 7f0e83400700 1 -- 172.21.15.26:0/1793210341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88016070 con 0x562894b4bd60 2024-06-13T07:01:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.551+0000 7f0e83400700 1 -- 172.21.15.26:0/1793210341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e88029070 con 0x562894b4bd60 2024-06-13T07:01:21.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.551+0000 7f0e83400700 1 -- 172.21.15.26:0/1793210341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88025b00 con 0x562894b4bd60 2024-06-13T07:01:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.552+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1793210341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4bd60 msgr2=0x562894b52c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.552+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4bd60 0x562894b52c50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x562894b7a870 tx=0x7f0e88012000 comp rx=0 tx=0).stop 2024-06-13T07:01:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.553+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1793210341 shutdown_connections 2024-06-13T07:01:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.553+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b57530 0x562894b7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.553+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b4bd60 0x562894b52c50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.553+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/1793210341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b54560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.553+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1793210341 >> 172.21.15.26:0/1793210341 conn(0x562894a79170 msgr2=0x562894b626a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.553+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1793210341 shutdown_connections 2024-06-13T07:01:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.553+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/1793210341 wait complete. 2024-06-13T07:01:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.554+0000 7f0ea80d3b80 1 -- start start 2024-06-13T07:01:21.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.554+0000 7f0ea80d3b80 1 Processor -- start 2024-06-13T07:01:21.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.555+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b5cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.555+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b5d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.555+0000 7f0ea80d3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b5d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.555+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7800d250 con 0x562894b57530 2024-06-13T07:01:21.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.555+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7800e6c0 con 0x562894b4c680 2024-06-13T07:01:21.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.555+0000 7f0ea80d3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7800d920 con 0x562894b4bd60 2024-06-13T07:01:21.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.555+0000 7f0ea80d3b80 10 client.0 _ll_register_callbacks cb 0x562894a34380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:21.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.555+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b5cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.556+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b5cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45984/0 (socket says 172.21.15.26:45984) 2024-06-13T07:01:21.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.556+0000 7f0e96a00700 1 -- 172.21.15.26:0/2545913059 learned_addr learned my addr 172.21.15.26:0/2545913059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:21.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.556+0000 7f0e96000700 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b5d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.556+0000 7f0e97400700 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b5d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.556+0000 7f0e96a00700 1 -- 172.21.15.26:0/2545913059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 msgr2=0x562894b5d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:21.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.556+0000 7f0e96a00700 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b5d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.556+0000 7f0e96a00700 1 -- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 msgr2=0x562894b5d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:21.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.557+0000 7f0e96a00700 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b5d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.557+0000 7f0e96a00700 1 -- 172.21.15.26:0/2545913059 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562894be18e0 con 0x562894b4bd60 2024-06-13T07:01:21.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.557+0000 7f0e97400700 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b5d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:21.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.558+0000 7f0e96a00700 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b5cd60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e88025700 tx=0x7f0e8801f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dab08e9e9007c620 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:21.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.558+0000 7f0e80c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:21.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.558+0000 7f0e80c00700 1 -- 172.21.15.26:0/2545913059 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88034070 con 0x562894b4bd60 2024-06-13T07:01:21.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.558+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/2545913059 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e88029070 con 0x562894b4bd60 2024-06-13T07:01:21.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.559+0000 7f0e80c00700 1 -- 172.21.15.26:0/2545913059 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0e8801fdf0 con 0x562894b4bd60 2024-06-13T07:01:21.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.559+0000 7f0e80c00700 1 -- 172.21.15.26:0/2545913059 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e88040050 con 0x562894b4bd60 2024-06-13T07:01:21.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.559+0000 7f0ea80d3b80 10 client.25318 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:21.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.559+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/2545913059 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562894be2b40 con 0x562894b4bd60 2024-06-13T07:01:21.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.560+0000 7f0e80c00700 1 -- 172.21.15.26:0/2545913059 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e88029070 con 0x562894b4bd60 2024-06-13T07:01:21.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.560+0000 7f0e80c00700 1 -- 172.21.15.26:0/2545913059 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0e88004750 con 0x562894b4bd60 2024-06-13T07:01:21.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.560+0000 7f0ea80d3b80 10 client.25318 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:21.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.560+0000 7f0ea80d3b80 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:21.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.560+0000 7f0ea80d3b80 20 client.25318 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:21.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.560+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562894be3110 0x562894be5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:21.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.560+0000 7f0ea80d3b80 4 client.25318 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:01:21.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.560+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/2545913059 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562894aa6580 con 0x562894be3110 2024-06-13T07:01:21.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.561+0000 7f0e96000700 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562894be3110 0x562894be5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:21.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.562+0000 7f0e96000700 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562894be3110 0x562894be5530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6c3c9d8e518241 server_cookie=5932cb1dc6a072c4 in_seq=0 out_seq=0 2024-06-13T07:01:21.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.562+0000 7f0e80c00700 10 client.25318 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:21.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.562+0000 7f0e80c00700 1 -- 172.21.15.26:0/2545913059 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562894aa6580 con 0x562894be3110 2024-06-13T07:01:21.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.562+0000 7f0e80c00700 10 client.25318 handle_command_reply: tid=0 2024-06-13T07:01:21.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.562+0000 7f0ea80d3b80 1 client.25318 shutdown 2024-06-13T07:01:21.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/2545913059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 msgr2=0x562894b5cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b5cd60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e88025700 tx=0x7f0e8801f8d0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/2545913059 shutdown_connections 2024-06-13T07:01:21.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562894be3110 0x562894be5530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562894b57530 0x562894b5d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562894b4bd60 0x562894b5cd60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0ea80d3b80 1 --2- 172.21.15.26:0/2545913059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562894b4c680 0x562894b5d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/2545913059 >> 172.21.15.26:0/2545913059 conn(0x562894a79170 msgr2=0x562894b53550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:21.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.563+0000 7f0e80c00700 0 client.25318 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:21.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.564+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/2545913059 shutdown_connections 2024-06-13T07:01:21.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.564+0000 7f0ea80d3b80 1 -- 172.21.15.26:0/2545913059 wait complete. 2024-06-13T07:01:21.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.564+0000 7f0ea80d3b80 20 client.25318 trim_cache size 0 max 16384 2024-06-13T07:01:21.565 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:01:21.565 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-06-13T07:01:21.565 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-06-13T07:01:21.566 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:01:21.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 -- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0e7c0a2ca0 msgr2=0x7f0e7c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0e7c0a2ca0 0x7f0e7c0a5160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005950 tx=0x7f0e8800a300 comp rx=0 tx=0).stop 2024-06-13T07:01:21.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 -- 172.21.15.26:0/1259061703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e98104f80 msgr2=0x7f0e9810e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:21.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e98104f80 0x7f0e9810e2c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e84005b40 tx=0x7f0e840038b0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 -- 172.21.15.26:0/1259061703 shutdown_connections 2024-06-13T07:01:21.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0e7c0a2ca0 0x7f0e7c0a5160 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9810a670 0x7f0e9810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e981045c0 0x7f0e981111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 --2- 172.21.15.26:0/1259061703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e98104f80 0x7f0e9810e2c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.565+0000 7f0e97e00700 1 -- 172.21.15.26:0/1259061703 >> 172.21.15.26:0/1259061703 conn(0x7f0e98100070 msgr2=0x7f0e98101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.566+0000 7f0e97e00700 1 -- 172.21.15.26:0/1259061703 shutdown_connections 2024-06-13T07:01:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:21.566+0000 7f0e97e00700 1 -- 172.21.15.26:0/1259061703 wait complete. 2024-06-13T07:01:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:21 smithi026 ceph-mon[32041]: pgmap v535: 129 pgs: 129 active+clean; 671 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-13T07:01:21.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:21 smithi135 ceph-mon[36276]: pgmap v535: 129 pgs: 129 active+clean; 671 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-13T07:01:21.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:21 smithi136 ceph-mon[37229]: pgmap v535: 129 pgs: 129 active+clean; 671 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-13T07:01:22.504 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:01:22.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:01:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:23.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:23 smithi135 ceph-mon[36276]: pgmap v536: 129 pgs: 129 active+clean; 671 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-13T07:01:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:23 smithi136 ceph-mon[37229]: pgmap v536: 129 pgs: 129 active+clean; 671 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-13T07:01:24.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:23 smithi026 ceph-mon[32041]: pgmap v536: 129 pgs: 129 active+clean; 671 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-13T07:01:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.588+0000 7f8486a00700 1 -- 172.21.15.26:0/2962359605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480074fe0 msgr2=0x7f8480113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.588+0000 7f8486a00700 1 --2- 172.21.15.26:0/2962359605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480074fe0 0x7f8480113120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f846c009990 tx=0x7f846c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.588+0000 7f8486a00700 1 -- 172.21.15.26:0/2962359605 shutdown_connections 2024-06-13T07:01:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.588+0000 7f8486a00700 1 --2- 172.21.15.26:0/2962359605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8480113660 0x7f8480115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.588+0000 7f8486a00700 1 --2- 172.21.15.26:0/2962359605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84800746c0 0x7f8480074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.588+0000 7f8486a00700 1 --2- 172.21.15.26:0/2962359605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480074fe0 0x7f8480113120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.588+0000 7f8486a00700 1 -- 172.21.15.26:0/2962359605 >> 172.21.15.26:0/2962359605 conn(0x7f84800795f0 msgr2=0x7f8480079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.589+0000 7f8486a00700 1 -- 172.21.15.26:0/2962359605 shutdown_connections 2024-06-13T07:01:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.589+0000 7f8486a00700 1 -- 172.21.15.26:0/2962359605 wait complete. 2024-06-13T07:01:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.590+0000 7f8486a00700 1 Processor -- start 2024-06-13T07:01:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.591+0000 7f8486a00700 1 -- start start 2024-06-13T07:01:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.591+0000 7f8486a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84800746c0 0x7f848019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.591+0000 7f8486a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8480074fe0 0x7f848019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.591+0000 7f8486a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480113660 0x7f84801a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.591+0000 7f8486a00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8480118df0 con 0x7f8480113660 2024-06-13T07:01:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.591+0000 7f8486a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8480118c70 con 0x7f84800746c0 2024-06-13T07:01:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.591+0000 7f8486a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8480118f70 con 0x7f8480074fe0 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84800746c0 0x7f848019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f8484a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480113660 0x7f84801a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8480074fe0 0x7f848019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84800746c0 0x7f848019f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36726/0 (socket says 172.21.15.26:36726) 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 -- 172.21.15.26:0/461927722 learned_addr learned my addr 172.21.15.26:0/461927722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 -- 172.21.15.26:0/461927722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8480074fe0 msgr2=0x7f848019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8480074fe0 0x7f848019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 -- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480113660 msgr2=0x7f84801a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480113660 0x7f84801a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 -- 172.21.15.26:0/461927722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84801a42b0 con 0x7f84800746c0 2024-06-13T07:01:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f8484a00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480113660 0x7f84801a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.592+0000 7f847fe00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84800746c0 0x7f848019f2a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f847000e9a0 tx=0x7f847000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.593+0000 7f847cc00700 1 -- 172.21.15.26:0/461927722 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847000cbd0 con 0x7f84800746c0 2024-06-13T07:01:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.593+0000 7f847cc00700 1 -- 172.21.15.26:0/461927722 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847000cd70 con 0x7f84800746c0 2024-06-13T07:01:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.593+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84801a45a0 con 0x7f84800746c0 2024-06-13T07:01:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.593+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84801abe20 con 0x7f84800746c0 2024-06-13T07:01:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.593+0000 7f847cc00700 1 -- 172.21.15.26:0/461927722 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470018750 con 0x7f84800746c0 2024-06-13T07:01:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.594+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8460005330 con 0x7f84800746c0 2024-06-13T07:01:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.596+0000 7f847cc00700 1 -- 172.21.15.26:0/461927722 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8470010430 con 0x7f84800746c0 2024-06-13T07:01:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.597+0000 7f847cc00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f84640a2ca0 0x7f84640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.598+0000 7f847cc00700 1 -- 172.21.15.26:0/461927722 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f84700cf630 con 0x7f84800746c0 2024-06-13T07:01:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.598+0000 7f847f400700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f84640a2ca0 0x7f84640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:25.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.599+0000 7f847f400700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f84640a2ca0 0x7f84640a5160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f846c00b2e0 tx=0x7f846c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.602+0000 7f847cc00700 1 -- 172.21.15.26:0/461927722 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f847009c140 con 0x7f84800746c0 2024-06-13T07:01:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:25 smithi135 ceph-mon[36276]: pgmap v537: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:25 smithi136 ceph-mon[37229]: pgmap v537: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.965+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8460005620 con 0x7f84800746c0 2024-06-13T07:01:25.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:25 smithi026 ceph-mon[32041]: pgmap v537: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.966+0000 7f847cc00700 1 -- 172.21.15.26:0/461927722 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f847009ba40 con 0x7f84800746c0 2024-06-13T07:01:25.966 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:01:25.966 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T07:00:22.609623+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24506,24511,24517]},"id":1}]} 2024-06-13T07:01:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.972+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f84640a2ca0 msgr2=0x7f84640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.972+0000 7f8486a00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f84640a2ca0 0x7f84640a5160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f846c00b2e0 tx=0x7f846c01a040 comp rx=0 tx=0).stop 2024-06-13T07:01:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.972+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84800746c0 msgr2=0x7f848019f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.972+0000 7f8486a00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84800746c0 0x7f848019f2a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f847000e9a0 tx=0x7f847000ecb0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.973+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 shutdown_connections 2024-06-13T07:01:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.973+0000 7f8486a00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f84640a2ca0 0x7f84640a5160 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.973+0000 7f8486a00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84800746c0 0x7f848019f2a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.973+0000 7f8486a00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8480113660 0x7f84801a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.973+0000 7f8486a00700 1 --2- 172.21.15.26:0/461927722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8480074fe0 0x7f848019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.973+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 >> 172.21.15.26:0/461927722 conn(0x7f84800795f0 msgr2=0x7f8480112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.973+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 shutdown_connections 2024-06-13T07:01:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:25.973+0000 7f8486a00700 1 -- 172.21.15.26:0/461927722 wait complete. 2024-06-13T07:01:25.974 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2024-06-13T07:01:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:26 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/461927722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:26.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:26 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/461927722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:26.845 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:01:26.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:26 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/461927722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:27.729 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:27 smithi026 ceph-mon[32041]: pgmap v538: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-06-13T07:01:27.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:27 smithi135 ceph-mon[36276]: pgmap v538: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-06-13T07:01:27.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:27 smithi136 ceph-mon[37229]: pgmap v538: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-06-13T07:01:29.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:29 smithi026 ceph-mon[32041]: pgmap v539: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 -- 172.21.15.26:0/2882292823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac0746c0 msgr2=0x7facac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 --2- 172.21.15.26:0/2882292823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac0746c0 0x7facac074aa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009990 tx=0x7fac9c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 -- 172.21.15.26:0/2882292823 shutdown_connections 2024-06-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 --2- 172.21.15.26:0/2882292823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac0746c0 0x7facac074aa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 --2- 172.21.15.26:0/2882292823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facac113660 0x7facac115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 --2- 172.21.15.26:0/2882292823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac074fe0 0x7facac113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 -- 172.21.15.26:0/2882292823 >> 172.21.15.26:0/2882292823 conn(0x7facac0795f0 msgr2=0x7facac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 -- 172.21.15.26:0/2882292823 shutdown_connections 2024-06-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.739+0000 7facb1800700 1 -- 172.21.15.26:0/2882292823 wait complete. 2024-06-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.740+0000 7facb1800700 1 Processor -- start 2024-06-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.741+0000 7facb1800700 1 -- start start 2024-06-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.741+0000 7facb1800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0746c0 0x7facac19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.741+0000 7facb1800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac074fe0 0x7facac19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.741+0000 7facb1800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facac113660 0x7facac1a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.741+0000 7facb1800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facac118d90 con 0x7facac113660 2024-06-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.741+0000 7facb1800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7facac118c10 con 0x7facac074fe0 2024-06-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.741+0000 7facb1800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7facac118f10 con 0x7facac0746c0 2024-06-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaaa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0746c0 0x7facac19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaaa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0746c0 0x7facac19f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60482/0 (socket says 172.21.15.26:60482) 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facab400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facac113660 0x7facac1a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaaa00700 1 -- 172.21.15.26:0/1851545009 learned_addr learned my addr 172.21.15.26:0/1851545009 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaa000700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac074fe0 0x7facac19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaa000700 1 -- 172.21.15.26:0/1851545009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0746c0 msgr2=0x7facac19f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaa000700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0746c0 0x7facac19f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaa000700 1 -- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facac113660 msgr2=0x7facac1a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaa000700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facac113660 0x7facac1a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.742+0000 7facaa000700 1 -- 172.21.15.26:0/1851545009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac1a42f0 con 0x7facac074fe0 2024-06-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.743+0000 7facaa000700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac074fe0 0x7facac19f820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fac9800e9f0 tx=0x7fac9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.743+0000 7faca3400700 1 -- 172.21.15.26:0/1851545009 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9800c9c0 con 0x7facac074fe0 2024-06-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.743+0000 7facb1800700 1 -- 172.21.15.26:0/1851545009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac1a45e0 con 0x7facac074fe0 2024-06-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.743+0000 7faca3400700 1 -- 172.21.15.26:0/1851545009 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9800cb60 con 0x7facac074fe0 2024-06-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.743+0000 7faca3400700 1 -- 172.21.15.26:0/1851545009 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98018680 con 0x7facac074fe0 2024-06-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.743+0000 7facb1800700 1 -- 172.21.15.26:0/1851545009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac1abe60 con 0x7facac074fe0 2024-06-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.749+0000 7faca3400700 1 -- 172.21.15.26:0/1851545009 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac98010430 con 0x7facac074fe0 2024-06-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.751+0000 7facbc0d1b80 1 Processor -- start 2024-06-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.751+0000 7facbc0d1b80 1 -- start start 2024-06-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.751+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba1f3e40 0x561bba277070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.751+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 0x561bba277a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.751+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 0x561bba2846f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.752+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561bba186a10 con 0x561bba1f3e40 2024-06-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.752+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561bba1c9b00 con 0x561bba282300 2024-06-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.752+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561bba1848f0 con 0x561bba2775b0 2024-06-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.752+0000 7faca3400700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fac900a2c20 0x7fac900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.752+0000 7faca3400700 1 -- 172.21.15.26:0/1851545009 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fac980cf5a0 con 0x7facac074fe0 2024-06-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.752+0000 7facab400700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fac900a2c20 0x7fac900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.753+0000 7facab400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba1f3e40 0x561bba277070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.753+0000 7facab400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba1f3e40 0x561bba277070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52844/0 (socket says 172.21.15.26:52844) 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.753+0000 7facab400700 1 -- 172.21.15.26:0/3686150962 learned_addr learned my addr 172.21.15.26:0/3686150962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.753+0000 7facaaa00700 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 0x561bba277a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.753+0000 7facaa000700 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 0x561bba2846f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.753+0000 7facab400700 1 -- 172.21.15.26:0/3686150962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 msgr2=0x561bba277a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.753+0000 7facab400700 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 0x561bba277a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.754+0000 7facab400700 1 -- 172.21.15.26:0/3686150962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 msgr2=0x561bba2846f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.754+0000 7facab400700 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 0x561bba2846f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.754+0000 7facab400700 1 -- 172.21.15.26:0/3686150962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bba171f90 con 0x561bba1f3e40 2024-06-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.754+0000 7facab400700 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba1f3e40 0x561bba277070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faca4018640 tx=0x7faca401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f453361534c1000 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.754+0000 7facab400700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fac900a2c20 0x7fac900a50e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faca4008920 tx=0x7faca4009350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.754+0000 7fac97e00700 1 -- 172.21.15.26:0/3686150962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4008020 con 0x561bba1f3e40 2024-06-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.755+0000 7fac97e00700 1 -- 172.21.15.26:0/3686150962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca40081c0 con 0x561bba1f3e40 2024-06-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.755+0000 7fac97e00700 1 -- 172.21.15.26:0/3686150962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca401ab90 con 0x561bba1f3e40 2024-06-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.757+0000 7facbc0d1b80 1 -- 172.21.15.26:0/3686150962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba1f3e40 msgr2=0x561bba277070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.757+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba1f3e40 0x561bba277070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faca4018640 tx=0x7faca401b000 comp rx=0 tx=0).stop 2024-06-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.758+0000 7facbc0d1b80 1 -- 172.21.15.26:0/3686150962 shutdown_connections 2024-06-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.758+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 0x561bba2846f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.758+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba1f3e40 0x561bba277070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.758+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/3686150962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 0x561bba277a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.758+0000 7facbc0d1b80 1 -- 172.21.15.26:0/3686150962 >> 172.21.15.26:0/3686150962 conn(0x561bba1a4170 msgr2=0x561bba2734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.758+0000 7facbc0d1b80 1 -- 172.21.15.26:0/3686150962 shutdown_connections 2024-06-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.758+0000 7facbc0d1b80 1 -- 172.21.15.26:0/3686150962 wait complete. 2024-06-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.759+0000 7facbc0d1b80 1 -- start start 2024-06-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.759+0000 7facbc0d1b80 1 Processor -- start 2024-06-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba1f3e40 0x561bba281b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 0x561bba27ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 0x561bba27f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561bba165f60 con 0x561bba282300 2024-06-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561bba1c5ac0 con 0x561bba1f3e40 2024-06-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561bba284f60 con 0x561bba2775b0 2024-06-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facbc0d1b80 10 client.0 _ll_register_callbacks cb 0x561bba163720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facaaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba1f3e40 0x561bba281b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facaaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba1f3e40 0x561bba281b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36760/0 (socket says 172.21.15.26:36760) 2024-06-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facaaa00700 1 -- 172.21.15.26:0/454958429 learned_addr learned my addr 172.21.15.26:0/454958429 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facaaa00700 1 -- 172.21.15.26:0/454958429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 msgr2=0x561bba27ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facaaa00700 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 0x561bba27ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facaaa00700 1 -- 172.21.15.26:0/454958429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 msgr2=0x561bba27f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facaaa00700 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 0x561bba27f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.760+0000 7facaaa00700 1 -- 172.21.15.26:0/454958429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bba1188d0 con 0x561bba1f3e40 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.761+0000 7facaaa00700 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba1f3e40 0x561bba281b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac9c005c70 tx=0x7fac9c00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d27782f9fe9ad6f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.761+0000 7fac94c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.761+0000 7fac94c00700 1 -- 172.21.15.26:0/454958429 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003680 con 0x561bba1f3e40 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.761+0000 7fac94c00700 1 -- 172.21.15.26:0/454958429 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fac9c003820 con 0x561bba1f3e40 2024-06-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.762+0000 7facbc0d1b80 1 -- 172.21.15.26:0/454958429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca40081c0 con 0x561bba1f3e40 2024-06-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.762+0000 7fac94c00700 1 -- 172.21.15.26:0/454958429 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac9c003a20 con 0x561bba1f3e40 2024-06-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.762+0000 7facbc0d1b80 10 client.25295 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.762+0000 7facbc0d1b80 1 -- 172.21.15.26:0/454958429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561bba28f760 con 0x561bba1f3e40 2024-06-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.762+0000 7fac94c00700 1 -- 172.21.15.26:0/454958429 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca40081c0 con 0x561bba1f3e40 2024-06-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.763+0000 7fac94c00700 1 -- 172.21.15.26:0/454958429 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fac9c024430 con 0x561bba1f3e40 2024-06-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.763+0000 7facbc0d1b80 10 client.25295 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.763+0000 7facbc0d1b80 10 client.25295 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.763+0000 7facbc0d1b80 20 client.25295 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.763+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x561bba291da0 0x561bba294190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.763+0000 7facbc0d1b80 4 client.25295 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.763+0000 7facbc0d1b80 1 -- 172.21.15.26:0/454958429 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bba1d7000 con 0x561bba291da0 2024-06-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.763+0000 7facaa000700 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x561bba291da0 0x561bba294190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.764+0000 7fac94c00700 10 client.25295 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.764+0000 7facaa000700 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x561bba291da0 0x561bba294190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=464648304742bc8e server_cookie=3a2e55b387405f5d in_seq=0 out_seq=0 2024-06-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.765+0000 7fac94c00700 1 -- 172.21.15.26:0/454958429 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561bba1d7000 con 0x561bba291da0 2024-06-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.765+0000 7fac94c00700 10 client.25295 handle_command_reply: tid=0 2024-06-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.765+0000 7facbc0d1b80 1 client.25295 shutdown 2024-06-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.765+0000 7facbc0d1b80 1 -- 172.21.15.26:0/454958429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba1f3e40 msgr2=0x561bba281b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.765+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba1f3e40 0x561bba281b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac9c005c70 tx=0x7fac9c00b610 comp rx=0 tx=0).stop 2024-06-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 1 -- 172.21.15.26:0/454958429 shutdown_connections 2024-06-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x561bba291da0 0x561bba294190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba1f3e40 0x561bba281b10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 0x561bba27f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/454958429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba2775b0 0x561bba27ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 1 -- 172.21.15.26:0/454958429 >> 172.21.15.26:0/454958429 conn(0x561bba1a4170 msgr2=0x561bba272740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7fac94c00700 0 client.25295 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 1 -- 172.21.15.26:0/454958429 shutdown_connections 2024-06-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 1 -- 172.21.15.26:0/454958429 wait complete. 2024-06-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.766+0000 7facbc0d1b80 20 client.25295 trim_cache size 0 max 16384 2024-06-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.798+0000 7facbc0d1b80 1 Processor -- start 2024-06-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.798+0000 7facbc0d1b80 1 -- start start 2024-06-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.798+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba273620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.798+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba2775b0 0x561bba27f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.798+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 0x561bba2725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.798+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b7320 con 0x561bba282300 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.798+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b4bb0 con 0x561bba2775b0 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.798+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b2a50 con 0x561bba1f3e40 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaaa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba273620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaa000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba2775b0 0x561bba27f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaaa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba273620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60510/0 (socket says 172.21.15.26:60510) 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaaa00700 1 -- 172.21.15.26:0/1758102997 learned_addr learned my addr 172.21.15.26:0/1758102997 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaaa00700 1 -- 172.21.15.26:0/1758102997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba2775b0 msgr2=0x561bba27f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaaa00700 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba2775b0 0x561bba27f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facab400700 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 0x561bba2725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaaa00700 1 -- 172.21.15.26:0/1758102997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 msgr2=0x561bba2725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaaa00700 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 0x561bba2725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facaaa00700 1 -- 172.21.15.26:0/1758102997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bba27d3c0 con 0x561bba1f3e40 2024-06-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.799+0000 7facab400700 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 0x561bba2725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.800+0000 7facaaa00700 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba273620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac9c024a90 tx=0x7fac9c009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4891f5de2b02fd35 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.800+0000 7fac97400700 1 -- 172.21.15.26:0/1758102997 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c03d070 con 0x561bba1f3e40 2024-06-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.800+0000 7fac97400700 1 -- 172.21.15.26:0/1758102997 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c024b10 con 0x561bba1f3e40 2024-06-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.800+0000 7fac97400700 1 -- 172.21.15.26:0/1758102997 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003e20 con 0x561bba1f3e40 2024-06-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.801+0000 7facbc0d1b80 1 -- 172.21.15.26:0/1758102997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 msgr2=0x561bba273620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.801+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba273620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac9c024a90 tx=0x7fac9c009290 comp rx=0 tx=0).stop 2024-06-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.801+0000 7facbc0d1b80 1 -- 172.21.15.26:0/1758102997 shutdown_connections 2024-06-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.801+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba282300 0x561bba2725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.801+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba273620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.801+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/1758102997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba2775b0 0x561bba27f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.801+0000 7facbc0d1b80 1 -- 172.21.15.26:0/1758102997 >> 172.21.15.26:0/1758102997 conn(0x561bba1a4170 msgr2=0x561bba271eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.801+0000 7facbc0d1b80 1 -- 172.21.15.26:0/1758102997 shutdown_connections 2024-06-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.802+0000 7facbc0d1b80 1 -- 172.21.15.26:0/1758102997 wait complete. 2024-06-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.802+0000 7facbc0d1b80 1 -- start start 2024-06-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.802+0000 7facbc0d1b80 1 Processor -- start 2024-06-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.803+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba28a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.803+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba2775b0 0x561bba28a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.803+0000 7facbc0d1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 0x561bba27c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.803+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b3070 con 0x561bba2775b0 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.803+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b44e0 con 0x561bba282300 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.803+0000 7facbc0d1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b3740 con 0x561bba1f3e40 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.803+0000 7facbc0d1b80 10 client.0 _ll_register_callbacks cb 0x561bba27fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaaa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba28a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaaa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba28a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60526/0 (socket says 172.21.15.26:60526) 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaaa00700 1 -- 172.21.15.26:0/2123921237 learned_addr learned my addr 172.21.15.26:0/2123921237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaa000700 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba2775b0 0x561bba28a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaaa00700 1 -- 172.21.15.26:0/2123921237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 msgr2=0x561bba27c0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaaa00700 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 0x561bba27c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaaa00700 1 -- 172.21.15.26:0/2123921237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba2775b0 msgr2=0x561bba28a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaaa00700 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba2775b0 0x561bba28a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaaa00700 1 -- 172.21.15.26:0/2123921237 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bba27c800 con 0x561bba1f3e40 2024-06-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.804+0000 7facaa000700 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba2775b0 0x561bba28a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.805+0000 7facaaa00700 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba28a3d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac9c026040 tx=0x7fac9c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83132465af2ac7d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.805+0000 7fac94c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.805+0000 7fac94c00700 1 -- 172.21.15.26:0/2123921237 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c03d070 con 0x561bba1f3e40 2024-06-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.805+0000 7facbc0d1b80 1 -- 172.21.15.26:0/2123921237 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac9c024b10 con 0x561bba1f3e40 2024-06-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.805+0000 7fac94c00700 1 -- 172.21.15.26:0/2123921237 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fac9c003bc0 con 0x561bba1f3e40 2024-06-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.805+0000 7fac94c00700 1 -- 172.21.15.26:0/2123921237 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac9c003e00 con 0x561bba1f3e40 2024-06-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.805+0000 7facbc0d1b80 10 client.25339 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.805+0000 7facbc0d1b80 1 -- 172.21.15.26:0/2123921237 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561bba283450 con 0x561bba1f3e40 2024-06-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.806+0000 7fac94c00700 1 -- 172.21.15.26:0/2123921237 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c024b10 con 0x561bba1f3e40 2024-06-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.807+0000 7fac94c00700 1 -- 172.21.15.26:0/2123921237 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fac9c00f460 con 0x561bba1f3e40 2024-06-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.807+0000 7facbc0d1b80 10 client.25339 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.807+0000 7facbc0d1b80 10 client.25339 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.807+0000 7facbc0d1b80 20 client.25339 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.807+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x561bba2fc9e0 0x561bba2fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.807+0000 7facbc0d1b80 4 client.25339 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.807+0000 7facbc0d1b80 1 -- 172.21.15.26:0/2123921237 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bba1d1580 con 0x561bba2fc9e0 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.807+0000 7facaa000700 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x561bba2fc9e0 0x561bba2fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.808+0000 7facaa000700 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x561bba2fc9e0 0x561bba2fedd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5aec57e5b94d7e59 server_cookie=635743d45ef0c56c in_seq=0 out_seq=0 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.808+0000 7fac94c00700 10 client.25339 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.808+0000 7fac94c00700 1 -- 172.21.15.26:0/2123921237 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bba1d1580 con 0x561bba2fc9e0 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.808+0000 7fac94c00700 10 client.25339 handle_command_reply: tid=0 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.808+0000 7facbc0d1b80 1 client.25339 shutdown 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 -- 172.21.15.26:0/2123921237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 msgr2=0x561bba28a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba28a3d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac9c026040 tx=0x7fac9c005e80 comp rx=0 tx=0).stop 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 -- 172.21.15.26:0/2123921237 shutdown_connections 2024-06-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x561bba2fc9e0 0x561bba2fedd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561bba282300 0x561bba27c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bba1f3e40 0x561bba28a3d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7fac94c00700 0 client.25339 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 --2- 172.21.15.26:0/2123921237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561bba2775b0 0x561bba28a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 -- 172.21.15.26:0/2123921237 >> 172.21.15.26:0/2123921237 conn(0x561bba1a4170 msgr2=0x561bba272740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 -- 172.21.15.26:0/2123921237 shutdown_connections 2024-06-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 1 -- 172.21.15.26:0/2123921237 wait complete. 2024-06-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.809+0000 7facbc0d1b80 20 client.25339 trim_cache size 0 max 16384 2024-06-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-06-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.810+0000 7facb1800700 1 -- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fac900a2c20 msgr2=0x7fac900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.810+0000 7facb1800700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fac900a2c20 0x7fac900a50e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faca4008920 tx=0x7faca4009350 comp rx=0 tx=0).stop 2024-06-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.810+0000 7facb1800700 1 -- 172.21.15.26:0/1851545009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac074fe0 msgr2=0x7facac19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.810+0000 7facb1800700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac074fe0 0x7facac19f820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fac9800e9f0 tx=0x7fac9800ed00 comp rx=0 tx=0).stop 2024-06-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.811+0000 7facb1800700 1 -- 172.21.15.26:0/1851545009 shutdown_connections 2024-06-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.811+0000 7facb1800700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fac900a2c20 0x7fac900a50e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.811+0000 7facb1800700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facac113660 0x7facac1a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.811+0000 7facb1800700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0746c0 0x7facac19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.811+0000 7facb1800700 1 --2- 172.21.15.26:0/1851545009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facac074fe0 0x7facac19f820 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.811+0000 7facb1800700 1 -- 172.21.15.26:0/1851545009 >> 172.21.15.26:0/1851545009 conn(0x7facac0795f0 msgr2=0x7facac112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.811+0000 7facb1800700 1 -- 172.21.15.26:0/1851545009 shutdown_connections 2024-06-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:29.811+0000 7facb1800700 1 -- 172.21.15.26:0/1851545009 wait complete. 2024-06-13T07:01:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:29 smithi136 ceph-mon[37229]: pgmap v539: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:30.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:29 smithi135 ceph-mon[36276]: pgmap v539: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:30.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:01:31.413 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:31.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:31 smithi026 ceph-mon[32041]: pgmap v540: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:31.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:31 smithi136 ceph-mon[37229]: pgmap v540: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:32.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:31 smithi135 ceph-mon[36276]: pgmap v540: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-13T07:01:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.076+0000 7f590ca00640 1 -- 172.21.15.26:0/968846744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908104c50 msgr2=0x7f59081050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.076+0000 7f590ca00640 1 --2- 172.21.15.26:0/968846744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908104c50 0x7f59081050d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009900 tx=0x7f58f402a400 comp rx=0 tx=0).stop 2024-06-13T07:01:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.076+0000 7f590ca00640 1 -- 172.21.15.26:0/968846744 shutdown_connections 2024-06-13T07:01:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.076+0000 7f590ca00640 1 --2- 172.21.15.26:0/968846744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908104c50 0x7f59081050d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.076+0000 7f590ca00640 1 --2- 172.21.15.26:0/968846744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5908105610 0x7f590810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.076+0000 7f590ca00640 1 --2- 172.21.15.26:0/968846744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5908103a60 0x7f5908103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.076+0000 7f590ca00640 1 -- 172.21.15.26:0/968846744 >> 172.21.15.26:0/968846744 conn(0x7f59080fefd0 msgr2=0x7f5908101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.077+0000 7f590ca00640 1 -- 172.21.15.26:0/968846744 shutdown_connections 2024-06-13T07:01:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.077+0000 7f590ca00640 1 -- 172.21.15.26:0/968846744 wait complete. 2024-06-13T07:01:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f590ca00640 1 Processor -- start 2024-06-13T07:01:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f590ca00640 1 -- start start 2024-06-13T07:01:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f590ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5908103a60 0x7f5908199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f590ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5908104c50 0x7f590819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f590ca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908105610 0x7f59081a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f590ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f590810f6c0 con 0x7f5908103a60 2024-06-13T07:01:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f590ca00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f590810f540 con 0x7f5908105610 2024-06-13T07:01:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f590ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f590810f840 con 0x7f5908104c50 2024-06-13T07:01:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f5905e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5908104c50 0x7f590819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f5905e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5908104c50 0x7f590819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39822/0 (socket says 172.21.15.26:39822) 2024-06-13T07:01:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.078+0000 7f5905e00640 1 -- 172.21.15.26:0/4012555213 learned_addr learned my addr 172.21.15.26:0/4012555213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5907200640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908105610 0x7f59081a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5905e00640 1 -- 172.21.15.26:0/4012555213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908105610 msgr2=0x7f59081a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5905e00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908105610 0x7f59081a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5905e00640 1 -- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5908103a60 msgr2=0x7f5908199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5906800640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5908103a60 0x7f5908199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5905e00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5908103a60 0x7f5908199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5905e00640 1 -- 172.21.15.26:0/4012555213 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590806b4f0 con 0x7f5908104c50 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5907200640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908105610 0x7f59081a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5906800640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5908103a60 0x7f5908199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.079+0000 7f5905e00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5908104c50 0x7f590819a290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009230 tx=0x7f58f4031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.080+0000 7f58ff400640 1 -- 172.21.15.26:0/4012555213 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f403f030 con 0x7f5908104c50 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.080+0000 7f58ff400640 1 -- 172.21.15.26:0/4012555213 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f4003ad0 con 0x7f5908104c50 2024-06-13T07:01:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.080+0000 7f590ca00640 1 -- 172.21.15.26:0/4012555213 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590806b780 con 0x7f5908104c50 2024-06-13T07:01:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.080+0000 7f58ff400640 1 -- 172.21.15.26:0/4012555213 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4035070 con 0x7f5908104c50 2024-06-13T07:01:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.080+0000 7f590ca00640 1 -- 172.21.15.26:0/4012555213 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590806bcc0 con 0x7f5908104c50 2024-06-13T07:01:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.085+0000 7f59101eb740 1 Processor -- start 2024-06-13T07:01:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.085+0000 7f59101eb740 1 -- start start 2024-06-13T07:01:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f58ff400640 1 -- 172.21.15.26:0/4012555213 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f58f4042030 con 0x7f5908104c50 2024-06-13T07:01:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f59101eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac7611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f59101eb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac7662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f59101eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac768db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f59101eb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563eac6c4420 con 0x563eac7617b0 2024-06-13T07:01:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f59101eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563eac66bc50 con 0x563eac760dc0 2024-06-13T07:01:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f59101eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563eac6c03e0 con 0x563eac766920 2024-06-13T07:01:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f5907200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac7611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f5907200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac7611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56720/0 (socket says 172.21.15.26:56720) 2024-06-13T07:01:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.086+0000 7f5907200640 1 -- 172.21.15.26:0/1786547954 learned_addr learned my addr 172.21.15.26:0/1786547954 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.087+0000 7f5906800640 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac7662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.087+0000 7f5905e00640 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac768db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.087+0000 7f5907200640 1 -- 172.21.15.26:0/1786547954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 msgr2=0x563eac768db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.087+0000 7f5907200640 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac768db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.087+0000 7f5907200640 1 -- 172.21.15.26:0/1786547954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 msgr2=0x563eac7662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.087+0000 7f5907200640 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac7662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.087+0000 7f5907200640 1 -- 172.21.15.26:0/1786547954 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eac3d8210 con 0x563eac760dc0 2024-06-13T07:01:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.088+0000 7f5907200640 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac7611e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5900009f90 tx=0x7f590000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2162dd533dff205 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.088+0000 7f58ff400640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58ec0a2c00 0x7f58ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.088+0000 7f58ff400640 1 -- 172.21.15.26:0/4012555213 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f58f40efb40 con 0x7f5908104c50 2024-06-13T07:01:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.088+0000 7f58f3400640 1 -- 172.21.15.26:0/1786547954 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900003b70 con 0x563eac760dc0 2024-06-13T07:01:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.088+0000 7f58f3400640 1 -- 172.21.15.26:0/1786547954 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900003d10 con 0x563eac760dc0 2024-06-13T07:01:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.088+0000 7f58f3400640 1 -- 172.21.15.26:0/1786547954 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900011ac0 con 0x563eac760dc0 2024-06-13T07:01:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.089+0000 7f5906800640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58ec0a2c00 0x7f58ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.089+0000 7f59101eb740 1 -- 172.21.15.26:0/1786547954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 msgr2=0x563eac7611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.089+0000 7f59101eb740 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac7611e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5900009f90 tx=0x7f590000f7f0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.089+0000 7f59101eb740 1 -- 172.21.15.26:0/1786547954 shutdown_connections 2024-06-13T07:01:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.090+0000 7f59101eb740 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac768db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.090+0000 7f59101eb740 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac7662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.090+0000 7f59101eb740 1 --2- 172.21.15.26:0/1786547954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac7611e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.090+0000 7f59101eb740 1 -- 172.21.15.26:0/1786547954 >> 172.21.15.26:0/1786547954 conn(0x563eac728fb0 msgr2=0x563eac75c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.090+0000 7f5906800640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58ec0a2c00 0x7f58ec0a50c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x563eac761bb0 tx=0x7f58f80094e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.090+0000 7f59101eb740 1 -- 172.21.15.26:0/1786547954 shutdown_connections 2024-06-13T07:01:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.090+0000 7f59101eb740 1 -- 172.21.15.26:0/1786547954 wait complete. 2024-06-13T07:01:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.091+0000 7f59101eb740 1 -- start start 2024-06-13T07:01:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.091+0000 7f59101eb740 1 Processor -- start 2024-06-13T07:01:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.091+0000 7f59101eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac76b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f59101eb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac76b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f59101eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac772710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f59101eb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563eac7698d0 con 0x563eac7617b0 2024-06-13T07:01:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f59101eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563eac67f0b0 con 0x563eac760dc0 2024-06-13T07:01:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f59101eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563eac769a50 con 0x563eac766920 2024-06-13T07:01:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f59101eb740 10 client.0 _ll_register_callbacks cb 0x563eac66af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f5906800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac76b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f5906800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac76b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56732/0 (socket says 172.21.15.26:56732) 2024-06-13T07:01:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f5906800640 1 -- 172.21.15.26:0/3384816939 learned_addr learned my addr 172.21.15.26:0/3384816939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f5907200640 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac772710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.092+0000 7f5905e00640 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac76b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.093+0000 7f5907200640 1 -- 172.21.15.26:0/3384816939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 msgr2=0x563eac76b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.093+0000 7f5907200640 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac76b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.093+0000 7f5907200640 1 -- 172.21.15.26:0/3384816939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 msgr2=0x563eac76b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.093+0000 7f5907200640 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac76b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.093+0000 7f5907200640 1 -- 172.21.15.26:0/3384816939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eac4a4620 con 0x563eac766920 2024-06-13T07:01:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.093+0000 7f5907200640 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac772710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f590000fcf0 tx=0x7f5900009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6f2239d4a3bb378 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.093+0000 7f58e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.094+0000 7f58e7e00640 1 -- 172.21.15.26:0/3384816939 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900004a40 con 0x563eac766920 2024-06-13T07:01:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.094+0000 7f58e7e00640 1 -- 172.21.15.26:0/3384816939 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5900004be0 con 0x563eac766920 2024-06-13T07:01:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.094+0000 7f59101eb740 1 -- 172.21.15.26:0/3384816939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5900003d10 con 0x563eac766920 2024-06-13T07:01:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.095+0000 7f58e7e00640 1 -- 172.21.15.26:0/3384816939 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f590002f050 con 0x563eac766920 2024-06-13T07:01:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.096+0000 7f59101eb740 10 client.25351 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.096+0000 7f59101eb740 1 -- 172.21.15.26:0/3384816939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563eac775a80 con 0x563eac766920 2024-06-13T07:01:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.096+0000 7f58e7e00640 1 -- 172.21.15.26:0/3384816939 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900003d10 con 0x563eac766920 2024-06-13T07:01:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.096+0000 7f58e7e00640 1 -- 172.21.15.26:0/3384816939 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5900004160 con 0x563eac766920 2024-06-13T07:01:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.097+0000 7f59101eb740 10 client.25351 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.097+0000 7f59101eb740 10 client.25351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.097+0000 7f59101eb740 20 client.25351 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.097+0000 7f59101eb740 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563eac776e90 0x563eac779280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.097+0000 7f59101eb740 4 client.25351 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.097+0000 7f59101eb740 1 -- 172.21.15.26:0/3384816939 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eac6d1b50 con 0x563eac776e90 2024-06-13T07:01:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.097+0000 7f5906800640 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563eac776e90 0x563eac779280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.098+0000 7f5906800640 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563eac776e90 0x563eac779280 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e57ce8fb42a08b server_cookie=73cdc586c590ed0e in_seq=0 out_seq=0 2024-06-13T07:01:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.098+0000 7f58e7e00640 10 client.25351 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f58e7e00640 1 -- 172.21.15.26:0/3384816939 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563eac6d1b50 con 0x563eac776e90 2024-06-13T07:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f58e7e00640 10 client.25351 handle_command_reply: tid=0 2024-06-13T07:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 client.25351 shutdown 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 -- 172.21.15.26:0/3384816939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 msgr2=0x563eac772710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac772710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f590000fcf0 tx=0x7f5900009330 comp rx=0 tx=0).stop 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 -- 172.21.15.26:0/3384816939 shutdown_connections 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563eac776e90 0x563eac779280 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac766920 0x563eac772710 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac76b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 --2- 172.21.15.26:0/3384816939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac760dc0 0x563eac76b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f59101eb740 1 -- 172.21.15.26:0/3384816939 >> 172.21.15.26:0/3384816939 conn(0x563eac728fb0 msgr2=0x563eac75b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.100+0000 7f58e7e00640 0 client.25351 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.101+0000 7f59101eb740 1 -- 172.21.15.26:0/3384816939 shutdown_connections 2024-06-13T07:01:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.101+0000 7f59101eb740 1 -- 172.21.15.26:0/3384816939 wait complete. 2024-06-13T07:01:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.102+0000 7f59101eb740 20 client.25351 trim_cache size 0 max 16384 2024-06-13T07:01:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.118+0000 7f59101eb740 1 Processor -- start 2024-06-13T07:01:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.118+0000 7f59101eb740 1 -- start start 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f59101eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac75fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f59101eb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac7726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f59101eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 0x563eac77d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f59101eb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b7940 con 0x563eac7617b0 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f59101eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b5210 con 0x563eac766920 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f59101eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b3150 con 0x563eac760dc0 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f5906800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac75fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f5906800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac75fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39852/0 (socket says 172.21.15.26:39852) 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f5906800640 1 -- 172.21.15.26:0/1068341265 learned_addr learned my addr 172.21.15.26:0/1068341265 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f5905e00640 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac7726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.119+0000 7f5907200640 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 0x563eac77d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.120+0000 7f5906800640 1 -- 172.21.15.26:0/1068341265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 msgr2=0x563eac77d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.120+0000 7f5906800640 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 0x563eac77d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.120+0000 7f5906800640 1 -- 172.21.15.26:0/1068341265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 msgr2=0x563eac7726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.120+0000 7f5906800640 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac7726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.120+0000 7f5906800640 1 -- 172.21.15.26:0/1068341265 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eac75d4f0 con 0x563eac760dc0 2024-06-13T07:01:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.120+0000 7f5906800640 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac75fe10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x563eac779930 tx=0x7f58f8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90094dc9e58139ae server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.121+0000 7f58f2a00640 1 -- 172.21.15.26:0/1068341265 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8005020 con 0x563eac760dc0 2024-06-13T07:01:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.121+0000 7f58f2a00640 1 -- 172.21.15.26:0/1068341265 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f80047a0 con 0x563eac760dc0 2024-06-13T07:01:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.121+0000 7f58f2a00640 1 -- 172.21.15.26:0/1068341265 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8028600 con 0x563eac760dc0 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 -- 172.21.15.26:0/1068341265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 msgr2=0x563eac75fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac75fe10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x563eac779930 tx=0x7f58f8014000 comp rx=0 tx=0).stop 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 -- 172.21.15.26:0/1068341265 shutdown_connections 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 0x563eac77d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac7726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 --2- 172.21.15.26:0/1068341265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac75fe10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 -- 172.21.15.26:0/1068341265 >> 172.21.15.26:0/1068341265 conn(0x563eac728fb0 msgr2=0x563eac75f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 -- 172.21.15.26:0/1068341265 shutdown_connections 2024-06-13T07:01:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 -- 172.21.15.26:0/1068341265 wait complete. 2024-06-13T07:01:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.122+0000 7f59101eb740 1 -- start start 2024-06-13T07:01:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.123+0000 7f59101eb740 1 Processor -- start 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.123+0000 7f59101eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac7704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.123+0000 7f59101eb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac770a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.123+0000 7f59101eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 0x563eac771a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.123+0000 7f59101eb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b36d0 con 0x563eac7617b0 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.123+0000 7f59101eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b4b40 con 0x563eac766920 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.123+0000 7f59101eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b3da0 con 0x563eac760dc0 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.123+0000 7f59101eb740 10 client.0 _ll_register_callbacks cb 0x7f5900015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5906800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac7704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5905e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac770a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5906800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac7704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39864/0 (socket says 172.21.15.26:39864) 2024-06-13T07:01:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5906800640 1 -- 172.21.15.26:0/378171584 learned_addr learned my addr 172.21.15.26:0/378171584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5907200640 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 0x563eac771a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5906800640 1 -- 172.21.15.26:0/378171584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 msgr2=0x563eac771a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5906800640 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 0x563eac771a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5906800640 1 -- 172.21.15.26:0/378171584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 msgr2=0x563eac770a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.124+0000 7f5906800640 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac770a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.125+0000 7f5906800640 1 -- 172.21.15.26:0/378171584 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eac772020 con 0x563eac760dc0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.125+0000 7f5906800640 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac7704e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f58f802b040 tx=0x7f58f8005f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fe0872e9c158988 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.125+0000 7f58e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.125+0000 7f58e7e00640 1 -- 172.21.15.26:0/378171584 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8036020 con 0x563eac760dc0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.126+0000 7f58e7e00640 1 -- 172.21.15.26:0/378171584 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f58f80286d0 con 0x563eac760dc0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.126+0000 7f59101eb740 1 -- 172.21.15.26:0/378171584 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f80047a0 con 0x563eac760dc0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.126+0000 7f58e7e00640 1 -- 172.21.15.26:0/378171584 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58f80288d0 con 0x563eac760dc0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.126+0000 7f59101eb740 10 client.25357 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.126+0000 7f59101eb740 1 -- 172.21.15.26:0/378171584 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563eac7e2100 con 0x563eac760dc0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f58e7e00640 1 -- 172.21.15.26:0/378171584 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f80047a0 con 0x563eac760dc0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f58e7e00640 1 -- 172.21.15.26:0/378171584 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f58f800f040 con 0x563eac760dc0 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f59101eb740 10 client.25357 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f59101eb740 10 client.25357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f59101eb740 20 client.25357 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f59101eb740 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563eac7e2a40 0x563eac7e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f59101eb740 4 client.25357 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f59101eb740 1 -- 172.21.15.26:0/378171584 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563eac6cbea0 con 0x563eac7e2a40 2024-06-13T07:01:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.127+0000 7f5905e00640 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563eac7e2a40 0x563eac7e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.128+0000 7f5905e00640 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563eac7e2a40 0x563eac7e4e60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8ac9f28f75b6b98 server_cookie=c6efd4403af3770c in_seq=0 out_seq=0 2024-06-13T07:01:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.128+0000 7f58e7e00640 10 client.25357 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.132+0000 7f58e7e00640 1 -- 172.21.15.26:0/378171584 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563eac6cbea0 con 0x563eac7e2a40 2024-06-13T07:01:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.132+0000 7f58e7e00640 10 client.25357 handle_command_reply: tid=0 2024-06-13T07:01:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.133+0000 7f59101eb740 1 client.25357 shutdown 2024-06-13T07:01:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.133+0000 7f59101eb740 1 -- 172.21.15.26:0/378171584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 msgr2=0x563eac7704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.133+0000 7f59101eb740 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac7704e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f58f802b040 tx=0x7f58f8005f60 comp rx=0 tx=0).stop 2024-06-13T07:01:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f59101eb740 1 -- 172.21.15.26:0/378171584 shutdown_connections 2024-06-13T07:01:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f59101eb740 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563eac7e2a40 0x563eac7e4e60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f59101eb740 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563eac766920 0x563eac771a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f59101eb740 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563eac7617b0 0x563eac770a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f59101eb740 1 --2- 172.21.15.26:0/378171584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eac760dc0 0x563eac7704e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f59101eb740 1 -- 172.21.15.26:0/378171584 >> 172.21.15.26:0/378171584 conn(0x563eac728fb0 msgr2=0x563eac75ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f58e7e00640 0 client.25357 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f59101eb740 1 -- 172.21.15.26:0/378171584 shutdown_connections 2024-06-13T07:01:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.134+0000 7f59101eb740 1 -- 172.21.15.26:0/378171584 wait complete. 2024-06-13T07:01:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.135+0000 7f59101eb740 20 client.25357 trim_cache size 0 max 16384 2024-06-13T07:01:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:01:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:01:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 -- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58ec0a2c00 msgr2=0x7f58ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58ec0a2c00 0x7f58ec0a50c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x563eac761bb0 tx=0x7f58f80094e0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 -- 172.21.15.26:0/4012555213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5908104c50 msgr2=0x7f590819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5908104c50 0x7f590819a290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009230 tx=0x7f58f4031b60 comp rx=0 tx=0).stop 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 -- 172.21.15.26:0/4012555213 shutdown_connections 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f58ec0a2c00 0x7f58ec0a50c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5908105610 0x7f59081a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5908104c50 0x7f590819a290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 --2- 172.21.15.26:0/4012555213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5908103a60 0x7f5908199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.136+0000 7f590ca00640 1 -- 172.21.15.26:0/4012555213 >> 172.21.15.26:0/4012555213 conn(0x7f59080fefd0 msgr2=0x7f5908107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.137+0000 7f590ca00640 1 -- 172.21.15.26:0/4012555213 shutdown_connections 2024-06-13T07:01:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:33.137+0000 7f590ca00640 1 -- 172.21.15.26:0/4012555213 wait complete. 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 -- 172.21.15.26:0/2410422780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c104f80 msgr2=0x7fcb4c111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 --2- 172.21.15.26:0/2410422780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c104f80 0x7fcb4c111440 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400b0d0 tx=0x7fcb4400b3e0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 -- 172.21.15.26:0/2410422780 shutdown_connections 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 --2- 172.21.15.26:0/2410422780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c104f80 0x7fcb4c111440 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 --2- 172.21.15.26:0/2410422780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb4c10a670 0x7fcb4c10aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 --2- 172.21.15.26:0/2410422780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb4c1045c0 0x7fcb4c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 -- 172.21.15.26:0/2410422780 >> 172.21.15.26:0/2410422780 conn(0x7fcb4c100070 msgr2=0x7fcb4c102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 -- 172.21.15.26:0/2410422780 shutdown_connections 2024-06-13T07:01:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.783+0000 7fcb52600700 1 -- 172.21.15.26:0/2410422780 wait complete. 2024-06-13T07:01:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.784+0000 7fcb52600700 1 Processor -- start 2024-06-13T07:01:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.785+0000 7fcb52600700 1 -- start start 2024-06-13T07:01:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.785+0000 7fcb52600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb4c1045c0 0x7fcb4c1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.785+0000 7fcb52600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb4c104f80 0x7fcb4c1a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.785+0000 7fcb52600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c10a670 0x7fcb4c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.785+0000 7fcb52600700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c114970 con 0x7fcb4c10a670 2024-06-13T07:01:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.785+0000 7fcb52600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1147f0 con 0x7fcb4c104f80 2024-06-13T07:01:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.785+0000 7fcb52600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c114af0 con 0x7fcb4c1045c0 2024-06-13T07:01:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.785+0000 7fcb51200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb4c1045c0 0x7fcb4c1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb4c1045c0 0x7fcb4c1a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39882/0 (socket says 172.21.15.26:39882) 2024-06-13T07:01:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51200700 1 -- 172.21.15.26:0/1711409292 learned_addr learned my addr 172.21.15.26:0/1711409292 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb4be00700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb4c104f80 0x7fcb4c1a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51200700 1 -- 172.21.15.26:0/1711409292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb4c104f80 msgr2=0x7fcb4c1a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51200700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb4c104f80 0x7fcb4c1a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51c00700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c10a670 0x7fcb4c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51200700 1 -- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c10a670 msgr2=0x7fcb4c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51200700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c10a670 0x7fcb4c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51200700 1 -- 172.21.15.26:0/1711409292 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c19b230 con 0x7fcb4c1045c0 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb4be00700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb4c104f80 0x7fcb4c1a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.786+0000 7fcb51c00700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c10a670 0x7fcb4c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.787+0000 7fcb51200700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb4c1045c0 0x7fcb4c1a0910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9a0 tx=0x7fcb4000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.787+0000 7fcb49600700 1 -- 172.21.15.26:0/1711409292 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000cbd0 con 0x7fcb4c1045c0 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.787+0000 7fcb49600700 1 -- 172.21.15.26:0/1711409292 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4000cd70 con 0x7fcb4c1045c0 2024-06-13T07:01:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.787+0000 7fcb52600700 1 -- 172.21.15.26:0/1711409292 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c19b520 con 0x7fcb4c1045c0 2024-06-13T07:01:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.787+0000 7fcb49600700 1 -- 172.21.15.26:0/1711409292 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40018780 con 0x7fcb4c1045c0 2024-06-13T07:01:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.787+0000 7fcb52600700 1 -- 172.21.15.26:0/1711409292 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c1a77e0 con 0x7fcb4c1045c0 2024-06-13T07:01:33.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:33 smithi026 ceph-mon[32041]: pgmap v541: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:01:33.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.792+0000 7fcb5e4adb80 1 Processor -- start 2024-06-13T07:01:33.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.792+0000 7fcb5e4adb80 1 -- start start 2024-06-13T07:01:33.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.792+0000 7fcb49600700 1 -- 172.21.15.26:0/1711409292 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb40018920 con 0x7fcb4c1045c0 2024-06-13T07:01:33.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.793+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7f98420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.793+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f98de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.793+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7fa1570 0x5566e7fa3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.793+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5566e7ea5a10 con 0x5566e7f12e40 2024-06-13T07:01:33.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.793+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5566e7ee8b00 con 0x5566e7f98960 2024-06-13T07:01:33.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.793+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5566e7ea38f0 con 0x5566e7fa1570 2024-06-13T07:01:33.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.794+0000 7fcb51c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7f98420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.794+0000 7fcb4be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7fa1570 0x5566e7fa3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.794+0000 7fcb4be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7fa1570 0x5566e7fa3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39896/0 (socket says 172.21.15.26:39896) 2024-06-13T07:01:33.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.794+0000 7fcb4be00700 1 -- 172.21.15.26:0/2436692727 learned_addr learned my addr 172.21.15.26:0/2436692727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.794+0000 7fcb51200700 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f98de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.795+0000 7fcb51200700 1 -- 172.21.15.26:0/2436692727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7fa1570 msgr2=0x5566e7fa3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.795+0000 7fcb51200700 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7fa1570 0x5566e7fa3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.795+0000 7fcb51200700 1 -- 172.21.15.26:0/2436692727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 msgr2=0x5566e7f98420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.795+0000 7fcb51200700 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7f98420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.795+0000 7fcb51200700 1 -- 172.21.15.26:0/2436692727 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e7e90f90 con 0x5566e7f98960 2024-06-13T07:01:33.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.795+0000 7fcb51c00700 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7f98420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:33.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.796+0000 7fcb51200700 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f98de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb400ccd20 tx=0x7fcb400d5a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc29ccc2e7e05b47 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.796+0000 7fcb49600700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcb340a2d20 0x7fcb340a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.796+0000 7fcb51c00700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcb340a2d20 0x7fcb340a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.796+0000 7fcb3e000700 1 -- 172.21.15.26:0/2436692727 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400e3070 con 0x5566e7f98960 2024-06-13T07:01:33.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.797+0000 7fcb3e000700 1 -- 172.21.15.26:0/2436692727 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4009c1e0 con 0x5566e7f98960 2024-06-13T07:01:33.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.797+0000 7fcb3e000700 1 -- 172.21.15.26:0/2436692727 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4009b620 con 0x5566e7f98960 2024-06-13T07:01:33.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.797+0000 7fcb49600700 1 -- 172.21.15.26:0/1711409292 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcb400cf6d0 con 0x7fcb4c1045c0 2024-06-13T07:01:33.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.799+0000 7fcb51c00700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcb340a2d20 0x7fcb340a51e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400b770 tx=0x7fcb4400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.799+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/2436692727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 msgr2=0x5566e7f98de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.799+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f98de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb400ccd20 tx=0x7fcb400d5a00 comp rx=0 tx=0).stop 2024-06-13T07:01:33.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.800+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/2436692727 shutdown_connections 2024-06-13T07:01:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.800+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7fa1570 0x5566e7fa3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.800+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7f98420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.800+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/2436692727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f98de0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.800+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/2436692727 >> 172.21.15.26:0/2436692727 conn(0x5566e7ec3170 msgr2=0x5566e7f92820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.800+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/2436692727 shutdown_connections 2024-06-13T07:01:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.800+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/2436692727 wait complete. 2024-06-13T07:01:33.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.801+0000 7fcb5e4adb80 1 -- start start 2024-06-13T07:01:33.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.802+0000 7fcb5e4adb80 1 Processor -- start 2024-06-13T07:01:33.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.802+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 0x5566e7fa0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.802+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f9e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.802+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7fa1570 0x5566e7f9e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.802+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5566e7e84f60 con 0x5566e7fa1570 2024-06-13T07:01:33.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.802+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5566e7ee4ac0 con 0x5566e7f98960 2024-06-13T07:01:33.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.802+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5566e7fa41e0 con 0x5566e7f12e40 2024-06-13T07:01:33.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.802+0000 7fcb5e4adb80 10 client.0 _ll_register_callbacks cb 0x5566e7e82720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:33.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.803+0000 7fcb51c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7fa1570 0x5566e7f9e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.803+0000 7fcb51c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7fa1570 0x5566e7f9e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52080/0 (socket says 172.21.15.26:52080) 2024-06-13T07:01:33.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.803+0000 7fcb51c00700 1 -- 172.21.15.26:0/3214041349 learned_addr learned my addr 172.21.15.26:0/3214041349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.804+0000 7fcb51200700 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 0x5566e7fa0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.804+0000 7fcb4be00700 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f9e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.804+0000 7fcb51c00700 1 -- 172.21.15.26:0/3214041349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 msgr2=0x5566e7fa0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.804+0000 7fcb51c00700 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 0x5566e7fa0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.804+0000 7fcb51c00700 1 -- 172.21.15.26:0/3214041349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 msgr2=0x5566e7f9e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.804+0000 7fcb51c00700 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f9e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.804+0000 7fcb51c00700 1 -- 172.21.15.26:0/3214041349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e7e378d0 con 0x5566e7fa1570 2024-06-13T07:01:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.805+0000 7fcb51c00700 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7fa1570 0x5566e7f9e5b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb44004cd0 tx=0x7fcb440099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8233ae9e158236c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.805+0000 7fcb32a00700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:01:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.805+0000 7fcb32a00700 1 -- 172.21.15.26:0/3214041349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44007970 con 0x5566e7fa1570 2024-06-13T07:01:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.805+0000 7fcb32a00700 1 -- 172.21.15.26:0/3214041349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcb44007b10 con 0x5566e7fa1570 2024-06-13T07:01:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.805+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3214041349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb4009c1e0 con 0x5566e7fa1570 2024-06-13T07:01:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb32a00700 1 -- 172.21.15.26:0/3214041349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb44007d10 con 0x5566e7fa1570 2024-06-13T07:01:33.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb5e4adb80 10 client.15225 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:33.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3214041349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566e7fb7260 con 0x5566e7fa1570 2024-06-13T07:01:33.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb32a00700 1 -- 172.21.15.26:0/3214041349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4009c1e0 con 0x5566e7fa1570 2024-06-13T07:01:33.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb32a00700 1 -- 172.21.15.26:0/3214041349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcb44011070 con 0x5566e7fa1570 2024-06-13T07:01:33.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb5e4adb80 10 client.15225 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:33.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb5e4adb80 10 client.15225 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-13T07:01:33.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb5e4adb80 20 client.15225 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:33.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x5566e7fb9940 0x5566e7fbbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb5e4adb80 4 client.15225 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:33.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.806+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3214041349 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566e7ef6000 con 0x5566e7fb9940 2024-06-13T07:01:33.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.807+0000 7fcb51200700 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x5566e7fb9940 0x5566e7fbbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.808+0000 7fcb51200700 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x5566e7fb9940 0x5566e7fbbd50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdaa77082e32e3c3 server_cookie=c3f061f7a5a15184 in_seq=0 out_seq=0 2024-06-13T07:01:33.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.808+0000 7fcb32a00700 10 client.15225 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T07:01:33.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb32a00700 1 -- 172.21.15.26:0/3214041349 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5566e7ef6000 con 0x5566e7fb9940 2024-06-13T07:01:33.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb32a00700 10 client.15225 handle_command_reply: tid=0 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 client.15225 shutdown 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3214041349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7fa1570 msgr2=0x5566e7f9e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7fa1570 0x5566e7f9e5b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb44004cd0 tx=0x7fcb440099e0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3214041349 shutdown_connections 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x5566e7fb9940 0x5566e7fbbd50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7fa1570 0x5566e7f9e5b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 0x5566e7fa0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3214041349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7f98960 0x5566e7f9e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.809+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3214041349 >> 172.21.15.26:0/3214041349 conn(0x5566e7ec3170 msgr2=0x5566e7f92410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.810+0000 7fcb32a00700 0 client.15225 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T07:01:33.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.810+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3214041349 shutdown_connections 2024-06-13T07:01:33.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.810+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3214041349 wait complete. 2024-06-13T07:01:33.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.810+0000 7fcb5e4adb80 20 client.15225 trim_cache size 0 max 16384 2024-06-13T07:01:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.826+0000 7fcb5e4adb80 1 Processor -- start 2024-06-13T07:01:33.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb5e4adb80 1 -- start start 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7fa3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f98960 0x5566e7f9e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c011500 con 0x5566e7f12e40 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c00ed90 con 0x5566e7fa1570 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c00cbe0 con 0x5566e7f98960 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb51200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7fa3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.827+0000 7fcb4be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f98960 0x5566e7f9e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.828+0000 7fcb4be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f98960 0x5566e7f9e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39918/0 (socket says 172.21.15.26:39918) 2024-06-13T07:01:33.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.828+0000 7fcb4be00700 1 -- 172.21.15.26:0/3801189902 learned_addr learned my addr 172.21.15.26:0/3801189902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.828+0000 7fcb51c00700 1 --2- 172.21.15.26:0/3801189902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.828+0000 7fcb51c00700 1 -- 172.21.15.26:0/3801189902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f98960 msgr2=0x5566e7f9e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.828+0000 7fcb51c00700 1 --2- 172.21.15.26:0/3801189902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f98960 0x5566e7f9e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.828+0000 7fcb51c00700 1 -- 172.21.15.26:0/3801189902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 msgr2=0x5566e7fa3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.828+0000 7fcb51c00700 1 --2- 172.21.15.26:0/3801189902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7fa3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.828+0000 7fcb51c00700 1 -- 172.21.15.26:0/3801189902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e7f91520 con 0x5566e7fa1570 2024-06-13T07:01:33.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.829+0000 7fcb51c00700 1 --2- 172.21.15.26:0/3801189902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9e570 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcb44029940 tx=0x7fcb44009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9afaa5bd0f7c260 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.829+0000 7fcb3d600700 1 -- 172.21.15.26:0/3801189902 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4403f020 con 0x5566e7fa1570 2024-06-13T07:01:33.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.829+0000 7fcb3d600700 1 -- 172.21.15.26:0/3801189902 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb44029bb0 con 0x5566e7fa1570 2024-06-13T07:01:33.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.829+0000 7fcb3d600700 1 -- 172.21.15.26:0/3801189902 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb440053c0 con 0x5566e7fa1570 2024-06-13T07:01:33.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.830+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3801189902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 msgr2=0x5566e7f9e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.830+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3801189902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9e570 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcb44029940 tx=0x7fcb44009dc0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.830+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3801189902 shutdown_connections 2024-06-13T07:01:33.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.830+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3801189902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9e570 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.830+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3801189902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f12e40 0x5566e7fa3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.830+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/3801189902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f98960 0x5566e7f9e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.830+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3801189902 >> 172.21.15.26:0/3801189902 conn(0x5566e7ec3170 msgr2=0x5566e7f99460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.831+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3801189902 shutdown_connections 2024-06-13T07:01:33.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.831+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/3801189902 wait complete. 2024-06-13T07:01:33.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.831+0000 7fcb5e4adb80 1 -- start start 2024-06-13T07:01:33.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.832+0000 7fcb5e4adb80 1 Processor -- start 2024-06-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 0x5566e7fb2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f98960 0x5566e7f9b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb5e4adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c00d250 con 0x5566e7f98960 2024-06-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c00e6c0 con 0x5566e7fa1570 2024-06-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb5e4adb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c00d920 con 0x5566e7f12e40 2024-06-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb5e4adb80 10 client.0 _ll_register_callbacks cb 0x5566e7e82720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:33.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb4be00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f98960 0x5566e7f9b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb4be00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f98960 0x5566e7f9b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52100/0 (socket says 172.21.15.26:52100) 2024-06-13T07:01:33.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb4be00700 1 -- 172.21.15.26:0/148527610 learned_addr learned my addr 172.21.15.26:0/148527610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:33.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.833+0000 7fcb51200700 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 0x5566e7fb2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.834+0000 7fcb51c00700 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.834+0000 7fcb51c00700 1 -- 172.21.15.26:0/148527610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 msgr2=0x5566e7fb2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.834+0000 7fcb51c00700 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 0x5566e7fb2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.834+0000 7fcb51c00700 1 -- 172.21.15.26:0/148527610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f98960 msgr2=0x5566e7f9b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:33.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.834+0000 7fcb51c00700 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f98960 0x5566e7f9b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.835+0000 7fcb51c00700 1 -- 172.21.15.26:0/148527610 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e7f9be30 con 0x5566e7fa1570 2024-06-13T07:01:33.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.835+0000 7fcb4be00700 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f98960 0x5566e7f9b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:33.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.835+0000 7fcb51c00700 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9b8c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb440298c0 tx=0x7fcb4400b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f243cb5538a951db server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:33.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.835+0000 7fcb32a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:33.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.837+0000 7fcb32a00700 1 -- 172.21.15.26:0/148527610 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4403f030 con 0x5566e7fa1570 2024-06-13T07:01:33.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.837+0000 7fcb32a00700 1 -- 172.21.15.26:0/148527610 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcb44004fc0 con 0x5566e7fa1570 2024-06-13T07:01:33.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.837+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/148527610 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb44029bb0 con 0x5566e7fa1570 2024-06-13T07:01:33.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.837+0000 7fcb32a00700 1 -- 172.21.15.26:0/148527610 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb44055050 con 0x5566e7fa1570 2024-06-13T07:01:33.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.837+0000 7fcb5e4adb80 10 client.25328 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:33.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.837+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/148527610 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566e7f9b0f0 con 0x5566e7fa1570 2024-06-13T07:01:33.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.838+0000 7fcb32a00700 1 -- 172.21.15.26:0/148527610 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb44029bb0 con 0x5566e7fa1570 2024-06-13T07:01:33.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.838+0000 7fcb32a00700 1 -- 172.21.15.26:0/148527610 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcb44011040 con 0x5566e7fa1570 2024-06-13T07:01:33.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.838+0000 7fcb5e4adb80 10 client.25328 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:33.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.838+0000 7fcb5e4adb80 10 client.25328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-13T07:01:33.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.838+0000 7fcb5e4adb80 20 client.25328 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:33.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.838+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x5566e8023a80 0x5566e8025ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:33.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.838+0000 7fcb5e4adb80 4 client.25328 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:01:33.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.838+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/148527610 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566e7ef0580 con 0x5566e8023a80 2024-06-13T07:01:33.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.839+0000 7fcb51200700 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x5566e8023a80 0x5566e8025ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:33.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.839+0000 7fcb51200700 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x5566e8023a80 0x5566e8025ea0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35911ee93ec5263e server_cookie=11d590b570c908e9 in_seq=0 out_seq=0 2024-06-13T07:01:33.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.839+0000 7fcb32a00700 10 client.25328 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb32a00700 1 -- 172.21.15.26:0/148527610 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566e7ef0580 con 0x5566e8023a80 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb32a00700 10 client.25328 handle_command_reply: tid=0 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 client.25328 shutdown 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/148527610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 msgr2=0x5566e7f9b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9b8c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb440298c0 tx=0x7fcb4400b7a0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/148527610 shutdown_connections 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x5566e8023a80 0x5566e8025ea0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566e7fa1570 0x5566e7f9b8c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5566e7f12e40 0x5566e7fb2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 --2- 172.21.15.26:0/148527610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566e7f98960 0x5566e7f9b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb32a00700 0 client.25328 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T07:01:33.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/148527610 >> 172.21.15.26:0/148527610 conn(0x5566e7ec3170 msgr2=0x5566e7f9dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.840+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/148527610 shutdown_connections 2024-06-13T07:01:33.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.841+0000 7fcb5e4adb80 1 -- 172.21.15.26:0/148527610 wait complete. 2024-06-13T07:01:33.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.841+0000 7fcb5e4adb80 20 client.25328 trim_cache size 0 max 16384 2024-06-13T07:01:33.841 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-06-13T07:01:33.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 -- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcb340a2d20 msgr2=0x7fcb340a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcb340a2d20 0x7fcb340a51e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400b770 tx=0x7fcb4400b040 comp rx=0 tx=0).stop 2024-06-13T07:01:33.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 -- 172.21.15.26:0/1711409292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb4c1045c0 msgr2=0x7fcb4c1a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:33.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb4c1045c0 0x7fcb4c1a0910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9a0 tx=0x7fcb4000ed60 comp rx=0 tx=0).stop 2024-06-13T07:01:33.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 -- 172.21.15.26:0/1711409292 shutdown_connections 2024-06-13T07:01:33.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcb340a2d20 0x7fcb340a51e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4c10a670 0x7fcb4c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb4c1045c0 0x7fcb4c1a0910 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9a0 tx=0x7fcb4000ed60 comp rx=0 tx=0).stop 2024-06-13T07:01:33.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 --2- 172.21.15.26:0/1711409292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb4c104f80 0x7fcb4c1a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:33.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 -- 172.21.15.26:0/1711409292 >> 172.21.15.26:0/1711409292 conn(0x7fcb4c100070 msgr2=0x7fcb4c1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:33.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 -- 172.21.15.26:0/1711409292 shutdown_connections 2024-06-13T07:01:33.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:33.842+0000 7fcb52600700 1 -- 172.21.15.26:0/1711409292 wait complete. 2024-06-13T07:01:33.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:33 smithi136 ceph-mon[37229]: pgmap v541: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:01:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:33 smithi135 ceph-mon[36276]: pgmap v541: 129 pgs: 129 active+clean; 672 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:01:34.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:01:35.416 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:35.834 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:35 smithi135 ceph-mon[36276]: pgmap v542: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:01:35.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:35 smithi136 ceph-mon[37229]: pgmap v542: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:01:35.916 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:35 smithi026 ceph-mon[32041]: pgmap v542: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:01:37.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.742+0000 7f82dd200700 1 -- 172.21.15.26:0/163781943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 msgr2=0x7f82d8106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.742+0000 7f82dd200700 1 --2- 172.21.15.26:0/163781943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 0x7f82d8106160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009990 tx=0x7f82c4009ca0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.742+0000 7f82dd200700 1 -- 172.21.15.26:0/163781943 shutdown_connections 2024-06-13T07:01:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.742+0000 7f82dd200700 1 --2- 172.21.15.26:0/163781943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82d8106780 0x7f82d8106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.742+0000 7f82dd200700 1 --2- 172.21.15.26:0/163781943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d8101370 0x7f82d8101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.742+0000 7f82dd200700 1 --2- 172.21.15.26:0/163781943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 0x7f82d8106160 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.742+0000 7f82dd200700 1 -- 172.21.15.26:0/163781943 >> 172.21.15.26:0/163781943 conn(0x7f82d80fcdb0 msgr2=0x7f82d80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.742+0000 7f82dd200700 1 -- 172.21.15.26:0/163781943 shutdown_connections 2024-06-13T07:01:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.743+0000 7f82dd200700 1 -- 172.21.15.26:0/163781943 wait complete. 2024-06-13T07:01:37.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.744+0000 7f82dd200700 1 Processor -- start 2024-06-13T07:01:37.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82dd200700 1 -- start start 2024-06-13T07:01:37.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82dd200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82d8101370 0x7f82d819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82dd200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 0x7f82d819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82dd200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d8106780 0x7f82d81a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82dd200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8118d20 con 0x7f82d8106780 2024-06-13T07:01:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82dd200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8118ba0 con 0x7f82d8101c90 2024-06-13T07:01:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82dd200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8118ea0 con 0x7f82d8101370 2024-06-13T07:01:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82d7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82d8101370 0x7f82d819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82d7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82d8101370 0x7f82d819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39956/0 (socket says 172.21.15.26:39956) 2024-06-13T07:01:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82d7400700 1 -- 172.21.15.26:0/3676074665 learned_addr learned my addr 172.21.15.26:0/3676074665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.745+0000 7f82d6a00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 0x7f82d819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.746+0000 7f82d7e00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d8106780 0x7f82d81a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.746+0000 7f82d6a00700 1 -- 172.21.15.26:0/3676074665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82d8101370 msgr2=0x7f82d819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.746+0000 7f82d6a00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82d8101370 0x7f82d819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.746+0000 7f82d6a00700 1 -- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d8106780 msgr2=0x7f82d81a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.746+0000 7f82d6a00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d8106780 0x7f82d81a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.746+0000 7f82d6a00700 1 -- 172.21.15.26:0/3676074665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d81a4170 con 0x7f82d8101c90 2024-06-13T07:01:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.746+0000 7f82d7e00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d8106780 0x7f82d81a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.746+0000 7f82d7400700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82d8101370 0x7f82d819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.747+0000 7f82d6a00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 0x7f82d819f6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f82c40098e0 tx=0x7f82c400b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.747+0000 7f82cfe00700 1 -- 172.21.15.26:0/3676074665 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4027070 con 0x7f82d8101c90 2024-06-13T07:01:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.747+0000 7f82dd200700 1 -- 172.21.15.26:0/3676074665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d81a4400 con 0x7f82d8101c90 2024-06-13T07:01:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.748+0000 7f82cfe00700 1 -- 172.21.15.26:0/3676074665 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c400be40 con 0x7f82d8101c90 2024-06-13T07:01:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.748+0000 7f82dd200700 1 -- 172.21.15.26:0/3676074665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d81abce0 con 0x7f82d8101c90 2024-06-13T07:01:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.748+0000 7f82cfe00700 1 -- 172.21.15.26:0/3676074665 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4022e60 con 0x7f82d8101c90 2024-06-13T07:01:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.753+0000 7f82cfe00700 1 -- 172.21.15.26:0/3676074665 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f82c4009d10 con 0x7f82d8101c90 2024-06-13T07:01:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.754+0000 7f82e920fb80 1 Processor -- start 2024-06-13T07:01:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.754+0000 7f82e920fb80 1 -- start start 2024-06-13T07:01:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.755+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0e6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.755+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0e6680 0x55c70c0f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.755+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0f1530 0x55c70c0f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.755+0000 7f82e920fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c70bff5a10 con 0x55c70c0e5d60 2024-06-13T07:01:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.755+0000 7f82e920fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c70c038b00 con 0x55c70c0f1530 2024-06-13T07:01:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.755+0000 7f82e920fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c70bff38f0 con 0x55c70c0e6680 2024-06-13T07:01:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.755+0000 7f82cfe00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82bc0a2ea0 0x7f82bc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.755+0000 7f82cfe00700 1 -- 172.21.15.26:0/3676074665 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f82c4022c00 con 0x7f82d8101c90 2024-06-13T07:01:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.756+0000 7f82d7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0e6680 0x55c70c0f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.756+0000 7f82d7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0e6680 0x55c70c0f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39960/0 (socket says 172.21.15.26:39960) 2024-06-13T07:01:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.756+0000 7f82d7400700 1 -- 172.21.15.26:0/264624594 learned_addr learned my addr 172.21.15.26:0/264624594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.756+0000 7f82d7e00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82bc0a2ea0 0x7f82bc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.756+0000 7f82d6a00700 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0f1530 0x55c70c0f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.756+0000 7f82d7e00700 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.757+0000 7f82d6a00700 1 -- 172.21.15.26:0/264624594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0e6680 msgr2=0x55c70c0f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.757+0000 7f82d6a00700 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0e6680 0x55c70c0f0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.757+0000 7f82d6a00700 1 -- 172.21.15.26:0/264624594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 msgr2=0x55c70c0e6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.757+0000 7f82d6a00700 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0e6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.757+0000 7f82d6a00700 1 -- 172.21.15.26:0/264624594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70bfe0f90 con 0x55c70c0f1530 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.757+0000 7f82d6a00700 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0f1530 0x55c70c0f3920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82c409c0f0 tx=0x7f82c4022930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d070fbbf117ee882 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.757+0000 7f82ccc00700 1 -- 172.21.15.26:0/264624594 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c409caf0 con 0x55c70c0f1530 2024-06-13T07:01:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.758+0000 7f82d7e00700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82bc0a2ea0 0x7f82bc0a5360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f82d0015b30 tx=0x7f82d0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.758+0000 7f82ccc00700 1 -- 172.21.15.26:0/264624594 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c409c3f0 con 0x55c70c0f1530 2024-06-13T07:01:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.758+0000 7f82ccc00700 1 -- 172.21.15.26:0/264624594 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c40a4b40 con 0x55c70c0f1530 2024-06-13T07:01:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 -- 172.21.15.26:0/264624594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0f1530 msgr2=0x55c70c0f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0f1530 0x55c70c0f3920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82c409c0f0 tx=0x7f82c4022930 comp rx=0 tx=0).stop 2024-06-13T07:01:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 -- 172.21.15.26:0/264624594 shutdown_connections 2024-06-13T07:01:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0f1530 0x55c70c0f3920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0e6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 --2- 172.21.15.26:0/264624594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0e6680 0x55c70c0f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 -- 172.21.15.26:0/264624594 >> 172.21.15.26:0/264624594 conn(0x55c70c013170 msgr2=0x55c70c0e2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 -- 172.21.15.26:0/264624594 shutdown_connections 2024-06-13T07:01:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.759+0000 7f82e920fb80 1 -- 172.21.15.26:0/264624594 wait complete. 2024-06-13T07:01:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.760+0000 7f82e920fb80 1 -- start start 2024-06-13T07:01:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.760+0000 7f82e920fb80 1 Processor -- start 2024-06-13T07:01:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0ee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82e920fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c70bfd4f60 con 0x55c70c0e5d60 2024-06-13T07:01:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82e920fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c70c034ac0 con 0x55c70c0e6680 2024-06-13T07:01:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82e920fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c70c038b00 con 0x55c70c0f1530 2024-06-13T07:01:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82e920fb80 10 client.0 _ll_register_callbacks cb 0x55c70bfce380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56862/0 (socket says 172.21.15.26:56862) 2024-06-13T07:01:37.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d6a00700 1 -- 172.21.15.26:0/67914926 learned_addr learned my addr 172.21.15.26:0/67914926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:37.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d7e00700 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0ee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d7400700 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d6a00700 1 -- 172.21.15.26:0/67914926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 msgr2=0x55c70c0ee5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:37.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d6a00700 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0ee5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d6a00700 1 -- 172.21.15.26:0/67914926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 msgr2=0x55c70c0f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:37.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d6a00700 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.761+0000 7f82d6a00700 1 -- 172.21.15.26:0/67914926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70bf878d0 con 0x55c70c0e6680 2024-06-13T07:01:37.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.762+0000 7f82d6a00700 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f82c4000c00 tx=0x7f82c4098050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e579c263a5155a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:37.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.762+0000 7f82c1600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.762+0000 7f82c1600700 1 -- 172.21.15.26:0/67914926 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c409caf0 con 0x55c70c0e6680 2024-06-13T07:01:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.762+0000 7f82e920fb80 1 -- 172.21.15.26:0/67914926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c409c3f0 con 0x55c70c0e6680 2024-06-13T07:01:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.762+0000 7f82c1600700 1 -- 172.21.15.26:0/67914926 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f82c40a4d50 con 0x55c70c0e6680 2024-06-13T07:01:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.762+0000 7f82c1600700 1 -- 172.21.15.26:0/67914926 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82c40b5220 con 0x55c70c0e6680 2024-06-13T07:01:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.762+0000 7f82e920fb80 10 client.25340 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:37.768 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:37 smithi026 ceph-mon[32041]: pgmap v543: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-13T07:01:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.762+0000 7f82e920fb80 1 -- 172.21.15.26:0/67914926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c70c0fe450 con 0x55c70c0e6680 2024-06-13T07:01:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.763+0000 7f82c1600700 1 -- 172.21.15.26:0/67914926 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c409c3f0 con 0x55c70c0e6680 2024-06-13T07:01:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.763+0000 7f82c1600700 1 -- 172.21.15.26:0/67914926 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f82c40bc0b0 con 0x55c70c0e6680 2024-06-13T07:01:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.763+0000 7f82e920fb80 10 client.25340 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.763+0000 7f82e920fb80 10 client.25340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.763+0000 7f82e920fb80 20 client.25340 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.763+0000 7f82e920fb80 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c70c100920 0x55c70c102d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.763+0000 7f82e920fb80 4 client.25340 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.763+0000 7f82e920fb80 1 -- 172.21.15.26:0/67914926 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c70c046000 con 0x55c70c100920 2024-06-13T07:01:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.764+0000 7f82d7400700 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c70c100920 0x55c70c102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.764+0000 7f82d7400700 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c70c100920 0x55c70c102d10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43517ad027673bc6 server_cookie=6ece418813294337 in_seq=0 out_seq=0 2024-06-13T07:01:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.764+0000 7f82c1600700 10 client.25340 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82c1600700 1 -- 172.21.15.26:0/67914926 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c70c046000 con 0x55c70c100920 2024-06-13T07:01:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82c1600700 10 client.25340 handle_command_reply: tid=0 2024-06-13T07:01:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 client.25340 shutdown 2024-06-13T07:01:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 -- 172.21.15.26:0/67914926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 msgr2=0x55c70c0ee0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f82c4000c00 tx=0x7f82c4098050 comp rx=0 tx=0).stop 2024-06-13T07:01:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 -- 172.21.15.26:0/67914926 shutdown_connections 2024-06-13T07:01:37.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c70c100920 0x55c70c102d10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0ee5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 --2- 172.21.15.26:0/67914926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee0a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.765+0000 7f82e920fb80 1 -- 172.21.15.26:0/67914926 >> 172.21.15.26:0/67914926 conn(0x55c70c013170 msgr2=0x55c70c0e1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:37.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.766+0000 7f82c1600700 0 client.25340 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.766+0000 7f82e920fb80 1 -- 172.21.15.26:0/67914926 shutdown_connections 2024-06-13T07:01:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.766+0000 7f82e920fb80 1 -- 172.21.15.26:0/67914926 wait complete. 2024-06-13T07:01:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.766+0000 7f82e920fb80 20 client.25340 trim_cache size 0 max 16384 2024-06-13T07:01:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.797+0000 7f82e920fb80 1 Processor -- start 2024-06-13T07:01:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.798+0000 7f82e920fb80 1 -- start start 2024-06-13T07:01:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.798+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.798+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.798+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c108370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.798+0000 7f82e920fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81ae950 con 0x55c70c0e5d60 2024-06-13T07:01:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.798+0000 7f82e920fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81ac1e0 con 0x55c70c0e6680 2024-06-13T07:01:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.798+0000 7f82e920fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82d80727e0 con 0x55c70c0f1530 2024-06-13T07:01:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d7400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d7400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0ee060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52162/0 (socket says 172.21.15.26:52162) 2024-06-13T07:01:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d7400700 1 -- 172.21.15.26:0/1544497411 learned_addr learned my addr 172.21.15.26:0/1544497411 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d6a00700 1 --2- 172.21.15.26:0/1544497411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d7400700 1 -- 172.21.15.26:0/1544497411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 msgr2=0x55c70c108370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:01:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d7400700 1 --2- 172.21.15.26:0/1544497411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c108370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d7400700 1 -- 172.21.15.26:0/1544497411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 msgr2=0x55c70c0ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d7400700 1 --2- 172.21.15.26:0/1544497411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.799+0000 7f82d7400700 1 -- 172.21.15.26:0/1544497411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70c0e2d50 con 0x55c70c0e5d60 2024-06-13T07:01:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.800+0000 7f82d7400700 1 --2- 172.21.15.26:0/1544497411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0ee060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f82c8004360 tx=0x7f82c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e835caa0a8df4d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.801+0000 7f82c3e00700 1 -- 172.21.15.26:0/1544497411 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8012d40 con 0x55c70c0e5d60 2024-06-13T07:01:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.801+0000 7f82c3e00700 1 -- 172.21.15.26:0/1544497411 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c80106b0 con 0x55c70c0e5d60 2024-06-13T07:01:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.801+0000 7f82c3e00700 1 -- 172.21.15.26:0/1544497411 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c80038e0 con 0x55c70c0e5d60 2024-06-13T07:01:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 -- 172.21.15.26:0/1544497411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 msgr2=0x55c70c0ee060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 --2- 172.21.15.26:0/1544497411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0ee060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f82c8004360 tx=0x7f82c8004750 comp rx=0 tx=0).stop 2024-06-13T07:01:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 -- 172.21.15.26:0/1544497411 shutdown_connections 2024-06-13T07:01:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 --2- 172.21.15.26:0/1544497411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 --2- 172.21.15.26:0/1544497411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e5d60 0x55c70c0ee060 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 --2- 172.21.15.26:0/1544497411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e6680 0x55c70c0ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 -- 172.21.15.26:0/1544497411 >> 172.21.15.26:0/1544497411 conn(0x55c70c013170 msgr2=0x55c70c0ed950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 -- 172.21.15.26:0/1544497411 shutdown_connections 2024-06-13T07:01:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.802+0000 7f82e920fb80 1 -- 172.21.15.26:0/1544497411 wait complete. 2024-06-13T07:01:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.803+0000 7f82e920fb80 1 -- start start 2024-06-13T07:01:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.803+0000 7f82e920fb80 1 Processor -- start 2024-06-13T07:01:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e5d60 0x55c70c0f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e6680 0x55c70c0f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82e920fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82e920fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8072e50 con 0x55c70c0e6680 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82e920fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82d80742c0 con 0x55c70c0e5d60 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82e920fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8073520 con 0x55c70c0f1530 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82e920fb80 10 client.0 _ll_register_callbacks cb 0x55c70bfce380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82d6a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e6680 0x55c70c0f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82d6a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e6680 0x55c70c0f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52178/0 (socket says 172.21.15.26:52178) 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.804+0000 7f82d6a00700 1 -- 172.21.15.26:0/193323023 learned_addr learned my addr 172.21.15.26:0/193323023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.805+0000 7f82d7e00700 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.805+0000 7f82d7400700 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e5d60 0x55c70c0f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.805+0000 7f82d7e00700 1 -- 172.21.15.26:0/193323023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e5d60 msgr2=0x55c70c0f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.805+0000 7f82d7e00700 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e5d60 0x55c70c0f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.805+0000 7f82d7e00700 1 -- 172.21.15.26:0/193323023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e6680 msgr2=0x55c70c0f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.805+0000 7f82d7e00700 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e6680 0x55c70c0f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.805+0000 7f82d7e00700 1 -- 172.21.15.26:0/193323023 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70c17a600 con 0x55c70c0f1530 2024-06-13T07:01:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.805+0000 7f82d6a00700 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e6680 0x55c70c0f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.806+0000 7f82d7e00700 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0f69e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82d00156e0 tx=0x7f82d0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96521631f5186630 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.806+0000 7f82c1600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.806+0000 7f82c1600700 1 -- 172.21.15.26:0/193323023 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0027030 con 0x55c70c0f1530 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.806+0000 7f82e920fb80 1 -- 172.21.15.26:0/193323023 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c80106b0 con 0x55c70c0f1530 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.806+0000 7f82c1600700 1 -- 172.21.15.26:0/193323023 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f82d0003a00 con 0x55c70c0f1530 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.807+0000 7f82c1600700 1 -- 172.21.15.26:0/193323023 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82d0033050 con 0x55c70c0f1530 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.807+0000 7f82e920fb80 10 client.25390 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.807+0000 7f82e920fb80 1 -- 172.21.15.26:0/193323023 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c70c17b930 con 0x55c70c0f1530 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.807+0000 7f82c1600700 1 -- 172.21.15.26:0/193323023 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c80106b0 con 0x55c70c0f1530 2024-06-13T07:01:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.807+0000 7f82c1600700 1 -- 172.21.15.26:0/193323023 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f82d0022040 con 0x55c70c0f1530 2024-06-13T07:01:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.808+0000 7f82e920fb80 10 client.25390 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.808+0000 7f82e920fb80 10 client.25390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.808+0000 7f82e920fb80 20 client.25390 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.808+0000 7f82e920fb80 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c70c17bff0 0x55c70c17e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.808+0000 7f82e920fb80 4 client.25390 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:01:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.808+0000 7f82e920fb80 1 -- 172.21.15.26:0/193323023 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c70c040580 con 0x55c70c17bff0 2024-06-13T07:01:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.808+0000 7f82d7400700 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c70c17bff0 0x55c70c17e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.809+0000 7f82d7400700 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c70c17bff0 0x55c70c17e410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732aaff32261f39e server_cookie=1e99ce108ce8a4f5 in_seq=0 out_seq=0 2024-06-13T07:01:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.809+0000 7f82c1600700 10 client.25390 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82c1600700 1 -- 172.21.15.26:0/193323023 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c70c040580 con 0x55c70c17bff0 2024-06-13T07:01:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82c1600700 10 client.25390 handle_command_reply: tid=0 2024-06-13T07:01:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 client.25390 shutdown 2024-06-13T07:01:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 -- 172.21.15.26:0/193323023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 msgr2=0x55c70c0f69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0f69e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82d00156e0 tx=0x7f82d0004ad0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 -- 172.21.15.26:0/193323023 shutdown_connections 2024-06-13T07:01:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c70c17bff0 0x55c70c17e410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70c0e5d60 0x55c70c0f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c70c0f1530 0x55c70c0f69e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 --2- 172.21.15.26:0/193323023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c70c0e6680 0x55c70c0f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.810+0000 7f82e920fb80 1 -- 172.21.15.26:0/193323023 >> 172.21.15.26:0/193323023 conn(0x55c70c013170 msgr2=0x55c70c0ee0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.811+0000 7f82c1600700 0 client.25390 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.811+0000 7f82e920fb80 1 -- 172.21.15.26:0/193323023 shutdown_connections 2024-06-13T07:01:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.811+0000 7f82e920fb80 1 -- 172.21.15.26:0/193323023 wait complete. 2024-06-13T07:01:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.811+0000 7f82e920fb80 20 client.25390 trim_cache size 0 max 16384 2024-06-13T07:01:37.812 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.812+0000 7f82dd200700 1 -- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82bc0a2ea0 msgr2=0x7f82bc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.812+0000 7f82dd200700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82bc0a2ea0 0x7f82bc0a5360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f82d0015b30 tx=0x7f82d0009920 comp rx=0 tx=0).stop 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.812+0000 7f82dd200700 1 -- 172.21.15.26:0/3676074665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 msgr2=0x7f82d819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.812+0000 7f82dd200700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 0x7f82d819f6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f82c40098e0 tx=0x7f82c400b880 comp rx=0 tx=0).stop 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.813+0000 7f82dd200700 1 -- 172.21.15.26:0/3676074665 shutdown_connections 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.813+0000 7f82dd200700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f82bc0a2ea0 0x7f82bc0a5360 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.813+0000 7f82dd200700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82d8106780 0x7f82d81a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.813+0000 7f82dd200700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82d8101370 0x7f82d819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.813+0000 7f82dd200700 1 --2- 172.21.15.26:0/3676074665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82d8101c90 0x7f82d819f6f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.813+0000 7f82dd200700 1 -- 172.21.15.26:0/3676074665 >> 172.21.15.26:0/3676074665 conn(0x7f82d80fcdb0 msgr2=0x7f82d8104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.813+0000 7f82dd200700 1 -- 172.21.15.26:0/3676074665 shutdown_connections 2024-06-13T07:01:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:37.813+0000 7f82dd200700 1 -- 172.21.15.26:0/3676074665 wait complete. 2024-06-13T07:01:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:37 smithi136 ceph-mon[37229]: pgmap v543: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-13T07:01:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:37 smithi135 ceph-mon[36276]: pgmap v543: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-13T07:01:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:39 smithi136 ceph-mon[37229]: pgmap v544: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:39 smithi026 ceph-mon[32041]: pgmap v544: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:39 smithi135 ceph-mon[36276]: pgmap v544: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:40.721 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:01:40.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:01:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:41.620 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:41 smithi026 ceph-mon[32041]: pgmap v545: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:41 smithi136 ceph-mon[37229]: pgmap v545: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:42.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:41 smithi135 ceph-mon[36276]: pgmap v545: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:42.920 INFO:tasks.workunit.client.0.smithi026.stderr:Updating files: 58% (7389/12572) Updating files: 59% (7418/12572) Updating files: 60% (7544/12572) Updating files: 61% (7669/12572) Updating files: 62% (7795/12572) Updating files: 63% (7921/12572) Updating files: 64% (8047/12572) Updating files: 65% (8172/12572) Updating files: 66% (8298/12572) Updating files: 67% (8424/12572) Updating files: 68% (8549/12572) Updating files: 69% (8675/12572) Updating files: 70% (8801/12572) Updating files: 71% (8927/12572) Updating files: 72% (9052/12572) Updating files: 73% (9178/12572) Updating files: 74% (9304/12572) Updating files: 75% (9429/12572) Updating files: 76% (9555/12572) Updating files: 77% (9681/12572) Updating files: 78% (9807/12572) Updating files: 79% (9932/12572) Updating files: 80% (10058/12572) Updating files: 81% (10184/12572) Updating files: 82% (10310/12572) Updating files: 83% (10435/12572) Updating files: 84% (10561/12572) Updating files: 85% (10687/12572) Updating files: 86% (10812/12572) Updating files: 87% (10938/12572) Updating files: 88% (11064/12572) Updating files: 89% (11190/12572) Updating files: 90% (11315/12572) Updating files: 91% (11441/12572) Updating files: 92% (11567/12572) Updating files: 92% (11611/12572) Updating files: 93% (11692/12572) Updating files: 94% (11818/12572) Updating files: 95% (11944/12572) Updating files: 96% (12070/12572) Updating files: 97% (12195/12572) Updating files: 98% (12321/12572) Updating files: 99% (12447/12572) Updating files: 100% (12572/12572) Updating files: 100% (12572/12572), done. 2024-06-13T07:01:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:43 smithi026 ceph-mon[32041]: pgmap v546: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.780+0000 7f7048e00700 1 -- 172.21.15.26:0/1913677841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 msgr2=0x7f7044074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.780+0000 7f7048e00700 1 --2- 172.21.15.26:0/1913677841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 0x7f7044074aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7034009990 tx=0x7f7034009ca0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.780+0000 7f7048e00700 1 -- 172.21.15.26:0/1913677841 shutdown_connections 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.780+0000 7f7048e00700 1 --2- 172.21.15.26:0/1913677841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7044113660 0x7f7044115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.781+0000 7f7048e00700 1 --2- 172.21.15.26:0/1913677841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 0x7f7044074aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.781+0000 7f7048e00700 1 --2- 172.21.15.26:0/1913677841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7044074fe0 0x7f7044113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.781+0000 7f7048e00700 1 -- 172.21.15.26:0/1913677841 >> 172.21.15.26:0/1913677841 conn(0x7f70440795f0 msgr2=0x7f7044079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.781+0000 7f7048e00700 1 -- 172.21.15.26:0/1913677841 shutdown_connections 2024-06-13T07:01:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.781+0000 7f7048e00700 1 -- 172.21.15.26:0/1913677841 wait complete. 2024-06-13T07:01:43.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.782+0000 7f7048e00700 1 Processor -- start 2024-06-13T07:01:43.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.782+0000 7f7048e00700 1 -- start start 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.782+0000 7f7048e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 0x7f704419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.782+0000 7f7048e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7044074fe0 0x7f704419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.782+0000 7f7048e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7044113660 0x7f70441a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.782+0000 7f7048e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7044118da0 con 0x7f7044074fe0 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.782+0000 7f7048e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7044118c20 con 0x7f7044113660 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.782+0000 7f7048e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7044118f20 con 0x7f70440746c0 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7044074fe0 0x7f704419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7044074fe0 0x7f704419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53096/0 (socket says 172.21.15.26:53096) 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 0x7f704419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041400700 1 -- 172.21.15.26:0/627539263 learned_addr learned my addr 172.21.15.26:0/627539263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7042800700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7044113660 0x7f70441a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041e00700 1 -- 172.21.15.26:0/627539263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7044113660 msgr2=0x7f70441a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7044113660 0x7f70441a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041e00700 1 -- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7044074fe0 msgr2=0x7f704419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7044074fe0 0x7f704419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041e00700 1 -- 172.21.15.26:0/627539263 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70441a42a0 con 0x7f70440746c0 2024-06-13T07:01:43.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.783+0000 7f7041e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 0x7f704419f290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7034006010 tx=0x7f703400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:43.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.784+0000 7f703aa00700 1 -- 172.21.15.26:0/627539263 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034027070 con 0x7f70440746c0 2024-06-13T07:01:43.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.784+0000 7f703aa00700 1 -- 172.21.15.26:0/627539263 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70340056d0 con 0x7f70440746c0 2024-06-13T07:01:43.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.784+0000 7f7048e00700 1 -- 172.21.15.26:0/627539263 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70441a4530 con 0x7f70440746c0 2024-06-13T07:01:43.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.784+0000 7f703aa00700 1 -- 172.21.15.26:0/627539263 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034022d90 con 0x7f70440746c0 2024-06-13T07:01:43.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.784+0000 7f7048e00700 1 -- 172.21.15.26:0/627539263 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70441abe10 con 0x7f70440746c0 2024-06-13T07:01:43.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.788+0000 7f703aa00700 1 -- 172.21.15.26:0/627539263 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f703402a070 con 0x7f70440746c0 2024-06-13T07:01:43.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.789+0000 7f70536eab80 1 Processor -- start 2024-06-13T07:01:43.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f70536eab80 1 -- start start 2024-06-13T07:01:43.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f703aa00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f70280a2ca0 0x7f70280a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f70536eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad066dec0 0x55dad06f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f70536eab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f70536eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f70536eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dad0600a90 con 0x55dad06fc380 2024-06-13T07:01:43.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f70536eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dad0643b80 con 0x55dad066dec0 2024-06-13T07:01:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f703aa00700 1 -- 172.21.15.26:0/627539263 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f70340d02b0 con 0x7f70440746c0 2024-06-13T07:01:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f70536eab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dad05fe970 con 0x55dad06f1630 2024-06-13T07:01:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.790+0000 7f7042800700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f70280a2ca0 0x7f70280a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43552/0 (socket says 172.21.15.26:43552) 2024-06-13T07:01:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 -- 172.21.15.26:0/1636932937 learned_addr learned my addr 172.21.15.26:0/1636932937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7042800700 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041e00700 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad066dec0 0x55dad06f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 -- 172.21.15.26:0/1636932937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad066dec0 msgr2=0x55dad06f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad066dec0 0x55dad06f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 -- 172.21.15.26:0/1636932937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 msgr2=0x55dad06fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 -- 172.21.15.26:0/1636932937 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad05ec010 con 0x55dad06f1630 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041e00700 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad066dec0 0x55dad06f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.791+0000 7f7041400700 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f1ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7030009df0 tx=0x7f7030011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51829995645981a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.792+0000 7f7042800700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f70280a2ca0 0x7f70280a5160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f703c009ec0 tx=0x7f703c0092f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.792+0000 7f702f400700 1 -- 172.21.15.26:0/1636932937 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030004230 con 0x55dad06f1630 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.792+0000 7f702f400700 1 -- 172.21.15.26:0/1636932937 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7030004d10 con 0x55dad06f1630 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.792+0000 7f702f400700 1 -- 172.21.15.26:0/1636932937 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030003680 con 0x55dad06f1630 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 -- 172.21.15.26:0/1636932937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 msgr2=0x55dad06f1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f1ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7030009df0 tx=0x7f7030011720 comp rx=0 tx=0).stop 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 -- 172.21.15.26:0/1636932937 shutdown_connections 2024-06-13T07:01:43.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad066dec0 0x55dad06f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 --2- 172.21.15.26:0/1636932937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f1ab0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 -- 172.21.15.26:0/1636932937 >> 172.21.15.26:0/1636932937 conn(0x55dad061e1f0 msgr2=0x55dad06ed530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:43.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 -- 172.21.15.26:0/1636932937 shutdown_connections 2024-06-13T07:01:43.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.793+0000 7f70536eab80 1 -- 172.21.15.26:0/1636932937 wait complete. 2024-06-13T07:01:43.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.794+0000 7f70536eab80 1 -- start start 2024-06-13T07:01:43.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.794+0000 7f70536eab80 1 Processor -- start 2024-06-13T07:01:43.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.794+0000 7f70536eab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.794+0000 7f70536eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06f1630 0x55dad06f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.794+0000 7f70536eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.794+0000 7f70536eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dad05dffe0 con 0x55dad06fc380 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f70536eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dad063fb40 con 0x55dad06f1630 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f70536eab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dad06ff040 con 0x55dad066dec0 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f70536eab80 10 client.0 _ll_register_callbacks cb 0x55dad05dd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7042800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06f1630 0x55dad06f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06f1630 0x55dad06f8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35480/0 (socket says 172.21.15.26:35480) 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43562/0 (socket says 172.21.15.26:43562) 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7042800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06f9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53122/0 (socket says 172.21.15.26:53122) 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041400700 1 -- 172.21.15.26:0/744760360 learned_addr learned my addr 172.21.15.26:0/744760360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041400700 1 -- 172.21.15.26:0/744760360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 msgr2=0x55dad06fbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041400700 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041400700 1 -- 172.21.15.26:0/744760360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 msgr2=0x55dad06f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:43.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041400700 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041400700 1 -- 172.21.15.26:0/744760360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad0592950 con 0x55dad06f1630 2024-06-13T07:01:43.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7041e00700 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:43.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.795+0000 7f7042800700 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:43.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.796+0000 7f7041400700 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06f1630 0x55dad06f8df0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7030004690 tx=0x7f7030005400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9303c5d7ed571ac8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:43.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.796+0000 7f7023e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:43.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.796+0000 7f7023e00700 1 -- 172.21.15.26:0/744760360 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030026020 con 0x55dad06f1630 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.796+0000 7f70536eab80 1 -- 172.21.15.26:0/744760360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7030004d10 con 0x55dad06f1630 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.796+0000 7f7023e00700 1 -- 172.21.15.26:0/744760360 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7030004750 con 0x55dad06f1630 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.796+0000 7f7023e00700 1 -- 172.21.15.26:0/744760360 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7030031050 con 0x55dad06f1630 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f70536eab80 10 client.25355 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f70536eab80 1 -- 172.21.15.26:0/744760360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dad0709770 con 0x55dad06f1630 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f7023e00700 1 -- 172.21.15.26:0/744760360 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7030004d10 con 0x55dad06f1630 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f7023e00700 1 -- 172.21.15.26:0/744760360 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7030022400 con 0x55dad06f1630 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f70536eab80 10 client.25355 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f70536eab80 10 client.25355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f70536eab80 20 client.25355 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f70536eab80 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dad070bca0 0x55dad070e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f70536eab80 4 client.25355 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:43.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.797+0000 7f70536eab80 1 -- 172.21.15.26:0/744760360 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dad0651080 con 0x55dad070bca0 2024-06-13T07:01:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.798+0000 7f7041e00700 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dad070bca0 0x55dad070e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.798+0000 7f7041e00700 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dad070bca0 0x55dad070e090 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=666f7e3829d2d581 server_cookie=3c75329872c56ae5 in_seq=0 out_seq=0 2024-06-13T07:01:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.798+0000 7f7023e00700 10 client.25355 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f7023e00700 1 -- 172.21.15.26:0/744760360 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dad0651080 con 0x55dad070bca0 2024-06-13T07:01:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f7023e00700 10 client.25355 handle_command_reply: tid=0 2024-06-13T07:01:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 client.25355 shutdown 2024-06-13T07:01:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 -- 172.21.15.26:0/744760360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06f1630 msgr2=0x55dad06f8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06f1630 0x55dad06f8df0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7030004690 tx=0x7f7030005400 comp rx=0 tx=0).stop 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 -- 172.21.15.26:0/744760360 shutdown_connections 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dad070bca0 0x55dad070e090 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06fc380 0x55dad06f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 --2- 172.21.15.26:0/744760360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06f1630 0x55dad06f8df0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f70536eab80 1 -- 172.21.15.26:0/744760360 >> 172.21.15.26:0/744760360 conn(0x55dad061e1f0 msgr2=0x55dad06ec7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.799+0000 7f7023e00700 0 client.25355 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.800+0000 7f70536eab80 1 -- 172.21.15.26:0/744760360 shutdown_connections 2024-06-13T07:01:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.800+0000 7f70536eab80 1 -- 172.21.15.26:0/744760360 wait complete. 2024-06-13T07:01:43.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.800+0000 7f70536eab80 20 client.25355 trim_cache size 0 max 16384 2024-06-13T07:01:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.822+0000 7f70536eab80 1 Processor -- start 2024-06-13T07:01:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.822+0000 7f70536eab80 1 -- start start 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f70536eab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f70536eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06f1630 0x55dad06fb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f70536eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f70536eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b7350 con 0x55dad06f1630 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f70536eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b4be0 con 0x55dad06fc380 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f70536eab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b2a30 con 0x55dad066dec0 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f7041400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06f1630 0x55dad06fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f7041400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06f1630 0x55dad06fb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53124/0 (socket says 172.21.15.26:53124) 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f7042800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f7041400700 1 -- 172.21.15.26:0/43558731 learned_addr learned my addr 172.21.15.26:0/43558731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.823+0000 7f7041e00700 1 --2- 172.21.15.26:0/43558731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f7041e00700 1 -- 172.21.15.26:0/43558731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 msgr2=0x55dad06f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f7041e00700 1 --2- 172.21.15.26:0/43558731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f7041e00700 1 -- 172.21.15.26:0/43558731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06f1630 msgr2=0x55dad06fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f7041e00700 1 --2- 172.21.15.26:0/43558731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06f1630 0x55dad06fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f7041e00700 1 -- 172.21.15.26:0/43558731 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad06ee100 con 0x55dad066dec0 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f7041e00700 1 --2- 172.21.15.26:0/43558731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fd770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7034000c00 tx=0x7f70340cd830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa654e9b9885076c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f702ea00700 1 -- 172.21.15.26:0/43558731 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703409c290 con 0x55dad066dec0 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f702ea00700 1 -- 172.21.15.26:0/43558731 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7034022470 con 0x55dad066dec0 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.824+0000 7f702ea00700 1 -- 172.21.15.26:0/43558731 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703409c0b0 con 0x55dad066dec0 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 -- 172.21.15.26:0/43558731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 msgr2=0x55dad06fd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 --2- 172.21.15.26:0/43558731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fd770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7034000c00 tx=0x7f70340cd830 comp rx=0 tx=0).stop 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 -- 172.21.15.26:0/43558731 shutdown_connections 2024-06-13T07:01:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 --2- 172.21.15.26:0/43558731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 --2- 172.21.15.26:0/43558731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad066dec0 0x55dad06fd770 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 --2- 172.21.15.26:0/43558731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad06f1630 0x55dad06fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 -- 172.21.15.26:0/43558731 >> 172.21.15.26:0/43558731 conn(0x55dad061e1f0 msgr2=0x55dad06f3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 -- 172.21.15.26:0/43558731 shutdown_connections 2024-06-13T07:01:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.825+0000 7f70536eab80 1 -- 172.21.15.26:0/43558731 wait complete. 2024-06-13T07:01:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.826+0000 7f70536eab80 1 -- start start 2024-06-13T07:01:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.826+0000 7f70536eab80 1 Processor -- start 2024-06-13T07:01:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.826+0000 7f70536eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad066dec0 0x55dad06f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f70536eab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f70536eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f70536eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b30a0 con 0x55dad066dec0 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f70536eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b4510 con 0x55dad06fc380 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f70536eab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b3770 con 0x55dad06f1630 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f70536eab80 10 client.0 _ll_register_callbacks cb 0x55dad05dd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad066dec0 0x55dad06f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad066dec0 0x55dad06f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53132/0 (socket says 172.21.15.26:53132) 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041e00700 1 -- 172.21.15.26:0/2611461372 learned_addr learned my addr 172.21.15.26:0/2611461372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7042800700 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041400700 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041e00700 1 -- 172.21.15.26:0/2611461372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 msgr2=0x55dad06f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041e00700 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041e00700 1 -- 172.21.15.26:0/2611461372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 msgr2=0x55dad06f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041e00700 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7041e00700 1 -- 172.21.15.26:0/2611461372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad0702ba0 con 0x55dad066dec0 2024-06-13T07:01:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.827+0000 7f7042800700 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.828+0000 7f7041e00700 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad066dec0 0x55dad06f5fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7034099120 tx=0x7f70340cd900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da2151f41a9fc20 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.828+0000 7f7023e00700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:01:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.828+0000 7f7023e00700 1 -- 172.21.15.26:0/2611461372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340b9070 con 0x55dad066dec0 2024-06-13T07:01:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.828+0000 7f7023e00700 1 -- 172.21.15.26:0/2611461372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f70340b0be0 con 0x55dad066dec0 2024-06-13T07:01:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.828+0000 7f70536eab80 1 -- 172.21.15.26:0/2611461372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dad064b600 con 0x55dad066dec0 2024-06-13T07:01:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.828+0000 7f7023e00700 1 -- 172.21.15.26:0/2611461372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f703409c0b0 con 0x55dad066dec0 2024-06-13T07:01:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.828+0000 7f70536eab80 10 client.15267 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.828+0000 7f70536eab80 1 -- 172.21.15.26:0/2611461372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dad0776cf0 con 0x55dad066dec0 2024-06-13T07:01:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f7023e00700 1 -- 172.21.15.26:0/2611461372 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dad064b600 con 0x55dad066dec0 2024-06-13T07:01:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f7023e00700 1 -- 172.21.15.26:0/2611461372 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f703409bba0 con 0x55dad066dec0 2024-06-13T07:01:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f70536eab80 10 client.15267 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f70536eab80 10 client.15267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f70536eab80 20 client.15267 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f70536eab80 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dad07772e0 0x55dad0779700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f70536eab80 4 client.15267 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:01:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f70536eab80 1 -- 172.21.15.26:0/2611461372 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dad04c52d0 con 0x55dad07772e0 2024-06-13T07:01:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.829+0000 7f7041400700 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dad07772e0 0x55dad0779700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.830+0000 7f7041400700 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dad07772e0 0x55dad0779700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b886d4fc561f435 server_cookie=a6c02d3e0ab68628 in_seq=0 out_seq=0 2024-06-13T07:01:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.830+0000 7f7023e00700 10 client.15267 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.831+0000 7f7023e00700 1 -- 172.21.15.26:0/2611461372 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dad04c52d0 con 0x55dad07772e0 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.831+0000 7f7023e00700 10 client.15267 handle_command_reply: tid=0 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.831+0000 7f70536eab80 1 client.15267 shutdown 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.831+0000 7f70536eab80 1 -- 172.21.15.26:0/2611461372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad066dec0 msgr2=0x55dad06f5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.831+0000 7f70536eab80 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad066dec0 0x55dad06f5fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7034099120 tx=0x7f70340cd900 comp rx=0 tx=0).stop 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 1 -- 172.21.15.26:0/2611461372 shutdown_connections 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dad07772e0 0x55dad0779700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dad06fc380 0x55dad06f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dad066dec0 0x55dad06f5fc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f7023e00700 0 client.15267 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 1 --2- 172.21.15.26:0/2611461372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dad06f1630 0x55dad06f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 1 -- 172.21.15.26:0/2611461372 >> 172.21.15.26:0/2611461372 conn(0x55dad061e1f0 msgr2=0x55dad06f38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 1 -- 172.21.15.26:0/2611461372 shutdown_connections 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 1 -- 172.21.15.26:0/2611461372 wait complete. 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.832+0000 7f70536eab80 20 client.15267 trim_cache size 0 max 16384 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "d257be87-06c5-440b-9149-0184f6f17163", 2024-06-13T07:01:43.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-06-13T07:01:43.834 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:01:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.833+0000 7f7048e00700 1 -- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f70280a2ca0 msgr2=0x7f70280a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.833+0000 7f7048e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f70280a2ca0 0x7f70280a5160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f703c009ec0 tx=0x7f703c0092f0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.833+0000 7f7048e00700 1 -- 172.21.15.26:0/627539263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 msgr2=0x7f704419f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.833+0000 7f7048e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 0x7f704419f290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7034006010 tx=0x7f703400bab0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.834+0000 7f7048e00700 1 -- 172.21.15.26:0/627539263 shutdown_connections 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.834+0000 7f7048e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f70280a2ca0 0x7f70280a5160 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.834+0000 7f7048e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7044113660 0x7f70441a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.834+0000 7f7048e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70440746c0 0x7f704419f290 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.834+0000 7f7048e00700 1 --2- 172.21.15.26:0/627539263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7044074fe0 0x7f704419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.834+0000 7f7048e00700 1 -- 172.21.15.26:0/627539263 >> 172.21.15.26:0/627539263 conn(0x7f70440795f0 msgr2=0x7f7044112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.834+0000 7f7048e00700 1 -- 172.21.15.26:0/627539263 shutdown_connections 2024-06-13T07:01:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:43.834+0000 7f7048e00700 1 -- 172.21.15.26:0/627539263 wait complete. 2024-06-13T07:01:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:43 smithi136 ceph-mon[37229]: pgmap v546: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:43.896 INFO:tasks.workunit.client.0.smithi026.stderr:Note: switching to '1373963c202323287783ccb099bbae1ec8f14a5d'. 2024-06-13T07:01:43.896 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-06-13T07:01:43.896 INFO:tasks.workunit.client.0.smithi026.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-13T07:01:43.896 INFO:tasks.workunit.client.0.smithi026.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-13T07:01:43.896 INFO:tasks.workunit.client.0.smithi026.stderr:state without impacting any branches by switching back to a branch. 2024-06-13T07:01:43.896 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr: git switch -c 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr:Or undo this operation with: 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr: git switch - 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-06-13T07:01:43.897 INFO:tasks.workunit.client.0.smithi026.stderr:HEAD is now at 1373963c202 Merge pull request #56700 from joscollin/wip-65318-squid 2024-06-13T07:01:43.907 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-13T07:01:44.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:43 smithi135 ceph-mon[36276]: pgmap v546: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-13T07:01:44.152 INFO:tasks.workunit.client.0.smithi026.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-13T07:01:44.163 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-13T07:01:44.164 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-13T07:01:44.841 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:44 smithi026 ceph-mon[32041]: scrub queued for path: / 2024-06-13T07:01:44.841 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:44 smithi026 ceph-mon[32041]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:44.841 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:44 smithi026 ceph-mon[32041]: scrub summary: active paths [/] 2024-06-13T07:01:44.841 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:44 smithi026 ceph-mon[32041]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:44.841 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:44 smithi026 ceph-mon[32041]: scrub summary: idle 2024-06-13T07:01:44.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:44 smithi136 ceph-mon[37229]: scrub queued for path: / 2024-06-13T07:01:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:44 smithi136 ceph-mon[37229]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:44 smithi136 ceph-mon[37229]: scrub summary: active paths [/] 2024-06-13T07:01:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:44 smithi136 ceph-mon[37229]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:44.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:44 smithi136 ceph-mon[37229]: scrub summary: idle 2024-06-13T07:01:45.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:44 smithi135 ceph-mon[36276]: scrub queued for path: / 2024-06-13T07:01:45.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:44 smithi135 ceph-mon[36276]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:45.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:44 smithi135 ceph-mon[36276]: scrub summary: active paths [/] 2024-06-13T07:01:45.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:44 smithi135 ceph-mon[36276]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:45.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:44 smithi135 ceph-mon[36276]: scrub summary: idle 2024-06-13T07:01:45.532 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:01:45.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:45 smithi026 ceph-mon[32041]: pgmap v547: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-13T07:01:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:45 smithi136 ceph-mon[37229]: pgmap v547: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-13T07:01:45.967 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-13T07:01:45.988 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:45 smithi135 ceph-mon[36276]: pgmap v547: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-13T07:01:46.076 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-13T07:01:46.165 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-13T07:01:46.170 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-13T07:01:46.170 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-13T07:01:46.263 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-13T07:01:46.273 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T07:01:46.274 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 -- 172.21.15.26:0/3748933112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d00759e0 msgr2=0x7f60d0073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 --2- 172.21.15.26:0/3748933112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d00759e0 0x7f60d0073e30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009900 tx=0x7f60c002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 -- 172.21.15.26:0/3748933112 shutdown_connections 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 --2- 172.21.15.26:0/3748933112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d01098d0 0x7f60d010bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 --2- 172.21.15.26:0/3748933112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d0074400 0x7f60d0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 --2- 172.21.15.26:0/3748933112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d00759e0 0x7f60d0073e30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 -- 172.21.15.26:0/3748933112 >> 172.21.15.26:0/3748933112 conn(0x7f60d00fcf60 msgr2=0x7f60d00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 -- 172.21.15.26:0/3748933112 shutdown_connections 2024-06-13T07:01:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.289+0000 7f60d5e00640 1 -- 172.21.15.26:0/3748933112 wait complete. 2024-06-13T07:01:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.290+0000 7f60d5e00640 1 Processor -- start 2024-06-13T07:01:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.290+0000 7f60d5e00640 1 -- start start 2024-06-13T07:01:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60d5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d0074400 0x7f60d0199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60d5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d00759e0 0x7f60d019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60d5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d01098d0 0x7f60d01a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60d5e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60d010f870 con 0x7f60d01098d0 2024-06-13T07:01:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60d5e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60d010f6f0 con 0x7f60d0074400 2024-06-13T07:01:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60d5e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60d010f9f0 con 0x7f60d00759e0 2024-06-13T07:01:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60ce000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d00759e0 0x7f60d019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60ce000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d00759e0 0x7f60d019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43604/0 (socket says 172.21.15.26:43604) 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60cf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d01098d0 0x7f60d01a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60ce000640 1 -- 172.21.15.26:0/1912771062 learned_addr learned my addr 172.21.15.26:0/1912771062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.291+0000 7f60cea00640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d0074400 0x7f60d0199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.292+0000 7f60ce000640 1 -- 172.21.15.26:0/1912771062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d0074400 msgr2=0x7f60d0199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.292+0000 7f60ce000640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d0074400 0x7f60d0199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.292+0000 7f60ce000640 1 -- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d01098d0 msgr2=0x7f60d01a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.292+0000 7f60ce000640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d01098d0 0x7f60d01a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.292+0000 7f60ce000640 1 -- 172.21.15.26:0/1912771062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d01a1a90 con 0x7f60d00759e0 2024-06-13T07:01:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.292+0000 7f60ce000640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d00759e0 0x7f60d019a400 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00e9f0 tx=0x7f60bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.292+0000 7f60d4a00640 1 -- 172.21.15.26:0/1912771062 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00cd80 con 0x7f60d00759e0 2024-06-13T07:01:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.293+0000 7f60d4a00640 1 -- 172.21.15.26:0/1912771062 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60bc010910 con 0x7f60d00759e0 2024-06-13T07:01:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.293+0000 7f60d5e00640 1 -- 172.21.15.26:0/1912771062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d01a1d80 con 0x7f60d00759e0 2024-06-13T07:01:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.293+0000 7f60d4a00640 1 -- 172.21.15.26:0/1912771062 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc018990 con 0x7f60d00759e0 2024-06-13T07:01:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.293+0000 7f60d5e00640 1 -- 172.21.15.26:0/1912771062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01a2250 con 0x7f60d00759e0 2024-06-13T07:01:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.296+0000 7f60d8c31740 1 Processor -- start 2024-06-13T07:01:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.296+0000 7f60d4a00640 1 -- 172.21.15.26:0/1912771062 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f60bc010ac0 con 0x7f60d00759e0 2024-06-13T07:01:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60d8c31740 1 -- start start 2024-06-13T07:01:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ebc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60d8c31740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f0e6e1a420 con 0x55f0e6ebc760 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60d8c31740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f0e6dc1c50 con 0x55f0e6eb6c00 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60d8c31740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f0e6e163e0 con 0x55f0e6eb75f0 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60cea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ebc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60cea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ebc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43606/0 (socket says 172.21.15.26:43606) 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60cea00640 1 -- 172.21.15.26:0/3243171732 learned_addr learned my addr 172.21.15.26:0/3243171732 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60ce000640 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.297+0000 7f60cf400640 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.298+0000 7f60cea00640 1 -- 172.21.15.26:0/3243171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 msgr2=0x55f0e6eb7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.298+0000 7f60cea00640 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.298+0000 7f60cea00640 1 -- 172.21.15.26:0/3243171732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 msgr2=0x55f0e6ebebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.298+0000 7f60cea00640 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.298+0000 7f60cea00640 1 -- 172.21.15.26:0/3243171732 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e6b2e210 con 0x55f0e6eb75f0 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.298+0000 7f60cf400640 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.298+0000 7f60cea00640 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ebc0f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60c002a910 tx=0x7f60c0005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f512a34d63817c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.298+0000 7f60d4a00640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60b40a2cb0 0x7f60b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.299+0000 7f60d4a00640 1 -- 172.21.15.26:0/1912771062 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f60bc0cf700 con 0x7f60d00759e0 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.299+0000 7f60cf400640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60b40a2cb0 0x7f60b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.299+0000 7f60bbe00640 1 -- 172.21.15.26:0/3243171732 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0040070 con 0x55f0e6eb75f0 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.299+0000 7f60bbe00640 1 -- 172.21.15.26:0/3243171732 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c0004030 con 0x55f0e6eb75f0 2024-06-13T07:01:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.299+0000 7f60bbe00640 1 -- 172.21.15.26:0/3243171732 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0035040 con 0x55f0e6eb75f0 2024-06-13T07:01:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 -- 172.21.15.26:0/3243171732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb75f0 msgr2=0x55f0e6ebc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ebc0f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60c002a910 tx=0x7f60c0005de0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60cf400640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60b40a2cb0 0x7f60b40a5170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55f0e6eb7e50 tx=0x7f60c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 -- 172.21.15.26:0/3243171732 shutdown_connections 2024-06-13T07:01:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ebc0f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 --2- 172.21.15.26:0/3243171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 -- 172.21.15.26:0/3243171732 >> 172.21.15.26:0/3243171732 conn(0x55f0e6e7efb0 msgr2=0x55f0e6eb2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 -- 172.21.15.26:0/3243171732 shutdown_connections 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.300+0000 7f60d8c31740 1 -- 172.21.15.26:0/3243171732 wait complete. 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 1 -- start start 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 1 Processor -- start 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f0e6ebf6b0 con 0x55f0e6ebc760 2024-06-13T07:01:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f0e6dd50b0 con 0x55f0e6eb75f0 2024-06-13T07:01:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f0e6ebf830 con 0x55f0e6eb6c00 2024-06-13T07:01:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.301+0000 7f60d8c31740 10 client.0 _ll_register_callbacks cb 0x55f0e6dc0f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60cf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35542/0 (socket says 172.21.15.26:35542) 2024-06-13T07:01:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 -- 172.21.15.26:0/4156735349 learned_addr learned my addr 172.21.15.26:0/4156735349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60cf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53176/0 (socket says 172.21.15.26:53176) 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60cea00640 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 -- 172.21.15.26:0/4156735349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb6c00 msgr2=0x55f0e6ec0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 -- 172.21.15.26:0/4156735349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 msgr2=0x55f0e6ec8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 -- 172.21.15.26:0/4156735349 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e6bfa620 con 0x55f0e6eb75f0 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60cf400640 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60ce000640 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec1450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f60bc098020 tx=0x7f60bc09ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4b1043f508f530 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.302+0000 7f60b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60b8c00640 1 -- 172.21.15.26:0/4156735349 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc09bbc0 con 0x55f0e6eb75f0 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60b8c00640 1 -- 172.21.15.26:0/4156735349 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f60bc018b30 con 0x55f0e6eb75f0 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60d8c31740 1 -- 172.21.15.26:0/4156735349 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c0004030 con 0x55f0e6eb75f0 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60b8c00640 1 -- 172.21.15.26:0/4156735349 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60bc09fec0 con 0x55f0e6eb75f0 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60d8c31740 10 client.25373 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60d8c31740 1 -- 172.21.15.26:0/4156735349 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0e6ecb870 con 0x55f0e6eb75f0 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60b8c00640 1 -- 172.21.15.26:0/4156735349 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c0004030 con 0x55f0e6eb75f0 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60b8c00640 1 -- 172.21.15.26:0/4156735349 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f60bc0cfaf0 con 0x55f0e6eb75f0 2024-06-13T07:01:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60d8c31740 10 client.25373 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.303+0000 7f60d8c31740 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.304+0000 7f60d8c31740 20 client.25373 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.304+0000 7f60d8c31740 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f0e6eccf00 0x55f0e6ecf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.304+0000 7f60d8c31740 4 client.25373 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.304+0000 7f60d8c31740 1 -- 172.21.15.26:0/4156735349 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0e6e27b50 con 0x55f0e6eccf00 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.304+0000 7f60cea00640 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f0e6eccf00 0x55f0e6ecf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.304+0000 7f60cea00640 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f0e6eccf00 0x55f0e6ecf2f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923d271110ad5f76 server_cookie=4f7c1ecf7da66e9c in_seq=0 out_seq=0 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.304+0000 7f60b8c00640 10 client.25373 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60b8c00640 1 -- 172.21.15.26:0/4156735349 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f0e6e27b50 con 0x55f0e6eccf00 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60b8c00640 10 client.25373 handle_command_reply: tid=0 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 client.25373 shutdown 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 -- 172.21.15.26:0/4156735349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 msgr2=0x55f0e6ec1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec1450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f60bc098020 tx=0x7f60bc09ba50 comp rx=0 tx=0).stop 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 -- 172.21.15.26:0/4156735349 shutdown_connections 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f0e6eccf00 0x55f0e6ecf2f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec1450 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60b8c00640 0 client.25373 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 --2- 172.21.15.26:0/4156735349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 -- 172.21.15.26:0/4156735349 >> 172.21.15.26:0/4156735349 conn(0x55f0e6e7efb0 msgr2=0x55f0e6eb1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 -- 172.21.15.26:0/4156735349 shutdown_connections 2024-06-13T07:01:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 1 -- 172.21.15.26:0/4156735349 wait complete. 2024-06-13T07:01:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.306+0000 7f60d8c31740 20 client.25373 trim_cache size 0 max 16384 2024-06-13T07:01:46.320 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-06-13T07:01:46.321 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-06-13T07:01:46.322 DEBUG:teuthology.orchestra.run.smithi026:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1373963c202323287783ccb099bbae1ec8f14a5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60d8c31740 1 Processor -- start 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60d8c31740 1 -- start start 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60d8c31740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01af250 con 0x55f0e6eb75f0 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60d8c31740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01acb20 con 0x55f0e6eb6c00 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60d8c31740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01aa970 con 0x55f0e6ebc760 2024-06-13T07:01:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.324+0000 7f60ce000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60ce000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53188/0 (socket says 172.21.15.26:53188) 2024-06-13T07:01:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60ce000640 1 -- 172.21.15.26:0/797557471 learned_addr learned my addr 172.21.15.26:0/797557471 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cf400640 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cea00640 1 -- 172.21.15.26:0/797557471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 msgr2=0x55f0e6ebbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cea00640 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cea00640 1 -- 172.21.15.26:0/797557471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb75f0 msgr2=0x55f0e6ec8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cea00640 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cea00640 1 -- 172.21.15.26:0/797557471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e6ead6d0 con 0x55f0e6eb6c00 2024-06-13T07:01:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cf400640 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60ce000640 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.325+0000 7f60cea00640 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb2810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55f0e6eb3530 tx=0x7f60c0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa9c1e8534dccc6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.326+0000 7f60bb400640 1 -- 172.21.15.26:0/797557471 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0035040 con 0x55f0e6eb6c00 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.326+0000 7f60bb400640 1 -- 172.21.15.26:0/797557471 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c00318b0 con 0x55f0e6eb6c00 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.326+0000 7f60bb400640 1 -- 172.21.15.26:0/797557471 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0040040 con 0x55f0e6eb6c00 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.326+0000 7f60d8c31740 1 -- 172.21.15.26:0/797557471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 msgr2=0x55f0e6eb2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.326+0000 7f60d8c31740 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb2810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55f0e6eb3530 tx=0x7f60c0005fd0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 -- 172.21.15.26:0/797557471 shutdown_connections 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ebbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 --2- 172.21.15.26:0/797557471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb6c00 0x55f0e6eb2810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 -- 172.21.15.26:0/797557471 >> 172.21.15.26:0/797557471 conn(0x55f0e6e7efb0 msgr2=0x55f0e6eb2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 -- 172.21.15.26:0/797557471 shutdown_connections 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 -- 172.21.15.26:0/797557471 wait complete. 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 -- start start 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.327+0000 7f60d8c31740 1 Processor -- start 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60d8c31740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60d8c31740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01aafe0 con 0x55f0e6eb6c00 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60d8c31740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01ac450 con 0x55f0e6eb75f0 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60d8c31740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01ab6b0 con 0x55f0e6ebc760 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60d8c31740 10 client.0 _ll_register_callbacks cb 0x7f60bc0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60ce000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53202/0 (socket says 172.21.15.26:53202) 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cea00640 1 -- 172.21.15.26:0/570234989 learned_addr learned my addr 172.21.15.26:0/570234989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cf400640 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cea00640 1 -- 172.21.15.26:0/570234989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 msgr2=0x55f0e6ec6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cea00640 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cea00640 1 -- 172.21.15.26:0/570234989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 msgr2=0x55f0e6ec5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cea00640 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60cea00640 1 -- 172.21.15.26:0/570234989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e6ec7120 con 0x55f0e6eb6c00 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.328+0000 7f60ce000640 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.329+0000 7f60cea00640 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec55e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60c00096c0 tx=0x7f60c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1f5199fbd33b51 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.329+0000 7f60b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:01:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.329+0000 7f60b8c00640 1 -- 172.21.15.26:0/570234989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c005d060 con 0x55f0e6eb6c00 2024-06-13T07:01:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.329+0000 7f60b8c00640 1 -- 172.21.15.26:0/570234989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f60c0004240 con 0x55f0e6eb6c00 2024-06-13T07:01:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.329+0000 7f60d8c31740 1 -- 172.21.15.26:0/570234989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c00318b0 con 0x55f0e6eb6c00 2024-06-13T07:01:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.329+0000 7f60b8c00640 1 -- 172.21.15.26:0/570234989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60c0040030 con 0x55f0e6eb6c00 2024-06-13T07:01:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.329+0000 7f60d8c31740 10 client.15285 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.329+0000 7f60d8c31740 1 -- 172.21.15.26:0/570234989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0e6ec8430 con 0x55f0e6eb6c00 2024-06-13T07:01:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60b8c00640 1 -- 172.21.15.26:0/570234989 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c00318b0 con 0x55f0e6eb6c00 2024-06-13T07:01:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60b8c00640 1 -- 172.21.15.26:0/570234989 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f60c0004810 con 0x55f0e6eb6c00 2024-06-13T07:01:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60d8c31740 10 client.15285 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60d8c31740 10 client.15285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60d8c31740 20 client.15285 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60d8c31740 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f0e6f37b90 0x55f0e6f39f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60d8c31740 4 client.15285 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60d8c31740 1 -- 172.21.15.26:0/570234989 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0e6e21ea0 con 0x55f0e6f37b90 2024-06-13T07:01:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.330+0000 7f60ce000640 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f0e6f37b90 0x55f0e6f39f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.331+0000 7f60ce000640 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f0e6f37b90 0x55f0e6f39f80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b9c53fbda2c9598 server_cookie=58bc87799fd3f8e3 in_seq=0 out_seq=0 2024-06-13T07:01:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.331+0000 7f60b8c00640 10 client.15285 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60b8c00640 1 -- 172.21.15.26:0/570234989 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f0e6e21ea0 con 0x55f0e6f37b90 2024-06-13T07:01:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60b8c00640 10 client.15285 handle_command_reply: tid=0 2024-06-13T07:01:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 client.15285 shutdown 2024-06-13T07:01:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 -- 172.21.15.26:0/570234989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb6c00 msgr2=0x55f0e6ec55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec55e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60c00096c0 tx=0x7f60c0000bc0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 -- 172.21.15.26:0/570234989 shutdown_connections 2024-06-13T07:01:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f0e6f37b90 0x55f0e6f39f80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f0e6eb75f0 0x55f0e6ec5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0e6ebc760 0x55f0e6ec6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 --2- 172.21.15.26:0/570234989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e6eb6c00 0x55f0e6ec55e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60b8c00640 0 client.15285 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.337+0000 7f60d8c31740 1 -- 172.21.15.26:0/570234989 >> 172.21.15.26:0/570234989 conn(0x55f0e6e7efb0 msgr2=0x55f0e6eb2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.338+0000 7f60d8c31740 1 -- 172.21.15.26:0/570234989 shutdown_connections 2024-06-13T07:01:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.338+0000 7f60d8c31740 1 -- 172.21.15.26:0/570234989 wait complete. 2024-06-13T07:01:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.338+0000 7f60d8c31740 20 client.15285 trim_cache size 0 max 16384 2024-06-13T07:01:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:01:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:01:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:01:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.338+0000 7f60d5e00640 1 -- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60b40a2cb0 msgr2=0x7f60b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.338+0000 7f60d5e00640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60b40a2cb0 0x7f60b40a5170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55f0e6eb7e50 tx=0x7f60c8009500 comp rx=0 tx=0).stop 2024-06-13T07:01:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.338+0000 7f60d5e00640 1 -- 172.21.15.26:0/1912771062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d00759e0 msgr2=0x7f60d019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.338+0000 7f60d5e00640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d00759e0 0x7f60d019a400 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00e9f0 tx=0x7f60bc00eeb0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.339+0000 7f60d5e00640 1 -- 172.21.15.26:0/1912771062 shutdown_connections 2024-06-13T07:01:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.339+0000 7f60d5e00640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f60b40a2cb0 0x7f60b40a5170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.339+0000 7f60d5e00640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d01098d0 0x7f60d01a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.339+0000 7f60d5e00640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60d00759e0 0x7f60d019a400 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.339+0000 7f60d5e00640 1 --2- 172.21.15.26:0/1912771062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d0074400 0x7f60d0199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.339+0000 7f60d5e00640 1 -- 172.21.15.26:0/1912771062 >> 172.21.15.26:0/1912771062 conn(0x7f60d00fcf60 msgr2=0x7f60d01057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.339+0000 7f60d5e00640 1 -- 172.21.15.26:0/1912771062 shutdown_connections 2024-06-13T07:01:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:46.339+0000 7f60d5e00640 1 -- 172.21.15.26:0/1912771062 wait complete. 2024-06-13T07:01:46.379 INFO:tasks.workunit.client.0.smithi026.stderr:+ dbench 1 2024-06-13T07:01:46.392 INFO:tasks.workunit.client.0.smithi026.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-13T07:01:46.392 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-06-13T07:01:46.392 INFO:tasks.workunit.client.0.smithi026.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-13T07:01:46.396 INFO:tasks.workunit.client.0.smithi026.stdout:0 of 1 processes prepared for launch 0 sec 2024-06-13T07:01:46.497 INFO:tasks.workunit.client.0.smithi026.stdout:1 of 1 processes prepared for launch 0 sec 2024-06-13T07:01:46.497 INFO:tasks.workunit.client.0.smithi026.stdout:releasing clients 2024-06-13T07:01:47.497 INFO:tasks.workunit.client.0.smithi026.stdout: 1 789 21.46 MB/sec warmup 1 sec latency 17.625 ms 2024-06-13T07:01:47.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:47 smithi136 ceph-mon[37229]: pgmap v548: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-13T07:01:47.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:47 smithi026 ceph-mon[32041]: pgmap v548: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-13T07:01:48.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:47 smithi135 ceph-mon[36276]: pgmap v548: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-13T07:01:48.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 2477 14.36 MB/sec warmup 2 sec latency 15.855 ms 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 -- 172.21.15.26:0/2170756601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f44104f80 msgr2=0x7f5f44111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/2170756601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f44104f80 0x7f5f44111440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00b0a0 tx=0x7f5f3c00b3b0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 -- 172.21.15.26:0/2170756601 shutdown_connections 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/2170756601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f4410a670 0x7f5f4410aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/2170756601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f44104f80 0x7f5f44111440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/2170756601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f441045c0 0x7f5f44104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 -- 172.21.15.26:0/2170756601 >> 172.21.15.26:0/2170756601 conn(0x7f5f44100070 msgr2=0x7f5f44102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 -- 172.21.15.26:0/2170756601 shutdown_connections 2024-06-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.524+0000 7f5f4aa00700 1 -- 172.21.15.26:0/2170756601 wait complete. 2024-06-13T07:01:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.525+0000 7f5f4aa00700 1 Processor -- start 2024-06-13T07:01:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4aa00700 1 -- start start 2024-06-13T07:01:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f441045c0 0x7f5f441a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4aa00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f44104f80 0x7f5f441a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f4410a670 0x7f5f4419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4aa00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44114930 con 0x7f5f44104f80 2024-06-13T07:01:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4aa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441147b0 con 0x7f5f4410a670 2024-06-13T07:01:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4aa00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44114ab0 con 0x7f5f441045c0 2024-06-13T07:01:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f48c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f44104f80 0x7f5f441a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f4410a670 0x7f5f4419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f4a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f4410a670 0x7f5f4419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35576/0 (socket says 172.21.15.26:35576) 2024-06-13T07:01:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.527+0000 7f5f4a000700 1 -- 172.21.15.26:0/1371778681 learned_addr learned my addr 172.21.15.26:0/1371778681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.527+0000 7f5f49600700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f441045c0 0x7f5f441a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.526+0000 7f5f48c00700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f44104f80 0x7f5f441a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53210/0 (socket says 172.21.15.26:53210) 2024-06-13T07:01:48.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.527+0000 7f5f4a000700 1 -- 172.21.15.26:0/1371778681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f441045c0 msgr2=0x7f5f441a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.527+0000 7f5f4a000700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f441045c0 0x7f5f441a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.527+0000 7f5f4a000700 1 -- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f44104f80 msgr2=0x7f5f441a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.527+0000 7f5f4a000700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f44104f80 0x7f5f441a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.527+0000 7f5f4a000700 1 -- 172.21.15.26:0/1371778681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4419b230 con 0x7f5f4410a670 2024-06-13T07:01:48.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.527+0000 7f5f48c00700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f44104f80 0x7f5f441a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:48.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.528+0000 7f5f4a000700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f4410a670 0x7f5f4419aad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c006070 tx=0x7f5f3c004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:48.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.528+0000 7f5f42000700 1 -- 172.21.15.26:0/1371778681 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c021070 con 0x7f5f4410a670 2024-06-13T07:01:48.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.528+0000 7f5f42000700 1 -- 172.21.15.26:0/1371778681 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c003c00 con 0x7f5f4410a670 2024-06-13T07:01:48.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.528+0000 7f5f4aa00700 1 -- 172.21.15.26:0/1371778681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4419b4c0 con 0x7f5f4410a670 2024-06-13T07:01:48.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.528+0000 7f5f42000700 1 -- 172.21.15.26:0/1371778681 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c004760 con 0x7f5f4410a670 2024-06-13T07:01:48.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.528+0000 7f5f4aa00700 1 -- 172.21.15.26:0/1371778681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f441a7790 con 0x7f5f4410a670 2024-06-13T07:01:48.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.532+0000 7f5f42000700 1 -- 172.21.15.26:0/1371778681 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f3c007980 con 0x7f5f4410a670 2024-06-13T07:01:48.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.533+0000 7f5f42000700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5f2c0a2ca0 0x7f5f2c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.533+0000 7f5f42000700 1 -- 172.21.15.26:0/1371778681 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5f3c00f020 con 0x7f5f4410a670 2024-06-13T07:01:48.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.533+0000 7f5f49600700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5f2c0a2ca0 0x7f5f2c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.534+0000 7f5f569d2b80 1 Processor -- start 2024-06-13T07:01:48.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.534+0000 7f5f569d2b80 1 -- start start 2024-06-13T07:01:48.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.534+0000 7f5f49600700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5f2c0a2ca0 0x7f5f2c0a5160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5f441095e0 tx=0x7f5f38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:48.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf8403420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8403de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf840e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563bf8310a10 con 0x563bf837de40 2024-06-13T07:01:48.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563bf8353b00 con 0x563bf840c570 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563bf830e8f0 con 0x563bf8403960 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f4a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf840e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f4a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf840e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35578/0 (socket says 172.21.15.26:35578) 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f4a000700 1 -- 172.21.15.26:0/780866346 learned_addr learned my addr 172.21.15.26:0/780866346 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f49600700 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf8403420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.535+0000 7f5f48c00700 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8403de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f4a000700 1 -- 172.21.15.26:0/780866346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 msgr2=0x563bf8403de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f4a000700 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8403de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f4a000700 1 -- 172.21.15.26:0/780866346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 msgr2=0x563bf8403420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:48.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f4a000700 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf8403420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f4a000700 1 -- 172.21.15.26:0/780866346 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bf82fbf90 con 0x563bf840c570 2024-06-13T07:01:48.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f48c00700 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8403de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:48.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f4a000700 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf840e960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0995c0 tx=0x7f5f3c004200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7daacd6aba80336 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:48.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f36a00700 1 -- 172.21.15.26:0/780866346 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c09d2f0 con 0x563bf840c570 2024-06-13T07:01:48.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f36a00700 1 -- 172.21.15.26:0/780866346 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c087040 con 0x563bf840c570 2024-06-13T07:01:48.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.536+0000 7f5f36a00700 1 -- 172.21.15.26:0/780866346 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c09af80 con 0x563bf840c570 2024-06-13T07:01:48.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 -- 172.21.15.26:0/780866346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 msgr2=0x563bf840e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf840e960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0995c0 tx=0x7f5f3c004200 comp rx=0 tx=0).stop 2024-06-13T07:01:48.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 -- 172.21.15.26:0/780866346 shutdown_connections 2024-06-13T07:01:48.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf840e960 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf8403420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/780866346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8403de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 -- 172.21.15.26:0/780866346 >> 172.21.15.26:0/780866346 conn(0x563bf832e170 msgr2=0x563bf83fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 -- 172.21.15.26:0/780866346 shutdown_connections 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.537+0000 7f5f569d2b80 1 -- 172.21.15.26:0/780866346 wait complete. 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.538+0000 7f5f569d2b80 1 -- start start 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.538+0000 7f5f569d2b80 1 Processor -- start 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf837de40 0x563bf840bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf8403960 0x563bf8409070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf840c570 0x563bf84095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563bf82eff60 con 0x563bf8403960 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563bf834fac0 con 0x563bf837de40 2024-06-13T07:01:48.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563bf840ef90 con 0x563bf840c570 2024-06-13T07:01:48.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f569d2b80 10 client.0 _ll_register_callbacks cb 0x563bf82ed720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:48.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f49600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf837de40 0x563bf840bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f49600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf837de40 0x563bf840bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35594/0 (socket says 172.21.15.26:35594) 2024-06-13T07:01:48.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.539+0000 7f5f49600700 1 -- 172.21.15.26:0/916881816 learned_addr learned my addr 172.21.15.26:0/916881816 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:48.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.540+0000 7f5f49600700 1 -- 172.21.15.26:0/916881816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf840c570 msgr2=0x563bf84095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.540+0000 7f5f49600700 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf840c570 0x563bf84095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.540+0000 7f5f49600700 1 -- 172.21.15.26:0/916881816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf8403960 msgr2=0x563bf8409070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.540+0000 7f5f49600700 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf8403960 0x563bf8409070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.540+0000 7f5f49600700 1 -- 172.21.15.26:0/916881816 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf82a28d0 con 0x563bf837de40 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.540+0000 7f5f49600700 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf837de40 0x563bf840bd70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f380045b0 tx=0x7f5f380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9508e18d1ffebe2c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.540+0000 7f5f2b400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.540+0000 7f5f2b400700 1 -- 172.21.15.26:0/916881816 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38004830 con 0x563bf837de40 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.541+0000 7f5f2b400700 1 -- 172.21.15.26:0/916881816 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5f380049d0 con 0x563bf837de40 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.541+0000 7f5f569d2b80 1 -- 172.21.15.26:0/916881816 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3c087040 con 0x563bf837de40 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.541+0000 7f5f2b400700 1 -- 172.21.15.26:0/916881816 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f38030050 con 0x563bf837de40 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.541+0000 7f5f569d2b80 10 client.25391 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.541+0000 7f5f569d2b80 1 -- 172.21.15.26:0/916881816 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563bf8419650 con 0x563bf837de40 2024-06-13T07:01:48.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.541+0000 7f5f2b400700 1 -- 172.21.15.26:0/916881816 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c087040 con 0x563bf837de40 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.541+0000 7f5f2b400700 1 -- 172.21.15.26:0/916881816 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5f3801d5b0 con 0x563bf837de40 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.542+0000 7f5f569d2b80 10 client.25391 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.542+0000 7f5f569d2b80 10 client.25391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.542+0000 7f5f569d2b80 20 client.25391 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.542+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563bf841bce0 0x563bf841e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.542+0000 7f5f569d2b80 4 client.25391 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.542+0000 7f5f569d2b80 1 -- 172.21.15.26:0/916881816 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bf8361000 con 0x563bf841bce0 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.542+0000 7f5f48c00700 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563bf841bce0 0x563bf841e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.543+0000 7f5f48c00700 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563bf841bce0 0x563bf841e1a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df49883d2c00289 server_cookie=7f86e31363326931 in_seq=0 out_seq=0 2024-06-13T07:01:48.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.543+0000 7f5f2b400700 10 client.25391 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:48.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f2b400700 1 -- 172.21.15.26:0/916881816 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563bf8361000 con 0x563bf841bce0 2024-06-13T07:01:48.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f2b400700 10 client.25391 handle_command_reply: tid=0 2024-06-13T07:01:48.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 client.25391 shutdown 2024-06-13T07:01:48.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 -- 172.21.15.26:0/916881816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf837de40 msgr2=0x563bf840bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf837de40 0x563bf840bd70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f380045b0 tx=0x7f5f380047c0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 -- 172.21.15.26:0/916881816 shutdown_connections 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563bf841bce0 0x563bf841e1a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf837de40 0x563bf840bd70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf840c570 0x563bf84095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/916881816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf8403960 0x563bf8409070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f2b400700 0 client.25391 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 -- 172.21.15.26:0/916881816 >> 172.21.15.26:0/916881816 conn(0x563bf832e170 msgr2=0x563bf83fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 -- 172.21.15.26:0/916881816 shutdown_connections 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.544+0000 7f5f569d2b80 1 -- 172.21.15.26:0/916881816 wait complete. 2024-06-13T07:01:48.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.545+0000 7f5f569d2b80 20 client.25391 trim_cache size 0 max 16384 2024-06-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.573+0000 7f5f569d2b80 1 Processor -- start 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.573+0000 7f5f569d2b80 1 -- start start 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf840e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf840b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8423a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441b2c10 con 0x563bf837de40 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441b04a0 con 0x563bf840c570 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441ae2f0 con 0x563bf8403960 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f49600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf840e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f48c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf840b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f49600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf840e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53240/0 (socket says 172.21.15.26:53240) 2024-06-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.574+0000 7f5f49600700 1 -- 172.21.15.26:0/3086780005 learned_addr learned my addr 172.21.15.26:0/3086780005 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.575+0000 7f5f4a000700 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8423a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.575+0000 7f5f49600700 1 -- 172.21.15.26:0/3086780005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 msgr2=0x563bf840b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.575+0000 7f5f49600700 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf840b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.575+0000 7f5f49600700 1 -- 172.21.15.26:0/3086780005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 msgr2=0x563bf8423a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.575+0000 7f5f49600700 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8423a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.575+0000 7f5f49600700 1 -- 172.21.15.26:0/3086780005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bf84075e0 con 0x563bf837de40 2024-06-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.575+0000 7f5f4a000700 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8423a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.576+0000 7f5f49600700 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf840e960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f380097c0 tx=0x7f5f380058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee91548754478c84 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.576+0000 7f5f36000700 1 -- 172.21.15.26:0/3086780005 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3801dbf0 con 0x563bf837de40 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.576+0000 7f5f36000700 1 -- 172.21.15.26:0/3086780005 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3801dd90 con 0x563bf837de40 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.576+0000 7f5f36000700 1 -- 172.21.15.26:0/3086780005 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38027470 con 0x563bf837de40 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 -- 172.21.15.26:0/3086780005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 msgr2=0x563bf840e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf840e960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f380097c0 tx=0x7f5f380058c0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 -- 172.21.15.26:0/3086780005 shutdown_connections 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8423a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf840e960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/3086780005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf840b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 -- 172.21.15.26:0/3086780005 >> 172.21.15.26:0/3086780005 conn(0x563bf832e170 msgr2=0x563bf8401aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 -- 172.21.15.26:0/3086780005 shutdown_connections 2024-06-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.577+0000 7f5f569d2b80 1 -- 172.21.15.26:0/3086780005 wait complete. 2024-06-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.578+0000 7f5f569d2b80 1 -- start start 2024-06-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.578+0000 7f5f569d2b80 1 Processor -- start 2024-06-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf84143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8414920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f569d2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8405b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441ae960 con 0x563bf837de40 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441afdd0 con 0x563bf840c570 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f569d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441af030 con 0x563bf8403960 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f4a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8405b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f48c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8414920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f49600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf84143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f569d2b80 10 client.0 _ll_register_callbacks cb 0x563bf82ed720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f4a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8405b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35602/0 (socket says 172.21.15.26:35602) 2024-06-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f4a000700 1 -- 172.21.15.26:0/2578676868 learned_addr learned my addr 172.21.15.26:0/2578676868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f4a000700 1 -- 172.21.15.26:0/2578676868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 msgr2=0x563bf8414920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f4a000700 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8414920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f4a000700 1 -- 172.21.15.26:0/2578676868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 msgr2=0x563bf84143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.579+0000 7f5f4a000700 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf84143e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.580+0000 7f5f4a000700 1 -- 172.21.15.26:0/2578676868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf8406370 con 0x563bf840c570 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.580+0000 7f5f4a000700 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8405b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c004fd0 tx=0x7f5f3c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43618d6de59335dd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.580+0000 7f5f2b400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.580+0000 7f5f2b400700 1 -- 172.21.15.26:0/2578676868 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c080060 con 0x563bf840c570 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.580+0000 7f5f2b400700 1 -- 172.21.15.26:0/2578676868 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5f3c09aef0 con 0x563bf840c570 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.580+0000 7f5f569d2b80 1 -- 172.21.15.26:0/2578676868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3801dd90 con 0x563bf840c570 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.580+0000 7f5f2b400700 1 -- 172.21.15.26:0/2578676868 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f3c0bb070 con 0x563bf840c570 2024-06-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f569d2b80 10 client.25397 fetch_fsmap learned FSMap version 27 2024-06-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f569d2b80 1 -- 172.21.15.26:0/2578676868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563bf84866b0 con 0x563bf840c570 2024-06-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f2b400700 1 -- 172.21.15.26:0/2578676868 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3801dd90 con 0x563bf840c570 2024-06-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f2b400700 1 -- 172.21.15.26:0/2578676868 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5f3c011070 con 0x563bf840c570 2024-06-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f569d2b80 10 client.25397 fetch_fsmap finished waiting for FSMap version 27 2024-06-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f569d2b80 10 client.25397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f569d2b80 20 client.25397 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563bf8486b30 0x563bf8488f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f569d2b80 4 client.25397 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.581+0000 7f5f569d2b80 1 -- 172.21.15.26:0/2578676868 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563bf835b580 con 0x563bf8486b30 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.582+0000 7f5f49600700 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563bf8486b30 0x563bf8488f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.582+0000 7f5f49600700 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563bf8486b30 0x563bf8488f20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff14b133d2acacd1 server_cookie=aeff6effcba5d6b2 in_seq=0 out_seq=0 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.582+0000 7f5f2b400700 10 client.25397 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f2b400700 1 -- 172.21.15.26:0/2578676868 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563bf835b580 con 0x563bf8486b30 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f2b400700 10 client.25397 handle_command_reply: tid=0 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 client.25397 shutdown 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 -- 172.21.15.26:0/2578676868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 msgr2=0x563bf8405b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8405b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c004fd0 tx=0x7f5f3c0099e0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 -- 172.21.15.26:0/2578676868 shutdown_connections 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563bf8486b30 0x563bf8488f20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bf840c570 0x563bf8405b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563bf837de40 0x563bf84143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 --2- 172.21.15.26:0/2578676868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563bf8403960 0x563bf8414920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f2b400700 0 client.25397 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 -- 172.21.15.26:0/2578676868 >> 172.21.15.26:0/2578676868 conn(0x563bf832e170 msgr2=0x563bf8408890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.583+0000 7f5f569d2b80 1 -- 172.21.15.26:0/2578676868 shutdown_connections 2024-06-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.584+0000 7f5f569d2b80 1 -- 172.21.15.26:0/2578676868 wait complete. 2024-06-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.584+0000 7f5f569d2b80 20 client.25397 trim_cache size 0 max 16384 2024-06-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-06-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-06-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 -- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5f2c0a2ca0 msgr2=0x7f5f2c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5f2c0a2ca0 0x7f5f2c0a5160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5f441095e0 tx=0x7f5f38009260 comp rx=0 tx=0).stop 2024-06-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 -- 172.21.15.26:0/1371778681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f4410a670 msgr2=0x7f5f4419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f4410a670 0x7f5f4419aad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c006070 tx=0x7f5f3c004c80 comp rx=0 tx=0).stop 2024-06-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 -- 172.21.15.26:0/1371778681 shutdown_connections 2024-06-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5f2c0a2ca0 0x7f5f2c0a5160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f4410a670 0x7f5f4419aad0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f441045c0 0x7f5f441a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 --2- 172.21.15.26:0/1371778681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f44104f80 0x7f5f441a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 -- 172.21.15.26:0/1371778681 >> 172.21.15.26:0/1371778681 conn(0x7f5f44100070 msgr2=0x7f5f44101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 -- 172.21.15.26:0/1371778681 shutdown_connections 2024-06-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:48.585+0000 7f5f4aa00700 1 -- 172.21.15.26:0/1371778681 wait complete. 2024-06-13T07:01:49.497 INFO:tasks.workunit.client.0.smithi026.stdout: 1 3836 13.25 MB/sec warmup 3 sec latency 15.335 ms 2024-06-13T07:01:49.699 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:01:49.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:01:49.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:49 smithi136 ceph-mon[37229]: pgmap v549: 129 pgs: 129 active+clean; 924 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-06-13T07:01:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:49 smithi026 ceph-mon[32041]: pgmap v549: 129 pgs: 129 active+clean; 924 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-06-13T07:01:50.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:49 smithi135 ceph-mon[36276]: pgmap v549: 129 pgs: 129 active+clean; 924 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-06-13T07:01:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:50.497 INFO:tasks.workunit.client.0.smithi026.stdout: 1 4738 10.53 MB/sec warmup 4 sec latency 34.289 ms 2024-06-13T07:01:51.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 6548 10.53 MB/sec warmup 5 sec latency 19.365 ms 2024-06-13T07:01:51.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:51 smithi026 ceph-mon[32041]: pgmap v550: 129 pgs: 129 active+clean; 1.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-06-13T07:01:51.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:51 smithi136 ceph-mon[37229]: pgmap v550: 129 pgs: 129 active+clean; 1.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-06-13T07:01:52.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:51 smithi135 ceph-mon[36276]: pgmap v550: 129 pgs: 129 active+clean; 1.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-06-13T07:01:52.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 7546 9.86 MB/sec warmup 6 sec latency 18.150 ms 2024-06-13T07:01:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 -- 172.21.15.26:0/1381938951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b90074fe0 msgr2=0x7f8b9010e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 --2- 172.21.15.26:0/1381938951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b90074fe0 0x7f8b9010e950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0098f0 tx=0x7f8b7c009c00 comp rx=0 tx=0).stop 2024-06-13T07:01:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 -- 172.21.15.26:0/1381938951 shutdown_connections 2024-06-13T07:01:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 --2- 172.21.15.26:0/1381938951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b9010ee90 0x7f8b90111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 --2- 172.21.15.26:0/1381938951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b900746c0 0x7f8b90074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 --2- 172.21.15.26:0/1381938951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b90074fe0 0x7f8b9010e950 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 -- 172.21.15.26:0/1381938951 >> 172.21.15.26:0/1381938951 conn(0x7f8b90079640 msgr2=0x7f8b90079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 -- 172.21.15.26:0/1381938951 shutdown_connections 2024-06-13T07:01:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.781+0000 7f8b97e00700 1 -- 172.21.15.26:0/1381938951 wait complete. 2024-06-13T07:01:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.783+0000 7f8b97e00700 1 Processor -- start 2024-06-13T07:01:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.783+0000 7f8b97e00700 1 -- start start 2024-06-13T07:01:52.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b97e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b900746c0 0x7f8b9019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:52.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b97e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90074fe0 0x7f8b9019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:52.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b97e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b9010ee90 0x7f8b9019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:52.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b97e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90114a30 con 0x7f8b900746c0 2024-06-13T07:01:52.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b97e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901148b0 con 0x7f8b9010ee90 2024-06-13T07:01:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b97e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90114bb0 con 0x7f8b90074fe0 2024-06-13T07:01:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b94a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90074fe0 0x7f8b9019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b94a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90074fe0 0x7f8b9019b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56096/0 (socket says 172.21.15.26:56096) 2024-06-13T07:01:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b94a00700 1 -- 172.21.15.26:0/3009990298 learned_addr learned my addr 172.21.15.26:0/3009990298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.784+0000 7f8b95400700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b900746c0 0x7f8b9019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.785+0000 7f8b95e00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b9010ee90 0x7f8b9019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.785+0000 7f8b94a00700 1 -- 172.21.15.26:0/3009990298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b9010ee90 msgr2=0x7f8b9019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.785+0000 7f8b94a00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b9010ee90 0x7f8b9019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.785+0000 7f8b94a00700 1 -- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b900746c0 msgr2=0x7f8b9019af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.785+0000 7f8b94a00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b900746c0 0x7f8b9019af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.785+0000 7f8b94a00700 1 -- 172.21.15.26:0/3009990298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9019ff40 con 0x7f8b90074fe0 2024-06-13T07:01:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.785+0000 7f8b95400700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b900746c0 0x7f8b9019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.786+0000 7f8b94a00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90074fe0 0x7f8b9019b4c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c005950 tx=0x7f8b7c0050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.786+0000 7f8b8e000700 1 -- 172.21.15.26:0/3009990298 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c005390 con 0x7f8b90074fe0 2024-06-13T07:01:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.786+0000 7f8b8e000700 1 -- 172.21.15.26:0/3009990298 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b7c005530 con 0x7f8b90074fe0 2024-06-13T07:01:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.786+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b901a01d0 con 0x7f8b90074fe0 2024-06-13T07:01:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.786+0000 7f8b8e000700 1 -- 172.21.15.26:0/3009990298 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c003c20 con 0x7f8b90074fe0 2024-06-13T07:01:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.787+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b901a7ab0 con 0x7f8b90074fe0 2024-06-13T07:01:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.788+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b90069180 con 0x7f8b90074fe0 2024-06-13T07:01:52.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.791+0000 7f8b8e000700 1 -- 172.21.15.26:0/3009990298 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b7c00b560 con 0x7f8b90074fe0 2024-06-13T07:01:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.793+0000 7f8b8e000700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b780a2ca0 0x7f8b780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.793+0000 7f8b8e000700 1 -- 172.21.15.26:0/3009990298 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8b7c0d0bf0 con 0x7f8b90074fe0 2024-06-13T07:01:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.793+0000 7f8b95400700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b780a2ca0 0x7f8b780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.794+0000 7f8b8e000700 1 -- 172.21.15.26:0/3009990298 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b7c013030 con 0x7f8b90074fe0 2024-06-13T07:01:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:52.795+0000 7f8b95400700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b780a2ca0 0x7f8b780a5160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b84005fd0 tx=0x7f8b84009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.142+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b9019c290 con 0x7f8b90074fe0 2024-06-13T07:01:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.143+0000 7f8b8e000700 1 -- 172.21.15.26:0/3009990298 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f8b7c09d790 con 0x7f8b90074fe0 2024-06-13T07:01:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:01:53.143 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T07:00:22.609623+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":29,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24506,24511,24517]},"id":1}]} 2024-06-13T07:01:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.146+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b780a2ca0 msgr2=0x7f8b780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.146+0000 7f8b97e00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b780a2ca0 0x7f8b780a5160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b84005fd0 tx=0x7f8b84009310 comp rx=0 tx=0).stop 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.146+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90074fe0 msgr2=0x7f8b9019b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.146+0000 7f8b97e00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90074fe0 0x7f8b9019b4c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c005950 tx=0x7f8b7c0050f0 comp rx=0 tx=0).stop 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.146+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 shutdown_connections 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.146+0000 7f8b97e00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b780a2ca0 0x7f8b780a5160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.147+0000 7f8b97e00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b9010ee90 0x7f8b9019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.147+0000 7f8b97e00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b900746c0 0x7f8b9019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.147+0000 7f8b97e00700 1 --2- 172.21.15.26:0/3009990298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b90074fe0 0x7f8b9019b4c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.147+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 >> 172.21.15.26:0/3009990298 conn(0x7f8b90079640 msgr2=0x7f8b900ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.147+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 shutdown_connections 2024-06-13T07:01:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:53.147+0000 7f8b97e00700 1 -- 172.21.15.26:0/3009990298 wait complete. 2024-06-13T07:01:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2024-06-13T07:01:53.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 8963 9.23 MB/sec warmup 7 sec latency 25.641 ms 2024-06-13T07:01:53.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:53 smithi136 ceph-mon[37229]: pgmap v551: 129 pgs: 129 active+clean; 2.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 148 KiB/s wr, 24 op/s 2024-06-13T07:01:53.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:53 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3009990298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:54.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:53 smithi026 ceph-mon[32041]: pgmap v551: 129 pgs: 129 active+clean; 2.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 148 KiB/s wr, 24 op/s 2024-06-13T07:01:54.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:53 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3009990298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:54.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:53 smithi135 ceph-mon[36276]: pgmap v551: 129 pgs: 129 active+clean; 2.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 148 KiB/s wr, 24 op/s 2024-06-13T07:01:54.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:53 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3009990298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:54.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 10457 9.21 MB/sec warmup 8 sec latency 43.888 ms 2024-06-13T07:01:55.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:01:55.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 11216 8.70 MB/sec warmup 9 sec latency 38.104 ms 2024-06-13T07:01:55.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:55 smithi136 ceph-mon[37229]: pgmap v552: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 417 KiB/s wr, 58 op/s 2024-06-13T07:01:55.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:55 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:01:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:56.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:55 smithi026 ceph-mon[32041]: pgmap v552: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 417 KiB/s wr, 58 op/s 2024-06-13T07:01:56.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:55 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:01:56.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:55 smithi135 ceph-mon[36276]: pgmap v552: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 417 KiB/s wr, 58 op/s 2024-06-13T07:01:56.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:55 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:01:56.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 12614 8.38 MB/sec warmup 10 sec latency 59.544 ms 2024-06-13T07:01:57.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 14015 8.44 MB/sec warmup 11 sec latency 20.622 ms 2024-06-13T07:01:57.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:57 smithi136 ceph-mon[37229]: pgmap v553: 129 pgs: 129 active+clean; 5.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 459 KiB/s wr, 60 op/s 2024-06-13T07:01:57.925 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:57 smithi026 ceph-mon[32041]: pgmap v553: 129 pgs: 129 active+clean; 5.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 459 KiB/s wr, 60 op/s 2024-06-13T07:01:58.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:57 smithi135 ceph-mon[36276]: pgmap v553: 129 pgs: 129 active+clean; 5.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 459 KiB/s wr, 60 op/s 2024-06-13T07:01:58.498 INFO:tasks.workunit.client.0.smithi026.stdout: 1 14804 8.12 MB/sec warmup 12 sec latency 29.968 ms 2024-06-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 -- 172.21.15.26:0/227766944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 msgr2=0x7fa67c1157c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 --2- 172.21.15.26:0/227766944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 0x7fa67c1157c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa674009f90 tx=0x7fa67400b330 comp rx=0 tx=0).stop 2024-06-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 -- 172.21.15.26:0/227766944 shutdown_connections 2024-06-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 --2- 172.21.15.26:0/227766944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 0x7fa67c1157c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 --2- 172.21.15.26:0/227766944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa67c074830 0x7fa67c074c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 --2- 172.21.15.26:0/227766944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c075150 0x7fa67c112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 -- 172.21.15.26:0/227766944 >> 172.21.15.26:0/227766944 conn(0x7fa67c0fdf10 msgr2=0x7fa67c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 -- 172.21.15.26:0/227766944 shutdown_connections 2024-06-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.527+0000 7fa683000700 1 -- 172.21.15.26:0/227766944 wait complete. 2024-06-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.529+0000 7fa683000700 1 Processor -- start 2024-06-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.529+0000 7fa683000700 1 -- start start 2024-06-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa683000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c074830 0x7fa67c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa683000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa67c075150 0x7fa67c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa683000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 0x7fa67c1a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa683000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c118d30 con 0x7fa67c075150 2024-06-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa683000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c118bb0 con 0x7fa67c1133d0 2024-06-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa683000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c118eb0 con 0x7fa67c074830 2024-06-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa682600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 0x7fa67c1a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa682600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 0x7fa67c1a3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59348/0 (socket says 172.21.15.26:59348) 2024-06-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa682600700 1 -- 172.21.15.26:0/3425853704 learned_addr learned my addr 172.21.15.26:0/3425853704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa681200700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa67c075150 0x7fa67c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa681c00700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c074830 0x7fa67c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa682600700 1 -- 172.21.15.26:0/3425853704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c074830 msgr2=0x7fa67c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa682600700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c074830 0x7fa67c19f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa682600700 1 -- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa67c075150 msgr2=0x7fa67c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa682600700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa67c075150 0x7fa67c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.530+0000 7fa682600700 1 -- 172.21.15.26:0/3425853704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c1a41b0 con 0x7fa67c1133d0 2024-06-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.531+0000 7fa681200700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa67c075150 0x7fa67c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.531+0000 7fa682600700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 0x7fa67c1a3a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa67400b6c0 tx=0x7fa67400cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.531+0000 7fa67a000700 1 -- 172.21.15.26:0/3425853704 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674021070 con 0x7fa67c1133d0 2024-06-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.531+0000 7fa67a000700 1 -- 172.21.15.26:0/3425853704 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa674007ac0 con 0x7fa67c1133d0 2024-06-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.531+0000 7fa67a000700 1 -- 172.21.15.26:0/3425853704 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6740052a0 con 0x7fa67c1133d0 2024-06-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.531+0000 7fa683000700 1 -- 172.21.15.26:0/3425853704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c1a4440 con 0x7fa67c1133d0 2024-06-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.531+0000 7fa683000700 1 -- 172.21.15.26:0/3425853704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c1abd20 con 0x7fa67c1133d0 2024-06-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.536+0000 7fa68ef64b80 1 Processor -- start 2024-06-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.537+0000 7fa67a000700 1 -- 172.21.15.26:0/3425853704 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa674024070 con 0x7fa67c1133d0 2024-06-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.537+0000 7fa68ef64b80 1 -- start start 2024-06-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9de9e40 0x558ab9e6f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e6f960 0x558ab9e6fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e7a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558ab9d7ca10 con 0x558ab9e6f960 2024-06-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558ab9dbfb00 con 0x558ab9de9e40 2024-06-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ab9d7a8f0 con 0x558ab9e78570 2024-06-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa681200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9de9e40 0x558ab9e6f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa681200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9de9e40 0x558ab9e6f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59364/0 (socket says 172.21.15.26:59364) 2024-06-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa681200700 1 -- 172.21.15.26:0/1000322843 learned_addr learned my addr 172.21.15.26:0/1000322843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.538+0000 7fa681200700 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e7a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa681200700 1 -- 172.21.15.26:0/1000322843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9de9e40 msgr2=0x558ab9e6f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa681200700 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9de9e40 0x558ab9e6f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa681200700 1 -- 172.21.15.26:0/1000322843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e6f960 msgr2=0x558ab9e6fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa681c00700 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e6f960 0x558ab9e6fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa681200700 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e6f960 0x558ab9e6fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa681200700 1 -- 172.21.15.26:0/1000322843 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab9d67f90 con 0x558ab9e78570 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa681c00700 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e6f960 0x558ab9e6fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa67a000700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2ca0 0x7fa6640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa681200700 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e7a960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa668012880 tx=0x7fa6680047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e408b86260e3701 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa66ea00700 1 -- 172.21.15.26:0/1000322843 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680120d0 con 0x558ab9e78570 2024-06-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.539+0000 7fa67a000700 1 -- 172.21.15.26:0/3425853704 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa67400f020 con 0x7fa67c1133d0 2024-06-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.540+0000 7fa66ea00700 1 -- 172.21.15.26:0/1000322843 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa668012510 con 0x558ab9e78570 2024-06-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.540+0000 7fa66ea00700 1 -- 172.21.15.26:0/1000322843 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66800f5e0 con 0x558ab9e78570 2024-06-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.540+0000 7fa681c00700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2ca0 0x7fa6640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1000322843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 msgr2=0x558ab9e7a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e7a960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa668012880 tx=0x7fa6680047c0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1000322843 shutdown_connections 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9de9e40 0x558ab9e6f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e7a960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/1000322843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e6f960 0x558ab9e6fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1000322843 >> 172.21.15.26:0/1000322843 conn(0x558ab9d9a170 msgr2=0x558ab9e69820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa681c00700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2ca0 0x7fa6640a5160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x558ab9e6a590 tx=0x7fa670009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1000322843 shutdown_connections 2024-06-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.541+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1000322843 wait complete. 2024-06-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.542+0000 7fa68ef64b80 1 -- start start 2024-06-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.542+0000 7fa68ef64b80 1 Processor -- start 2024-06-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.542+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 0x558ab9e77d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.542+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e6f960 0x558ab9e75050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.542+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e78570 0x558ab9e75590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.542+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558ab9d5bf60 con 0x558ab9de9e40 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.542+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558ab9dbbac0 con 0x558ab9e78570 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.542+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ab9e7b120 con 0x558ab9e6f960 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa68ef64b80 10 client.0 _ll_register_callbacks cb 0x558ab9d59720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e78570 0x558ab9e75590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e78570 0x558ab9e75590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59368/0 (socket says 172.21.15.26:59368) 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 -- 172.21.15.26:0/553459600 learned_addr learned my addr 172.21.15.26:0/553459600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa681c00700 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 0x558ab9e77d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa681200700 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e6f960 0x558ab9e75050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 -- 172.21.15.26:0/553459600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e6f960 msgr2=0x558ab9e75050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e6f960 0x558ab9e75050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 -- 172.21.15.26:0/553459600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 msgr2=0x558ab9e77d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 0x558ab9e77d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 -- 172.21.15.26:0/553459600 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab9d0e8d0 con 0x558ab9e78570 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa681200700 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e6f960 0x558ab9e75050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa682600700 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e78570 0x558ab9e75590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6740095d0 tx=0x7fa67409c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee5f02ef05faecbb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.543+0000 7fa663400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa663400700 1 -- 172.21.15.26:0/553459600 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67409d500 con 0x558ab9e78570 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa663400700 1 -- 172.21.15.26:0/553459600 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa6740a5550 con 0x558ab9e78570 2024-06-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 1 -- 172.21.15.26:0/553459600 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa668012510 con 0x558ab9e78570 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa663400700 1 -- 172.21.15.26:0/553459600 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa67409ce00 con 0x558ab9e78570 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 10 client.25415 fetch_fsmap learned FSMap version 28 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 1 -- 172.21.15.26:0/553459600 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ab9e85850 con 0x558ab9e78570 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa663400700 1 -- 172.21.15.26:0/553459600 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa668012510 con 0x558ab9e78570 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa663400700 1 -- 172.21.15.26:0/553459600 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fa674011070 con 0x558ab9e78570 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 10 client.25415 fetch_fsmap finished waiting for FSMap version 28 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 10 client.25415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 20 client.25415 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x558ab9e87eb0 0x558ab9e8a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 4 client.25415 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.544+0000 7fa68ef64b80 1 -- 172.21.15.26:0/553459600 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ab9dcd000 con 0x558ab9e87eb0 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.545+0000 7fa681c00700 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x558ab9e87eb0 0x558ab9e8a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.545+0000 7fa681c00700 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x558ab9e87eb0 0x558ab9e8a2d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3396827e6e492c48 server_cookie=beca3a452249397c in_seq=0 out_seq=0 2024-06-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.545+0000 7fa663400700 10 client.25415 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa663400700 1 -- 172.21.15.26:0/553459600 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558ab9dcd000 con 0x558ab9e87eb0 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa663400700 10 client.25415 handle_command_reply: tid=0 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 client.25415 shutdown 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 -- 172.21.15.26:0/553459600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e78570 msgr2=0x558ab9e75590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e78570 0x558ab9e75590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6740095d0 tx=0x7fa67409c760 comp rx=0 tx=0).stop 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 -- 172.21.15.26:0/553459600 shutdown_connections 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x558ab9e87eb0 0x558ab9e8a2d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e78570 0x558ab9e75590 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 0x558ab9e77d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/553459600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e6f960 0x558ab9e75050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.546+0000 7fa68ef64b80 1 -- 172.21.15.26:0/553459600 >> 172.21.15.26:0/553459600 conn(0x558ab9d9a170 msgr2=0x558ab9e69410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.547+0000 7fa663400700 0 client.25415 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.547+0000 7fa68ef64b80 1 -- 172.21.15.26:0/553459600 shutdown_connections 2024-06-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.547+0000 7fa68ef64b80 1 -- 172.21.15.26:0/553459600 wait complete. 2024-06-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.547+0000 7fa68ef64b80 20 client.25415 trim_cache size 0 max 16384 2024-06-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.566+0000 7fa68ef64b80 1 Processor -- start 2024-06-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.566+0000 7fa68ef64b80 1 -- start start 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 0x558ab9e77890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e8fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b71e0 con 0x558ab9de9e40 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b4a70 con 0x558ab9e6f960 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b28c0 con 0x558ab9e78570 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa681200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e8fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa682600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa681200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e8fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59376/0 (socket says 172.21.15.26:59376) 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa681c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 0x558ab9e77890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.567+0000 7fa681200700 1 -- 172.21.15.26:0/1721121608 learned_addr learned my addr 172.21.15.26:0/1721121608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.568+0000 7fa681200700 1 -- 172.21.15.26:0/1721121608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 msgr2=0x558ab9e900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.568+0000 7fa681200700 1 --2- 172.21.15.26:0/1721121608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.568+0000 7fa681200700 1 -- 172.21.15.26:0/1721121608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 msgr2=0x558ab9e77890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.568+0000 7fa681200700 1 --2- 172.21.15.26:0/1721121608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 0x558ab9e77890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.568+0000 7fa681200700 1 -- 172.21.15.26:0/1721121608 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab9e8a8b0 con 0x558ab9e6f960 2024-06-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.568+0000 7fa682600700 1 --2- 172.21.15.26:0/1721121608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.568+0000 7fa681200700 1 --2- 172.21.15.26:0/1721121608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e8fba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa668012820 tx=0x7fa668009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57836bf98c3f42fc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.568+0000 7fa66e000700 1 -- 172.21.15.26:0/1721121608 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680120d0 con 0x558ab9e6f960 2024-06-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa66e000700 1 -- 172.21.15.26:0/1721121608 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa668012270 con 0x558ab9e6f960 2024-06-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa66e000700 1 -- 172.21.15.26:0/1721121608 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680106a0 con 0x558ab9e6f960 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1721121608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 msgr2=0x558ab9e8fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/1721121608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e8fba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa668012820 tx=0x7fa668009650 comp rx=0 tx=0).stop 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1721121608 shutdown_connections 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/1721121608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9e78570 0x558ab9e900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/1721121608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9de9e40 0x558ab9e77890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/1721121608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e8fba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.569+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1721121608 >> 172.21.15.26:0/1721121608 conn(0x558ab9d9a170 msgr2=0x558ab9e73960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.570+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1721121608 shutdown_connections 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.570+0000 7fa68ef64b80 1 -- 172.21.15.26:0/1721121608 wait complete. 2024-06-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.570+0000 7fa68ef64b80 1 -- start start 2024-06-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.570+0000 7fa68ef64b80 1 Processor -- start 2024-06-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9de9e40 0x558ab9e72da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e803e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa68ef64b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e78570 0x558ab9e80b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b2f30 con 0x558ab9e78570 2024-06-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b43a0 con 0x558ab9e6f960 2024-06-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa68ef64b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b3600 con 0x558ab9de9e40 2024-06-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa68ef64b80 10 client.0 _ll_register_callbacks cb 0x558ab9d59720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa681c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9de9e40 0x558ab9e72da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa681c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9de9e40 0x558ab9e72da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56154/0 (socket says 172.21.15.26:56154) 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa681c00700 1 -- 172.21.15.26:0/317713826 learned_addr learned my addr 172.21.15.26:0/317713826 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa681200700 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e803e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.571+0000 7fa682600700 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e78570 0x558ab9e80b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa681c00700 1 -- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 msgr2=0x558ab9e803e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa681c00700 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e803e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa681c00700 1 -- 172.21.15.26:0/317713826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e78570 msgr2=0x558ab9e80b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa681c00700 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e78570 0x558ab9e80b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa681c00700 1 -- 172.21.15.26:0/317713826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab9e713b0 con 0x558ab9de9e40 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa681200700 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e803e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa681c00700 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9de9e40 0x558ab9e72da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa670011aa0 tx=0x7fa670004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f116e8ffe9de236f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa663400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa663400700 1 -- 172.21.15.26:0/317713826 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670011bf0 con 0x558ab9de9e40 2024-06-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa68ef64b80 1 -- 172.21.15.26:0/317713826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa668012270 con 0x558ab9de9e40 2024-06-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.572+0000 7fa663400700 1 -- 172.21.15.26:0/317713826 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa670004020 con 0x558ab9de9e40 2024-06-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.573+0000 7fa663400700 1 -- 172.21.15.26:0/317713826 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa67003a050 con 0x558ab9de9e40 2024-06-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.573+0000 7fa68ef64b80 10 client.25462 fetch_fsmap learned FSMap version 28 2024-06-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.573+0000 7fa68ef64b80 1 -- 172.21.15.26:0/317713826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ab9ef1f30 con 0x558ab9de9e40 2024-06-13T07:01:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.573+0000 7fa663400700 1 -- 172.21.15.26:0/317713826 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa668012270 con 0x558ab9de9e40 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.573+0000 7fa663400700 1 -- 172.21.15.26:0/317713826 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fa670029390 con 0x558ab9de9e40 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.573+0000 7fa68ef64b80 10 client.25462 fetch_fsmap finished waiting for FSMap version 28 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.573+0000 7fa68ef64b80 10 client.25462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.573+0000 7fa68ef64b80 20 client.25462 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.574+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x558ab9ef2620 0x558ab9ef4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.574+0000 7fa68ef64b80 4 client.25462 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.574+0000 7fa68ef64b80 1 -- 172.21.15.26:0/317713826 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ab9dc7580 con 0x558ab9ef2620 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.574+0000 7fa681200700 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x558ab9ef2620 0x558ab9ef4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa681200700 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x558ab9ef2620 0x558ab9ef4a40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25aef6d76ce44abf server_cookie=6348bcc56ba10e07 in_seq=0 out_seq=0 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa663400700 10 client.25462 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa663400700 1 -- 172.21.15.26:0/317713826 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ab9dc7580 con 0x558ab9ef2620 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa663400700 10 client.25462 handle_command_reply: tid=0 2024-06-13T07:01:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 client.25462 shutdown 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 -- 172.21.15.26:0/317713826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9de9e40 msgr2=0x558ab9e72da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9de9e40 0x558ab9e72da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa670011aa0 tx=0x7fa670004750 comp rx=0 tx=0).stop 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 -- 172.21.15.26:0/317713826 shutdown_connections 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x558ab9ef2620 0x558ab9ef4a40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ab9e78570 0x558ab9e80b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ab9de9e40 0x558ab9e72da0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 --2- 172.21.15.26:0/317713826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ab9e6f960 0x558ab9e803e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa68ef64b80 1 -- 172.21.15.26:0/317713826 >> 172.21.15.26:0/317713826 conn(0x558ab9d9a170 msgr2=0x558ab9e75550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.575+0000 7fa663400700 0 client.25462 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.576+0000 7fa68ef64b80 1 -- 172.21.15.26:0/317713826 shutdown_connections 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.576+0000 7fa68ef64b80 1 -- 172.21.15.26:0/317713826 wait complete. 2024-06-13T07:01:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.576+0000 7fa68ef64b80 20 client.25462 trim_cache size 0 max 16384 2024-06-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 -- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2ca0 msgr2=0x7fa6640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2ca0 0x7fa6640a5160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x558ab9e6a590 tx=0x7fa670009310 comp rx=0 tx=0).stop 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 -- 172.21.15.26:0/3425853704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 msgr2=0x7fa67c1a3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 0x7fa67c1a3a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa67400b6c0 tx=0x7fa67400cfa0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 -- 172.21.15.26:0/3425853704 shutdown_connections 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa6640a2ca0 0x7fa6640a5160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c1133d0 0x7fa67c1a3a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c074830 0x7fa67c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 --2- 172.21.15.26:0/3425853704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa67c075150 0x7fa67c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 -- 172.21.15.26:0/3425853704 >> 172.21.15.26:0/3425853704 conn(0x7fa67c0fdf10 msgr2=0x7fa67c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 -- 172.21.15.26:0/3425853704 shutdown_connections 2024-06-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:01:58.577+0000 7fa683000700 1 -- 172.21.15.26:0/3425853704 wait complete. 2024-06-13T07:01:59.499 INFO:tasks.workunit.client.0.smithi026.stdout: 1 15971 7.90 MB/sec warmup 13 sec latency 31.745 ms 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 -- 172.21.15.26:0/3560860871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c1013f0 msgr2=0x7f126c101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 --2- 172.21.15.26:0/3560860871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c1013f0 0x7f126c101870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1264009ca0 tx=0x7f126402a8b0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 -- 172.21.15.26:0/3560860871 shutdown_connections 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 --2- 172.21.15.26:0/3560860871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f126c100a50 0x7f126c100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 --2- 172.21.15.26:0/3560860871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c1013f0 0x7f126c101870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 --2- 172.21.15.26:0/3560860871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126c1000f0 0x7f126c100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 -- 172.21.15.26:0/3560860871 >> 172.21.15.26:0/3560860871 conn(0x7f126c0fbc90 msgr2=0x7f126c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 -- 172.21.15.26:0/3560860871 shutdown_connections 2024-06-13T07:01:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.513+0000 7f126b200640 1 -- 172.21.15.26:0/3560860871 wait complete. 2024-06-13T07:01:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.515+0000 7f126b200640 1 Processor -- start 2024-06-13T07:01:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.515+0000 7f126b200640 1 -- start start 2024-06-13T07:01:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f126b200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f126c1000f0 0x7f126c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f126b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c100a50 0x7f126c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f126b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126c1013f0 0x7f126c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f126b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10f690 con 0x7f126c1013f0 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f126b200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10f510 con 0x7f126c1000f0 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f126b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10f810 con 0x7f126c100a50 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f1269e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f126c1000f0 0x7f126c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f1269400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c100a50 0x7f126c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f1269e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f126c1000f0 0x7f126c199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59404/0 (socket says 172.21.15.26:59404) 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f1269400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c100a50 0x7f126c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56162/0 (socket says 172.21.15.26:56162) 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f1269e00640 1 -- 172.21.15.26:0/1457709917 learned_addr learned my addr 172.21.15.26:0/1457709917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f126a800640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126c1013f0 0x7f126c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f1269e00640 1 -- 172.21.15.26:0/1457709917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c100a50 msgr2=0x7f126c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f1269e00640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c100a50 0x7f126c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.516+0000 7f1269e00640 1 -- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126c1013f0 msgr2=0x7f126c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.517+0000 7f1269e00640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126c1013f0 0x7f126c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.517+0000 7f1269e00640 1 -- 172.21.15.26:0/1457709917 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c1a1930 con 0x7f126c1000f0 2024-06-13T07:01:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.517+0000 7f1269e00640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f126c1000f0 0x7f126c199d20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f125c00e9f0 tx=0x7f125c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.517+0000 7f1262a00640 1 -- 172.21.15.26:0/1457709917 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c00cd80 con 0x7f126c1000f0 2024-06-13T07:01:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.517+0000 7f1262a00640 1 -- 172.21.15.26:0/1457709917 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125c010910 con 0x7f126c1000f0 2024-06-13T07:01:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.517+0000 7f1262a00640 1 -- 172.21.15.26:0/1457709917 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c018990 con 0x7f126c1000f0 2024-06-13T07:01:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.517+0000 7f126b200640 1 -- 172.21.15.26:0/1457709917 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c1a1c20 con 0x7f126c1000f0 2024-06-13T07:01:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.517+0000 7f126b200640 1 -- 172.21.15.26:0/1457709917 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c1a21c0 con 0x7f126c1000f0 2024-06-13T07:01:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.521+0000 7f1273f79740 1 Processor -- start 2024-06-13T07:01:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.521+0000 7f1273f79740 1 -- start start 2024-06-13T07:01:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.521+0000 7f1273f79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.521+0000 7f1273f79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb8eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.521+0000 7f1273f79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.521+0000 7f1273f79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f74cae4420 con 0x55f74cb89fc0 2024-06-13T07:01:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.521+0000 7f1273f79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f74ca8bc50 con 0x55f74cb895d0 2024-06-13T07:01:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.521+0000 7f1273f79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f74cae03e0 con 0x55f74cb8f130 2024-06-13T07:01:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f1262a00640 1 -- 172.21.15.26:0/1457709917 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f125c010ac0 con 0x7f126c1000f0 2024-06-13T07:01:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f126a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb899f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f126a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb899f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59414/0 (socket says 172.21.15.26:59414) 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f126a800640 1 -- 172.21.15.26:0/2692041412 learned_addr learned my addr 172.21.15.26:0/2692041412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f1269400640 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f1269e00640 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb8eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f126a800640 1 -- 172.21.15.26:0/2692041412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 msgr2=0x55f74cb915c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f126a800640 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb915c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f126a800640 1 -- 172.21.15.26:0/2692041412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 msgr2=0x55f74cb8eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f126a800640 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb8eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f126a800640 1 -- 172.21.15.26:0/2692041412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74c7f8210 con 0x55f74cb895d0 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.522+0000 7f1269400640 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb915c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.523+0000 7f126a800640 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb899f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1264009ca0 tx=0x7f1264009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cff80d0e0c7f7e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.523+0000 7f1257400640 1 -- 172.21.15.26:0/2692041412 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264030a30 con 0x55f74cb895d0 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.523+0000 7f1257400640 1 -- 172.21.15.26:0/2692041412 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1264036030 con 0x55f74cb895d0 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.523+0000 7f1257400640 1 -- 172.21.15.26:0/2692041412 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264004630 con 0x55f74cb895d0 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1262a00640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f12500a2c00 0x7f12500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1262a00640 1 -- 172.21.15.26:0/1457709917 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f125c0cf790 con 0x7f126c1000f0 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 -- 172.21.15.26:0/2692041412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 msgr2=0x55f74cb899f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb899f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1264009ca0 tx=0x7f1264009a20 comp rx=0 tx=0).stop 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1269e00640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f12500a2c00 0x7f12500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 -- 172.21.15.26:0/2692041412 shutdown_connections 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb915c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb8eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 --2- 172.21.15.26:0/2692041412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb899f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 -- 172.21.15.26:0/2692041412 >> 172.21.15.26:0/2692041412 conn(0x55f74cb48fb0 msgr2=0x55f74cb85050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 -- 172.21.15.26:0/2692041412 shutdown_connections 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.524+0000 7f1273f79740 1 -- 172.21.15.26:0/2692041412 wait complete. 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.525+0000 7f1269e00640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f12500a2c00 0x7f12500a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f125c09b950 tx=0x7f125c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.525+0000 7f1273f79740 1 -- start start 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.525+0000 7f1273f79740 1 Processor -- start 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1273f79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1273f79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb93e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1273f79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb8f130 0x55f74cb9ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1273f79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f74cb92080 con 0x55f74cb8f130 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1273f79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f74ca9f0b0 con 0x55f74cb895d0 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1273f79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f74cb92200 con 0x55f74cb89fc0 2024-06-13T07:01:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1273f79740 10 client.0 _ll_register_callbacks cb 0x55f74ca8af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1269400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb93e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f126a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1269400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb93e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56176/0 (socket says 172.21.15.26:56176) 2024-06-13T07:01:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f1269400640 1 -- 172.21.15.26:0/927261134 learned_addr learned my addr 172.21.15.26:0/927261134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.526+0000 7f126a800640 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb8f130 0x55f74cb9ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.527+0000 7f1269400640 1 -- 172.21.15.26:0/927261134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 msgr2=0x55f74cb938d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.527+0000 7f1269400640 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb938d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.527+0000 7f1269400640 1 -- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb8f130 msgr2=0x55f74cb9ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.527+0000 7f1269400640 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb8f130 0x55f74cb9ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.527+0000 7f1269400640 1 -- 172.21.15.26:0/927261134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74c8c4620 con 0x55f74cb89fc0 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.527+0000 7f126a800640 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb8f130 0x55f74cb9ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.527+0000 7f1269400640 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb93e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1258012740 tx=0x7f1258012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e313aa5cb45646 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.527+0000 7f124be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.528+0000 7f124be00640 1 -- 172.21.15.26:0/927261134 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125800f8c0 con 0x55f74cb89fc0 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.528+0000 7f124be00640 1 -- 172.21.15.26:0/927261134 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f125800fa60 con 0x55f74cb89fc0 2024-06-13T07:01:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.528+0000 7f1273f79740 1 -- 172.21.15.26:0/927261134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1264036030 con 0x55f74cb89fc0 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.528+0000 7f124be00640 1 -- 172.21.15.26:0/927261134 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f125800fc60 con 0x55f74cb89fc0 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.528+0000 7f1273f79740 10 client.25474 fetch_fsmap learned FSMap version 28 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.528+0000 7f1273f79740 1 -- 172.21.15.26:0/927261134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f74cb9e2e0 con 0x55f74cb89fc0 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.529+0000 7f124be00640 1 -- 172.21.15.26:0/927261134 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1264036030 con 0x55f74cb89fc0 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.529+0000 7f124be00640 1 -- 172.21.15.26:0/927261134 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f1258010430 con 0x55f74cb89fc0 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.530+0000 7f1273f79740 10 client.25474 fetch_fsmap finished waiting for FSMap version 28 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.530+0000 7f1273f79740 10 client.25474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.530+0000 7f1273f79740 20 client.25474 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.530+0000 7f1273f79740 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f74cb9fa40 0x55f74cba1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.530+0000 7f1273f79740 4 client.25474 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.530+0000 7f1273f79740 1 -- 172.21.15.26:0/927261134 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f74caf1b50 con 0x55f74cb9fa40 2024-06-13T07:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.530+0000 7f126a800640 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f74cb9fa40 0x55f74cba1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.531+0000 7f126a800640 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f74cb9fa40 0x55f74cba1e60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01616bf140a0834 server_cookie=15327b27f59d68f5 in_seq=0 out_seq=0 2024-06-13T07:01:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.531+0000 7f124be00640 10 client.25474 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f124be00640 1 -- 172.21.15.26:0/927261134 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f74caf1b50 con 0x55f74cb9fa40 2024-06-13T07:01:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f124be00640 10 client.25474 handle_command_reply: tid=0 2024-06-13T07:01:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 client.25474 shutdown 2024-06-13T07:01:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 -- 172.21.15.26:0/927261134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 msgr2=0x55f74cb93e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb93e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1258012740 tx=0x7f1258012c00 comp rx=0 tx=0).stop 2024-06-13T07:01:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 -- 172.21.15.26:0/927261134 shutdown_connections 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f74cb9fa40 0x55f74cba1e60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb8f130 0x55f74cb9ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb93e10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 --2- 172.21.15.26:0/927261134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb938d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f124be00640 0 client.25474 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.533+0000 7f1273f79740 1 -- 172.21.15.26:0/927261134 >> 172.21.15.26:0/927261134 conn(0x55f74cb48fb0 msgr2=0x55f74cb7b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.534+0000 7f1273f79740 1 -- 172.21.15.26:0/927261134 shutdown_connections 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.534+0000 7f1273f79740 1 -- 172.21.15.26:0/927261134 wait complete. 2024-06-13T07:01:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.534+0000 7f1273f79740 20 client.25474 trim_cache size 0 max 16384 2024-06-13T07:01:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1273f79740 1 Processor -- start 2024-06-13T07:01:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1273f79740 1 -- start start 2024-06-13T07:01:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1273f79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb85240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1273f79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb9ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1273f79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb92ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1273f79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1aac00 con 0x55f74cb89fc0 2024-06-13T07:01:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1273f79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1a84d0 con 0x55f74cb895d0 2024-06-13T07:01:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1273f79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1a6320 con 0x55f74cb8f130 2024-06-13T07:01:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f126a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb85240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f1269400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb9ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.553+0000 7f126a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb85240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59434/0 (socket says 172.21.15.26:59434) 2024-06-13T07:01:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f126a800640 1 -- 172.21.15.26:0/2966469372 learned_addr learned my addr 172.21.15.26:0/2966469372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f126a800640 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb92ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f126a800640 1 -- 172.21.15.26:0/2966469372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 msgr2=0x55f74cb92ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f126a800640 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb92ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f126a800640 1 -- 172.21.15.26:0/2966469372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 msgr2=0x55f74cb9ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f126a800640 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb9ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f126a800640 1 -- 172.21.15.26:0/2966469372 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74cb85d00 con 0x55f74cb895d0 2024-06-13T07:01:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f1269400640 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb9ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.554+0000 7f126a800640 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb85240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1264030860 tx=0x7f1264004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9850f38774072373 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.555+0000 7f1256a00640 1 -- 172.21.15.26:0/2966469372 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264030a30 con 0x55f74cb895d0 2024-06-13T07:01:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.555+0000 7f1256a00640 1 -- 172.21.15.26:0/2966469372 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1264036050 con 0x55f74cb895d0 2024-06-13T07:01:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.555+0000 7f1256a00640 1 -- 172.21.15.26:0/2966469372 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264004690 con 0x55f74cb895d0 2024-06-13T07:01:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 -- 172.21.15.26:0/2966469372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 msgr2=0x55f74cb85240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb85240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1264030860 tx=0x7f1264004040 comp rx=0 tx=0).stop 2024-06-13T07:01:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 -- 172.21.15.26:0/2966469372 shutdown_connections 2024-06-13T07:01:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb8f130 0x55f74cb92ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb89fc0 0x55f74cb9ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 --2- 172.21.15.26:0/2966469372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb895d0 0x55f74cb85240 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 -- 172.21.15.26:0/2966469372 >> 172.21.15.26:0/2966469372 conn(0x55f74cb48fb0 msgr2=0x55f74cb84b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 -- 172.21.15.26:0/2966469372 shutdown_connections 2024-06-13T07:01:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 -- 172.21.15.26:0/2966469372 wait complete. 2024-06-13T07:01:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.556+0000 7f1273f79740 1 -- start start 2024-06-13T07:01:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1273f79740 1 Processor -- start 2024-06-13T07:01:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1273f79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb895d0 0x55f74cb97f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1273f79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1273f79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb8f130 0x55f74cb994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1273f79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1a6990 con 0x55f74cb895d0 2024-06-13T07:01:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1273f79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1a7e00 con 0x55f74cb8f130 2024-06-13T07:01:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1273f79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1a7060 con 0x55f74cb89fc0 2024-06-13T07:01:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1273f79740 10 client.0 _ll_register_callbacks cb 0x7f1258009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1269400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1269400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56196/0 (socket says 172.21.15.26:56196) 2024-06-13T07:01:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f1269400640 1 -- 172.21.15.26:0/2363407192 learned_addr learned my addr 172.21.15.26:0/2363407192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:01:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f126a800640 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb895d0 0x55f74cb97f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.557+0000 7f126a800640 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb8f130 0x55f74cb994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f1269400640 1 -- 172.21.15.26:0/2363407192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb8f130 msgr2=0x55f74cb994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f1269400640 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb8f130 0x55f74cb994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f1269400640 1 -- 172.21.15.26:0/2363407192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb895d0 msgr2=0x55f74cb97f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f1269400640 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb895d0 0x55f74cb97f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f1269400640 1 -- 172.21.15.26:0/2363407192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74cb78000 con 0x55f74cb89fc0 2024-06-13T07:01:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f126a800640 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb8f130 0x55f74cb994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f1269400640 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb984a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55f74cb85ed0 tx=0x7f1258010dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b458a60afe9a665c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f124be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:01:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f124be00640 1 -- 172.21.15.26:0/2363407192 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258010430 con 0x55f74cb89fc0 2024-06-13T07:01:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f124be00640 1 -- 172.21.15.26:0/2363407192 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1258018070 con 0x55f74cb89fc0 2024-06-13T07:01:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.558+0000 7f1273f79740 1 -- 172.21.15.26:0/2363407192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1264036050 con 0x55f74cb89fc0 2024-06-13T07:01:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f124be00640 1 -- 172.21.15.26:0/2363407192 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1258034050 con 0x55f74cb89fc0 2024-06-13T07:01:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f1273f79740 10 client.25480 fetch_fsmap learned FSMap version 28 2024-06-13T07:01:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f1273f79740 1 -- 172.21.15.26:0/2363407192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f74cb9ac80 con 0x55f74cb89fc0 2024-06-13T07:01:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f124be00640 1 -- 172.21.15.26:0/2363407192 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1264036050 con 0x55f74cb89fc0 2024-06-13T07:01:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f124be00640 1 -- 172.21.15.26:0/2363407192 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f12580035f0 con 0x55f74cb89fc0 2024-06-13T07:01:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f1273f79740 10 client.25480 fetch_fsmap finished waiting for FSMap version 28 2024-06-13T07:01:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f1273f79740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:01:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f1273f79740 20 client.25480 populate_metadata read hostname 'smithi026' 2024-06-13T07:01:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f1273f79740 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f74cc0aaa0 0x55f74cc0cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f1273f79740 4 client.25480 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f1273f79740 1 -- 172.21.15.26:0/2363407192 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f74caebea0 con 0x55f74cc0aaa0 2024-06-13T07:01:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.559+0000 7f126a800640 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f74cc0aaa0 0x55f74cc0cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.560+0000 7f126a800640 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f74cc0aaa0 0x55f74cc0cec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bb41ce44cdabca0 server_cookie=ac81627992cd61c2 in_seq=0 out_seq=0 2024-06-13T07:01:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.560+0000 7f124be00640 10 client.25480 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:59.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:01:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f124be00640 1 -- 172.21.15.26:0/2363407192 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f74caebea0 con 0x55f74cc0aaa0 2024-06-13T07:01:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f124be00640 10 client.25480 handle_command_reply: tid=0 2024-06-13T07:01:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 client.25480 shutdown 2024-06-13T07:01:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 -- 172.21.15.26:0/2363407192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 msgr2=0x55f74cb984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb984a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55f74cb85ed0 tx=0x7f1258010dd0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 -- 172.21.15.26:0/2363407192 shutdown_connections 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f74cc0aaa0 0x55f74cc0cec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f74cb8f130 0x55f74cb994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f74cb89fc0 0x55f74cb984a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f124be00640 0 client.25480 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 --2- 172.21.15.26:0/2363407192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f74cb895d0 0x55f74cb97f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 -- 172.21.15.26:0/2363407192 >> 172.21.15.26:0/2363407192 conn(0x55f74cb48fb0 msgr2=0x55f74cb87f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 -- 172.21.15.26:0/2363407192 shutdown_connections 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.791+0000 7f1273f79740 1 -- 172.21.15.26:0/2363407192 wait complete. 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.792+0000 7f1273f79740 20 client.25480 trim_cache size 0 max 16384 2024-06-13T07:01:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:01:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:01:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:01:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 -- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f12500a2c00 msgr2=0x7f12500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f12500a2c00 0x7f12500a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f125c09b950 tx=0x7f125c000bc0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 -- 172.21.15.26:0/1457709917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f126c1000f0 msgr2=0x7f126c199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f126c1000f0 0x7f126c199d20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f125c00e9f0 tx=0x7f125c00eeb0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 -- 172.21.15.26:0/1457709917 shutdown_connections 2024-06-13T07:01:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f12500a2c00 0x7f12500a50c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126c1013f0 0x7f126c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f126c100a50 0x7f126c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 --2- 172.21.15.26:0/1457709917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f126c1000f0 0x7f126c199d20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 -- 172.21.15.26:0/1457709917 >> 172.21.15.26:0/1457709917 conn(0x7f126c0fbc90 msgr2=0x7f126c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 -- 172.21.15.26:0/1457709917 shutdown_connections 2024-06-13T07:01:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:01:59.793+0000 7f126b200640 1 -- 172.21.15.26:0/1457709917 wait complete. 2024-06-13T07:01:59.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:01:59 smithi136 ceph-mon[37229]: pgmap v554: 129 pgs: 129 active+clean; 7.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 676 KiB/s wr, 96 op/s 2024-06-13T07:02:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:01:59 smithi026 ceph-mon[32041]: pgmap v554: 129 pgs: 129 active+clean; 7.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 676 KiB/s wr, 96 op/s 2024-06-13T07:02:00.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:01:59 smithi135 ceph-mon[36276]: pgmap v554: 129 pgs: 129 active+clean; 7.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 676 KiB/s wr, 96 op/s 2024-06-13T07:02:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:00.499 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16530 7.45 MB/sec warmup 14 sec latency 645.504 ms 2024-06-13T07:02:01.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:00 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:01.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:00 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:01.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:00 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:01.499 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16530 6.95 MB/sec warmup 15 sec latency 1645.663 ms 2024-06-13T07:02:02.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:01 smithi026 ceph-mon[32041]: pgmap v555: 129 pgs: 129 active+clean; 7.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 696 KiB/s wr, 114 op/s 2024-06-13T07:02:02.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:01 smithi135 ceph-mon[36276]: pgmap v555: 129 pgs: 129 active+clean; 7.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 696 KiB/s wr, 114 op/s 2024-06-13T07:02:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:01 smithi136 ceph-mon[37229]: pgmap v555: 129 pgs: 129 active+clean; 7.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 696 KiB/s wr, 114 op/s 2024-06-13T07:02:02.499 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16584 6.53 MB/sec warmup 16 sec latency 2569.515 ms 2024-06-13T07:02:03.499 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16584 6.14 MB/sec warmup 17 sec latency 1013.424 ms 2024-06-13T07:02:03.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:03 smithi026 ceph-mon[32041]: pgmap v556: 129 pgs: 129 active+clean; 11 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 982 KiB/s wr, 151 op/s 2024-06-13T07:02:04.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:03 smithi135 ceph-mon[36276]: pgmap v556: 129 pgs: 129 active+clean; 11 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 982 KiB/s wr, 151 op/s 2024-06-13T07:02:04.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:03 smithi136 ceph-mon[37229]: pgmap v556: 129 pgs: 129 active+clean; 11 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 982 KiB/s wr, 151 op/s 2024-06-13T07:02:04.500 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16584 5.80 MB/sec warmup 18 sec latency 2013.519 ms 2024-06-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 -- 172.21.15.26:0/241198363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e806aac0 msgr2=0x7f41e810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 --2- 172.21.15.26:0/241198363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e806aac0 0x7f41e810ea00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f41d40098f0 tx=0x7f41d4009c00 comp rx=0 tx=0).stop 2024-06-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 -- 172.21.15.26:0/241198363 shutdown_connections 2024-06-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 --2- 172.21.15.26:0/241198363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e810ef40 0x7f41e8111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 --2- 172.21.15.26:0/241198363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e806a1a0 0x7f41e806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 --2- 172.21.15.26:0/241198363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e806aac0 0x7f41e810ea00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 -- 172.21.15.26:0/241198363 >> 172.21.15.26:0/241198363 conn(0x7f41e8078340 msgr2=0x7f41e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 -- 172.21.15.26:0/241198363 shutdown_connections 2024-06-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.545+0000 7f41edc00700 1 -- 172.21.15.26:0/241198363 wait complete. 2024-06-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41edc00700 1 Processor -- start 2024-06-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41edc00700 1 -- start start 2024-06-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41edc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e806a1a0 0x7f41e819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41edc00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e806aac0 0x7f41e819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41edc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e810ef40 0x7f41e819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41edc00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41e81149f0 con 0x7f41e806aac0 2024-06-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41edc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8114870 con 0x7f41e810ef40 2024-06-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41edc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8114b70 con 0x7f41e806a1a0 2024-06-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41e6a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e806a1a0 0x7f41e819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41e6000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e806aac0 0x7f41e819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.547+0000 7f41e6a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e806a1a0 0x7f41e819af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58086/0 (socket says 172.21.15.26:58086) 2024-06-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e6a00700 1 -- 172.21.15.26:0/2692724700 learned_addr learned my addr 172.21.15.26:0/2692724700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e7400700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e810ef40 0x7f41e819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e6a00700 1 -- 172.21.15.26:0/2692724700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e810ef40 msgr2=0x7f41e819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e6a00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e810ef40 0x7f41e819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e6a00700 1 -- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e806aac0 msgr2=0x7f41e819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e6a00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e806aac0 0x7f41e819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e6a00700 1 -- 172.21.15.26:0/2692724700 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e819ff10 con 0x7f41e806a1a0 2024-06-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e6000700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e806aac0 0x7f41e819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.548+0000 7f41e6a00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e806a1a0 0x7f41e819af00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41d800ec10 tx=0x7f41d800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.549+0000 7f41df400700 1 -- 172.21.15.26:0/2692724700 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d800cc90 con 0x7f41e806a1a0 2024-06-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.549+0000 7f41edc00700 1 -- 172.21.15.26:0/2692724700 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e81a0200 con 0x7f41e806a1a0 2024-06-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.549+0000 7f41df400700 1 -- 172.21.15.26:0/2692724700 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d800ce30 con 0x7f41e806a1a0 2024-06-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.549+0000 7f41df400700 1 -- 172.21.15.26:0/2692724700 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8010690 con 0x7f41e806a1a0 2024-06-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.549+0000 7f41edc00700 1 -- 172.21.15.26:0/2692724700 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e81a7a80 con 0x7f41e806a1a0 2024-06-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.554+0000 7f41df400700 1 -- 172.21.15.26:0/2692724700 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41d8003680 con 0x7f41e806a1a0 2024-06-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.555+0000 7f41f85adb80 1 Processor -- start 2024-06-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.555+0000 7f41f85adb80 1 -- start start 2024-06-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.555+0000 7f41df400700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f41cc0a2ca0 0x7f41cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41df400700 1 -- 172.21.15.26:0/2692724700 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f41d80cf780 con 0x7f41e806a1a0 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184e63e40 0x565184ee7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ee7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184ef46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41f85adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565184df6a10 con 0x565184e63e40 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41e7400700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f41cc0a2ca0 0x7f41cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41f85adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565184e39b00 con 0x565184ee75b0 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41f85adb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565184df48f0 con 0x565184ef2300 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41e6000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ee7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41e6a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184e63e40 0x565184ee7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41e6000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ee7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50140/0 (socket says 172.21.15.26:50140) 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41e7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184ef46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.556+0000 7f41e6000700 1 -- 172.21.15.26:0/1778731455 learned_addr learned my addr 172.21.15.26:0/1778731455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.557+0000 7f41e6000700 1 -- 172.21.15.26:0/1778731455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 msgr2=0x565184ef46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.557+0000 7f41e6000700 1 --2- 172.21.15.26:0/1778731455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184ef46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.557+0000 7f41e6000700 1 -- 172.21.15.26:0/1778731455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184e63e40 msgr2=0x565184ee7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.557+0000 7f41e6000700 1 --2- 172.21.15.26:0/1778731455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184e63e40 0x565184ee7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.557+0000 7f41e6000700 1 -- 172.21.15.26:0/1778731455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565184de1f90 con 0x565184ee75b0 2024-06-13T07:02:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.557+0000 7f41e7400700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f41cc0a2ca0 0x7f41cc0a5160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009550 tx=0x7f41e0004040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.557+0000 7f41e6000700 1 --2- 172.21.15.26:0/1778731455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ee7a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41d40098f0 tx=0x7f41d400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dd8e727defd0c4d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.557+0000 7f41d3e00700 1 -- 172.21.15.26:0/1778731455 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d400f820 con 0x565184ee75b0 2024-06-13T07:02:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41d3e00700 1 -- 172.21.15.26:0/1778731455 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d400f9c0 con 0x565184ee75b0 2024-06-13T07:02:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41d3e00700 1 -- 172.21.15.26:0/1778731455 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4004cc0 con 0x565184ee75b0 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41f85adb80 1 -- 172.21.15.26:0/1778731455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 msgr2=0x565184ee7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1778731455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ee7a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41d40098f0 tx=0x7f41d400bf90 comp rx=0 tx=0).stop 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41f85adb80 1 -- 172.21.15.26:0/1778731455 shutdown_connections 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1778731455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184ef46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1778731455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184e63e40 0x565184ee7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1778731455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ee7a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41f85adb80 1 -- 172.21.15.26:0/1778731455 >> 172.21.15.26:0/1778731455 conn(0x565184e14170 msgr2=0x565184ee34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.558+0000 7f41f85adb80 1 -- 172.21.15.26:0/1778731455 shutdown_connections 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.559+0000 7f41f85adb80 1 -- 172.21.15.26:0/1778731455 wait complete. 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.559+0000 7f41f85adb80 1 -- start start 2024-06-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.559+0000 7f41f85adb80 1 Processor -- start 2024-06-13T07:02:04.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184eeed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184eef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41f85adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565184dd5f60 con 0x565184ee75b0 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41f85adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565184e35ac0 con 0x565184e63e40 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41f85adb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565184ef4f60 con 0x565184ef2300 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41f85adb80 10 client.0 _ll_register_callbacks cb 0x565184dd3720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41e6000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184eeed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41e6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41e7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184eef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41e6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50142/0 (socket says 172.21.15.26:50142) 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.560+0000 7f41e6a00700 1 -- 172.21.15.26:0/1331618784 learned_addr learned my addr 172.21.15.26:0/1331618784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41e6a00700 1 -- 172.21.15.26:0/1331618784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 msgr2=0x565184eef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41e6a00700 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184eef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41e6a00700 1 -- 172.21.15.26:0/1331618784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 msgr2=0x565184eeed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41e6a00700 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184eeed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41e6a00700 1 -- 172.21.15.26:0/1331618784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565184d888d0 con 0x565184e63e40 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41e6000700 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184eeed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41e6a00700 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef1a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009b40 tx=0x7f41d809b570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b40f6543ed4dff2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41d0c00700 1 -- 172.21.15.26:0/1331618784 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d809bff0 con 0x565184e63e40 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41d0c00700 1 -- 172.21.15.26:0/1331618784 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f41d80a4370 con 0x565184e63e40 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41f85adb80 1 -- 172.21.15.26:0/1331618784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d400f9c0 con 0x565184e63e40 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41d0c00700 1 -- 172.21.15.26:0/1331618784 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41d8082050 con 0x565184e63e40 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41f85adb80 10 client.25451 fetch_fsmap learned FSMap version 29 2024-06-13T07:02:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.561+0000 7f41f85adb80 1 -- 172.21.15.26:0/1331618784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565184eff6a0 con 0x565184e63e40 2024-06-13T07:02:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41d0c00700 1 -- 172.21.15.26:0/1331618784 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d400f9c0 con 0x565184e63e40 2024-06-13T07:02:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41d0c00700 1 -- 172.21.15.26:0/1331618784 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f41d80ce040 con 0x565184e63e40 2024-06-13T07:02:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41f85adb80 10 client.25451 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T07:02:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41f85adb80 10 client.25451 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-13T07:02:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41f85adb80 20 client.25451 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x565184f01ce0 0x565184f04130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41f85adb80 4 client.25451 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41f85adb80 1 -- 172.21.15.26:0/1331618784 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565184e47000 con 0x565184f01ce0 2024-06-13T07:02:04.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.562+0000 7f41e6000700 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x565184f01ce0 0x565184f04130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.563+0000 7f41e6000700 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x565184f01ce0 0x565184f04130 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ced1234acdaced5 server_cookie=c3f24e00a1f82720 in_seq=0 out_seq=0 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.563+0000 7f41d0c00700 10 client.25451 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41d0c00700 1 -- 172.21.15.26:0/1331618784 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565184e47000 con 0x565184f01ce0 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41d0c00700 10 client.25451 handle_command_reply: tid=0 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 client.25451 shutdown 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 -- 172.21.15.26:0/1331618784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 msgr2=0x565184ef1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef1a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009b40 tx=0x7f41d809b570 comp rx=0 tx=0).stop 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 -- 172.21.15.26:0/1331618784 shutdown_connections 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x565184f01ce0 0x565184f04130 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef1a30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184eef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 --2- 172.21.15.26:0/1331618784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184eeed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41d0c00700 0 client.25451 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T07:02:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.564+0000 7f41f85adb80 1 -- 172.21.15.26:0/1331618784 >> 172.21.15.26:0/1331618784 conn(0x565184e14170 msgr2=0x565184ee2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.565+0000 7f41f85adb80 1 -- 172.21.15.26:0/1331618784 shutdown_connections 2024-06-13T07:02:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.565+0000 7f41f85adb80 1 -- 172.21.15.26:0/1331618784 wait complete. 2024-06-13T07:02:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.565+0000 7f41f85adb80 20 client.25451 trim_cache size 0 max 16384 2024-06-13T07:02:04.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.582+0000 7f41f85adb80 1 Processor -- start 2024-06-13T07:02:04.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.582+0000 7f41f85adb80 1 -- start start 2024-06-13T07:02:04.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.582+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184e63e40 0x565184eedc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.582+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ef1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.582+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ef2300 0x565184ef36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.582+0000 7f41f85adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41e81b2f30 con 0x565184ef2300 2024-06-13T07:02:04.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.582+0000 7f41f85adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41e81b07c0 con 0x565184ee75b0 2024-06-13T07:02:04.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.582+0000 7f41f85adb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41e81ae610 con 0x565184e63e40 2024-06-13T07:02:04.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e7400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ef2300 0x565184ef36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ef1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184e63e40 0x565184eedc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ef1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50150/0 (socket says 172.21.15.26:50150) 2024-06-13T07:02:04.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6000700 1 -- 172.21.15.26:0/3799382053 learned_addr learned my addr 172.21.15.26:0/3799382053 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:04.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6a00700 1 -- 172.21.15.26:0/3799382053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 msgr2=0x565184ef1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:04.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6a00700 1 --2- 172.21.15.26:0/3799382053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ef1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6a00700 1 -- 172.21.15.26:0/3799382053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ef2300 msgr2=0x565184ef36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:04.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6a00700 1 --2- 172.21.15.26:0/3799382053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ef2300 0x565184ef36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6a00700 1 -- 172.21.15.26:0/3799382053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565184eed2a0 con 0x565184e63e40 2024-06-13T07:02:04.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6000700 1 --2- 172.21.15.26:0/3799382053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ef1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.583+0000 7f41e6a00700 1 --2- 172.21.15.26:0/3799382053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184e63e40 0x565184eedc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f41d809bb20 tx=0x7f41d8097e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7883706d18880a5e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41d3400700 1 -- 172.21.15.26:0/3799382053 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d809bb90 con 0x565184e63e40 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41d3400700 1 -- 172.21.15.26:0/3799382053 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d80b9070 con 0x565184e63e40 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41d3400700 1 -- 172.21.15.26:0/3799382053 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d809c050 con 0x565184e63e40 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41f85adb80 1 -- 172.21.15.26:0/3799382053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184e63e40 msgr2=0x565184eedc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41f85adb80 1 --2- 172.21.15.26:0/3799382053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184e63e40 0x565184eedc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f41d809bb20 tx=0x7f41d8097e30 comp rx=0 tx=0).stop 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41f85adb80 1 -- 172.21.15.26:0/3799382053 shutdown_connections 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41f85adb80 1 --2- 172.21.15.26:0/3799382053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ef2300 0x565184ef36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41f85adb80 1 --2- 172.21.15.26:0/3799382053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184e63e40 0x565184eedc00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41f85adb80 1 --2- 172.21.15.26:0/3799382053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184ee75b0 0x565184ef1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.584+0000 7f41f85adb80 1 -- 172.21.15.26:0/3799382053 >> 172.21.15.26:0/3799382053 conn(0x565184e14170 msgr2=0x565184eed4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.585+0000 7f41f85adb80 1 -- 172.21.15.26:0/3799382053 shutdown_connections 2024-06-13T07:02:04.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.585+0000 7f41f85adb80 1 -- 172.21.15.26:0/3799382053 wait complete. 2024-06-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.585+0000 7f41f85adb80 1 -- start start 2024-06-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41f85adb80 1 Processor -- start 2024-06-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184ef3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41f85adb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184ef3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41f85adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41e81aec80 con 0x565184ee75b0 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41f85adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41e81b00f0 con 0x565184e63e40 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41f85adb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41e81af350 con 0x565184ef2300 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41f85adb80 10 client.0 _ll_register_callbacks cb 0x565184dd3720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41e6000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184ef3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41e6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41e6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50158/0 (socket says 172.21.15.26:50158) 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41e6a00700 1 -- 172.21.15.26:0/545996080 learned_addr learned my addr 172.21.15.26:0/545996080 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.586+0000 7f41e7400700 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184ef3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41e6a00700 1 -- 172.21.15.26:0/545996080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 msgr2=0x565184ef3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41e6a00700 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184ef3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41e6a00700 1 -- 172.21.15.26:0/545996080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 msgr2=0x565184ef3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41e6a00700 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184ef3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41e6a00700 1 -- 172.21.15.26:0/545996080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565184ef91f0 con 0x565184e63e40 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41e6a00700 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef2ce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f41d8097ec0 tx=0x7f41d8097d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36efed9ca7f3c8b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41d0c00700 1 -- 172.21.15.26:0/545996080 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d80a4800 con 0x565184e63e40 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41d0c00700 1 -- 172.21.15.26:0/545996080 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f41d80a4370 con 0x565184e63e40 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.587+0000 7f41f85adb80 1 -- 172.21.15.26:0/545996080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d80b9070 con 0x565184e63e40 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41d0c00700 1 -- 172.21.15.26:0/545996080 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41d809bb90 con 0x565184e63e40 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41f85adb80 10 client.25457 fetch_fsmap learned FSMap version 29 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41f85adb80 1 -- 172.21.15.26:0/545996080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565184ef91f0 con 0x565184e63e40 2024-06-13T07:02:04.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41d0c00700 1 -- 172.21.15.26:0/545996080 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d80b9070 con 0x565184e63e40 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41d0c00700 1 -- 172.21.15.26:0/545996080 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f41d80ce070 con 0x565184e63e40 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41f85adb80 10 client.25457 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41f85adb80 10 client.25457 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41f85adb80 20 client.25457 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41f85adb80 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x565184efa660 0x565184efaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41f85adb80 4 client.25457 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.588+0000 7f41f85adb80 1 -- 172.21.15.26:0/545996080 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565184e41580 con 0x565184efa660 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.589+0000 7f41e6000700 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x565184efa660 0x565184efaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.589+0000 7f41e6000700 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x565184efa660 0x565184efaa70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd1dd2528d42663c server_cookie=ecd3c334d221239 in_seq=0 out_seq=0 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.589+0000 7f41d0c00700 10 client.25457 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.589+0000 7f41d0c00700 1 -- 172.21.15.26:0/545996080 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565184e41580 con 0x565184efa660 2024-06-13T07:02:04.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.589+0000 7f41d0c00700 10 client.25457 handle_command_reply: tid=0 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.589+0000 7f41f85adb80 1 client.25457 shutdown 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.589+0000 7f41f85adb80 1 -- 172.21.15.26:0/545996080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 msgr2=0x565184ef2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.589+0000 7f41f85adb80 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef2ce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f41d8097ec0 tx=0x7f41d8097d20 comp rx=0 tx=0).stop 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 1 -- 172.21.15.26:0/545996080 shutdown_connections 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x565184efa660 0x565184efaa70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565184e63e40 0x565184ef2ce0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565184ef2300 0x565184ef3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 1 --2- 172.21.15.26:0/545996080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565184ee75b0 0x565184ef3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41d0c00700 0 client.25457 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 1 -- 172.21.15.26:0/545996080 >> 172.21.15.26:0/545996080 conn(0x565184e14170 msgr2=0x565184ee1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 1 -- 172.21.15.26:0/545996080 shutdown_connections 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 1 -- 172.21.15.26:0/545996080 wait complete. 2024-06-13T07:02:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.590+0000 7f41f85adb80 20 client.25457 trim_cache size 0 max 16384 2024-06-13T07:02:04.593 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-06-13T07:02:04.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 -- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f41cc0a2ca0 msgr2=0x7f41cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f41cc0a2ca0 0x7f41cc0a5160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009550 tx=0x7f41e0004040 comp rx=0 tx=0).stop 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 -- 172.21.15.26:0/2692724700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e806a1a0 msgr2=0x7f41e819af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e806a1a0 0x7f41e819af00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41d800ec10 tx=0x7f41d800ef20 comp rx=0 tx=0).stop 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 -- 172.21.15.26:0/2692724700 shutdown_connections 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f41cc0a2ca0 0x7f41cc0a5160 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e810ef40 0x7f41e819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e806a1a0 0x7f41e819af00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 --2- 172.21.15.26:0/2692724700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e806aac0 0x7f41e819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 -- 172.21.15.26:0/2692724700 >> 172.21.15.26:0/2692724700 conn(0x7f41e8078340 msgr2=0x7f41e8110410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 -- 172.21.15.26:0/2692724700 shutdown_connections 2024-06-13T07:02:04.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:04.591+0000 7f41edc00700 1 -- 172.21.15.26:0/2692724700 wait complete. 2024-06-13T07:02:05.500 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16584 5.50 MB/sec warmup 19 sec latency 3014.064 ms 2024-06-13T07:02:05.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:02:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:05 smithi026 ceph-mon[32041]: pgmap v557: 129 pgs: 129 active+clean; 15 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-06-13T07:02:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:05 smithi135 ceph-mon[36276]: pgmap v557: 129 pgs: 129 active+clean; 15 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-06-13T07:02:06.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:05 smithi136 ceph-mon[37229]: pgmap v557: 129 pgs: 129 active+clean; 15 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-06-13T07:02:06.181 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:06.500 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16584 5.22 MB/sec warmup 20 sec latency 4014.194 ms 2024-06-13T07:02:07.500 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16596 4.97 MB/sec warmup 21 sec latency 4986.401 ms 2024-06-13T07:02:07.987 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:07 smithi026 ceph-mon[32041]: pgmap v558: 129 pgs: 129 active+clean; 23 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-06-13T07:02:07.987 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:07 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:02:08.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:07 smithi135 ceph-mon[36276]: pgmap v558: 129 pgs: 129 active+clean; 23 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-06-13T07:02:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:07 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:02:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:07 smithi136 ceph-mon[37229]: pgmap v558: 129 pgs: 129 active+clean; 23 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-06-13T07:02:08.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:07 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:02:08.500 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16685 4.78 MB/sec warmup 22 sec latency 816.349 ms 2024-06-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.534+0000 7f8edb200700 1 -- 172.21.15.26:0/1647759146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc102fd0 msgr2=0x7f8edc107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.534+0000 7f8edb200700 1 --2- 172.21.15.26:0/1647759146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc102fd0 0x7f8edc107020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009990 tx=0x7f8ec8009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.535+0000 7f8edb200700 1 -- 172.21.15.26:0/1647759146 shutdown_connections 2024-06-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.535+0000 7f8edb200700 1 --2- 172.21.15.26:0/1647759146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc107700 0x7f8edc10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.535+0000 7f8edb200700 1 --2- 172.21.15.26:0/1647759146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc102620 0x7f8edc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.535+0000 7f8edb200700 1 --2- 172.21.15.26:0/1647759146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc102fd0 0x7f8edc107020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.535+0000 7f8edb200700 1 -- 172.21.15.26:0/1647759146 >> 172.21.15.26:0/1647759146 conn(0x7f8edc0fdf10 msgr2=0x7f8edc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.535+0000 7f8edb200700 1 -- 172.21.15.26:0/1647759146 shutdown_connections 2024-06-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.535+0000 7f8edb200700 1 -- 172.21.15.26:0/1647759146 wait complete. 2024-06-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.537+0000 7f8edb200700 1 Processor -- start 2024-06-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.537+0000 7f8edb200700 1 -- start start 2024-06-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8edb200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc102620 0x7f8edc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8edb200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc102fd0 0x7f8edc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8edb200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc107700 0x7f8edc19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8edb200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1126c0 con 0x7f8edc102fd0 2024-06-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8edb200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc112540 con 0x7f8edc107700 2024-06-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8edb200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc112840 con 0x7f8edc102620 2024-06-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8ed9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc102620 0x7f8edc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8ed9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc102620 0x7f8edc19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58132/0 (socket says 172.21.15.26:58132) 2024-06-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.538+0000 7f8ed9e00700 1 -- 172.21.15.26:0/1936884556 learned_addr learned my addr 172.21.15.26:0/1936884556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8ed9400700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc102fd0 0x7f8edc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8eda800700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc107700 0x7f8edc19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8ed9e00700 1 -- 172.21.15.26:0/1936884556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc107700 msgr2=0x7f8edc19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc107700 0x7f8edc19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8ed9e00700 1 -- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc102fd0 msgr2=0x7f8edc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc102fd0 0x7f8edc19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8ed9e00700 1 -- 172.21.15.26:0/1936884556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc19fe20 con 0x7f8edc102620 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8eda800700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc107700 0x7f8edc19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.539+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc102620 0x7f8edc19ae10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00d950 tx=0x7f8ecc00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.540+0000 7f8ed2a00700 1 -- 172.21.15.26:0/1936884556 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc016070 con 0x7f8edc102620 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.540+0000 7f8ed2a00700 1 -- 172.21.15.26:0/1936884556 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ecc009b40 con 0x7f8edc102620 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.540+0000 7f8ed2a00700 1 -- 172.21.15.26:0/1936884556 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00f620 con 0x7f8edc102620 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.540+0000 7f8edb200700 1 -- 172.21.15.26:0/1936884556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc1a0110 con 0x7f8edc102620 2024-06-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.540+0000 7f8edb200700 1 -- 172.21.15.26:0/1936884556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc1a7990 con 0x7f8edc102620 2024-06-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.543+0000 7f8ed2a00700 1 -- 172.21.15.26:0/1936884556 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ecc019070 con 0x7f8edc102620 2024-06-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.544+0000 7f8ed2a00700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.545+0000 7f8ed2a00700 1 -- 172.21.15.26:0/1936884556 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8ecc0cf450 con 0x7f8edc102620 2024-06-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.545+0000 7f8eeb3cdb80 1 Processor -- start 2024-06-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.545+0000 7f8ed9400700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.545+0000 7f8eeb3cdb80 1 -- start start 2024-06-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.545+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b2d60 0x5626876b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876bdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 0x5626876c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5626875c2a10 con 0x5626876be530 2024-06-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562687605b00 con 0x5626876b3680 2024-06-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5626875c08f0 con 0x5626876b2d60 2024-06-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b2d60 0x5626876b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b2d60 0x5626876b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58146/0 (socket says 172.21.15.26:58146) 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9e00700 1 -- 172.21.15.26:0/87309090 learned_addr learned my addr 172.21.15.26:0/87309090 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8eda800700 1 --2- 172.21.15.26:0/87309090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 0x5626876c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9e00700 1 -- 172.21.15.26:0/87309090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 msgr2=0x5626876bdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/87309090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876bdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9e00700 1 -- 172.21.15.26:0/87309090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 msgr2=0x5626876c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/87309090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 0x5626876c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9e00700 1 -- 172.21.15.26:0/87309090 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626875adf90 con 0x5626876b2d60 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.546+0000 7f8ed9400700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8006010 tx=0x7f8ec801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.547+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/87309090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b2d60 0x5626876b3140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc097cc0 tx=0x7f8ecc09b240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c5c9a9f75e5a588 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.547+0000 7f8ec7400700 1 -- 172.21.15.26:0/87309090 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc020070 con 0x5626876b2d60 2024-06-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.547+0000 7f8ec7400700 1 -- 172.21.15.26:0/87309090 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ecc09fc10 con 0x5626876b2d60 2024-06-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.547+0000 7f8ec7400700 1 -- 172.21.15.26:0/87309090 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0860b0 con 0x5626876b2d60 2024-06-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/87309090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b2d60 msgr2=0x5626876b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/87309090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b2d60 0x5626876b3140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc097cc0 tx=0x7f8ecc09b240 comp rx=0 tx=0).stop 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/87309090 shutdown_connections 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/87309090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 0x5626876c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/87309090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b2d60 0x5626876b3140 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/87309090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876bdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/87309090 >> 172.21.15.26:0/87309090 conn(0x5626875e0170 msgr2=0x5626876af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/87309090 shutdown_connections 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.548+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/87309090 wait complete. 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.549+0000 7f8eeb3cdb80 1 -- start start 2024-06-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.549+0000 7f8eeb3cdb80 1 Processor -- start 2024-06-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876bdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b3680 0x5626876bb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 0x5626876bb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5626875a1f60 con 0x5626876b3680 2024-06-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562687601ac0 con 0x5626876b2d60 2024-06-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5626876c0f60 con 0x5626876be530 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8eeb3cdb80 10 client.0 _ll_register_callbacks cb 0x56268759b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8ed9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876bdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8ed9400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b3680 0x5626876bb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8ed9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876bdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50188/0 (socket says 172.21.15.26:50188) 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8ed9e00700 1 -- 172.21.15.26:0/648493987 learned_addr learned my addr 172.21.15.26:0/648493987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.550+0000 7f8eda800700 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 0x5626876bb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ed9e00700 1 -- 172.21.15.26:0/648493987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 msgr2=0x5626876bb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 0x5626876bb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ed9e00700 1 -- 172.21.15.26:0/648493987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b3680 msgr2=0x5626876bb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b3680 0x5626876bb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ed9e00700 1 -- 172.21.15.26:0/648493987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626875548d0 con 0x5626876b2d60 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876bdd40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc000c00 tx=0x7f8ecc010ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e87add6c12fb34e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ebbe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ebbe00700 1 -- 172.21.15.26:0/648493987 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0b3070 con 0x5626876b2d60 2024-06-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ebbe00700 1 -- 172.21.15.26:0/648493987 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8ecc0cf170 con 0x5626876b2d60 2024-06-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/648493987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ecc09fc10 con 0x5626876b2d60 2024-06-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.551+0000 7f8ebbe00700 1 -- 172.21.15.26:0/648493987 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ecc020070 con 0x5626876b2d60 2024-06-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8eeb3cdb80 10 client.25466 fetch_fsmap learned FSMap version 29 2024-06-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/648493987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626876cb6f0 con 0x5626876b2d60 2024-06-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8ebbe00700 1 -- 172.21.15.26:0/648493987 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ecc09fc10 con 0x5626876b2d60 2024-06-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8ebbe00700 1 -- 172.21.15.26:0/648493987 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ecc09bac0 con 0x5626876b2d60 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8eeb3cdb80 10 client.25466 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8eeb3cdb80 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8eeb3cdb80 20 client.25466 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5626876cdbc0 0x5626876cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8eeb3cdb80 4 client.25466 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.552+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/648493987 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562687613000 con 0x5626876cdbc0 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.553+0000 7f8ed9400700 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5626876cdbc0 0x5626876cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.553+0000 7f8ed9400700 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5626876cdbc0 0x5626876cffb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f3373875934fe2 server_cookie=b75dc50cfcf36609 in_seq=0 out_seq=0 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.553+0000 7f8ebbe00700 10 client.25466 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.554+0000 7f8ebbe00700 1 -- 172.21.15.26:0/648493987 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562687613000 con 0x5626876cdbc0 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.554+0000 7f8ebbe00700 10 client.25466 handle_command_reply: tid=0 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.554+0000 7f8eeb3cdb80 1 client.25466 shutdown 2024-06-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/648493987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 msgr2=0x5626876bdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876bdd40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc000c00 tx=0x7f8ecc010ed0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/648493987 shutdown_connections 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5626876cdbc0 0x5626876cffb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876bdd40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 0x5626876bb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/648493987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b3680 0x5626876bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8ebbe00700 0 client.25466 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/648493987 >> 172.21.15.26:0/648493987 conn(0x5626875e0170 msgr2=0x5626876aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/648493987 shutdown_connections 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/648493987 wait complete. 2024-06-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.555+0000 7f8eeb3cdb80 20 client.25466 trim_cache size 0 max 16384 2024-06-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.572+0000 7f8eeb3cdb80 1 Processor -- start 2024-06-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8eeb3cdb80 1 -- start start 2024-06-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876b9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b3680 0x5626876b9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 0x5626876bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b2e80 con 0x5626876be530 2024-06-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b0710 con 0x5626876b2d60 2024-06-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1ae560 con 0x5626876b3680 2024-06-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8ed9400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b3680 0x5626876b9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8eda800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 0x5626876bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8ed9400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b3680 0x5626876b9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58150/0 (socket says 172.21.15.26:58150) 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8ed9400700 1 -- 172.21.15.26:0/452367604 learned_addr learned my addr 172.21.15.26:0/452367604 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.573+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/452367604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.574+0000 7f8ed9400700 1 -- 172.21.15.26:0/452367604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 msgr2=0x5626876b9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.574+0000 7f8ed9400700 1 --2- 172.21.15.26:0/452367604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876b9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.574+0000 7f8ed9400700 1 -- 172.21.15.26:0/452367604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 msgr2=0x5626876bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.574+0000 7f8ed9400700 1 --2- 172.21.15.26:0/452367604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 0x5626876bb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.574+0000 7f8ed9400700 1 -- 172.21.15.26:0/452367604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626876b95b0 con 0x5626876b3680 2024-06-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.574+0000 7f8ed9400700 1 --2- 172.21.15.26:0/452367604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b3680 0x5626876b9e70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8edc06ada0 tx=0x7f8ec8026040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d837670859c0903b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.574+0000 7f8ec6a00700 1 -- 172.21.15.26:0/452367604 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8036070 con 0x5626876b3680 2024-06-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8ec6a00700 1 -- 172.21.15.26:0/452367604 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec800be80 con 0x5626876b3680 2024-06-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8ec6a00700 1 -- 172.21.15.26:0/452367604 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8023670 con 0x5626876b3680 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/452367604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b3680 msgr2=0x5626876b9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/452367604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b3680 0x5626876b9e70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8edc06ada0 tx=0x7f8ec8026040 comp rx=0 tx=0).stop 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/452367604 shutdown_connections 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/452367604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b2d60 0x5626876b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/452367604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876be530 0x5626876bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/452367604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876b3680 0x5626876b9e70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.575+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/452367604 >> 172.21.15.26:0/452367604 conn(0x5626875e0170 msgr2=0x5626876aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.576+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/452367604 shutdown_connections 2024-06-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.576+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/452367604 wait complete. 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.576+0000 7f8eeb3cdb80 1 -- start start 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.577+0000 7f8eeb3cdb80 1 Processor -- start 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.577+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b2d60 0x5626876d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.577+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.577+0000 7f8eeb3cdb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 0x5626877364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.577+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1aebd0 con 0x5626876b2d60 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.577+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b0040 con 0x5626876b3680 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.577+0000 7f8eeb3cdb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1af2a0 con 0x5626876be530 2024-06-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8eeb3cdb80 10 client.0 _ll_register_callbacks cb 0x56268759b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b2d60 0x5626876d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50206/0 (socket says 172.21.15.26:50206) 2024-06-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9400700 1 -- 172.21.15.26:0/1368770160 learned_addr learned my addr 172.21.15.26:0/1368770160 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8eda800700 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 0x5626877364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9400700 1 -- 172.21.15.26:0/1368770160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 msgr2=0x5626877364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9400700 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 0x5626877364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9400700 1 -- 172.21.15.26:0/1368770160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b2d60 msgr2=0x5626876d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9400700 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b2d60 0x5626876d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.578+0000 7f8ed9400700 1 -- 172.21.15.26:0/1368770160 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562687736a80 con 0x5626876b3680 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8ed9400700 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876d0a40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80098e0 tx=0x7f8ec8030c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47356224e6a920b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8ebbe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8ebbe00700 1 -- 172.21.15.26:0/1368770160 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8036040 con 0x5626876b3680 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/1368770160 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56268760d580 con 0x5626876b3680 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8ebbe00700 1 -- 172.21.15.26:0/1368770160 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8ec800fd20 con 0x5626876b3680 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8ebbe00700 1 -- 172.21.15.26:0/1368770160 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8030440 con 0x5626876b3680 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8eeb3cdb80 10 client.25472 fetch_fsmap learned FSMap version 29 2024-06-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/1368770160 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562687737d80 con 0x5626876b3680 2024-06-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.579+0000 7f8ebbe00700 1 -- 172.21.15.26:0/1368770160 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56268760d580 con 0x5626876b3680 2024-06-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.580+0000 7f8ebbe00700 1 -- 172.21.15.26:0/1368770160 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ec8023c10 con 0x5626876b3680 2024-06-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.580+0000 7f8eeb3cdb80 10 client.25472 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.580+0000 7f8eeb3cdb80 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.580+0000 7f8eeb3cdb80 20 client.25472 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.580+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5626877383f0 0x56268773a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.580+0000 7f8eeb3cdb80 4 client.25472 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.580+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/1368770160 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562687487250 con 0x5626877383f0 2024-06-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.580+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5626877383f0 0x56268773a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8ed9e00700 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5626877383f0 0x56268773a810 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe10f0921c30742 server_cookie=3897eae2191c48ad in_seq=0 out_seq=0 2024-06-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8ebbe00700 10 client.25472 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8ebbe00700 1 -- 172.21.15.26:0/1368770160 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562687487250 con 0x5626877383f0 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8ebbe00700 10 client.25472 handle_command_reply: tid=0 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 client.25472 shutdown 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/1368770160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 msgr2=0x5626876d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876d0a40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80098e0 tx=0x7f8ec8030c20 comp rx=0 tx=0).stop 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/1368770160 shutdown_connections 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5626877383f0 0x56268773a810 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626876be530 0x5626877364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626876b2d60 0x5626876d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 --2- 172.21.15.26:0/1368770160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626876b3680 0x5626876d0a40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.581+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/1368770160 >> 172.21.15.26:0/1368770160 conn(0x5626875e0170 msgr2=0x5626876b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.582+0000 7f8ebbe00700 0 client.25472 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.582+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/1368770160 shutdown_connections 2024-06-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.582+0000 7f8eeb3cdb80 1 -- 172.21.15.26:0/1368770160 wait complete. 2024-06-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.582+0000 7f8eeb3cdb80 20 client.25472 trim_cache size 0 max 16384 2024-06-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-06-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 -- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ec00a2ca0 msgr2=0x7f8ec00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8006010 tx=0x7f8ec801a040 comp rx=0 tx=0).stop 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 -- 172.21.15.26:0/1936884556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc102620 msgr2=0x7f8edc19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc102620 0x7f8edc19ae10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00d950 tx=0x7f8ecc00dd10 comp rx=0 tx=0).stop 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 -- 172.21.15.26:0/1936884556 shutdown_connections 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8edc107700 0x7f8edc19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8edc102620 0x7f8edc19ae10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 --2- 172.21.15.26:0/1936884556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8edc102fd0 0x7f8edc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 -- 172.21.15.26:0/1936884556 >> 172.21.15.26:0/1936884556 conn(0x7f8edc0fdf10 msgr2=0x7f8edc100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 -- 172.21.15.26:0/1936884556 shutdown_connections 2024-06-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:08.583+0000 7f8edb200700 1 -- 172.21.15.26:0/1936884556 wait complete. 2024-06-13T07:02:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:08 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:02:09.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:08 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:02:09.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:08 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:02:09.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:08 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:02:09.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:08 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:02:09.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:08 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:02:09.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:08 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:02:09.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:08 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:02:09.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:08 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:02:09.500 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16685 4.57 MB/sec warmup 23 sec latency 1816.434 ms 2024-06-13T07:02:10.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:09 smithi026 ceph-mon[32041]: pgmap v559: 129 pgs: 129 active+clean; 26 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-06-13T07:02:10.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:09 smithi135 ceph-mon[36276]: pgmap v559: 129 pgs: 129 active+clean; 26 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-06-13T07:02:10.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:09 smithi136 ceph-mon[37229]: pgmap v559: 129 pgs: 129 active+clean; 26 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-06-13T07:02:10.500 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16685 4.38 MB/sec warmup 24 sec latency 2816.603 ms 2024-06-13T07:02:10.709 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:02:10.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:02:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:11.501 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16685 4.21 MB/sec warmup 25 sec latency 3816.745 ms 2024-06-13T07:02:12.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:11 smithi026 ceph-mon[32041]: pgmap v560: 129 pgs: 129 active+clean; 26 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-06-13T07:02:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:11 smithi135 ceph-mon[36276]: pgmap v560: 129 pgs: 129 active+clean; 26 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-06-13T07:02:12.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:11 smithi136 ceph-mon[37229]: pgmap v560: 129 pgs: 129 active+clean; 26 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-06-13T07:02:12.501 INFO:tasks.workunit.client.0.smithi026.stdout: 1 16754 4.05 MB/sec warmup 26 sec latency 4755.947 ms 2024-06-13T07:02:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 -- 172.21.15.26:0/740587398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8104c50 msgr2=0x7fbfc81050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 --2- 172.21.15.26:0/740587398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8104c50 0x7fbfc81050d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009900 tx=0x7fbfb002a410 comp rx=0 tx=0).stop 2024-06-13T07:02:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 -- 172.21.15.26:0/740587398 shutdown_connections 2024-06-13T07:02:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 --2- 172.21.15.26:0/740587398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc8105610 0x7fbfc810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 --2- 172.21.15.26:0/740587398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8104c50 0x7fbfc81050d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 --2- 172.21.15.26:0/740587398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfc8103a60 0x7fbfc8103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 -- 172.21.15.26:0/740587398 >> 172.21.15.26:0/740587398 conn(0x7fbfc80fefd0 msgr2=0x7fbfc8101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 -- 172.21.15.26:0/740587398 shutdown_connections 2024-06-13T07:02:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.956+0000 7fbfcda00640 1 -- 172.21.15.26:0/740587398 wait complete. 2024-06-13T07:02:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.957+0000 7fbfcda00640 1 Processor -- start 2024-06-13T07:02:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfcda00640 1 -- start start 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfcda00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8103a60 0x7fbfc8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfcda00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfc8104c50 0x7fbfc819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfcda00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc8105610 0x7fbfc81a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfcda00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc810f710 con 0x7fbfc8105610 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfcda00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc810f590 con 0x7fbfc8104c50 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfcda00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc810f890 con 0x7fbfc8103a60 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8103a60 0x7fbfc8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfbf400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfc8104c50 0x7fbfc819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8103a60 0x7fbfc8199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55856/0 (socket says 172.21.15.26:55856) 2024-06-13T07:02:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfc7e00640 1 -- 172.21.15.26:0/1982810494 learned_addr learned my addr 172.21.15.26:0/1982810494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.958+0000 7fbfcd000640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc8105610 0x7fbfc81a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.959+0000 7fbfbf400640 1 -- 172.21.15.26:0/1982810494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8103a60 msgr2=0x7fbfc8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.959+0000 7fbfbf400640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8103a60 0x7fbfc8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.959+0000 7fbfbf400640 1 -- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc8105610 msgr2=0x7fbfc81a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.959+0000 7fbfbf400640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc8105610 0x7fbfc81a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.959+0000 7fbfbf400640 1 -- 172.21.15.26:0/1982810494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc81a1950 con 0x7fbfc8104c50 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.959+0000 7fbfcd000640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc8105610 0x7fbfc81a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.959+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8103a60 0x7fbfc8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.959+0000 7fbfbf400640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfc8104c50 0x7fbfc819a280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbfb002a930 tx=0x7fbfb0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.960+0000 7fbfc5600640 1 -- 172.21.15.26:0/1982810494 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb003f030 con 0x7fbfc8104c50 2024-06-13T07:02:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.960+0000 7fbfcda00640 1 -- 172.21.15.26:0/1982810494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc81a1be0 con 0x7fbfc8104c50 2024-06-13T07:02:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.960+0000 7fbfc5600640 1 -- 172.21.15.26:0/1982810494 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb0003ce0 con 0x7fbfc8104c50 2024-06-13T07:02:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.960+0000 7fbfc5600640 1 -- 172.21.15.26:0/1982810494 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0035070 con 0x7fbfc8104c50 2024-06-13T07:02:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.960+0000 7fbfcda00640 1 -- 172.21.15.26:0/1982810494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc81a20a0 con 0x7fbfc8104c50 2024-06-13T07:02:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.963+0000 7fbfc5600640 1 -- 172.21.15.26:0/1982810494 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbfb0042030 con 0x7fbfc8104c50 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.964+0000 7fbfd1e65740 1 Processor -- start 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.964+0000 7fbfd1e65740 1 -- start start 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.964+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdbfb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.964+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdc002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.964+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc02db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.964+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5648fdb5e420 con 0x5648fdbfb7b0 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.964+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648fdb05c50 con 0x5648fdbfadc0 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.964+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5648fdb5a3e0 con 0x5648fdc00920 2024-06-13T07:02:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfcd000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdbfb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfcd000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdbfb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37432/0 (socket says 172.21.15.26:37432) 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfcd000640 1 -- 172.21.15.26:0/3715958079 learned_addr learned my addr 172.21.15.26:0/3715958079 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdc002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfbf400640 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc02db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfcd000640 1 -- 172.21.15.26:0/3715958079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 msgr2=0x5648fdc02db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfcd000640 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc02db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfcd000640 1 -- 172.21.15.26:0/3715958079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 msgr2=0x5648fdc002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfcd000640 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdc002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.965+0000 7fbfcd000640 1 -- 172.21.15.26:0/3715958079 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fd872210 con 0x5648fdbfadc0 2024-06-13T07:02:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.966+0000 7fbfcd000640 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdbfb1e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009fd0 tx=0x7fbfc00108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8dd206c778dd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.966+0000 7fbfc5600640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfac0a2b80 0x7fbfac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.966+0000 7fbfbd600640 1 -- 172.21.15.26:0/3715958079 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0011760 con 0x5648fdbfadc0 2024-06-13T07:02:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.966+0000 7fbfbd600640 1 -- 172.21.15.26:0/3715958079 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc0011de0 con 0x5648fdbfadc0 2024-06-13T07:02:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.966+0000 7fbfbd600640 1 -- 172.21.15.26:0/3715958079 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000e3c0 con 0x5648fdbfadc0 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.966+0000 7fbfc5600640 1 -- 172.21.15.26:0/1982810494 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbfb00ef930 con 0x7fbfc8104c50 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.966+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfac0a2b80 0x7fbfac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 -- 172.21.15.26:0/3715958079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 msgr2=0x5648fdbfb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdbfb1e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009fd0 tx=0x7fbfc00108e0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfac0a2b80 0x7fbfac0a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5648fdbfbbb0 tx=0x7fbfb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 -- 172.21.15.26:0/3715958079 shutdown_connections 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc02db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdc002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/3715958079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdbfb1e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 -- 172.21.15.26:0/3715958079 >> 172.21.15.26:0/3715958079 conn(0x5648fdbc2fb0 msgr2=0x5648fdbf6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 -- 172.21.15.26:0/3715958079 shutdown_connections 2024-06-13T07:02:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.967+0000 7fbfd1e65740 1 -- 172.21.15.26:0/3715958079 wait complete. 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 1 -- start start 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 1 Processor -- start 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdc05670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc0c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5648fdc038d0 con 0x5648fdbfb7b0 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648fdb190b0 con 0x5648fdbfadc0 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5648fdc03a50 con 0x5648fdc00920 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.968+0000 7fbfd1e65740 10 client.0 _ll_register_callbacks cb 0x5648fdb04f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc0c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc05130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfbf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdc05670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc0c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55864/0 (socket says 172.21.15.26:55864) 2024-06-13T07:02:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 -- 172.21.15.26:0/2025516140 learned_addr learned my addr 172.21.15.26:0/2025516140 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 -- 172.21.15.26:0/2025516140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 msgr2=0x5648fdc05130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc05130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 -- 172.21.15.26:0/2025516140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 msgr2=0x5648fdc05670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdc05670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 -- 172.21.15.26:0/2025516140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fd93e620 con 0x5648fdc00920 2024-06-13T07:02:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfcd000640 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc0c6b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0006010 tx=0x7fbfc0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10821ffddbfc8c02 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.969+0000 7fbfb6000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfb6000640 1 -- 172.21.15.26:0/2025516140 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc002b060 con 0x5648fdc00920 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfb6000640 1 -- 172.21.15.26:0/2025516140 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbfc0011560 con 0x5648fdc00920 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2025516140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc0011de0 con 0x5648fdc00920 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfb6000640 1 -- 172.21.15.26:0/2025516140 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbfc0011760 con 0x5648fdc00920 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfd1e65740 10 client.25525 fetch_fsmap learned FSMap version 30 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2025516140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648fdc0fa20 con 0x5648fdc00920 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfb6000640 1 -- 172.21.15.26:0/2025516140 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc0011de0 con 0x5648fdc00920 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfb6000640 1 -- 172.21.15.26:0/2025516140 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fbfc0026470 con 0x5648fdc00920 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfd1e65740 10 client.25525 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.970+0000 7fbfd1e65740 10 client.25525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.971+0000 7fbfd1e65740 20 client.25525 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.971+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5648fdc10e20 0x5648fdc13210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.971+0000 7fbfd1e65740 4 client.25525 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.971+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2025516140 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648fdb6bb50 con 0x5648fdc10e20 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.971+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5648fdc10e20 0x5648fdc13210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.972+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5648fdc10e20 0x5648fdc13210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd683a04561d9d server_cookie=3d9ec4318e4b0ef6 in_seq=0 out_seq=0 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.972+0000 7fbfb6000640 10 client.25525 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.972+0000 7fbfb6000640 1 -- 172.21.15.26:0/2025516140 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5648fdb6bb50 con 0x5648fdc10e20 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.972+0000 7fbfb6000640 10 client.25525 handle_command_reply: tid=0 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 client.25525 shutdown 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2025516140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 msgr2=0x5648fdc0c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc0c6b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0006010 tx=0x7fbfc0004a40 comp rx=0 tx=0).stop 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2025516140 shutdown_connections 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5648fdc10e20 0x5648fdc13210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdc00920 0x5648fdc0c6b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdc05670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2025516140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc05130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfb6000640 0 client.25525 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2025516140 >> 172.21.15.26:0/2025516140 conn(0x5648fdbc2fb0 msgr2=0x5648fdbf59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2025516140 shutdown_connections 2024-06-13T07:02:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2025516140 wait complete. 2024-06-13T07:02:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.973+0000 7fbfd1e65740 20 client.25525 trim_cache size 0 max 16384 2024-06-13T07:02:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfd1e65740 1 Processor -- start 2024-06-13T07:02:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfd1e65740 1 -- start start 2024-06-13T07:02:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdbf97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdbf9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81af0f0 con 0x5648fdbfb7b0 2024-06-13T07:02:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ac990 con 0x5648fdbfadc0 2024-06-13T07:02:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81aa7e0 con 0x5648fdbc2fb0 2024-06-13T07:02:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfbf400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdbf97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdbf97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55880/0 (socket says 172.21.15.26:55880) 2024-06-13T07:02:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfbf400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc0c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37450/0 (socket says 172.21.15.26:37450) 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfc7e00640 1 -- 172.21.15.26:0/2238339594 learned_addr learned my addr 172.21.15.26:0/2238339594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.992+0000 7fbfcd000640 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdbf9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.993+0000 7fbfbf400640 1 -- 172.21.15.26:0/2238339594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 msgr2=0x5648fdbf97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.993+0000 7fbfbf400640 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdbf97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.993+0000 7fbfbf400640 1 -- 172.21.15.26:0/2238339594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 msgr2=0x5648fdbf9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.993+0000 7fbfbf400640 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdbf9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.993+0000 7fbfbf400640 1 -- 172.21.15.26:0/2238339594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fdc03b90 con 0x5648fdbfadc0 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.993+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdbf97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.993+0000 7fbfbf400640 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc0c670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00ecfe0 tx=0x7fbfb002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4c4c5a43222a6e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.993+0000 7fbfbcc00640 1 -- 172.21.15.26:0/2238339594 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0049040 con 0x5648fdbfadc0 2024-06-13T07:02:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfbcc00640 1 -- 172.21.15.26:0/2238339594 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb00bc440 con 0x5648fdbfadc0 2024-06-13T07:02:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfbcc00640 1 -- 172.21.15.26:0/2238339594 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00c00f0 con 0x5648fdbfadc0 2024-06-13T07:02:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2238339594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 msgr2=0x5648fdc0c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc0c670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00ecfe0 tx=0x7fbfb002a410 comp rx=0 tx=0).stop 2024-06-13T07:02:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2238339594 shutdown_connections 2024-06-13T07:02:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfadc0 0x5648fdc0c670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfb7b0 0x5648fdbf9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/2238339594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdbf97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.994+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2238339594 >> 172.21.15.26:0/2238339594 conn(0x5648fdc004d0 msgr2=0x5648fdc05590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.995+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2238339594 shutdown_connections 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.995+0000 7fbfd1e65740 1 -- 172.21.15.26:0/2238339594 wait complete. 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.995+0000 7fbfd1e65740 1 -- start start 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.995+0000 7fbfd1e65740 1 Processor -- start 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdc09b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfadc0 0x5648fdc0a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfd1e65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfb7b0 0x5648fdc0b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81aae50 con 0x5648fdbfadc0 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ac2c0 con 0x5648fdbfb7b0 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfd1e65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ab520 con 0x5648fdbc2fb0 2024-06-13T07:02:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfd1e65740 10 client.0 _ll_register_callbacks cb 0x7fbfb8009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdc09b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfc7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdc09b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55890/0 (socket says 172.21.15.26:55890) 2024-06-13T07:02:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfc7e00640 1 -- 172.21.15.26:0/1277879941 learned_addr learned my addr 172.21.15.26:0/1277879941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfcd000640 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfb7b0 0x5648fdc0b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfbf400640 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfadc0 0x5648fdc0a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfc7e00640 1 -- 172.21.15.26:0/1277879941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfb7b0 msgr2=0x5648fdc0b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfb7b0 0x5648fdc0b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.996+0000 7fbfc7e00640 1 -- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfadc0 msgr2=0x5648fdc0a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.997+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfadc0 0x5648fdc0a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.997+0000 7fbfc7e00640 1 -- 172.21.15.26:0/1277879941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fdc0b6c0 con 0x5648fdbc2fb0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.997+0000 7fbfbf400640 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfadc0 0x5648fdc0a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.997+0000 7fbfc7e00640 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdc09b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbfc819b210 tx=0x7fbfb80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45cea603ecb1575c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.997+0000 7fbfb6000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.997+0000 7fbfb6000640 1 -- 172.21.15.26:0/1277879941 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb802b380 con 0x5648fdbc2fb0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.997+0000 7fbfd1e65740 1 -- 172.21.15.26:0/1277879941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb00bc440 con 0x5648fdbc2fb0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.997+0000 7fbfb6000640 1 -- 172.21.15.26:0/1277879941 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbfb802b520 con 0x5648fdbc2fb0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfb6000640 1 -- 172.21.15.26:0/1277879941 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbfb803e050 con 0x5648fdbc2fb0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfd1e65740 10 client.25531 fetch_fsmap learned FSMap version 30 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfd1e65740 1 -- 172.21.15.26:0/1277879941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648fdc7ba90 con 0x5648fdbc2fb0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfb6000640 1 -- 172.21.15.26:0/1277879941 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb00bc440 con 0x5648fdbc2fb0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfb6000640 1 -- 172.21.15.26:0/1277879941 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fbfb801f910 con 0x5648fdbc2fb0 2024-06-13T07:02:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfd1e65740 10 client.25531 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T07:02:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfd1e65740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfd1e65740 20 client.25531 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5648fdc00920 0x5648fdc7e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfd1e65740 4 client.25531 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfd1e65740 1 -- 172.21.15.26:0/1277879941 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648fdb65ea0 con 0x5648fdc00920 2024-06-13T07:02:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.998+0000 7fbfbf400640 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5648fdc00920 0x5648fdc7e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.999+0000 7fbfbf400640 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5648fdc00920 0x5648fdc7e4a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe3759bdcf2abb5 server_cookie=c3775e3995bc021c in_seq=0 out_seq=0 2024-06-13T07:02:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:12.999+0000 7fbfb6000640 10 client.25531 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfb6000640 1 -- 172.21.15.26:0/1277879941 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5648fdb65ea0 con 0x5648fdc00920 2024-06-13T07:02:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfb6000640 10 client.25531 handle_command_reply: tid=0 2024-06-13T07:02:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 client.25531 shutdown 2024-06-13T07:02:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 -- 172.21.15.26:0/1277879941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 msgr2=0x5648fdc09b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdc09b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbfc819b210 tx=0x7fbfb80046c0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 -- 172.21.15.26:0/1277879941 shutdown_connections 2024-06-13T07:02:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5648fdc00920 0x5648fdc7e4a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fdbfb7b0 0x5648fdc0b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648fdbfadc0 0x5648fdc0a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 --2- 172.21.15.26:0/1277879941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648fdbc2fb0 0x5648fdc09b80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfb6000640 0 client.25531 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.020+0000 7fbfd1e65740 1 -- 172.21.15.26:0/1277879941 >> 172.21.15.26:0/1277879941 conn(0x5648fdc004d0 msgr2=0x5648fdc17ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.021+0000 7fbfd1e65740 1 -- 172.21.15.26:0/1277879941 shutdown_connections 2024-06-13T07:02:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.021+0000 7fbfd1e65740 1 -- 172.21.15.26:0/1277879941 wait complete. 2024-06-13T07:02:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.021+0000 7fbfd1e65740 20 client.25531 trim_cache size 0 max 16384 2024-06-13T07:02:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:02:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:02:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 -- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfac0a2b80 msgr2=0x7fbfac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfac0a2b80 0x7fbfac0a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5648fdbfbbb0 tx=0x7fbfb8005f90 comp rx=0 tx=0).stop 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 -- 172.21.15.26:0/1982810494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfc8104c50 msgr2=0x7fbfc819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfc8104c50 0x7fbfc819a280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbfb002a930 tx=0x7fbfb0032ce0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 -- 172.21.15.26:0/1982810494 shutdown_connections 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbfac0a2b80 0x7fbfac0a5040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfc8104c50 0x7fbfc819a280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc8105610 0x7fbfc81a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 --2- 172.21.15.26:0/1982810494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfc8103a60 0x7fbfc8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 -- 172.21.15.26:0/1982810494 >> 172.21.15.26:0/1982810494 conn(0x7fbfc80fefd0 msgr2=0x7fbfc8107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 -- 172.21.15.26:0/1982810494 shutdown_connections 2024-06-13T07:02:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:13.022+0000 7fbfcda00640 1 -- 172.21.15.26:0/1982810494 wait complete. 2024-06-13T07:02:13.501 INFO:tasks.workunit.client.0.smithi026.stdout: 1 17975 4.30 MB/sec warmup 27 sec latency 215.220 ms 2024-06-13T07:02:13.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:13 smithi026 ceph-mon[32041]: pgmap v561: 129 pgs: 129 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 2.8 MiB/s wr, 168 op/s 2024-06-13T07:02:13.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:13 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 -- 172.21.15.26:0/3293889360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c101690 msgr2=0x7f612c1129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 --2- 172.21.15.26:0/3293889360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c101690 0x7f612c1129d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6118009990 tx=0x7f6118009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 -- 172.21.15.26:0/3293889360 shutdown_connections 2024-06-13T07:02:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 --2- 172.21.15.26:0/3293889360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f612c113090 0x7f612c115530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 --2- 172.21.15.26:0/3293889360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c100ce0 0x7f612c1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 --2- 172.21.15.26:0/3293889360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c101690 0x7f612c1129d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 -- 172.21.15.26:0/3293889360 >> 172.21.15.26:0/3293889360 conn(0x7f612c077a00 msgr2=0x7f612c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 -- 172.21.15.26:0/3293889360 shutdown_connections 2024-06-13T07:02:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.009+0000 7f6132e00700 1 -- 172.21.15.26:0/3293889360 wait complete. 2024-06-13T07:02:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.011+0000 7f6132e00700 1 Processor -- start 2024-06-13T07:02:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.011+0000 7f6132e00700 1 -- start start 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.011+0000 7f6132e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c100ce0 0x7f612c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.011+0000 7f6132e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c101690 0x7f612c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.011+0000 7f6132e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f612c113090 0x7f612c1a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.011+0000 7f6132e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f612c118df0 con 0x7f612c101690 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.011+0000 7f6132e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f612c118c70 con 0x7f612c100ce0 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.011+0000 7f6132e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f612c118f70 con 0x7f612c113090 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c101690 0x7f612c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f6130e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f612c113090 0x7f612c1a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c100ce0 0x7f612c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f6130e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f612c113090 0x7f612c1a3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55906/0 (socket says 172.21.15.26:55906) 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c101690 0x7f612c19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56000/0 (socket says 172.21.15.26:56000) 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f6130e00700 1 -- 172.21.15.26:0/4017460508 learned_addr learned my addr 172.21.15.26:0/4017460508 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612be00700 1 -- 172.21.15.26:0/4017460508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f612c113090 msgr2=0x7f612c1a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612be00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f612c113090 0x7f612c1a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612be00700 1 -- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c101690 msgr2=0x7f612c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612be00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c101690 0x7f612c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612be00700 1 -- 172.21.15.26:0/4017460508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c1a4260 con 0x7f612c100ce0 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f6130e00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f612c113090 0x7f612c1a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:14.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.012+0000 7f612b400700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c101690 0x7f612c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:14.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.013+0000 7f612be00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c100ce0 0x7f612c19f2a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e9f0 tx=0x7f611c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:14.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.013+0000 7f6128c00700 1 -- 172.21.15.26:0/4017460508 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00cb70 con 0x7f612c100ce0 2024-06-13T07:02:14.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.013+0000 7f6132e00700 1 -- 172.21.15.26:0/4017460508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c1a4550 con 0x7f612c100ce0 2024-06-13T07:02:14.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.013+0000 7f6128c00700 1 -- 172.21.15.26:0/4017460508 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f611c00ee90 con 0x7f612c100ce0 2024-06-13T07:02:14.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.013+0000 7f6128c00700 1 -- 172.21.15.26:0/4017460508 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c003dc0 con 0x7f612c100ce0 2024-06-13T07:02:14.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.013+0000 7f6132e00700 1 -- 172.21.15.26:0/4017460508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c1abdd0 con 0x7f612c100ce0 2024-06-13T07:02:14.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.017+0000 7f6128c00700 1 -- 172.21.15.26:0/4017460508 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f611c004000 con 0x7f612c100ce0 2024-06-13T07:02:14.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.018+0000 7f6128c00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f61100a2c20 0x7f61100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.019+0000 7f6128c00700 1 -- 172.21.15.26:0/4017460508 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f611c0cf5f0 con 0x7f612c100ce0 2024-06-13T07:02:14.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.019+0000 7f612b400700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f61100a2c20 0x7f61100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.019+0000 7f613d7beb80 1 Processor -- start 2024-06-13T07:02:14.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.019+0000 7f613d7beb80 1 -- start start 2024-06-13T07:02:14.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f612b400700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f61100a2c20 0x7f61100a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f612c112a50 tx=0x7f611800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa3950ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 0x563fa394ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f613d7beb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563fa3859a90 con 0x563fa3951530 2024-06-13T07:02:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f613d7beb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563fa389cb80 con 0x563fa394b2c0 2024-06-13T07:02:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f613d7beb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563fa3857970 con 0x563fa3950c10 2024-06-13T07:02:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f6130e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f612be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa3950ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.020+0000 7f612be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa3950ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55922/0 (socket says 172.21.15.26:55922) 2024-06-13T07:02:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f612be00700 1 -- 172.21.15.26:0/45514583 learned_addr learned my addr 172.21.15.26:0/45514583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f6130e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37486/0 (socket says 172.21.15.26:37486) 2024-06-13T07:02:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f612b400700 1 --2- 172.21.15.26:0/45514583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 0x563fa394ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f6130e00700 1 -- 172.21.15.26:0/45514583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 msgr2=0x563fa3950ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f6130e00700 1 --2- 172.21.15.26:0/45514583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa3950ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f6130e00700 1 -- 172.21.15.26:0/45514583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 msgr2=0x563fa394ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f6130e00700 1 --2- 172.21.15.26:0/45514583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 0x563fa394ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f6130e00700 1 -- 172.21.15.26:0/45514583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa3845010 con 0x563fa394b2c0 2024-06-13T07:02:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f6130e00700 1 --2- 172.21.15.26:0/45514583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394b700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6124009430 tx=0x7f6124009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af0af3033393547 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.021+0000 7f6121600700 1 -- 172.21.15.26:0/45514583 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124019070 con 0x563fa394b2c0 2024-06-13T07:02:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.022+0000 7f6121600700 1 -- 172.21.15.26:0/45514583 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6124004db0 con 0x563fa394b2c0 2024-06-13T07:02:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.022+0000 7f6121600700 1 -- 172.21.15.26:0/45514583 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124005520 con 0x563fa394b2c0 2024-06-13T07:02:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.022+0000 7f613d7beb80 1 -- 172.21.15.26:0/45514583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 msgr2=0x563fa394b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.022+0000 7f613d7beb80 1 --2- 172.21.15.26:0/45514583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394b700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6124009430 tx=0x7f6124009920 comp rx=0 tx=0).stop 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.023+0000 7f613d7beb80 1 -- 172.21.15.26:0/45514583 shutdown_connections 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.023+0000 7f613d7beb80 1 --2- 172.21.15.26:0/45514583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394b700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.023+0000 7f613d7beb80 1 --2- 172.21.15.26:0/45514583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa3950ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.023+0000 7f613d7beb80 1 --2- 172.21.15.26:0/45514583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 0x563fa394ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.023+0000 7f613d7beb80 1 -- 172.21.15.26:0/45514583 >> 172.21.15.26:0/45514583 conn(0x563fa38771f0 msgr2=0x563fa3946070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.023+0000 7f613d7beb80 1 -- 172.21.15.26:0/45514583 shutdown_connections 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.023+0000 7f613d7beb80 1 -- 172.21.15.26:0/45514583 wait complete. 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.023+0000 7f613d7beb80 1 -- start start 2024-06-13T07:02:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.024+0000 7f613d7beb80 1 Processor -- start 2024-06-13T07:02:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.024+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa39572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.024+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3950c10 0x563fa39545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.024+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa3954b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.024+0000 7f613d7beb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563fa3838fe0 con 0x563fa3950c10 2024-06-13T07:02:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.024+0000 7f613d7beb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563fa3898b40 con 0x563fa394b2c0 2024-06-13T07:02:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.024+0000 7f613d7beb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563fa3957f10 con 0x563fa3951530 2024-06-13T07:02:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.024+0000 7f613d7beb80 10 client.0 _ll_register_callbacks cb 0x563fa3832400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa39572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612b400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3950c10 0x563fa39545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa39572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37494/0 (socket says 172.21.15.26:37494) 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 -- 172.21.15.26:0/707420616 learned_addr learned my addr 172.21.15.26:0/707420616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f6130e00700 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa3954b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 -- 172.21.15.26:0/707420616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 msgr2=0x563fa3954b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa3954b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 -- 172.21.15.26:0/707420616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3950c10 msgr2=0x563fa39545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3950c10 0x563fa39545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 -- 172.21.15.26:0/707420616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa37eb950 con 0x563fa394b2c0 2024-06-13T07:02:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f612be00700 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa39572e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e9c0 tx=0x7f611c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3520be809cf38c70 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.025+0000 7f6116000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.026+0000 7f6116000700 1 -- 172.21.15.26:0/707420616 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b3070 con 0x563fa394b2c0 2024-06-13T07:02:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.026+0000 7f6116000700 1 -- 172.21.15.26:0/707420616 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f611c009400 con 0x563fa394b2c0 2024-06-13T07:02:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.026+0000 7f613d7beb80 1 -- 172.21.15.26:0/707420616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6124004db0 con 0x563fa394b2c0 2024-06-13T07:02:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.026+0000 7f6116000700 1 -- 172.21.15.26:0/707420616 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f611c09fdb0 con 0x563fa394b2c0 2024-06-13T07:02:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.026+0000 7f613d7beb80 10 client.25502 fetch_fsmap learned FSMap version 30 2024-06-13T07:02:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.026+0000 7f613d7beb80 1 -- 172.21.15.26:0/707420616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563fa3962730 con 0x563fa394b2c0 2024-06-13T07:02:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.026+0000 7f6116000700 1 -- 172.21.15.26:0/707420616 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6124004db0 con 0x563fa394b2c0 2024-06-13T07:02:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.026+0000 7f6116000700 1 -- 172.21.15.26:0/707420616 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f611c09bc60 con 0x563fa394b2c0 2024-06-13T07:02:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.027+0000 7f613d7beb80 10 client.25502 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T07:02:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.027+0000 7f613d7beb80 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.027+0000 7f613d7beb80 20 client.25502 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.027+0000 7f613d7beb80 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563fa3964d10 0x563fa3967140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.027+0000 7f613d7beb80 4 client.25502 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.027+0000 7f613d7beb80 1 -- 172.21.15.26:0/707420616 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fa38aa080 con 0x563fa3964d10 2024-06-13T07:02:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.027+0000 7f612b400700 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563fa3964d10 0x563fa3967140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.028+0000 7f612b400700 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563fa3964d10 0x563fa3967140 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90159c393532b2d0 server_cookie=45e06444862d5793 in_seq=0 out_seq=0 2024-06-13T07:02:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.028+0000 7f6116000700 10 client.25502 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.029+0000 7f6116000700 1 -- 172.21.15.26:0/707420616 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563fa38aa080 con 0x563fa3964d10 2024-06-13T07:02:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.029+0000 7f6116000700 10 client.25502 handle_command_reply: tid=0 2024-06-13T07:02:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.029+0000 7f613d7beb80 1 client.25502 shutdown 2024-06-13T07:02:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.029+0000 7f613d7beb80 1 -- 172.21.15.26:0/707420616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 msgr2=0x563fa39572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.029+0000 7f613d7beb80 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa39572e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e9c0 tx=0x7f611c003820 comp rx=0 tx=0).stop 2024-06-13T07:02:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 1 -- 172.21.15.26:0/707420616 shutdown_connections 2024-06-13T07:02:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563fa3964d10 0x563fa3967140 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa39572e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa3954b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 1 --2- 172.21.15.26:0/707420616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3950c10 0x563fa39545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f6116000700 0 client.25502 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 1 -- 172.21.15.26:0/707420616 >> 172.21.15.26:0/707420616 conn(0x563fa38771f0 msgr2=0x563fa393f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 1 -- 172.21.15.26:0/707420616 shutdown_connections 2024-06-13T07:02:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 1 -- 172.21.15.26:0/707420616 wait complete. 2024-06-13T07:02:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.030+0000 7f613d7beb80 20 client.25502 trim_cache size 0 max 16384 2024-06-13T07:02:14.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:13 smithi135 ceph-mon[36276]: pgmap v561: 129 pgs: 129 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 2.8 MiB/s wr, 168 op/s 2024-06-13T07:02:14.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:13 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.057+0000 7f613d7beb80 1 Processor -- start 2024-06-13T07:02:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.057+0000 7f613d7beb80 1 -- start start 2024-06-13T07:02:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa39401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 0x563fa39534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f613d7beb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1b7400 con 0x563fa3951530 2024-06-13T07:02:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f613d7beb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1b4c60 con 0x563fa394b2c0 2024-06-13T07:02:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f613d7beb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1b2ab0 con 0x563fa3950c10 2024-06-13T07:02:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f612be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f612be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37498/0 (socket says 172.21.15.26:37498) 2024-06-13T07:02:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f612be00700 1 -- 172.21.15.26:0/517117325 learned_addr learned my addr 172.21.15.26:0/517117325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f612b400700 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa39401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.058+0000 7f6130e00700 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 0x563fa39534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f612b400700 1 -- 172.21.15.26:0/517117325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 msgr2=0x563fa394ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f612b400700 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f612b400700 1 -- 172.21.15.26:0/517117325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 msgr2=0x563fa39534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f612b400700 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 0x563fa39534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f612b400700 1 -- 172.21.15.26:0/517117325 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa39677c0 con 0x563fa3950c10 2024-06-13T07:02:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f612b400700 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa39401b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61180098e0 tx=0x7f6118020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eddfb6623d19773 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f6120c00700 1 -- 172.21.15.26:0/517117325 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118040070 con 0x563fa3950c10 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f6120c00700 1 -- 172.21.15.26:0/517117325 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f611800f460 con 0x563fa3950c10 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.059+0000 7f6120c00700 1 -- 172.21.15.26:0/517117325 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800bea0 con 0x563fa3950c10 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 -- 172.21.15.26:0/517117325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 msgr2=0x563fa39401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa39401b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61180098e0 tx=0x7f6118020000 comp rx=0 tx=0).stop 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 -- 172.21.15.26:0/517117325 shutdown_connections 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa394b2c0 0x563fa394ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa3951530 0x563fa39534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 --2- 172.21.15.26:0/517117325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3950c10 0x563fa39401b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 -- 172.21.15.26:0/517117325 >> 172.21.15.26:0/517117325 conn(0x563fa38771f0 msgr2=0x563fa3952da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 -- 172.21.15.26:0/517117325 shutdown_connections 2024-06-13T07:02:14.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.060+0000 7f613d7beb80 1 -- 172.21.15.26:0/517117325 wait complete. 2024-06-13T07:02:14.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.061+0000 7f613d7beb80 1 -- start start 2024-06-13T07:02:14.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.061+0000 7f613d7beb80 1 Processor -- start 2024-06-13T07:02:14.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa394b2c0 0x563fa395d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa3950c10 0x563fa395d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f613d7beb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa395a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f613d7beb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1b3120 con 0x563fa394b2c0 2024-06-13T07:02:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f613d7beb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1b4590 con 0x563fa3950c10 2024-06-13T07:02:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f613d7beb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1b37f0 con 0x563fa3951530 2024-06-13T07:02:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f613d7beb80 10 client.0 _ll_register_callbacks cb 0x563fa3832400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa3950c10 0x563fa395d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa3950c10 0x563fa395d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37506/0 (socket says 172.21.15.26:37506) 2024-06-13T07:02:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f6130e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa395a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612be00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa394b2c0 0x563fa395d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612b400700 1 -- 172.21.15.26:0/887352031 learned_addr learned my addr 172.21.15.26:0/887352031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612b400700 1 -- 172.21.15.26:0/887352031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 msgr2=0x563fa395a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612b400700 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa395a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612b400700 1 -- 172.21.15.26:0/887352031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa394b2c0 msgr2=0x563fa395d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612b400700 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa394b2c0 0x563fa395d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.062+0000 7f612b400700 1 -- 172.21.15.26:0/887352031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa395aa60 con 0x563fa3950c10 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.063+0000 7f6130e00700 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa395a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.063+0000 7f612b400700 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa3950c10 0x563fa395d9d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f61180098e0 tx=0x7f6118009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503ce5875b100892 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.063+0000 7f6116000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.063+0000 7f6116000700 1 -- 172.21.15.26:0/887352031 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118040070 con 0x563fa3950c10 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.063+0000 7f6116000700 1 -- 172.21.15.26:0/887352031 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6118003c60 con 0x563fa3950c10 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.063+0000 7f613d7beb80 1 -- 172.21.15.26:0/887352031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611800f460 con 0x563fa3950c10 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f6116000700 1 -- 172.21.15.26:0/887352031 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f611804a050 con 0x563fa3950c10 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f613d7beb80 10 client.25508 fetch_fsmap learned FSMap version 30 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f613d7beb80 1 -- 172.21.15.26:0/887352031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563fa39d05f0 con 0x563fa3950c10 2024-06-13T07:02:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f6116000700 1 -- 172.21.15.26:0/887352031 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f611800f460 con 0x563fa3950c10 2024-06-13T07:02:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f6116000700 1 -- 172.21.15.26:0/887352031 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f611803a030 con 0x563fa3950c10 2024-06-13T07:02:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f613d7beb80 10 client.25508 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T07:02:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f613d7beb80 10 client.25508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f613d7beb80 20 client.25508 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f613d7beb80 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563fa39d0cf0 0x563fa39d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f613d7beb80 4 client.25508 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:02:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.064+0000 7f613d7beb80 1 -- 172.21.15.26:0/887352031 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563fa394f080 con 0x563fa39d0cf0 2024-06-13T07:02:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.065+0000 7f612be00700 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563fa39d0cf0 0x563fa39d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.065+0000 7f612be00700 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563fa39d0cf0 0x563fa39d3110 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48f122447a766fc server_cookie=91c66e1eadb7d163 in_seq=0 out_seq=0 2024-06-13T07:02:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.065+0000 7f6116000700 10 client.25508 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.066+0000 7f6116000700 1 -- 172.21.15.26:0/887352031 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563fa394f080 con 0x563fa39d0cf0 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.066+0000 7f6116000700 10 client.25508 handle_command_reply: tid=0 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.066+0000 7f613d7beb80 1 client.25508 shutdown 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.066+0000 7f613d7beb80 1 -- 172.21.15.26:0/887352031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa3950c10 msgr2=0x563fa395d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.066+0000 7f613d7beb80 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa3950c10 0x563fa395d9d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f61180098e0 tx=0x7f6118009380 comp rx=0 tx=0).stop 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f613d7beb80 1 -- 172.21.15.26:0/887352031 shutdown_connections 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f613d7beb80 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563fa39d0cf0 0x563fa39d3110 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f613d7beb80 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563fa3951530 0x563fa395a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f6116000700 0 client.25508 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f613d7beb80 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fa394b2c0 0x563fa395d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f613d7beb80 1 --2- 172.21.15.26:0/887352031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fa3950c10 0x563fa395d9d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f613d7beb80 1 -- 172.21.15.26:0/887352031 >> 172.21.15.26:0/887352031 conn(0x563fa38771f0 msgr2=0x563fa3953630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f613d7beb80 1 -- 172.21.15.26:0/887352031 shutdown_connections 2024-06-13T07:02:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.067+0000 7f613d7beb80 1 -- 172.21.15.26:0/887352031 wait complete. 2024-06-13T07:02:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.068+0000 7f613d7beb80 20 client.25508 trim_cache size 0 max 16384 2024-06-13T07:02:14.073 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:02:14.073 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-06-13T07:02:14.073 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "3f365b2e-2520-4732-aa5b-70db1827d5a6", 2024-06-13T07:02:14.073 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-06-13T07:02:14.074 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:02:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.068+0000 7f6132e00700 1 -- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f61100a2c20 msgr2=0x7f61100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.068+0000 7f6132e00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f61100a2c20 0x7f61100a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f612c112a50 tx=0x7f611800b260 comp rx=0 tx=0).stop 2024-06-13T07:02:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.068+0000 7f6132e00700 1 -- 172.21.15.26:0/4017460508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c100ce0 msgr2=0x7f612c19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.068+0000 7f6132e00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c100ce0 0x7f612c19f2a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e9f0 tx=0x7f611c00ed00 comp rx=0 tx=0).stop 2024-06-13T07:02:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.069+0000 7f6132e00700 1 -- 172.21.15.26:0/4017460508 shutdown_connections 2024-06-13T07:02:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.069+0000 7f6132e00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f61100a2c20 0x7f61100a50e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.069+0000 7f6132e00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f612c100ce0 0x7f612c19f2a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.069+0000 7f6132e00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f612c113090 0x7f612c1a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.069+0000 7f6132e00700 1 --2- 172.21.15.26:0/4017460508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c101690 0x7f612c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.069+0000 7f6132e00700 1 -- 172.21.15.26:0/4017460508 >> 172.21.15.26:0/4017460508 conn(0x7f612c077a00 msgr2=0x7f612c111970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.069+0000 7f6132e00700 1 -- 172.21.15.26:0/4017460508 shutdown_connections 2024-06-13T07:02:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:14.069+0000 7f6132e00700 1 -- 172.21.15.26:0/4017460508 wait complete. 2024-06-13T07:02:14.081 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:13 smithi136 ceph-mon[37229]: pgmap v561: 129 pgs: 129 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 2.8 MiB/s wr, 168 op/s 2024-06-13T07:02:14.081 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:13 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:14.377 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: 2024-06-13T07:02:14.079+0000 7f61709d2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000226 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS auth v604 ap=1 s=260096 n(v0 rc2024-06-13T07:01:46.876003+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@76},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55d186aa5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v2801>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000226:[<0x10000000225/T1.XLS v604>,<0x10000000009/PM v3266>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.377 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: 2024-06-13T07:02:14.080+0000 7f61709d2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000227 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC auth v606 ap=1 s=260096 n(v0 rc2024-06-13T07:01:46.889003+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@76},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55d186ab4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v2801>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000227:[<0x10000000225/EVENTS.DOC v606>,<0x10000000009/PM v3266>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.377 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: 2024-06-13T07:02:14.080+0000 7f61709d2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000228 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC auth v602 ap=1 s=260096 n(v0 rc2024-06-13T07:01:46.901003+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@76},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55d186ab4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v2801>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000228:[<0x10000000225/MOVED.DOC v602>,<0x10000000009/PM v3266>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.501 INFO:tasks.workunit.client.0.smithi026.stdout: 1 18528 4.18 MB/sec warmup 28 sec latency 21.588 ms 2024-06-13T07:02:14.534 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.076+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS auth{0=1} v414 ap=1 s=260096 n(v0 rc2024-06-13T07:01:59.824815+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@24},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5649ec918000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v2785>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v414>,<0x10000000009/EXCEL v3074>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.534 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.076+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS auth{0=1} v496 ap=1 s=260096 n(v0 rc2024-06-13T07:02:12.461632+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@30},l=25144 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5649ec918580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v2785>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000020e:[<0x1000000020c/SALES.XLS v496>,<0x10000000009/EXCEL v3074>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.078+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022b [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX auth v504 ap=1 s=260096 n(v0 rc2024-06-13T07:01:59.557819+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ec91a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022b:[<0x10000000229/COURSES.PX v4>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000022b:[<0x10000000229/COURSES.PX v504>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.078+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022c [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04 auth v496 ap=1 s=260096 n(v0 rc2024-06-13T07:01:59.557819+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ec91b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022c:[<0x10000000229/COURSES.X04 v6>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000022c:[<0x10000000229/COURSES.X04 v496>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.079+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000231 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0 auth v498 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.001001+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ec91b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000231:[<0x10000000229/STUDENTS.YG0 v16>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000231:[<0x10000000229/STUDENTS.YG0 v498>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.079+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042c [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.DB auth v518 ap=1 s=19584 n(v0 rc2024-06-13T07:01:59.564819+0000 b19584 1=1+0) caps={25144=pAsLsXsFsc/-@1},l=25144(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ec91ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000042c:[<0x10000000229/ANSWER.DB v440>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000042c:[<0x10000000229/ANSWER.DB v518>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.079+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000237 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX auth v508 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.082000+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbaa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000237:[<0x10000000229/ENROLL.PX v28>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000237:[<0x10000000229/ENROLL.PX v508>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.079+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022f [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX auth v492 ap=1 s=260096 n(v0 rc2024-06-13T07:01:59.556819+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbaab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022f:[<0x10000000229/STUDENTS.PX v12>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000022f:[<0x10000000229/STUDENTS.PX v492>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.080+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000426 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/CHANGED.DB auth v524 ap=1 s=4096 n(v0 rc2024-06-13T07:01:59.365822+0000 b4096 1=1+0) caps={25144=pAsLsXsFsc/-@1},l=25144(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbaa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000426:[<0x10000000229/CHANGED.DB v414>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000426:[<0x10000000229/CHANGED.DB v524>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.080+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023b [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV auth v512 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.137999+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbab080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023b:[<0x10000000229/STUDENTS.TV v36>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000023b:[<0x10000000229/STUDENTS.TV v512>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.080+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023c [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL auth v516 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.149999+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbab600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023c:[<0x10000000229/STUDENTS.FSL v38>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000023c:[<0x10000000229/STUDENTS.FSL v516>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.080+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000233 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1 auth v480 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.028001+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbabb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000233:[<0x10000000229/STUDENTS.YG1 v20>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000233:[<0x10000000229/STUDENTS.YG1 v480>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.535 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.081+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000239 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03 auth v510 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.109000+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbac100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000239:[<0x10000000229/ENROLL.Y03 v32>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000239:[<0x10000000229/ENROLL.Y03 v510>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.081+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023d [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL auth v488 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.162999+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbac680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023d:[<0x10000000229/COURSES.FSL v40>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000023d:[<0x10000000229/COURSES.FSL v488>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.081+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000234 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL auth v500 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.041001+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbacc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000234:[<0x10000000229/STUDENTS.VAL v22>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000234:[<0x10000000229/STUDENTS.VAL v500>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.081+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022a [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB auth v502 ap=1 s=260096 n(v0 rc2024-06-13T07:01:59.557819+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022a:[<0x10000000229/COURSES.DB v2>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000022a:[<0x10000000229/COURSES.DB v502>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.081+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022d [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04 auth v506 ap=1 s=260096 n(v0 rc2024-06-13T07:01:59.557819+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbad700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022d:[<0x10000000229/COURSES.Y04 v8>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000022d:[<0x10000000229/COURSES.Y04 v506>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.082+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000238 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03 auth v494 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.094000+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000238:[<0x10000000229/ENROLL.X03 v30>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000238:[<0x10000000229/ENROLL.X03 v494>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.082+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000235 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB auth v486 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.055000+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000235:[<0x10000000229/STUDENTS.MB v24>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000235:[<0x10000000229/STUDENTS.MB v486>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.082+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023a [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL auth v514 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.123999+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023a:[<0x10000000229/ENROLL.VAL v34>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000023a:[<0x10000000229/ENROLL.VAL v514>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.083+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000236 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB auth v484 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.068000+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000236:[<0x10000000229/ENROLL.DB v26>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000236:[<0x10000000229/ENROLL.DB v484>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.083+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000232 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1 auth v490 ap=1 s=260096 n(v0 rc2024-06-13T07:01:47.014001+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000232:[<0x10000000229/STUDENTS.XG1 v18>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000232:[<0x10000000229/STUDENTS.XG1 v490>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.083+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042b [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.MB auth v522 ap=1 s=4096 n(v0 rc2024-06-13T07:01:59.468820+0000 b4096 1=1+0) caps={25144=pAsLsXsFsc/-@1},l=25144(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000042b:[<0x10000000229/ANSWER.MB v436>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000042b:[<0x10000000229/ANSWER.MB v522>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.536 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.083+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000230 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0 auth v482 ap=1 s=260096 n(v0 rc2024-06-13T07:01:46.988001+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000230:[<0x10000000229/STUDENTS.XG0 v14>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000230:[<0x10000000229/STUDENTS.XG0 v482>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.537 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.083+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000424 [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.VAL auth v520 ap=1 s=543 n(v0 rc2024-06-13T07:01:59.271823+0000 b543 1=1+0) caps={25144=pAsLsXsFsc/-@1},l=25144(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000424:[<0x10000000229/COURSES.VAL v404>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x10000000424:[<0x10000000229/COURSES.VAL v520>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.537 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:02:14.083+0000 7f0080d99700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022e [2,head] /volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB auth v478 ap=1 s=260096 n(v0 rc2024-06-13T07:01:59.556819+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25144=0-4194304@1} caps={25144=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25144 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5649ecbb4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022e:[<0x10000000229/STUDENTS.DB v10>,<0x10000000009/PARADOX v2971>,<0x10000000008/~dmtmp v159>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v42>,<0x10000000001/sv_1 v64>,<0x10000000000/qa v55>,<0x1/volumes v52>]//[3]","memoryvalue":"(4)0x1000000022e:[<0x10000000229/STUDENTS.DB v478>,<0x10000000009/PARADOX v3033>,<0x10000000008/~dmtmp v217>,<0x10000000007/client0 v64>,<0x10000000006/clients v46>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v46>,<0x100000001fa/1d7d31dc-0547-4061-aea7-beb40a9acb25 v60>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v30>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-13T07:02:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: scrub queued for path: / 2024-06-13T07:02:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: scrub summary: active paths [/] 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: scrub summary: active paths [/] 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: scrub summary: active paths [/] 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.i log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/CHANGED.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.i log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.i log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.MB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:14.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:14 smithi026 ceph-mon[32041]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: scrub queued for path: / 2024-06-13T07:02:15.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: scrub summary: active paths [/] 2024-06-13T07:02:15.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: scrub summary: active paths [/] 2024-06-13T07:02:15.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: scrub summary: active paths [/] 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.i log and `damage ls` output for details 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/CHANGED.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.i log and `damage ls` output for details 2024-06-13T07:02:15.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.i log and `damage ls` output for details 2024-06-13T07:02:15.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.MB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.038 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.038 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.038 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.038 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:14 smithi135 ceph-mon[36276]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.077 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:02:15.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: scrub queued for path: / 2024-06-13T07:02:15.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: scrub summary: active paths [/] 2024-06-13T07:02:15.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: scrub summary: active paths [/] 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: scrub summary: active paths [/] 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.i log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/CHANGED.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.i log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.i log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.MB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.VAL) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.131 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.b log and `damage ls` output for details 2024-06-13T07:02:15.131 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.131 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:14 smithi136 ceph-mon[37229]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:15.501 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20118 4.26 MB/sec warmup 29 sec latency 32.580 ms 2024-06-13T07:02:15.562 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:16.006 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:15 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:02:15.695+0000 7fc609800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-13T07:02:16.006 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:15 smithi026 ceph-mon[32041]: pgmap v562: 129 pgs: 129 active+clean; 23 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-06-13T07:02:16.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:15 smithi135 ceph-mon[36276]: pgmap v562: 129 pgs: 129 active+clean; 23 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-06-13T07:02:16.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:15 smithi136 ceph-mon[37229]: pgmap v562: 129 pgs: 129 active+clean; 23 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-06-13T07:02:16.501 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20119 4.12 MB/sec warmup 30 sec latency 999.326 ms 2024-06-13T07:02:17.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:16 smithi026 ceph-mon[32041]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-13T07:02:17.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:16 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:active 2024-06-13T07:02:17.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:16 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:16 smithi135 ceph-mon[36276]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-13T07:02:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:16 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:active 2024-06-13T07:02:17.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:16 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:17.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:16 smithi136 ceph-mon[37229]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-13T07:02:17.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:16 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] up:active 2024-06-13T07:02:17.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:16 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:17.501 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20173 3.99 MB/sec warmup 31 sec latency 1931.270 ms 2024-06-13T07:02:18.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:17 smithi026 ceph-mon[32041]: pgmap v563: 129 pgs: 129 active+clean; 23 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-06-13T07:02:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.009+0000 7f3e1c000700 1 -- 172.21.15.26:0/1490992149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e141084e0 msgr2=0x7f3e141088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.009+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1490992149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e141084e0 0x7f3e141088c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009990 tx=0x7f3e08009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.010+0000 7f3e1c000700 1 -- 172.21.15.26:0/1490992149 shutdown_connections 2024-06-13T07:02:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.010+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1490992149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e141084e0 0x7f3e141088c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.010+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1490992149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e1406ab10 0x7f3e1410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.010+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1490992149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e1406a170 0x7f3e1406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.010+0000 7f3e1c000700 1 -- 172.21.15.26:0/1490992149 >> 172.21.15.26:0/1490992149 conn(0x7f3e140782c0 msgr2=0x7f3e140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.010+0000 7f3e1c000700 1 -- 172.21.15.26:0/1490992149 shutdown_connections 2024-06-13T07:02:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.010+0000 7f3e1c000700 1 -- 172.21.15.26:0/1490992149 wait complete. 2024-06-13T07:02:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.011+0000 7f3e1c000700 1 Processor -- start 2024-06-13T07:02:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.012+0000 7f3e1c000700 1 -- start start 2024-06-13T07:02:18.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.012+0000 7f3e1c000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e1406a170 0x7f3e14101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.012+0000 7f3e1c000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e1406ab10 0x7f3e141019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.012+0000 7f3e1c000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e141084e0 0x7f3e14101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e1c000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14112790 con 0x7f3e141084e0 2024-06-13T07:02:18.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e1c000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14112610 con 0x7f3e1406ab10 2024-06-13T07:02:18.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e1c000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14112910 con 0x7f3e1406a170 2024-06-13T07:02:18.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e19600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e1406a170 0x7f3e14101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e18c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e1406ab10 0x7f3e141019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e19600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e1406a170 0x7f3e14101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55968/0 (socket says 172.21.15.26:55968) 2024-06-13T07:02:18.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e19600700 1 -- 172.21.15.26:0/1244159189 learned_addr learned my addr 172.21.15.26:0/1244159189 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:18.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e1a000700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e141084e0 0x7f3e14101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e18c00700 1 -- 172.21.15.26:0/1244159189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e1406a170 msgr2=0x7f3e14101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e18c00700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e1406a170 0x7f3e14101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e18c00700 1 -- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e141084e0 msgr2=0x7f3e14101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e18c00700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e141084e0 0x7f3e14101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.013+0000 7f3e18c00700 1 -- 172.21.15.26:0/1244159189 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e141a7230 con 0x7f3e1406ab10 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.014+0000 7f3e18c00700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e1406ab10 0x7f3e141019a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000e9f0 tx=0x7f3e0000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.014+0000 7f3e12000700 1 -- 172.21.15.26:0/1244159189 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000c9c0 con 0x7f3e1406ab10 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.014+0000 7f3e12000700 1 -- 172.21.15.26:0/1244159189 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0000cb60 con 0x7f3e1406ab10 2024-06-13T07:02:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.014+0000 7f3e12000700 1 -- 172.21.15.26:0/1244159189 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00018680 con 0x7f3e1406ab10 2024-06-13T07:02:18.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.014+0000 7f3e1c000700 1 -- 172.21.15.26:0/1244159189 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e141a7520 con 0x7f3e1406ab10 2024-06-13T07:02:18.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.014+0000 7f3e1c000700 1 -- 172.21.15.26:0/1244159189 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e141a7ab0 con 0x7f3e1406ab10 2024-06-13T07:02:18.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.017+0000 7f3e12000700 1 -- 172.21.15.26:0/1244159189 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e00010430 con 0x7f3e1406ab10 2024-06-13T07:02:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.020+0000 7f3e12000700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3dfc0a2ca0 0x7f3dfc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.020+0000 7f3e2691fb80 1 Processor -- start 2024-06-13T07:02:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.020+0000 7f3e12000700 1 -- 172.21.15.26:0/1244159189 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f3e000cf510 con 0x7f3e1406ab10 2024-06-13T07:02:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.020+0000 7f3e19600700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3dfc0a2ca0 0x7f3dfc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.020+0000 7f3e2691fb80 1 -- start start 2024-06-13T07:02:18.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97ee9ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97ee98d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97ee99680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a97eda7a10 con 0x55a97ee9f4b0 2024-06-13T07:02:18.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a97edeab00 con 0x55a97ee9eb90 2024-06-13T07:02:18.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a97eda58f0 con 0x55a97ee99240 2024-06-13T07:02:18.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e18c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97ee98d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e19600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97ee9ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e1a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97ee99680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e19600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97ee9ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37542/0 (socket says 172.21.15.26:37542) 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e1a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97ee99680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55976/0 (socket says 172.21.15.26:55976) 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e19600700 1 -- 172.21.15.26:0/1838150527 learned_addr learned my addr 172.21.15.26:0/1838150527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.021+0000 7f3e18c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97ee98d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56046/0 (socket says 172.21.15.26:56046) 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e19600700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3dfc0a2ca0 0x7f3dfc0a5160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009750 tx=0x7f3e0800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e1a000700 1 -- 172.21.15.26:0/1838150527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 msgr2=0x55a97ee9ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e1a000700 1 --2- 172.21.15.26:0/1838150527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97ee9ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e1a000700 1 -- 172.21.15.26:0/1838150527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 msgr2=0x55a97ee98d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e1a000700 1 --2- 172.21.15.26:0/1838150527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97ee98d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e1a000700 1 -- 172.21.15.26:0/1838150527 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a97ed92f90 con 0x55a97ee99240 2024-06-13T07:02:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e18c00700 1 --2- 172.21.15.26:0/1838150527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97ee98d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e1a000700 1 --2- 172.21.15.26:0/1838150527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97ee99680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c011580 tx=0x7f3e0c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=807b874ad9f28a5e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.022+0000 7f3e06a00700 1 -- 172.21.15.26:0/1838150527 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004db0 con 0x55a97ee99240 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e06a00700 1 -- 172.21.15.26:0/1838150527 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0c0090d0 con 0x55a97ee99240 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e06a00700 1 -- 172.21.15.26:0/1838150527 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004720 con 0x55a97ee99240 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1838150527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 msgr2=0x55a97ee99680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1838150527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97ee99680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c011580 tx=0x7f3e0c009920 comp rx=0 tx=0).stop 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1838150527 shutdown_connections 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1838150527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97ee9ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1838150527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97ee99680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1838150527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97ee98d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1838150527 >> 172.21.15.26:0/1838150527 conn(0x55a97edc5170 msgr2=0x55a97ee93ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.023+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1838150527 shutdown_connections 2024-06-13T07:02:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.024+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1838150527 wait complete. 2024-06-13T07:02:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.024+0000 7f3e2691fb80 1 -- start start 2024-06-13T07:02:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.024+0000 7f3e2691fb80 1 Processor -- start 2024-06-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97eea51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97eea24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a97ed86f60 con 0x55a97ee9f4b0 2024-06-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a97ede6ac0 con 0x55a97ee9eb90 2024-06-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a97eea5e90 con 0x55a97ee99240 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e2691fb80 10 client.0 _ll_register_callbacks cb 0x55a97ed80380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e19600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97eea51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e1a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e19600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97eea51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55980/0 (socket says 172.21.15.26:55980) 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e19600700 1 -- 172.21.15.26:0/1900837811 learned_addr learned my addr 172.21.15.26:0/1900837811 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.025+0000 7f3e18c00700 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97eea24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3e19600700 1 -- 172.21.15.26:0/1900837811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 msgr2=0x55a97eea24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3e19600700 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97eea24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3e19600700 1 -- 172.21.15.26:0/1900837811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 msgr2=0x55a97eea29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3e19600700 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3e19600700 1 -- 172.21.15.26:0/1900837811 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a97ed398d0 con 0x55a97ee99240 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3e19600700 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97eea51a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005570 tx=0x7f3e08009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c813cdabf7fb523 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3dfb400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3dfb400700 1 -- 172.21.15.26:0/1900837811 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08028040 con 0x55a97ee99240 2024-06-13T07:02:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3dfb400700 1 -- 172.21.15.26:0/1900837811 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f3e08003d10 con 0x55a97ee99240 2024-06-13T07:02:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.026+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1900837811 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0c0090d0 con 0x55a97ee99240 2024-06-13T07:02:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.027+0000 7f3dfb400700 1 -- 172.21.15.26:0/1900837811 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e0803a050 con 0x55a97ee99240 2024-06-13T07:02:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.027+0000 7f3e2691fb80 10 client.25561 fetch_fsmap learned FSMap version 31 2024-06-13T07:02:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.027+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1900837811 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a97eeb05a0 con 0x55a97ee99240 2024-06-13T07:02:18.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.027+0000 7f3dfb400700 1 -- 172.21.15.26:0/1900837811 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0c0090d0 con 0x55a97ee99240 2024-06-13T07:02:18.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.027+0000 7f3dfb400700 1 -- 172.21.15.26:0/1900837811 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f3e0802c490 con 0x55a97ee99240 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.027+0000 7f3e2691fb80 10 client.25561 fetch_fsmap finished waiting for FSMap version 31 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.027+0000 7f3e2691fb80 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.028+0000 7f3e2691fb80 20 client.25561 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.028+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a97eeb2c30 0x55a97eeb5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.028+0000 7f3e2691fb80 4 client.25561 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.028+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1900837811 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a97edf8000 con 0x55a97eeb2c30 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.028+0000 7f3e18c00700 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a97eeb2c30 0x55a97eeb5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.028+0000 7f3e18c00700 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a97eeb2c30 0x55a97eeb5070 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1facf009a6d2e309 server_cookie=9aa44b9ec635755 in_seq=0 out_seq=0 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.028+0000 7f3dfb400700 10 client.25561 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.029+0000 7f3dfb400700 1 -- 172.21.15.26:0/1900837811 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a97edf8000 con 0x55a97eeb2c30 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.029+0000 7f3dfb400700 10 client.25561 handle_command_reply: tid=0 2024-06-13T07:02:18.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.029+0000 7f3e2691fb80 1 client.25561 shutdown 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1900837811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 msgr2=0x55a97eea51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97eea51a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005570 tx=0x7f3e08009310 comp rx=0 tx=0).stop 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1900837811 shutdown_connections 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a97eeb2c30 0x55a97eeb5070 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3dfb400700 0 client.25561 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee99240 0x55a97eea51a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/1900837811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee9eb90 0x55a97eea24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1900837811 >> 172.21.15.26:0/1900837811 conn(0x55a97edc5170 msgr2=0x55a97ee8d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1900837811 shutdown_connections 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 1 -- 172.21.15.26:0/1900837811 wait complete. 2024-06-13T07:02:18.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.030+0000 7f3e2691fb80 20 client.25561 trim_cache size 0 max 16384 2024-06-13T07:02:18.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:17 smithi135 ceph-mon[36276]: pgmap v563: 129 pgs: 129 active+clean; 23 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-06-13T07:02:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.053+0000 7f3e2691fb80 1 Processor -- start 2024-06-13T07:02:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.053+0000 7f3e2691fb80 1 -- start start 2024-06-13T07:02:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.053+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97ee98d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.053+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9eb90 0x55a97ee941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.053+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9f4b0 0x55a97ee8e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.053+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141b30a0 con 0x55a97ee9eb90 2024-06-13T07:02:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.053+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141b0900 con 0x55a97ee99240 2024-06-13T07:02:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.053+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ae750 con 0x55a97ee9f4b0 2024-06-13T07:02:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97ee98d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e18c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9eb90 0x55a97ee941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97ee98d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37554/0 (socket says 172.21.15.26:37554) 2024-06-13T07:02:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 -- 172.21.15.26:0/3448202448 learned_addr learned my addr 172.21.15.26:0/3448202448 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e1a000700 1 --2- 172.21.15.26:0/3448202448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9f4b0 0x55a97ee8e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 -- 172.21.15.26:0/3448202448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9f4b0 msgr2=0x55a97ee8e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 --2- 172.21.15.26:0/3448202448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9f4b0 0x55a97ee8e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 -- 172.21.15.26:0/3448202448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9eb90 msgr2=0x55a97ee941a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 --2- 172.21.15.26:0/3448202448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9eb90 0x55a97ee941a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 -- 172.21.15.26:0/3448202448 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a97eea5980 con 0x55a97ee99240 2024-06-13T07:02:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e19600700 1 --2- 172.21.15.26:0/3448202448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97ee98d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e080098e0 tx=0x7f3e08005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a8f9d2c4b0f1f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e06000700 1 -- 172.21.15.26:0/3448202448 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08043070 con 0x55a97ee99240 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e06000700 1 -- 172.21.15.26:0/3448202448 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e08031070 con 0x55a97ee99240 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.054+0000 7f3e06000700 1 -- 172.21.15.26:0/3448202448 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08028030 con 0x55a97ee99240 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3448202448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 msgr2=0x55a97ee98d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3448202448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97ee98d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e080098e0 tx=0x7f3e08005980 comp rx=0 tx=0).stop 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3448202448 shutdown_connections 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3448202448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97ee98d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3448202448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9f4b0 0x55a97ee8e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3448202448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9eb90 0x55a97ee941a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3448202448 >> 172.21.15.26:0/3448202448 conn(0x55a97edc5170 msgr2=0x55a97ee93a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3448202448 shutdown_connections 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.055+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3448202448 wait complete. 2024-06-13T07:02:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.056+0000 7f3e2691fb80 1 -- start start 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.056+0000 7f3e2691fb80 1 Processor -- start 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97eeab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9eb90 0x55a97eeab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e2691fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141aedc0 con 0x55a97ee9f4b0 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141b0230 con 0x55a97ee99240 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e2691fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141af490 con 0x55a97ee9eb90 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e2691fb80 10 client.0 _ll_register_callbacks cb 0x55a97ed80380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:18.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e19600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97eeab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e19600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97eeab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37570/0 (socket says 172.21.15.26:37570) 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e19600700 1 -- 172.21.15.26:0/3724946317 learned_addr learned my addr 172.21.15.26:0/3724946317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e18c00700 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9eb90 0x55a97eeab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e1a000700 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e19600700 1 -- 172.21.15.26:0/3724946317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9eb90 msgr2=0x55a97eeab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e19600700 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9eb90 0x55a97eeab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e19600700 1 -- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 msgr2=0x55a97eea8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e19600700 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e19600700 1 -- 172.21.15.26:0/3724946317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a97eea8660 con 0x55a97ee99240 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.057+0000 7f3e1a000700 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.058+0000 7f3e19600700 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97eeab150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3e08041040 tx=0x7f3e08026000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f9bec1becf20a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.058+0000 7f3dfb400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.058+0000 7f3dfb400700 1 -- 172.21.15.26:0/3724946317 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08043070 con 0x55a97ee99240 2024-06-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.058+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3724946317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e08031070 con 0x55a97ee99240 2024-06-13T07:02:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.058+0000 7f3dfb400700 1 -- 172.21.15.26:0/3724946317 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f3e08003b60 con 0x55a97ee99240 2024-06-13T07:02:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.058+0000 7f3dfb400700 1 -- 172.21.15.26:0/3724946317 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e08050050 con 0x55a97ee99240 2024-06-13T07:02:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3e2691fb80 10 client.25526 fetch_fsmap learned FSMap version 31 2024-06-13T07:02:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3724946317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a97ef1e090 con 0x55a97ee99240 2024-06-13T07:02:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3dfb400700 1 -- 172.21.15.26:0/3724946317 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e08031070 con 0x55a97ee99240 2024-06-13T07:02:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3dfb400700 1 -- 172.21.15.26:0/3724946317 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f3e080364e0 con 0x55a97ee99240 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3e2691fb80 10 client.25526 fetch_fsmap finished waiting for FSMap version 31 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3e2691fb80 10 client.25526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3e2691fb80 20 client.25526 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a97ef1e6e0 0x55a97ef20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3e2691fb80 4 client.25526 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.059+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3724946317 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a97edf2580 con 0x55a97ef1e6e0 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.060+0000 7f3e18c00700 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a97ef1e6e0 0x55a97ef20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.060+0000 7f3e18c00700 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a97ef1e6e0 0x55a97ef20b00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d09f85fdf0f7d1 server_cookie=3020a21c9c642667 in_seq=0 out_seq=0 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.060+0000 7f3dfb400700 10 client.25526 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.060+0000 7f3dfb400700 1 -- 172.21.15.26:0/3724946317 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a97edf2580 con 0x55a97ef1e6e0 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.060+0000 7f3dfb400700 10 client.25526 handle_command_reply: tid=0 2024-06-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 client.25526 shutdown 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3724946317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 msgr2=0x55a97eeab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97eeab150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3e08041040 tx=0x7f3e08026000 comp rx=0 tx=0).stop 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3724946317 shutdown_connections 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a97ef1e6e0 0x55a97ef20b00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a97ee99240 0x55a97eeab150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a97ee9f4b0 0x55a97eea8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 --2- 172.21.15.26:0/3724946317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a97ee9eb90 0x55a97eeab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3dfb400700 0 client.25526 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3724946317 >> 172.21.15.26:0/3724946317 conn(0x55a97edc5170 msgr2=0x55a97ee9f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3724946317 shutdown_connections 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 1 -- 172.21.15.26:0/3724946317 wait complete. 2024-06-13T07:02:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.061+0000 7f3e2691fb80 20 client.25526 trim_cache size 0 max 16384 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout: "3f365b2e-2520-4732-aa5b-70db1827d5a6": { 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "3f365b2e-2520-4732-aa5b-70db1827d5a6", 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:02:18.064 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:02:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 -- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3dfc0a2ca0 msgr2=0x7f3dfc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3dfc0a2ca0 0x7f3dfc0a5160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009750 tx=0x7f3e0800b280 comp rx=0 tx=0).stop 2024-06-13T07:02:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 -- 172.21.15.26:0/1244159189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e1406ab10 msgr2=0x7f3e141019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e1406ab10 0x7f3e141019a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000e9f0 tx=0x7f3e0000ed00 comp rx=0 tx=0).stop 2024-06-13T07:02:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 -- 172.21.15.26:0/1244159189 shutdown_connections 2024-06-13T07:02:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f3dfc0a2ca0 0x7f3dfc0a5160 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e141084e0 0x7f3e14101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e1406a170 0x7f3e14101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 --2- 172.21.15.26:0/1244159189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e1406ab10 0x7f3e141019a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:18.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 -- 172.21.15.26:0/1244159189 >> 172.21.15.26:0/1244159189 conn(0x7f3e140782c0 msgr2=0x7f3e140ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:18.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 -- 172.21.15.26:0/1244159189 shutdown_connections 2024-06-13T07:02:18.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:18.062+0000 7f3e1c000700 1 -- 172.21.15.26:0/1244159189 wait complete. 2024-06-13T07:02:18.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:17 smithi136 ceph-mon[37229]: pgmap v563: 129 pgs: 129 active+clean; 23 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-06-13T07:02:18.501 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20274 3.89 MB/sec warmup 32 sec latency 785.119 ms 2024-06-13T07:02:19.169 INFO:tasks.cephfs.filesystem:scrub status for tag:3f365b2e-2520-4732-aa5b-70db1827d5a6 - {'path': '/', 'tag': '3f365b2e-2520-4732-aa5b-70db1827d5a6', 'options': 'recursive,force'} 2024-06-13T07:02:19.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20274 3.77 MB/sec warmup 33 sec latency 1785.256 ms 2024-06-13T07:02:20.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:19 smithi026 ceph-mon[32041]: pgmap v564: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-06-13T07:02:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:19 smithi026 ceph-mon[32041]: scrub summary: idle 2024-06-13T07:02:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:19 smithi026 ceph-mon[32041]: scrub summary: idle 2024-06-13T07:02:20.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:19 smithi026 ceph-mon[32041]: scrub summary: idle 2024-06-13T07:02:20.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:19 smithi135 ceph-mon[36276]: pgmap v564: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-06-13T07:02:20.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:19 smithi135 ceph-mon[36276]: scrub summary: idle 2024-06-13T07:02:20.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:19 smithi135 ceph-mon[36276]: scrub summary: idle 2024-06-13T07:02:20.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:19 smithi135 ceph-mon[36276]: scrub summary: idle 2024-06-13T07:02:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:19 smithi136 ceph-mon[37229]: pgmap v564: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-06-13T07:02:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:19 smithi136 ceph-mon[37229]: scrub summary: idle 2024-06-13T07:02:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:19 smithi136 ceph-mon[37229]: scrub summary: idle 2024-06-13T07:02:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:19 smithi136 ceph-mon[37229]: scrub summary: idle 2024-06-13T07:02:20.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20274 3.66 MB/sec warmup 34 sec latency 2785.400 ms 2024-06-13T07:02:21.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:20 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:02:20.729+0000 7fc609800700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-13T07:02:21.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:20 smithi026 ceph-mon[32041]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-13T07:02:21.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:20 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:active 2024-06-13T07:02:21.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:20 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:21.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:20 smithi135 ceph-mon[36276]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-13T07:02:21.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:20 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:active 2024-06-13T07:02:21.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:20 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:21.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:20 smithi136 ceph-mon[37229]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-13T07:02:21.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:20 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] up:active 2024-06-13T07:02:21.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:20 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:21.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20274 3.55 MB/sec warmup 35 sec latency 3785.524 ms 2024-06-13T07:02:22.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:21 smithi136 ceph-mon[37229]: pgmap v565: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 590 KiB/s wr, 83 op/s 2024-06-13T07:02:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:21 smithi026 ceph-mon[32041]: pgmap v565: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 590 KiB/s wr, 83 op/s 2024-06-13T07:02:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:21 smithi135 ceph-mon[36276]: pgmap v565: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 590 KiB/s wr, 83 op/s 2024-06-13T07:02:22.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 20354 3.46 MB/sec warmup 36 sec latency 4728.493 ms 2024-06-13T07:02:23.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 21561 3.66 MB/sec warmup 37 sec latency 21.784 ms 2024-06-13T07:02:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:23 smithi136 ceph-mon[37229]: pgmap v566: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 579 KiB/s wr, 79 op/s 2024-06-13T07:02:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:23 smithi026 ceph-mon[32041]: pgmap v566: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 579 KiB/s wr, 79 op/s 2024-06-13T07:02:24.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:23 smithi135 ceph-mon[36276]: pgmap v566: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 579 KiB/s wr, 79 op/s 2024-06-13T07:02:24.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 22041 3.58 MB/sec warmup 38 sec latency 35.269 ms 2024-06-13T07:02:25.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 23099 3.62 MB/sec warmup 39 sec latency 24.714 ms 2024-06-13T07:02:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:25 smithi136 ceph-mon[37229]: pgmap v567: 129 pgs: 129 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 200 KiB/s wr, 62 op/s 2024-06-13T07:02:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 -- 172.21.15.26:0/1817384577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76c01076e0 msgr2=0x7f76c0109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1817384577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76c01076e0 0x7f76c0109b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009860 tx=0x7f76b802a340 comp rx=0 tx=0).stop 2024-06-13T07:02:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 -- 172.21.15.26:0/1817384577 shutdown_connections 2024-06-13T07:02:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1817384577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76c01076e0 0x7f76c0109b30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1817384577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0104db0 0x7f76c01071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1817384577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c0102480 0x7f76c0104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 -- 172.21.15.26:0/1817384577 >> 172.21.15.26:0/1817384577 conn(0x7f76c00fbe70 msgr2=0x7f76c00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 -- 172.21.15.26:0/1817384577 shutdown_connections 2024-06-13T07:02:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.181+0000 7f76c6c00640 1 -- 172.21.15.26:0/1817384577 wait complete. 2024-06-13T07:02:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.183+0000 7f76c6c00640 1 Processor -- start 2024-06-13T07:02:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.183+0000 7f76c6c00640 1 -- start start 2024-06-13T07:02:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.183+0000 7f76c6c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c0102480 0x7f76c0197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.183+0000 7f76c6c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0104db0 0x7f76c0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.183+0000 7f76c6c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76c01076e0 0x7f76c019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.183+0000 7f76c6c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010d680 con 0x7f76c0102480 2024-06-13T07:02:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.183+0000 7f76c6c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010d500 con 0x7f76c01076e0 2024-06-13T07:02:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.183+0000 7f76c6c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010d800 con 0x7f76c0104db0 2024-06-13T07:02:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76b7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0104db0 0x7f76c0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76b7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0104db0 0x7f76c0198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:46348/0 (socket says 172.21.15.26:46348) 2024-06-13T07:02:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76b7400640 1 -- 172.21.15.26:0/1217906524 learned_addr learned my addr 172.21.15.26:0/1217906524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76c4c00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76c01076e0 0x7f76c019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c0102480 0x7f76c0197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76b7400640 1 -- 172.21.15.26:0/1217906524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76c01076e0 msgr2=0x7f76c019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76b7400640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76c01076e0 0x7f76c019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76b7400640 1 -- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c0102480 msgr2=0x7f76c0197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76b7400640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c0102480 0x7f76c0197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76b7400640 1 -- 172.21.15.26:0/1217906524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c019f800 con 0x7f76c0104db0 2024-06-13T07:02:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.184+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c0102480 0x7f76c0197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.185+0000 7f76b7400640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0104db0 0x7f76c0198120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76a800e9a0 tx=0x7f76a800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.185+0000 7f76bd600640 1 -- 172.21.15.26:0/1217906524 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a800cd80 con 0x7f76c0104db0 2024-06-13T07:02:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.185+0000 7f76bd600640 1 -- 172.21.15.26:0/1217906524 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a80035f0 con 0x7f76c0104db0 2024-06-13T07:02:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.185+0000 7f76bd600640 1 -- 172.21.15.26:0/1217906524 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8010670 con 0x7f76c0104db0 2024-06-13T07:02:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.185+0000 7f76c6c00640 1 -- 172.21.15.26:0/1217906524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c019faf0 con 0x7f76c0104db0 2024-06-13T07:02:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.185+0000 7f76c6c00640 1 -- 172.21.15.26:0/1217906524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c019fff0 con 0x7f76c0104db0 2024-06-13T07:02:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.189+0000 7f76bd600640 1 -- 172.21.15.26:0/1217906524 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76a80037a0 con 0x7f76c0104db0 2024-06-13T07:02:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.189+0000 7f76c99dc740 1 Processor -- start 2024-06-13T07:02:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.189+0000 7f76c99dc740 1 -- start start 2024-06-13T07:02:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f985380430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985385500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f985388000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c99dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f9852df420 con 0x55f985380010 2024-06-13T07:02:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c99dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f985286c50 con 0x55f985385b70 2024-06-13T07:02:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c99dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f9852db3e0 con 0x55f985380a00 2024-06-13T07:02:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f985380430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f985380430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34498/0 (socket says 172.21.15.26:34498) 2024-06-13T07:02:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76c4c00640 1 -- 172.21.15.26:0/1799886776 learned_addr learned my addr 172.21.15.26:0/1799886776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.190+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985385500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76b7400640 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f985388000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76bfe00640 1 -- 172.21.15.26:0/1799886776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 msgr2=0x55f985388000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f985388000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76bfe00640 1 -- 172.21.15.26:0/1799886776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 msgr2=0x55f985380430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f985380430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76bfe00640 1 -- 172.21.15.26:0/1799886776 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f984ff3210 con 0x55f985380a00 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76bd600640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76a40a2c30 0x7f76a40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76b7400640 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f985388000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76bd600640 1 -- 172.21.15.26:0/1217906524 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f76a80cf740 con 0x7f76c0104db0 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.191+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985385500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f76b000f3e0 tx=0x7f76b000f8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f89f5c30ea418a53 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.192+0000 7f76b7400640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76a40a2c30 0x7f76a40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.192+0000 7f76b5200640 1 -- 172.21.15.26:0/1799886776 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0004d20 con 0x55f985380a00 2024-06-13T07:02:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.192+0000 7f76b5200640 1 -- 172.21.15.26:0/1799886776 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b00119e0 con 0x55f985380a00 2024-06-13T07:02:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.192+0000 7f76b5200640 1 -- 172.21.15.26:0/1799886776 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0005610 con 0x55f985380a00 2024-06-13T07:02:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 -- 172.21.15.26:0/1799886776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 msgr2=0x55f985385500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76b7400640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76a40a2c30 0x7f76a40a50f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f76a809bae0 tx=0x7f76a8003f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985385500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f76b000f3e0 tx=0x7f76b000f8a0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 -- 172.21.15.26:0/1799886776 shutdown_connections 2024-06-13T07:02:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f985388000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985385500 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1799886776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f985380430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 -- 172.21.15.26:0/1799886776 >> 172.21.15.26:0/1799886776 conn(0x55f985343fb0 msgr2=0x55f98537ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 -- 172.21.15.26:0/1799886776 shutdown_connections 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 -- 172.21.15.26:0/1799886776 wait complete. 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.193+0000 7f76c99dc740 1 -- start start 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c99dc740 1 Processor -- start 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98538a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f98538a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c99dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f985388b20 con 0x55f985380010 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c99dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f98529a0b0 con 0x55f985385b70 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c99dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f985388ca0 con 0x55f985380a00 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c99dc740 10 client.0 _ll_register_callbacks cb 0x55f985285f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f98538a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76c4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98538a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f98538a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:46356/0 (socket says 172.21.15.26:46356) 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.194+0000 7f76bfe00640 1 -- 172.21.15.26:0/58505267 learned_addr learned my addr 172.21.15.26:0/58505267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76c4c00640 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76bfe00640 1 -- 172.21.15.26:0/58505267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 msgr2=0x55f9853918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76bfe00640 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853918b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76bfe00640 1 -- 172.21.15.26:0/58505267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 msgr2=0x55f98538a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76bfe00640 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98538a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76bfe00640 1 -- 172.21.15.26:0/58505267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9850bf620 con 0x55f985380a00 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76c4c00640 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98538a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76bfe00640 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f98538a8e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76b0010010 tx=0x7f76b0012dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46bda029fbc7194d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76ad600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76ad600640 1 -- 172.21.15.26:0/58505267 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b002b060 con 0x55f985380a00 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76c99dc740 1 -- 172.21.15.26:0/58505267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b00119e0 con 0x55f985380a00 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.195+0000 7f76ad600640 1 -- 172.21.15.26:0/58505267 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f76b0011710 con 0x55f985380a00 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.196+0000 7f76ad600640 1 -- 172.21.15.26:0/58505267 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f76b0032050 con 0x55f985380a00 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.196+0000 7f76c99dc740 10 client.25579 fetch_fsmap learned FSMap version 32 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.196+0000 7f76c99dc740 1 -- 172.21.15.26:0/58505267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f985394be0 con 0x55f985380a00 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.196+0000 7f76ad600640 1 -- 172.21.15.26:0/58505267 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b00119e0 con 0x55f985380a00 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.196+0000 7f76ad600640 1 -- 172.21.15.26:0/58505267 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f76b0005400 con 0x55f985380a00 2024-06-13T07:02:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.196+0000 7f76c99dc740 10 client.25579 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:02:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.196+0000 7f76c99dc740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.196+0000 7f76c99dc740 20 client.25579 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.197+0000 7f76c99dc740 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f985395fe0 0x55f985398410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.197+0000 7f76c99dc740 4 client.25579 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.197+0000 7f76c99dc740 1 -- 172.21.15.26:0/58505267 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9852ecb50 con 0x55f985395fe0 2024-06-13T07:02:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.197+0000 7f76c4c00640 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f985395fe0 0x55f985398410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.197+0000 7f76c4c00640 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f985395fe0 0x55f985398410 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6846a98bcd311807 server_cookie=5009463a619f951b in_seq=0 out_seq=0 2024-06-13T07:02:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.197+0000 7f76ad600640 10 client.25579 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.198+0000 7f76ad600640 1 -- 172.21.15.26:0/58505267 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f9852ecb50 con 0x55f985395fe0 2024-06-13T07:02:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.198+0000 7f76ad600640 10 client.25579 handle_command_reply: tid=0 2024-06-13T07:02:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.198+0000 7f76c99dc740 1 client.25579 shutdown 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.198+0000 7f76c99dc740 1 -- 172.21.15.26:0/58505267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 msgr2=0x55f98538a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.198+0000 7f76c99dc740 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f98538a8e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76b0010010 tx=0x7f76b0012dd0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 1 -- 172.21.15.26:0/58505267 shutdown_connections 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f985395fe0 0x55f985398410 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f98538a8e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 1 --2- 172.21.15.26:0/58505267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98538a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 1 -- 172.21.15.26:0/58505267 >> 172.21.15.26:0/58505267 conn(0x55f985343fb0 msgr2=0x55f985376840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76ad600640 0 client.25579 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 1 -- 172.21.15.26:0/58505267 shutdown_connections 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 1 -- 172.21.15.26:0/58505267 wait complete. 2024-06-13T07:02:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.199+0000 7f76c99dc740 20 client.25579 trim_cache size 0 max 16384 2024-06-13T07:02:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.217+0000 7f76c99dc740 1 Processor -- start 2024-06-13T07:02:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.217+0000 7f76c99dc740 1 -- start start 2024-06-13T07:02:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.217+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98537bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.217+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985391870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.217+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f98537e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.217+0000 7f76c99dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01aadb0 con 0x55f985380010 2024-06-13T07:02:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.217+0000 7f76c99dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01a8650 con 0x55f985385b70 2024-06-13T07:02:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.217+0000 7f76c99dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01a64a0 con 0x55f985380a00 2024-06-13T07:02:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985391870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985391870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:46364/0 (socket says 172.21.15.26:46364) 2024-06-13T07:02:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 -- 172.21.15.26:0/3323611621 learned_addr learned my addr 172.21.15.26:0/3323611621 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76c4c00640 1 --2- 172.21.15.26:0/3323611621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98537bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 -- 172.21.15.26:0/3323611621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 msgr2=0x55f98537e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 --2- 172.21.15.26:0/3323611621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f98537e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 -- 172.21.15.26:0/3323611621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 msgr2=0x55f98537bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 --2- 172.21.15.26:0/3323611621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98537bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 -- 172.21.15.26:0/3323611621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f98537c740 con 0x55f985380a00 2024-06-13T07:02:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.218+0000 7f76bfe00640 1 --2- 172.21.15.26:0/3323611621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985391870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76b0000f80 tx=0x7f76b0009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a28740b4313ca4f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.219+0000 7f76afe00640 1 -- 172.21.15.26:0/3323611621 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b002b070 con 0x55f985380a00 2024-06-13T07:02:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.219+0000 7f76afe00640 1 -- 172.21.15.26:0/3323611621 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b0004040 con 0x55f985380a00 2024-06-13T07:02:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.219+0000 7f76afe00640 1 -- 172.21.15.26:0/3323611621 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0004580 con 0x55f985380a00 2024-06-13T07:02:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.219+0000 7f76c99dc740 1 -- 172.21.15.26:0/3323611621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 msgr2=0x55f985391870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.219+0000 7f76c99dc740 1 --2- 172.21.15.26:0/3323611621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985391870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76b0000f80 tx=0x7f76b0009ea0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 -- 172.21.15.26:0/3323611621 shutdown_connections 2024-06-13T07:02:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 --2- 172.21.15.26:0/3323611621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f98537e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 --2- 172.21.15.26:0/3323611621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380a00 0x55f985391870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 --2- 172.21.15.26:0/3323611621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380010 0x55f98537bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 -- 172.21.15.26:0/3323611621 >> 172.21.15.26:0/3323611621 conn(0x55f985343fb0 msgr2=0x55f98537b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 -- 172.21.15.26:0/3323611621 shutdown_connections 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 -- 172.21.15.26:0/3323611621 wait complete. 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 -- start start 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.220+0000 7f76c99dc740 1 Processor -- start 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380010 0x55f98538e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380a00 0x55f98538ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c99dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853feca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c99dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01a6b10 con 0x55f985380a00 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c99dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01a7f80 con 0x55f985385b70 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c99dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01a71e0 con 0x55f985380010 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c99dc740 10 client.0 _ll_register_callbacks cb 0x7f76b0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76bfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380a00 0x55f98538ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76bfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380a00 0x55f98538ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34530/0 (socket says 172.21.15.26:34530) 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76bfe00640 1 -- 172.21.15.26:0/1486099678 learned_addr learned my addr 172.21.15.26:0/1486099678 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c4c00640 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380010 0x55f98538e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76bfe00640 1 -- 172.21.15.26:0/1486099678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380010 msgr2=0x55f98538e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76c4c00640 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380010 0x55f98538e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76bfe00640 1 -- 172.21.15.26:0/1486099678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 msgr2=0x55f9853feca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853feca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.221+0000 7f76bfe00640 1 -- 172.21.15.26:0/1486099678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9853ff260 con 0x55f985380a00 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.222+0000 7f76c4c00640 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.222+0000 7f76bfe00640 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380a00 0x55f98538ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f76b0005940 tx=0x7f76b0003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06f21565e32af44 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.222+0000 7f76ad600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.222+0000 7f76ad600640 1 -- 172.21.15.26:0/1486099678 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b002b070 con 0x55f985380a00 2024-06-13T07:02:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.222+0000 7f76c99dc740 1 -- 172.21.15.26:0/1486099678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b0004040 con 0x55f985380a00 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.222+0000 7f76ad600640 1 -- 172.21.15.26:0/1486099678 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f76b0004c10 con 0x55f985380a00 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.222+0000 7f76ad600640 1 -- 172.21.15.26:0/1486099678 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f76b0044050 con 0x55f985380a00 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.222+0000 7f76c99dc740 10 client.15447 fetch_fsmap learned FSMap version 32 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76c99dc740 1 -- 172.21.15.26:0/1486099678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f985400570 con 0x55f985380a00 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76ad600640 1 -- 172.21.15.26:0/1486099678 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b0004040 con 0x55f985380a00 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76ad600640 1 -- 172.21.15.26:0/1486099678 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f76b0005200 con 0x55f985380a00 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76c99dc740 10 client.15447 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76c99dc740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76c99dc740 20 client.15447 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f985400eb0 0x55f9854032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76c99dc740 4 client.15447 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.223+0000 7f76c99dc740 1 -- 172.21.15.26:0/1486099678 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9852e6ea0 con 0x55f985400eb0 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.224+0000 7f76c4c00640 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f985400eb0 0x55f9854032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.224+0000 7f76c4c00640 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f985400eb0 0x55f9854032d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703f08b933b0ebdb server_cookie=74236dd13371b26f in_seq=0 out_seq=0 2024-06-13T07:02:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.224+0000 7f76ad600640 10 client.15447 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:26.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:25 smithi026 ceph-mon[32041]: pgmap v567: 129 pgs: 129 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 200 KiB/s wr, 62 op/s 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.263+0000 7f76ad600640 1 -- 172.21.15.26:0/1486099678 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f9852e6ea0 con 0x55f985400eb0 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.263+0000 7f76ad600640 10 client.15447 handle_command_reply: tid=0 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 client.15447 shutdown 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 -- 172.21.15.26:0/1486099678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380a00 msgr2=0x55f98538ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380a00 0x55f98538ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f76b0005940 tx=0x7f76b0003c00 comp rx=0 tx=0).stop 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 -- 172.21.15.26:0/1486099678 shutdown_connections 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f985400eb0 0x55f9854032d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f985385b70 0x55f9853feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f985380a00 0x55f98538ee40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 --2- 172.21.15.26:0/1486099678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f985380010 0x55f98538e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76ad600640 0 client.15447 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 -- 172.21.15.26:0/1486099678 >> 172.21.15.26:0/1486099678 conn(0x55f985343fb0 msgr2=0x55f98537e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 -- 172.21.15.26:0/1486099678 shutdown_connections 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 1 -- 172.21.15.26:0/1486099678 wait complete. 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.264+0000 7f76c99dc740 20 client.15447 trim_cache size 0 max 16384 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:02:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:02:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 -- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76a40a2c30 msgr2=0x7f76a40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76a40a2c30 0x7f76a40a50f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f76a809bae0 tx=0x7f76a8003f10 comp rx=0 tx=0).stop 2024-06-13T07:02:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 -- 172.21.15.26:0/1217906524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0104db0 msgr2=0x7f76c0198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0104db0 0x7f76c0198120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76a800e9a0 tx=0x7f76a800ee60 comp rx=0 tx=0).stop 2024-06-13T07:02:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 -- 172.21.15.26:0/1217906524 shutdown_connections 2024-06-13T07:02:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76a40a2c30 0x7f76a40a50f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76c01076e0 0x7f76c019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0104db0 0x7f76c0198120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 --2- 172.21.15.26:0/1217906524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c0102480 0x7f76c0197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 -- 172.21.15.26:0/1217906524 >> 172.21.15.26:0/1217906524 conn(0x7f76c00fbe70 msgr2=0x7f76c0102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 -- 172.21.15.26:0/1217906524 shutdown_connections 2024-06-13T07:02:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:26.265+0000 7f76c6c00640 1 -- 172.21.15.26:0/1217906524 wait complete. 2024-06-13T07:02:26.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:25 smithi135 ceph-mon[36276]: pgmap v567: 129 pgs: 129 active+clean; 24 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 200 KiB/s wr, 62 op/s 2024-06-13T07:02:26.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 24628 3.75 MB/sec warmup 40 sec latency 16.511 ms 2024-06-13T07:02:27.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 25302 3.78 MB/sec warmup 41 sec latency 16.872 ms 2024-06-13T07:02:28.057 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:27 smithi026 ceph-mon[32041]: pgmap v568: 129 pgs: 129 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 205 KiB/s wr, 58 op/s 2024-06-13T07:02:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:27 smithi136 ceph-mon[37229]: pgmap v568: 129 pgs: 129 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 205 KiB/s wr, 58 op/s 2024-06-13T07:02:28.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:27 smithi135 ceph-mon[36276]: pgmap v568: 129 pgs: 129 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 205 KiB/s wr, 58 op/s 2024-06-13T07:02:28.502 INFO:tasks.workunit.client.0.smithi026.stdout: 1 26132 3.74 MB/sec warmup 42 sec latency 21.673 ms 2024-06-13T07:02:29.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 27753 3.86 MB/sec warmup 43 sec latency 26.536 ms 2024-06-13T07:02:30.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:29 smithi136 ceph-mon[37229]: pgmap v569: 129 pgs: 129 active+clean; 26 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 362 KiB/s wr, 101 op/s 2024-06-13T07:02:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:29 smithi026 ceph-mon[32041]: pgmap v569: 129 pgs: 129 active+clean; 26 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 362 KiB/s wr, 101 op/s 2024-06-13T07:02:30.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:29 smithi135 ceph-mon[36276]: pgmap v569: 129 pgs: 129 active+clean; 26 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 362 KiB/s wr, 101 op/s 2024-06-13T07:02:30.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 28711 3.95 MB/sec warmup 44 sec latency 19.892 ms 2024-06-13T07:02:31.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 29224 3.88 MB/sec warmup 45 sec latency 30.564 ms 2024-06-13T07:02:32.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:31 smithi136 ceph-mon[37229]: pgmap v570: 129 pgs: 129 active+clean; 26 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 252 KiB/s wr, 70 op/s 2024-06-13T07:02:32.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:31 smithi026 ceph-mon[32041]: pgmap v570: 129 pgs: 129 active+clean; 26 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 252 KiB/s wr, 70 op/s 2024-06-13T07:02:32.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:31 smithi135 ceph-mon[36276]: pgmap v570: 129 pgs: 129 active+clean; 26 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 252 KiB/s wr, 70 op/s 2024-06-13T07:02:32.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 30373 3.90 MB/sec warmup 46 sec latency 27.051 ms 2024-06-13T07:02:33.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 31605 3.98 MB/sec warmup 47 sec latency 18.023 ms 2024-06-13T07:02:34.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:33 smithi136 ceph-mon[37229]: pgmap v571: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 409 KiB/s wr, 111 op/s 2024-06-13T07:02:34.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:33 smithi026 ceph-mon[32041]: pgmap v571: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 409 KiB/s wr, 111 op/s 2024-06-13T07:02:34.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:33 smithi135 ceph-mon[36276]: pgmap v571: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 409 KiB/s wr, 111 op/s 2024-06-13T07:02:34.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 32412 4.02 MB/sec warmup 48 sec latency 26.926 ms 2024-06-13T07:02:35.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 33185 3.98 MB/sec warmup 49 sec latency 19.670 ms 2024-06-13T07:02:36.084 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:35 smithi135 ceph-mon[36276]: pgmap v572: 129 pgs: 129 active+clean; 29 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 478 KiB/s wr, 123 op/s 2024-06-13T07:02:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:35 smithi136 ceph-mon[37229]: pgmap v572: 129 pgs: 129 active+clean; 29 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 478 KiB/s wr, 123 op/s 2024-06-13T07:02:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:35 smithi026 ceph-mon[32041]: pgmap v572: 129 pgs: 129 active+clean; 29 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 478 KiB/s wr, 123 op/s 2024-06-13T07:02:36.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 34455 4.02 MB/sec warmup 50 sec latency 26.266 ms 2024-06-13T07:02:37.503 INFO:tasks.workunit.client.0.smithi026.stdout: 1 35711 4.15 MB/sec warmup 51 sec latency 17.519 ms 2024-06-13T07:02:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:37 smithi136 ceph-mon[37229]: pgmap v573: 129 pgs: 129 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 529 KiB/s wr, 124 op/s 2024-06-13T07:02:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:37 smithi026 ceph-mon[32041]: pgmap v573: 129 pgs: 129 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 529 KiB/s wr, 124 op/s 2024-06-13T07:02:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:37 smithi135 ceph-mon[36276]: pgmap v573: 129 pgs: 129 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 529 KiB/s wr, 124 op/s 2024-06-13T07:02:38.505 INFO:tasks.workunit.client.0.smithi026.stdout: 1 36109 4.08 MB/sec warmup 52 sec latency 27.112 ms 2024-06-13T07:02:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 -- 172.21.15.26:0/573648798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0109be0 msgr2=0x7f76b0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 --2- 172.21.15.26:0/573648798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0109be0 0x7f76b0110470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009ca0 tx=0x7f76ac02a8c0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 -- 172.21.15.26:0/573648798 shutdown_connections 2024-06-13T07:02:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 --2- 172.21.15.26:0/573648798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0109be0 0x7f76b0110470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 --2- 172.21.15.26:0/573648798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b0109220 0x7f76b01096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 --2- 172.21.15.26:0/573648798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76b0108030 0x7f76b0108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 -- 172.21.15.26:0/573648798 >> 172.21.15.26:0/573648798 conn(0x7f76b0076ff0 msgr2=0x7f76b0079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 -- 172.21.15.26:0/573648798 shutdown_connections 2024-06-13T07:02:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.418+0000 7f76b8800640 1 -- 172.21.15.26:0/573648798 wait complete. 2024-06-13T07:02:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.419+0000 7f76b8800640 1 Processor -- start 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b8800640 1 -- start start 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b8800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0108030 0x7f76b019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b8800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76b0109220 0x7f76b019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b8800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b0109be0 0x7f76b01a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b8800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0113d10 con 0x7f76b0109220 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b8800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0113b90 con 0x7f76b0108030 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b8800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0113e90 con 0x7f76b0109be0 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0108030 0x7f76b019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0108030 0x7f76b019e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53548/0 (socket says 172.21.15.26:53548) 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.420+0000 7f76b5e00640 1 -- 172.21.15.26:0/2852659056 learned_addr learned my addr 172.21.15.26:0/2852659056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b6800640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b0109be0 0x7f76b01a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b5400640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76b0109220 0x7f76b019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b5e00640 1 -- 172.21.15.26:0/2852659056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b0109be0 msgr2=0x7f76b01a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b5e00640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b0109be0 0x7f76b01a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b5e00640 1 -- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76b0109220 msgr2=0x7f76b019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b5e00640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76b0109220 0x7f76b019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b5e00640 1 -- 172.21.15.26:0/2852659056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b01a5e60 con 0x7f76b0108030 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b5400640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76b0109220 0x7f76b019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76b5e00640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0108030 0x7f76b019e240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f76a400e9c0 tx=0x7f76a400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76aaa00640 1 -- 172.21.15.26:0/2852659056 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a400cd80 con 0x7f76b0108030 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.421+0000 7f76aaa00640 1 -- 172.21.15.26:0/2852659056 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a40035f0 con 0x7f76b0108030 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.422+0000 7f76b8800640 1 -- 172.21.15.26:0/2852659056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b01a6150 con 0x7f76b0108030 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.422+0000 7f76aaa00640 1 -- 172.21.15.26:0/2852659056 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4010690 con 0x7f76b0108030 2024-06-13T07:02:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.422+0000 7f76b8800640 1 -- 172.21.15.26:0/2852659056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b01a66f0 con 0x7f76b0108030 2024-06-13T07:02:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.425+0000 7f76aaa00640 1 -- 172.21.15.26:0/2852659056 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76a40037a0 con 0x7f76b0108030 2024-06-13T07:02:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.425+0000 7f76bb608740 1 Processor -- start 2024-06-13T07:02:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76bb608740 1 -- start start 2024-06-13T07:02:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76bb608740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4ec00 0x55ce79e4f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76bb608740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e4f5f0 0x55ce79e540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76bb608740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e54760 0x55ce79e56bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76bb608740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ce79db2420 con 0x55ce79e4f5f0 2024-06-13T07:02:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76bb608740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ce79d59c50 con 0x55ce79e54760 2024-06-13T07:02:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76bb608740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ce79dae3e0 con 0x55ce79e4ec00 2024-06-13T07:02:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76b6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4ec00 0x55ce79e4f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76b5400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e54760 0x55ce79e56bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76b6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4ec00 0x55ce79e4f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37856/0 (socket says 172.21.15.26:37856) 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76b6800640 1 -- 172.21.15.26:0/2697876564 learned_addr learned my addr 172.21.15.26:0/2697876564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.426+0000 7f76b5400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e54760 0x55ce79e56bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53550/0 (socket says 172.21.15.26:53550) 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b5e00640 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e4f5f0 0x55ce79e540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b6800640 1 -- 172.21.15.26:0/2697876564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e54760 msgr2=0x55ce79e56bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b6800640 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e54760 0x55ce79e56bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b6800640 1 -- 172.21.15.26:0/2697876564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e4f5f0 msgr2=0x55ce79e540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b6800640 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e4f5f0 0x55ce79e540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b6800640 1 -- 172.21.15.26:0/2697876564 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce79ac6210 con 0x55ce79e4ec00 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b5400640 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e54760 0x55ce79e56bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b5e00640 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e4f5f0 0x55ce79e540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76aaa00640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76980a2cb0 0x7f76980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b6800640 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4ec00 0x55ce79e4f020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009ca0 tx=0x7f76ac0093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2b8f847d08867ec server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76aaa00640 1 -- 172.21.15.26:0/2852659056 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f76a40cf8a0 con 0x7f76b0108030 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.427+0000 7f76b5e00640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76980a2cb0 0x7f76980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.428+0000 7f769f000640 1 -- 172.21.15.26:0/2697876564 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac031650 con 0x55ce79e4ec00 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.428+0000 7f769f000640 1 -- 172.21.15.26:0/2697876564 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ac036030 con 0x55ce79e4ec00 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.428+0000 7f769f000640 1 -- 172.21.15.26:0/2697876564 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac004700 con 0x55ce79e4ec00 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.428+0000 7f76bb608740 1 -- 172.21.15.26:0/2697876564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4ec00 msgr2=0x55ce79e4f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.428+0000 7f76bb608740 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4ec00 0x55ce79e4f020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009ca0 tx=0x7f76ac0093c0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.428+0000 7f76b5e00640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76980a2cb0 0x7f76980a5170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f76a4098110 tx=0x7f76a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 -- 172.21.15.26:0/2697876564 shutdown_connections 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e54760 0x55ce79e56bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e4f5f0 0x55ce79e540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 --2- 172.21.15.26:0/2697876564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4ec00 0x55ce79e4f020 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 -- 172.21.15.26:0/2697876564 >> 172.21.15.26:0/2697876564 conn(0x55ce79e16fb0 msgr2=0x55ce79e4a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 -- 172.21.15.26:0/2697876564 shutdown_connections 2024-06-13T07:02:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 -- 172.21.15.26:0/2697876564 wait complete. 2024-06-13T07:02:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 -- start start 2024-06-13T07:02:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.429+0000 7f76bb608740 1 Processor -- start 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76bb608740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e58d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76bb608740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e59290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76bb608740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76bb608740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ce79e57530 con 0x55ce79e54760 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76bb608740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ce79d6d0b0 con 0x55ce79e4ec00 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76bb608740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ce79e576b0 con 0x55ce79e4f5f0 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76bb608740 10 client.0 _ll_register_callbacks cb 0x55ce79d58f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76b5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e59290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76b5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e59290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37862/0 (socket says 172.21.15.26:37862) 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76b5400640 1 -- 172.21.15.26:0/3972881789 learned_addr learned my addr 172.21.15.26:0/3972881789 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.430+0000 7f76b6800640 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e58d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f76b6800640 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f76b5400640 1 -- 172.21.15.26:0/3972881789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 msgr2=0x55ce79e58d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f76b5400640 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e58d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f76b5400640 1 -- 172.21.15.26:0/3972881789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 msgr2=0x55ce79e602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f76b5400640 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f76b5400640 1 -- 172.21.15.26:0/3972881789 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce79b92620 con 0x55ce79e4f5f0 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f76b6800640 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e58d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f76b5400640 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e59290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f76a0012770 tx=0x7f76a0012c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6373c4e77d3b0fad server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f7693400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.431+0000 7f7693400640 1 -- 172.21.15.26:0/3972881789 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011b50 con 0x55ce79e4f5f0 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.432+0000 7f76bb608740 1 -- 172.21.15.26:0/3972881789 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ac036030 con 0x55ce79e4f5f0 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.432+0000 7f7693400640 1 -- 172.21.15.26:0/3972881789 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f76a0011cf0 con 0x55ce79e4f5f0 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.432+0000 7f7693400640 1 -- 172.21.15.26:0/3972881789 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f76a000f460 con 0x55ce79e4f5f0 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.432+0000 7f76bb608740 10 client.25597 fetch_fsmap learned FSMap version 32 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.432+0000 7f76bb608740 1 -- 172.21.15.26:0/3972881789 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ce79e63710 con 0x55ce79e4f5f0 2024-06-13T07:02:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.432+0000 7f7693400640 1 -- 172.21.15.26:0/3972881789 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ac036030 con 0x55ce79e4f5f0 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.432+0000 7f7693400640 1 -- 172.21.15.26:0/3972881789 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f76a001c7b0 con 0x55ce79e4f5f0 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.433+0000 7f76bb608740 10 client.25597 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.433+0000 7f76bb608740 10 client.25597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.433+0000 7f76bb608740 20 client.25597 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.433+0000 7f76bb608740 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce79e64f60 0x55ce79e67380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.433+0000 7f76bb608740 4 client.25597 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.433+0000 7f76bb608740 1 -- 172.21.15.26:0/3972881789 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce79dbfb50 con 0x55ce79e64f60 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.433+0000 7f76b6800640 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce79e64f60 0x55ce79e67380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.434+0000 7f76b6800640 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce79e64f60 0x55ce79e67380 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6c121d6485b734 server_cookie=a0f4400c83639b67 in_seq=0 out_seq=0 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.434+0000 7f7693400640 10 client.25597 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.434+0000 7f7693400640 1 -- 172.21.15.26:0/3972881789 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ce79dbfb50 con 0x55ce79e64f60 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.434+0000 7f7693400640 10 client.25597 handle_command_reply: tid=0 2024-06-13T07:02:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 client.25597 shutdown 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 -- 172.21.15.26:0/3972881789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 msgr2=0x55ce79e59290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e59290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f76a0012770 tx=0x7f76a0012c30 comp rx=0 tx=0).stop 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 -- 172.21.15.26:0/3972881789 shutdown_connections 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce79e64f60 0x55ce79e67380 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e59290 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 --2- 172.21.15.26:0/3972881789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e58d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f7693400640 0 client.25597 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 -- 172.21.15.26:0/3972881789 >> 172.21.15.26:0/3972881789 conn(0x55ce79e16fb0 msgr2=0x55ce79e49840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 -- 172.21.15.26:0/3972881789 shutdown_connections 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 1 -- 172.21.15.26:0/3972881789 wait complete. 2024-06-13T07:02:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.435+0000 7f76bb608740 20 client.25597 trim_cache size 0 max 16384 2024-06-13T07:02:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76bb608740 1 Processor -- start 2024-06-13T07:02:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76bb608740 1 -- start start 2024-06-13T07:02:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76bb608740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e4a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76bb608740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e60290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76bb608740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e4d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76bb608740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01b3620 con 0x55ce79e54760 2024-06-13T07:02:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76bb608740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01b0ec0 con 0x55ce79e4ec00 2024-06-13T07:02:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76bb608740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01aed10 con 0x55ce79e4f5f0 2024-06-13T07:02:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76b6800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e4a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.454+0000 7f76b5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e60290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e4a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53570/0 (socket says 172.21.15.26:53570) 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 -- 172.21.15.26:0/1486799303 learned_addr learned my addr 172.21.15.26:0/1486799303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 --2- 172.21.15.26:0/1486799303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e4d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 -- 172.21.15.26:0/1486799303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 msgr2=0x55ce79e60290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 --2- 172.21.15.26:0/1486799303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e60290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 -- 172.21.15.26:0/1486799303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 msgr2=0x55ce79e4d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 --2- 172.21.15.26:0/1486799303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e4d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 -- 172.21.15.26:0/1486799303 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce79e53ba0 con 0x55ce79e4ec00 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.455+0000 7f76b6800640 1 --2- 172.21.15.26:0/1486799303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e4a870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f76ac006010 tx=0x7f76ac031bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b71225f534d3918 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.456+0000 7f769e600640 1 -- 172.21.15.26:0/1486799303 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0041c0 con 0x55ce79e4ec00 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.456+0000 7f769e600640 1 -- 172.21.15.26:0/1486799303 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ac036040 con 0x55ce79e4ec00 2024-06-13T07:02:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.456+0000 7f769e600640 1 -- 172.21.15.26:0/1486799303 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac003ed0 con 0x55ce79e4ec00 2024-06-13T07:02:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 -- 172.21.15.26:0/1486799303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 msgr2=0x55ce79e4a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 --2- 172.21.15.26:0/1486799303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e4a870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f76ac006010 tx=0x7f76ac031bb0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 -- 172.21.15.26:0/1486799303 shutdown_connections 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 --2- 172.21.15.26:0/1486799303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e4d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 --2- 172.21.15.26:0/1486799303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e60290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 --2- 172.21.15.26:0/1486799303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e4a870 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 -- 172.21.15.26:0/1486799303 >> 172.21.15.26:0/1486799303 conn(0x55ce79e16fb0 msgr2=0x55ce79e4a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 -- 172.21.15.26:0/1486799303 shutdown_connections 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 -- 172.21.15.26:0/1486799303 wait complete. 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.457+0000 7f76bb608740 1 -- start start 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76bb608740 1 Processor -- start 2024-06-13T07:02:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76bb608740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e5d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76bb608740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e5d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76bb608740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e5e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76bb608740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01af380 con 0x55ce79e54760 2024-06-13T07:02:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76bb608740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01b07f0 con 0x55ce79e4ec00 2024-06-13T07:02:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76bb608740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01afa50 con 0x55ce79e4f5f0 2024-06-13T07:02:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76bb608740 10 client.0 _ll_register_callbacks cb 0x7f76a0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.458+0000 7f76b6800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e5d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e5d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b6800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e5d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53582/0 (socket says 172.21.15.26:53582) 2024-06-13T07:02:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b6800640 1 -- 172.21.15.26:0/3837354426 learned_addr learned my addr 172.21.15.26:0/3837354426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b6800640 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e5e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b5400640 1 -- 172.21.15.26:0/3837354426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 msgr2=0x55ce79e5d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b5400640 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e5d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b5400640 1 -- 172.21.15.26:0/3837354426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 msgr2=0x55ce79e5e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b5400640 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e5e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b5400640 1 -- 172.21.15.26:0/3837354426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce79e5ef50 con 0x55ce79e4f5f0 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b6800640 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e5d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f76b5400640 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e5d9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f76a001ce10 tx=0x7f76a0005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5363c14aa6c05b2b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.459+0000 7f7693400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.460+0000 7f7693400640 1 -- 172.21.15.26:0/3837354426 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a002e020 con 0x55ce79e4f5f0 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.460+0000 7f76bb608740 1 -- 172.21.15.26:0/3837354426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ac036040 con 0x55ce79e4f5f0 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.460+0000 7f7693400640 1 -- 172.21.15.26:0/3837354426 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f76a0005250 con 0x55ce79e4f5f0 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.460+0000 7f7693400640 1 -- 172.21.15.26:0/3837354426 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f76a0039050 con 0x55ce79e4f5f0 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.460+0000 7f76bb608740 10 client.25603 fetch_fsmap learned FSMap version 32 2024-06-13T07:02:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.460+0000 7f76bb608740 1 -- 172.21.15.26:0/3837354426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ce79e60260 con 0x55ce79e4f5f0 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f7693400640 1 -- 172.21.15.26:0/3837354426 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ac036040 con 0x55ce79e4f5f0 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f7693400640 1 -- 172.21.15.26:0/3837354426 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f76a0004780 con 0x55ce79e4f5f0 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f76bb608740 10 client.25603 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f76bb608740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f76bb608740 20 client.25603 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f76bb608740 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce79ed00f0 0x55ce79ed2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f76bb608740 4 client.25603 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f76bb608740 1 -- 172.21.15.26:0/3837354426 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce79e58b40 con 0x55ce79ed00f0 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.461+0000 7f76b6800640 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce79ed00f0 0x55ce79ed2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.462+0000 7f76b6800640 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce79ed00f0 0x55ce79ed2510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab5f2ae53c9dc2e server_cookie=3f7f0a94dd241280 in_seq=0 out_seq=0 2024-06-13T07:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.462+0000 7f7693400640 10 client.25603 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:39.506 INFO:tasks.workunit.client.0.smithi026.stdout: 1 37097 4.10 MB/sec warmup 53 sec latency 29.121 ms 2024-06-13T07:02:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.537+0000 7f7693400640 1 -- 172.21.15.26:0/3837354426 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ce79e58b40 con 0x55ce79ed00f0 2024-06-13T07:02:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.537+0000 7f7693400640 10 client.25603 handle_command_reply: tid=0 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.537+0000 7f76bb608740 1 client.25603 shutdown 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f76bb608740 1 -- 172.21.15.26:0/3837354426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 msgr2=0x55ce79e5d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f76bb608740 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e5d9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f76a001ce10 tx=0x7f76a0005bd0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f76bb608740 1 -- 172.21.15.26:0/3837354426 shutdown_connections 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f76bb608740 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ce79ed00f0 0x55ce79ed2510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f76bb608740 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce79e54760 0x55ce79e5e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f76bb608740 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce79e4f5f0 0x55ce79e5d9f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f76bb608740 1 --2- 172.21.15.26:0/3837354426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ce79e4ec00 0x55ce79e5d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f76bb608740 1 -- 172.21.15.26:0/3837354426 >> 172.21.15.26:0/3837354426 conn(0x55ce79e16fb0 msgr2=0x55ce79e4d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.538+0000 7f7693400640 0 client.25603 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.539+0000 7f76bb608740 1 -- 172.21.15.26:0/3837354426 shutdown_connections 2024-06-13T07:02:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.539+0000 7f76bb608740 1 -- 172.21.15.26:0/3837354426 wait complete. 2024-06-13T07:02:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.539+0000 7f76bb608740 20 client.25603 trim_cache size 0 max 16384 2024-06-13T07:02:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:02:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:02:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:02:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 -- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76980a2cb0 msgr2=0x7f76980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76980a2cb0 0x7f76980a5170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f76a4098110 tx=0x7f76a4000bc0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 -- 172.21.15.26:0/2852659056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0108030 msgr2=0x7f76b019e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0108030 0x7f76b019e240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f76a400e9c0 tx=0x7f76a400ee80 comp rx=0 tx=0).stop 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 -- 172.21.15.26:0/2852659056 shutdown_connections 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f76980a2cb0 0x7f76980a5170 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b0109be0 0x7f76b01a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76b0109220 0x7f76b019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 --2- 172.21.15.26:0/2852659056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76b0108030 0x7f76b019e240 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 -- 172.21.15.26:0/2852659056 >> 172.21.15.26:0/2852659056 conn(0x7f76b0076ff0 msgr2=0x7f76b010c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 -- 172.21.15.26:0/2852659056 shutdown_connections 2024-06-13T07:02:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:39.540+0000 7f76b8800640 1 -- 172.21.15.26:0/2852659056 wait complete. 2024-06-13T07:02:40.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:39 smithi136 ceph-mon[37229]: pgmap v574: 129 pgs: 129 active+clean; 31 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 744 KiB/s wr, 166 op/s 2024-06-13T07:02:40.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:39 smithi026 ceph-mon[32041]: pgmap v574: 129 pgs: 129 active+clean; 31 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 744 KiB/s wr, 166 op/s 2024-06-13T07:02:40.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:39 smithi135 ceph-mon[36276]: pgmap v574: 129 pgs: 129 active+clean; 31 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 744 KiB/s wr, 166 op/s 2024-06-13T07:02:40.506 INFO:tasks.workunit.client.0.smithi026.stdout: 1 38509 4.16 MB/sec warmup 54 sec latency 18.531 ms 2024-06-13T07:02:41.506 INFO:tasks.workunit.client.0.smithi026.stdout: 1 39478 4.21 MB/sec warmup 55 sec latency 21.158 ms 2024-06-13T07:02:42.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:41 smithi136 ceph-mon[37229]: pgmap v575: 129 pgs: 129 active+clean; 32 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 601 KiB/s wr, 123 op/s 2024-06-13T07:02:42.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:41 smithi026 ceph-mon[32041]: pgmap v575: 129 pgs: 129 active+clean; 32 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 601 KiB/s wr, 123 op/s 2024-06-13T07:02:42.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:41 smithi135 ceph-mon[36276]: pgmap v575: 129 pgs: 129 active+clean; 32 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 601 KiB/s wr, 123 op/s 2024-06-13T07:02:42.506 INFO:tasks.workunit.client.0.smithi026.stdout: 1 40239 4.17 MB/sec warmup 56 sec latency 20.977 ms 2024-06-13T07:02:43.506 INFO:tasks.workunit.client.0.smithi026.stdout: 1 41525 4.19 MB/sec warmup 57 sec latency 21.806 ms 2024-06-13T07:02:44.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:43 smithi136 ceph-mon[37229]: pgmap v576: 129 pgs: 129 active+clean; 33 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 748 KiB/s wr, 167 op/s 2024-06-13T07:02:44.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:43 smithi026 ceph-mon[32041]: pgmap v576: 129 pgs: 129 active+clean; 33 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 748 KiB/s wr, 167 op/s 2024-06-13T07:02:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:43 smithi135 ceph-mon[36276]: pgmap v576: 129 pgs: 129 active+clean; 33 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 748 KiB/s wr, 167 op/s 2024-06-13T07:02:44.506 INFO:tasks.workunit.client.0.smithi026.stdout: 1 42763 4.30 MB/sec warmup 58 sec latency 18.870 ms 2024-06-13T07:02:45.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 43265 4.25 MB/sec warmup 59 sec latency 20.276 ms 2024-06-13T07:02:46.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:45 smithi136 ceph-mon[37229]: pgmap v577: 129 pgs: 129 active+clean; 33 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 689 KiB/s wr, 154 op/s 2024-06-13T07:02:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:45 smithi026 ceph-mon[32041]: pgmap v577: 129 pgs: 129 active+clean; 33 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 689 KiB/s wr, 154 op/s 2024-06-13T07:02:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:45 smithi135 ceph-mon[36276]: pgmap v577: 129 pgs: 129 active+clean; 33 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 689 KiB/s wr, 154 op/s 2024-06-13T07:02:46.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 44256 4.26 MB/sec warmup 60 sec latency 42.869 ms 2024-06-13T07:02:47.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 45579 4.31 MB/sec warmup 61 sec latency 16.729 ms 2024-06-13T07:02:48.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:47 smithi136 ceph-mon[37229]: pgmap v578: 129 pgs: 129 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 158 op/s 2024-06-13T07:02:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:47 smithi026 ceph-mon[32041]: pgmap v578: 129 pgs: 129 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 158 op/s 2024-06-13T07:02:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:47 smithi135 ceph-mon[36276]: pgmap v578: 129 pgs: 129 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 158 op/s 2024-06-13T07:02:48.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 46530 4.35 MB/sec warmup 62 sec latency 37.329 ms 2024-06-13T07:02:49.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:02:49.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 47245 4.31 MB/sec warmup 63 sec latency 20.983 ms 2024-06-13T07:02:49.681 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:50.116 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:49 smithi026 ceph-mon[32041]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 844 KiB/s wr, 187 op/s 2024-06-13T07:02:50.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:49 smithi136 ceph-mon[37229]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 844 KiB/s wr, 187 op/s 2024-06-13T07:02:50.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:49 smithi135 ceph-mon[36276]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 844 KiB/s wr, 187 op/s 2024-06-13T07:02:50.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 48742 4.34 MB/sec warmup 64 sec latency 24.107 ms 2024-06-13T07:02:51.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 49928 4.44 MB/sec warmup 65 sec latency 20.933 ms 2024-06-13T07:02:51.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:51 smithi026 ceph-mon[32041]: pgmap v580: 129 pgs: 129 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 KiB/s wr, 167 op/s 2024-06-13T07:02:52.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:51 smithi136 ceph-mon[37229]: pgmap v580: 129 pgs: 129 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 KiB/s wr, 167 op/s 2024-06-13T07:02:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.272+0000 7f32e2000700 1 -- 172.21.15.26:0/688894216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc10ec00 msgr2=0x7f32dc111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.272+0000 7f32e2000700 1 --2- 172.21.15.26:0/688894216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc10ec00 0x7f32dc111070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b0a0 tx=0x7f32d400b3b0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.272+0000 7f32e2000700 1 -- 172.21.15.26:0/688894216 shutdown_connections 2024-06-13T07:02:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.272+0000 7f32e2000700 1 --2- 172.21.15.26:0/688894216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc10ec00 0x7f32dc111070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.272+0000 7f32e2000700 1 --2- 172.21.15.26:0/688894216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32dc0748c0 0x7f32dc074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.272+0000 7f32e2000700 1 --2- 172.21.15.26:0/688894216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc0751e0 0x7f32dc10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.272+0000 7f32e2000700 1 -- 172.21.15.26:0/688894216 >> 172.21.15.26:0/688894216 conn(0x7f32dc0fdf10 msgr2=0x7f32dc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.272+0000 7f32e2000700 1 -- 172.21.15.26:0/688894216 shutdown_connections 2024-06-13T07:02:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.273+0000 7f32e2000700 1 -- 172.21.15.26:0/688894216 wait complete. 2024-06-13T07:02:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.274+0000 7f32e2000700 1 Processor -- start 2024-06-13T07:02:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.274+0000 7f32e2000700 1 -- start start 2024-06-13T07:02:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.274+0000 7f32e2000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0748c0 0x7f32dc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.274+0000 7f32e2000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32dc0751e0 0x7f32dc19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.274+0000 7f32e2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc10ec00 0x7f32dc19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.274+0000 7f32e2000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc114930 con 0x7f32dc0748c0 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.274+0000 7f32e2000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1147b0 con 0x7f32dc10ec00 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.274+0000 7f32e2000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc114ab0 con 0x7f32dc0751e0 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32e1600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc10ec00 0x7f32dc19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32e0c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0748c0 0x7f32dc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32e1600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc10ec00 0x7f32dc19f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49514/0 (socket says 172.21.15.26:49514) 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32e1600700 1 -- 172.21.15.26:0/512232662 learned_addr learned my addr 172.21.15.26:0/512232662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32dbe00700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32dc0751e0 0x7f32dc19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32dbe00700 1 -- 172.21.15.26:0/512232662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc10ec00 msgr2=0x7f32dc19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32dbe00700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc10ec00 0x7f32dc19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32dbe00700 1 -- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0748c0 msgr2=0x7f32dc19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32dbe00700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0748c0 0x7f32dc19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32dbe00700 1 -- 172.21.15.26:0/512232662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc19fd20 con 0x7f32dc0751e0 2024-06-13T07:02:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.275+0000 7f32e1600700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc10ec00 0x7f32dc19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.276+0000 7f32dbe00700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32dc0751e0 0x7f32dc19b2a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f32c800e9f0 tx=0x7f32c800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.276+0000 7f32d9600700 1 -- 172.21.15.26:0/512232662 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c800c9c0 con 0x7f32dc0751e0 2024-06-13T07:02:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.276+0000 7f32d9600700 1 -- 172.21.15.26:0/512232662 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c800cb60 con 0x7f32dc0751e0 2024-06-13T07:02:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.276+0000 7f32e2000700 1 -- 172.21.15.26:0/512232662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc1a0010 con 0x7f32dc0751e0 2024-06-13T07:02:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.276+0000 7f32d9600700 1 -- 172.21.15.26:0/512232662 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8018680 con 0x7f32dc0751e0 2024-06-13T07:02:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.276+0000 7f32e2000700 1 -- 172.21.15.26:0/512232662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc1a7890 con 0x7f32dc0751e0 2024-06-13T07:02:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.281+0000 7f32ee024b80 1 Processor -- start 2024-06-13T07:02:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.282+0000 7f32d9600700 1 -- 172.21.15.26:0/512232662 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32c8010430 con 0x7f32dc0751e0 2024-06-13T07:02:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.282+0000 7f32ee024b80 1 -- start start 2024-06-13T07:02:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.282+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.282+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6cb960 0x55d50f6cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.282+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6d6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.282+0000 7f32ee024b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d50f5d8a10 con 0x55d50f6d4570 2024-06-13T07:02:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.282+0000 7f32ee024b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d50f61bb00 con 0x55d50f645e40 2024-06-13T07:02:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.282+0000 7f32ee024b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d50f5d68f0 con 0x55d50f6cb960 2024-06-13T07:02:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e1600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e1600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49528/0 (socket says 172.21.15.26:49528) 2024-06-13T07:02:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e1600700 1 -- 172.21.15.26:0/4174549875 learned_addr learned my addr 172.21.15.26:0/4174549875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e0c00700 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6cb960 0x55d50f6cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:51 smithi135 ceph-mon[36276]: pgmap v580: 129 pgs: 129 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 KiB/s wr, 167 op/s 2024-06-13T07:02:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32dbe00700 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6d6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e1600700 1 -- 172.21.15.26:0/4174549875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6cb960 msgr2=0x55d50f6cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e1600700 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6cb960 0x55d50f6cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e1600700 1 -- 172.21.15.26:0/4174549875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 msgr2=0x55d50f6d6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e1600700 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6d6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.283+0000 7f32e1600700 1 -- 172.21.15.26:0/4174549875 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50f5c3f90 con 0x55d50f645e40 2024-06-13T07:02:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.284+0000 7f32e1600700 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6cb420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009e40 tx=0x7f32d400cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256b824ad4c072b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.284+0000 7f32d9600700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32c40a2ca0 0x7f32c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.284+0000 7f32d9600700 1 -- 172.21.15.26:0/512232662 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f32c80cf420 con 0x7f32dc0751e0 2024-06-13T07:02:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.284+0000 7f32e0c00700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32c40a2ca0 0x7f32c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.284+0000 7f32ce000700 1 -- 172.21.15.26:0/4174549875 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4021070 con 0x55d50f645e40 2024-06-13T07:02:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.284+0000 7f32ce000700 1 -- 172.21.15.26:0/4174549875 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32d4009460 con 0x55d50f645e40 2024-06-13T07:02:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.284+0000 7f32ce000700 1 -- 172.21.15.26:0/4174549875 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40256b0 con 0x55d50f645e40 2024-06-13T07:02:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.285+0000 7f32ee024b80 1 -- 172.21.15.26:0/4174549875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 msgr2=0x55d50f6cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.285+0000 7f32ee024b80 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6cb420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009e40 tx=0x7f32d400cf30 comp rx=0 tx=0).stop 2024-06-13T07:02:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.285+0000 7f32e0c00700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32c40a2ca0 0x7f32c40a5160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55d50f6c6590 tx=0x7f32d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.286+0000 7f32ee024b80 1 -- 172.21.15.26:0/4174549875 shutdown_connections 2024-06-13T07:02:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.286+0000 7f32ee024b80 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6cb420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.286+0000 7f32ee024b80 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6d6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.286+0000 7f32ee024b80 1 --2- 172.21.15.26:0/4174549875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6cb960 0x55d50f6cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.286+0000 7f32ee024b80 1 -- 172.21.15.26:0/4174549875 >> 172.21.15.26:0/4174549875 conn(0x55d50f5f6170 msgr2=0x55d50f6c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.286+0000 7f32ee024b80 1 -- 172.21.15.26:0/4174549875 shutdown_connections 2024-06-13T07:02:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.286+0000 7f32ee024b80 1 -- 172.21.15.26:0/4174549875 wait complete. 2024-06-13T07:02:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.286+0000 7f32ee024b80 1 -- start start 2024-06-13T07:02:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.287+0000 7f32ee024b80 1 Processor -- start 2024-06-13T07:02:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.287+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.287+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 0x55d50f6d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.287+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.287+0000 7f32ee024b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d50f5b7f60 con 0x55d50f6cb960 2024-06-13T07:02:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.287+0000 7f32ee024b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d50f617ac0 con 0x55d50f645e40 2024-06-13T07:02:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.287+0000 7f32ee024b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d50f6d7180 con 0x55d50f6d4570 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.287+0000 7f32ee024b80 10 client.0 _ll_register_callbacks cb 0x55d50f5b5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44994/0 (socket says 172.21.15.26:44994) 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 -- 172.21.15.26:0/173792511 learned_addr learned my addr 172.21.15.26:0/173792511 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6d3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49534/0 (socket says 172.21.15.26:49534) 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32dbe00700 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 0x55d50f6d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 -- 172.21.15.26:0/173792511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 msgr2=0x55d50f6d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 -- 172.21.15.26:0/173792511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 msgr2=0x55d50f6d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 0x55d50f6d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 -- 172.21.15.26:0/173792511 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50f56a8d0 con 0x55d50f6d4570 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e0c00700 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32e1600700 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6d1590 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f32d4007dc0 tx=0x7f32d40076d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e33744d17634d443 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.288+0000 7f32c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.289+0000 7f32c2a00700 1 -- 172.21.15.26:0/173792511 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d402b070 con 0x55d50f6d4570 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.289+0000 7f32c2a00700 1 -- 172.21.15.26:0/173792511 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f32d4025860 con 0x55d50f6d4570 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.289+0000 7f32ee024b80 1 -- 172.21.15.26:0/173792511 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d4009460 con 0x55d50f6d4570 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.289+0000 7f32c2a00700 1 -- 172.21.15.26:0/173792511 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32d4038050 con 0x55d50f6d4570 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.289+0000 7f32ee024b80 10 client.25615 fetch_fsmap learned FSMap version 32 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.289+0000 7f32ee024b80 1 -- 172.21.15.26:0/173792511 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d50f6e1910 con 0x55d50f6d4570 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32c2a00700 1 -- 172.21.15.26:0/173792511 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32d4009460 con 0x55d50f6d4570 2024-06-13T07:02:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32c2a00700 1 -- 172.21.15.26:0/173792511 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f32d4011070 con 0x55d50f6d4570 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32ee024b80 10 client.25615 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32ee024b80 10 client.25615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32ee024b80 20 client.25615 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32ee024b80 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d50f6e3de0 0x55d50f6e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32ee024b80 4 client.25615 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32ee024b80 1 -- 172.21.15.26:0/173792511 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d50f629000 con 0x55d50f6e3de0 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.290+0000 7f32e0c00700 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d50f6e3de0 0x55d50f6e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.291+0000 7f32e0c00700 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d50f6e3de0 0x55d50f6e61d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12fd90793e64532 server_cookie=6f0126732d03b059 in_seq=0 out_seq=0 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.291+0000 7f32c2a00700 10 client.25615 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32c2a00700 1 -- 172.21.15.26:0/173792511 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d50f629000 con 0x55d50f6e3de0 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32c2a00700 10 client.25615 handle_command_reply: tid=0 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 client.25615 shutdown 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 -- 172.21.15.26:0/173792511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 msgr2=0x55d50f6d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6d1590 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f32d4007dc0 tx=0x7f32d40076d0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 -- 172.21.15.26:0/173792511 shutdown_connections 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d50f6e3de0 0x55d50f6e61d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6d1590 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 --2- 172.21.15.26:0/173792511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 0x55d50f6d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32c2a00700 0 client.25615 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 -- 172.21.15.26:0/173792511 >> 172.21.15.26:0/173792511 conn(0x55d50f5f6170 msgr2=0x55d50f6c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 -- 172.21.15.26:0/173792511 shutdown_connections 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.292+0000 7f32ee024b80 1 -- 172.21.15.26:0/173792511 wait complete. 2024-06-13T07:02:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.293+0000 7f32ee024b80 20 client.25615 trim_cache size 0 max 16384 2024-06-13T07:02:52.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.322+0000 7f32ee024b80 1 Processor -- start 2024-06-13T07:02:52.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.322+0000 7f32ee024b80 1 -- start start 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f645e40 0x55d50f6d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f6cb960 0x55d50f6e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6e9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32ee024b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1b2c90 con 0x55d50f6d4570 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32ee024b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1b04f0 con 0x55d50f6cb960 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32ee024b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1ae340 con 0x55d50f645e40 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32dbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f6cb960 0x55d50f6e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32e1600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32dbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f6cb960 0x55d50f6e91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50236/0 (socket says 172.21.15.26:50236) 2024-06-13T07:02:52.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32e1600700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6e9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54556/0 (socket says 172.21.15.26:54556) 2024-06-13T07:02:52.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32dbe00700 1 -- 172.21.15.26:0/935501425 learned_addr learned my addr 172.21.15.26:0/935501425 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.323+0000 7f32e0c00700 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f645e40 0x55d50f6d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.324+0000 7f32dbe00700 1 -- 172.21.15.26:0/935501425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f645e40 msgr2=0x55d50f6d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.324+0000 7f32dbe00700 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f645e40 0x55d50f6d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.324+0000 7f32dbe00700 1 -- 172.21.15.26:0/935501425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 msgr2=0x55d50f6e9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.324+0000 7f32dbe00700 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6e9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.324+0000 7f32dbe00700 1 -- 172.21.15.26:0/935501425 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50f6cf5c0 con 0x55d50f6cb960 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.324+0000 7f32e0c00700 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f645e40 0x55d50f6d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.324+0000 7f32dbe00700 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f6cb960 0x55d50f6e91f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f32c809b820 tx=0x7f32c8010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=741cf81ae3574827 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.324+0000 7f32cd600700 1 -- 172.21.15.26:0/935501425 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c809b660 con 0x55d50f6cb960 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32cd600700 1 -- 172.21.15.26:0/935501425 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c800cd10 con 0x55d50f6cb960 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32cd600700 1 -- 172.21.15.26:0/935501425 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c809b1f0 con 0x55d50f6cb960 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32ee024b80 1 -- 172.21.15.26:0/935501425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f6cb960 msgr2=0x55d50f6e91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32ee024b80 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f6cb960 0x55d50f6e91f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f32c809b820 tx=0x7f32c8010ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32ee024b80 1 -- 172.21.15.26:0/935501425 shutdown_connections 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32ee024b80 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6d4570 0x55d50f6e9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32ee024b80 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f645e40 0x55d50f6d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32ee024b80 1 --2- 172.21.15.26:0/935501425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f6cb960 0x55d50f6e91f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.325+0000 7f32ee024b80 1 -- 172.21.15.26:0/935501425 >> 172.21.15.26:0/935501425 conn(0x55d50f5f6170 msgr2=0x55d50f6d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.326+0000 7f32ee024b80 1 -- 172.21.15.26:0/935501425 shutdown_connections 2024-06-13T07:02:52.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.326+0000 7f32ee024b80 1 -- 172.21.15.26:0/935501425 wait complete. 2024-06-13T07:02:52.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.326+0000 7f32ee024b80 1 -- start start 2024-06-13T07:02:52.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32ee024b80 1 Processor -- start 2024-06-13T07:02:52.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 0x55d50f6e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32ee024b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6e7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32ee024b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1ae9b0 con 0x55d50f6cb960 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32ee024b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1afe20 con 0x55d50f645e40 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32ee024b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1af080 con 0x55d50f6d4570 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32ee024b80 10 client.0 _ll_register_callbacks cb 0x55d50f5b5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.327+0000 7f32e0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6e9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50238/0 (socket says 172.21.15.26:50238) 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e0c00700 1 -- 172.21.15.26:0/2351461288 learned_addr learned my addr 172.21.15.26:0/2351461288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32dbe00700 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 0x55d50f6e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e1600700 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e0c00700 1 -- 172.21.15.26:0/2351461288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 msgr2=0x55d50f6e7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e0c00700 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6e7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e0c00700 1 -- 172.21.15.26:0/2351461288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 msgr2=0x55d50f6e6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e0c00700 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 0x55d50f6e6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e0c00700 1 -- 172.21.15.26:0/2351461288 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50f6e7660 con 0x55d50f645e40 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32e0c00700 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6e9770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55d50f617af0 tx=0x7f32d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58bd520687eb8d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.328+0000 7f32c2a00700 1 -- 172.21.15.26:0/2351461288 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0037060 con 0x55d50f645e40 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32c2a00700 1 -- 172.21.15.26:0/2351461288 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f32d0003cc0 con 0x55d50f645e40 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 1 -- 172.21.15.26:0/2351461288 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d50f623580 con 0x55d50f645e40 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32c2a00700 1 -- 172.21.15.26:0/2351461288 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32d003e050 con 0x55d50f645e40 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 10 client.25571 fetch_fsmap learned FSMap version 32 2024-06-13T07:02:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 1 -- 172.21.15.26:0/2351461288 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d50f756df0 con 0x55d50f645e40 2024-06-13T07:02:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32c2a00700 1 -- 172.21.15.26:0/2351461288 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d50f623580 con 0x55d50f645e40 2024-06-13T07:02:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32c2a00700 1 -- 172.21.15.26:0/2351461288 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f32d001e520 con 0x55d50f645e40 2024-06-13T07:02:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 10 client.25571 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:02:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 20 client.25571 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d50f7574d0 0x55d50f7598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 4 client.25571 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.329+0000 7f32ee024b80 1 -- 172.21.15.26:0/2351461288 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d50f49d250 con 0x55d50f7574d0 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.330+0000 7f32dbe00700 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d50f7574d0 0x55d50f7598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.330+0000 7f32dbe00700 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d50f7574d0 0x55d50f7598f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63adf25ad34e0c26 server_cookie=7faf31f90f38e9d7 in_seq=0 out_seq=0 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.330+0000 7f32c2a00700 10 client.25571 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32c2a00700 1 -- 172.21.15.26:0/2351461288 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d50f49d250 con 0x55d50f7574d0 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32c2a00700 10 client.25571 handle_command_reply: tid=0 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 client.25571 shutdown 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 -- 172.21.15.26:0/2351461288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 msgr2=0x55d50f6e9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6e9770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55d50f617af0 tx=0x7f32d0004750 comp rx=0 tx=0).stop 2024-06-13T07:02:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 -- 172.21.15.26:0/2351461288 shutdown_connections 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d50f7574d0 0x55d50f7598f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d50f645e40 0x55d50f6e9770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d50f6d4570 0x55d50f6e7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 --2- 172.21.15.26:0/2351461288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d50f6cb960 0x55d50f6e6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 -- 172.21.15.26:0/2351461288 >> 172.21.15.26:0/2351461288 conn(0x55d50f5f6170 msgr2=0x55d50f6c99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32c2a00700 0 client.25571 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 -- 172.21.15.26:0/2351461288 shutdown_connections 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.331+0000 7f32ee024b80 1 -- 172.21.15.26:0/2351461288 wait complete. 2024-06-13T07:02:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.332+0000 7f32ee024b80 20 client.25571 trim_cache size 0 max 16384 2024-06-13T07:02:52.335 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:02:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-06-13T07:02:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-06-13T07:02:52.335 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:02:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.332+0000 7f32e2000700 1 -- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32c40a2ca0 msgr2=0x7f32c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.332+0000 7f32e2000700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32c40a2ca0 0x7f32c40a5160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55d50f6c6590 tx=0x7f32d0009260 comp rx=0 tx=0).stop 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.332+0000 7f32e2000700 1 -- 172.21.15.26:0/512232662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32dc0751e0 msgr2=0x7f32dc19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.332+0000 7f32e2000700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32dc0751e0 0x7f32dc19b2a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f32c800e9f0 tx=0x7f32c800ed00 comp rx=0 tx=0).stop 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.333+0000 7f32e2000700 1 -- 172.21.15.26:0/512232662 shutdown_connections 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.333+0000 7f32e2000700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32c40a2ca0 0x7f32c40a5160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.333+0000 7f32e2000700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc10ec00 0x7f32dc19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.333+0000 7f32e2000700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0748c0 0x7f32dc19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.333+0000 7f32e2000700 1 --2- 172.21.15.26:0/512232662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32dc0751e0 0x7f32dc19b2a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.333+0000 7f32e2000700 1 -- 172.21.15.26:0/512232662 >> 172.21.15.26:0/512232662 conn(0x7f32dc0fdf10 msgr2=0x7f32dc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.333+0000 7f32e2000700 1 -- 172.21.15.26:0/512232662 shutdown_connections 2024-06-13T07:02:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:52.333+0000 7f32e2000700 1 -- 172.21.15.26:0/512232662 wait complete. 2024-06-13T07:02:52.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 50478 4.38 MB/sec warmup 66 sec latency 17.151 ms 2024-06-13T07:02:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.700+0000 7fbb9d000640 1 -- 172.21.15.26:0/863119896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb981022c0 msgr2=0x7fbb981046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.700+0000 7fbb9d000640 1 --2- 172.21.15.26:0/863119896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb981022c0 0x7fbb981046b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009900 tx=0x7fbb8802a400 comp rx=0 tx=0).stop 2024-06-13T07:02:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.700+0000 7fbb9d000640 1 -- 172.21.15.26:0/863119896 shutdown_connections 2024-06-13T07:02:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.700+0000 7fbb9d000640 1 --2- 172.21.15.26:0/863119896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb98104bf0 0x7fbb98106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.700+0000 7fbb9d000640 1 --2- 172.21.15.26:0/863119896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb98107520 0x7fbb98109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.700+0000 7fbb9d000640 1 --2- 172.21.15.26:0/863119896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb981022c0 0x7fbb981046b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.700+0000 7fbb9d000640 1 -- 172.21.15.26:0/863119896 >> 172.21.15.26:0/863119896 conn(0x7fbb980fbc90 msgr2=0x7fbb980fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.701+0000 7fbb9d000640 1 -- 172.21.15.26:0/863119896 shutdown_connections 2024-06-13T07:02:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.701+0000 7fbb9d000640 1 -- 172.21.15.26:0/863119896 wait complete. 2024-06-13T07:02:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.703+0000 7fbb9d000640 1 Processor -- start 2024-06-13T07:02:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.704+0000 7fbb9d000640 1 -- start start 2024-06-13T07:02:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.705+0000 7fbb9d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb981022c0 0x7fbb98195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.705+0000 7fbb9d000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb98104bf0 0x7fbb98195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.705+0000 7fbb9d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb98107520 0x7fbb9819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.705+0000 7fbb9d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9810b2f0 con 0x7fbb981022c0 2024-06-13T07:02:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.705+0000 7fbb9d000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9810b170 con 0x7fbb98104bf0 2024-06-13T07:02:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.705+0000 7fbb9d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9810b470 con 0x7fbb98107520 2024-06-13T07:02:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.705+0000 7fbb96a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb98104bf0 0x7fbb98195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.706+0000 7fbb96a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb98104bf0 0x7fbb98195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50254/0 (socket says 172.21.15.26:50254) 2024-06-13T07:02:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.706+0000 7fbb96a00640 1 -- 172.21.15.26:0/3337125926 learned_addr learned my addr 172.21.15.26:0/3337125926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.706+0000 7fbb97e00640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb98107520 0x7fbb9819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.706+0000 7fbb96a00640 1 -- 172.21.15.26:0/3337125926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb98107520 msgr2=0x7fbb9819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.706+0000 7fbb96a00640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb98107520 0x7fbb9819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.706+0000 7fbb96a00640 1 -- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb981022c0 msgr2=0x7fbb98195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.706+0000 7fbb96a00640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb981022c0 0x7fbb98195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.706+0000 7fbb96a00640 1 -- 172.21.15.26:0/3337125926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9819d5a0 con 0x7fbb98104bf0 2024-06-13T07:02:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.707+0000 7fbb96a00640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb98104bf0 0x7fbb98195ec0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e9a0 tx=0x7fbb8400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.707+0000 7fbb8fe00640 1 -- 172.21.15.26:0/3337125926 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400cd80 con 0x7fbb98104bf0 2024-06-13T07:02:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.708+0000 7fbb8fe00640 1 -- 172.21.15.26:0/3337125926 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb840035f0 con 0x7fbb98104bf0 2024-06-13T07:02:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.708+0000 7fbb9d000640 1 -- 172.21.15.26:0/3337125926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9819d890 con 0x7fbb98104bf0 2024-06-13T07:02:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.708+0000 7fbb8fe00640 1 -- 172.21.15.26:0/3337125926 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84010670 con 0x7fbb98104bf0 2024-06-13T07:02:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.708+0000 7fbb9d000640 1 -- 172.21.15.26:0/3337125926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9819de30 con 0x7fbb98104bf0 2024-06-13T07:02:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.718+0000 7fbba1467740 1 Processor -- start 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbba1467740 1 -- start start 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbba1467740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbba1467740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbba1467740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c26920 0x55db72c28db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbba1467740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55db72b84420 con 0x55db72c217b0 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbba1467740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db72b2bc50 con 0x55db72c26920 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbba1467740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55db72b803e0 con 0x55db72c20dc0 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbb8fe00640 1 -- 172.21.15.26:0/3337125926 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb840037a0 con 0x7fbb98104bf0 2024-06-13T07:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbb97e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbb97e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37492/0 (socket says 172.21.15.26:37492) 2024-06-13T07:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.719+0000 7fbb97e00640 1 -- 172.21.15.26:0/497274935 learned_addr learned my addr 172.21.15.26:0/497274935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.720+0000 7fbb96a00640 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c26920 0x55db72c28db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.720+0000 7fbb97400640 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.721+0000 7fbb96a00640 1 -- 172.21.15.26:0/497274935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 msgr2=0x55db72c211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.721+0000 7fbb96a00640 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.721+0000 7fbb96a00640 1 -- 172.21.15.26:0/497274935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 msgr2=0x55db72c262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.721+0000 7fbb96a00640 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.721+0000 7fbb96a00640 1 -- 172.21.15.26:0/497274935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db72898210 con 0x55db72c26920 2024-06-13T07:02:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.721+0000 7fbb97400640 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.722+0000 7fbb8fe00640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbb7c0a2c40 0x7fbb7c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.722+0000 7fbb8fe00640 1 -- 172.21.15.26:0/3337125926 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbb840cf770 con 0x7fbb98104bf0 2024-06-13T07:02:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.722+0000 7fbb96a00640 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c26920 0x55db72c28db0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb840cce50 tx=0x7fbb840667b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2482346f55bd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.723+0000 7fbb8cc00640 1 -- 172.21.15.26:0/497274935 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84086030 con 0x55db72c26920 2024-06-13T07:02:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.723+0000 7fbb8cc00640 1 -- 172.21.15.26:0/497274935 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb84066cc0 con 0x55db72c26920 2024-06-13T07:02:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.723+0000 7fbb8cc00640 1 -- 172.21.15.26:0/497274935 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8409b540 con 0x55db72c26920 2024-06-13T07:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 -- 172.21.15.26:0/497274935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c26920 msgr2=0x55db72c28db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c26920 0x55db72c28db0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb840cce50 tx=0x7fbb840667b0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbb97e00640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbb7c0a2c40 0x7fbb7c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 -- 172.21.15.26:0/497274935 shutdown_connections 2024-06-13T07:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c26920 0x55db72c28db0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 --2- 172.21.15.26:0/497274935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 -- 172.21.15.26:0/497274935 >> 172.21.15.26:0/497274935 conn(0x55db72be8fb0 msgr2=0x55db72c1c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 -- 172.21.15.26:0/497274935 shutdown_connections 2024-06-13T07:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.724+0000 7fbba1467740 1 -- 172.21.15.26:0/497274935 wait complete. 2024-06-13T07:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.725+0000 7fbb97e00640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbb7c0a2c40 0x7fbb7c0a5100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55db72c22010 tx=0x7fbb9000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.725+0000 7fbba1467740 1 -- start start 2024-06-13T07:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.725+0000 7fbba1467740 1 Processor -- start 2024-06-13T07:02:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.726+0000 7fbba1467740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c3c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.727+0000 7fbb97400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c3c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.727+0000 7fbba1467740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c217b0 0x55db72c3c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.727+0000 7fbba1467740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c26920 0x55db72c43620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.727+0000 7fbba1467740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55db72c3a880 con 0x55db72c26920 2024-06-13T07:02:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.727+0000 7fbba1467740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db72b3f0b0 con 0x55db72c20dc0 2024-06-13T07:02:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.727+0000 7fbba1467740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55db72c3aa00 con 0x55db72c217b0 2024-06-13T07:02:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.727+0000 7fbba1467740 10 client.0 _ll_register_callbacks cb 0x55db72b2af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.727+0000 7fbb97400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c3c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50278/0 (socket says 172.21.15.26:50278) 2024-06-13T07:02:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.728+0000 7fbb97e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c26920 0x55db72c43620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.728+0000 7fbb97400640 1 -- 172.21.15.26:0/4180199913 learned_addr learned my addr 172.21.15.26:0/4180199913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.728+0000 7fbb96a00640 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c217b0 0x55db72c3c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.729+0000 7fbb96a00640 1 -- 172.21.15.26:0/4180199913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 msgr2=0x55db72c3c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.729+0000 7fbb96a00640 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c3c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.729+0000 7fbb96a00640 1 -- 172.21.15.26:0/4180199913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c26920 msgr2=0x55db72c43620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.729+0000 7fbb96a00640 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c26920 0x55db72c43620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.729+0000 7fbb96a00640 1 -- 172.21.15.26:0/4180199913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db72964620 con 0x55db72c217b0 2024-06-13T07:02:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.729+0000 7fbb97e00640 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c26920 0x55db72c43620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.729+0000 7fbb96a00640 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c217b0 0x55db72c3c5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009380 tx=0x7fbb84009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9967dbf47d961b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.729+0000 7fbb81600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:02:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.730+0000 7fbb81600640 1 -- 172.21.15.26:0/4180199913 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84086040 con 0x55db72c217b0 2024-06-13T07:02:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.730+0000 7fbb81600640 1 -- 172.21.15.26:0/4180199913 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbb84096490 con 0x55db72c217b0 2024-06-13T07:02:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.730+0000 7fbba1467740 1 -- 172.21.15.26:0/4180199913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb84066cc0 con 0x55db72c217b0 2024-06-13T07:02:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.730+0000 7fbb81600640 1 -- 172.21.15.26:0/4180199913 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb8409b540 con 0x55db72c217b0 2024-06-13T07:02:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.731+0000 7fbba1467740 10 client.25630 fetch_fsmap learned FSMap version 32 2024-06-13T07:02:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.731+0000 7fbba1467740 1 -- 172.21.15.26:0/4180199913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db72c3a8d0 con 0x55db72c217b0 2024-06-13T07:02:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.731+0000 7fbb81600640 1 -- 172.21.15.26:0/4180199913 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb84066cc0 con 0x55db72c217b0 2024-06-13T07:02:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.731+0000 7fbb81600640 1 -- 172.21.15.26:0/4180199913 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fbb840d5020 con 0x55db72c217b0 2024-06-13T07:02:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.731+0000 7fbba1467740 10 client.25630 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:02:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.731+0000 7fbba1467740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.732+0000 7fbba1467740 20 client.25630 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.732+0000 7fbba1467740 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db72c50440 0x55db72c52830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.732+0000 7fbba1467740 4 client.25630 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.732+0000 7fbba1467740 1 -- 172.21.15.26:0/4180199913 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db72b91b50 con 0x55db72c50440 2024-06-13T07:02:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.733+0000 7fbb97400640 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db72c50440 0x55db72c52830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.734+0000 7fbb97400640 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db72c50440 0x55db72c52830 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7d60e54ed52b65 server_cookie=58f530a2aec4d9f7 in_seq=0 out_seq=0 2024-06-13T07:02:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.734+0000 7fbb81600640 10 client.25630 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.735+0000 7fbb81600640 1 -- 172.21.15.26:0/4180199913 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db72b91b50 con 0x55db72c50440 2024-06-13T07:02:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.735+0000 7fbb81600640 10 client.25630 handle_command_reply: tid=0 2024-06-13T07:02:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.735+0000 7fbba1467740 1 client.25630 shutdown 2024-06-13T07:02:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.735+0000 7fbba1467740 1 -- 172.21.15.26:0/4180199913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c217b0 msgr2=0x55db72c3c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.735+0000 7fbba1467740 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c217b0 0x55db72c3c5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009380 tx=0x7fbb84009310 comp rx=0 tx=0).stop 2024-06-13T07:02:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.735+0000 7fbba1467740 1 -- 172.21.15.26:0/4180199913 shutdown_connections 2024-06-13T07:02:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.735+0000 7fbba1467740 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db72c50440 0x55db72c52830 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.736+0000 7fbba1467740 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c26920 0x55db72c43620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.736+0000 7fbba1467740 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c217b0 0x55db72c3c5e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.736+0000 7fbba1467740 1 --2- 172.21.15.26:0/4180199913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c3c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.736+0000 7fbba1467740 1 -- 172.21.15.26:0/4180199913 >> 172.21.15.26:0/4180199913 conn(0x55db72be8fb0 msgr2=0x55db72c1b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.736+0000 7fbb81600640 0 client.25630 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.736+0000 7fbba1467740 1 -- 172.21.15.26:0/4180199913 shutdown_connections 2024-06-13T07:02:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.737+0000 7fbba1467740 1 -- 172.21.15.26:0/4180199913 wait complete. 2024-06-13T07:02:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.737+0000 7fbba1467740 20 client.25630 trim_cache size 0 max 16384 2024-06-13T07:02:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbba1467740 1 Processor -- start 2024-06-13T07:02:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbba1467740 1 -- start start 2024-06-13T07:02:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbba1467740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72be8fb0 0x55db72c1fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbba1467740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbba1467740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c217b0 0x55db72c570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbba1467740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbb981a2610 con 0x55db72be8fb0 2024-06-13T07:02:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbba1467740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9819feb0 con 0x55db72c217b0 2024-06-13T07:02:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbba1467740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98073810 con 0x55db72c20dc0 2024-06-13T07:02:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.763+0000 7fbb97e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c217b0 0x55db72c570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb97e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c217b0 0x55db72c570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50284/0 (socket says 172.21.15.26:50284) 2024-06-13T07:02:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb97e00640 1 -- 172.21.15.26:0/1793575979 learned_addr learned my addr 172.21.15.26:0/1793575979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb96a00640 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb97400640 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72be8fb0 0x55db72c1fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb97e00640 1 -- 172.21.15.26:0/1793575979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 msgr2=0x55db72c435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb97e00640 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb97e00640 1 -- 172.21.15.26:0/1793575979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72be8fb0 msgr2=0x55db72c1fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb97e00640 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72be8fb0 0x55db72c1fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.764+0000 7fbb97e00640 1 -- 172.21.15.26:0/1793575979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db72c1d1f0 con 0x55db72c217b0 2024-06-13T07:02:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.765+0000 7fbb97e00640 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c217b0 0x55db72c570f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbb9001a8b0 tx=0x7fbb90018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f0bcb36a2a31b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.765+0000 7fbb83e00640 1 -- 172.21.15.26:0/1793575979 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90021040 con 0x55db72c217b0 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.765+0000 7fbb83e00640 1 -- 172.21.15.26:0/1793575979 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb9001ad70 con 0x55db72c217b0 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.765+0000 7fbb83e00640 1 -- 172.21.15.26:0/1793575979 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb900052c0 con 0x55db72c217b0 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.766+0000 7fbba1467740 1 -- 172.21.15.26:0/1793575979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c217b0 msgr2=0x55db72c570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.766+0000 7fbba1467740 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c217b0 0x55db72c570f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbb9001a8b0 tx=0x7fbb90018000 comp rx=0 tx=0).stop 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.766+0000 7fbba1467740 1 -- 172.21.15.26:0/1793575979 shutdown_connections 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.766+0000 7fbba1467740 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c217b0 0x55db72c570f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.766+0000 7fbba1467740 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72c20dc0 0x55db72c435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.766+0000 7fbba1467740 1 --2- 172.21.15.26:0/1793575979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72be8fb0 0x55db72c1fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.766+0000 7fbba1467740 1 -- 172.21.15.26:0/1793575979 >> 172.21.15.26:0/1793575979 conn(0x55db72c3bc60 msgr2=0x55db72c1f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.766+0000 7fbba1467740 1 -- 172.21.15.26:0/1793575979 shutdown_connections 2024-06-13T07:02:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.767+0000 7fbba1467740 1 -- 172.21.15.26:0/1793575979 wait complete. 2024-06-13T07:02:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.767+0000 7fbba1467740 1 -- start start 2024-06-13T07:02:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.767+0000 7fbba1467740 1 Processor -- start 2024-06-13T07:02:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbba1467740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72be8fb0 0x55db72c411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbba1467740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbba1467740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbba1467740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9819e370 con 0x55db72c217b0 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbba1467740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9819f7e0 con 0x55db72c20dc0 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbba1467740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9819ea40 con 0x55db72be8fb0 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbba1467740 10 client.0 _ll_register_callbacks cb 0x7fbb9000d270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb96a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb96a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50294/0 (socket says 172.21.15.26:50294) 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb96a00640 1 -- 172.21.15.26:0/4257078192 learned_addr learned my addr 172.21.15.26:0/4257078192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb97400640 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72be8fb0 0x55db72c411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb97e00640 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb96a00640 1 -- 172.21.15.26:0/4257078192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72be8fb0 msgr2=0x55db72c411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb96a00640 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72be8fb0 0x55db72c411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb96a00640 1 -- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 msgr2=0x55db72c426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb96a00640 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb96a00640 1 -- 172.21.15.26:0/4257078192 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db72c1d4f0 con 0x55db72c20dc0 2024-06-13T07:02:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.768+0000 7fbb97e00640 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.769+0000 7fbb96a00640 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c416e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbb840a4950 tx=0x7fbb84003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a6ab206bc066f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.769+0000 7fbb81600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:02:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.769+0000 7fbb81600640 1 -- 172.21.15.26:0/4257078192 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84086040 con 0x55db72c20dc0 2024-06-13T07:02:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.769+0000 7fbba1467740 1 -- 172.21.15.26:0/4257078192 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9001ad70 con 0x55db72c20dc0 2024-06-13T07:02:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.769+0000 7fbb81600640 1 -- 172.21.15.26:0/4257078192 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbb840a43c0 con 0x55db72c20dc0 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.769+0000 7fbb81600640 1 -- 172.21.15.26:0/4257078192 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb8406f040 con 0x55db72c20dc0 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.769+0000 7fbba1467740 10 client.25589 fetch_fsmap learned FSMap version 32 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.769+0000 7fbba1467740 1 -- 172.21.15.26:0/4257078192 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db72cbb3d0 con 0x55db72c20dc0 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.770+0000 7fbb81600640 1 -- 172.21.15.26:0/4257078192 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb9001ad70 con 0x55db72c20dc0 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.770+0000 7fbb81600640 1 -- 172.21.15.26:0/4257078192 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fbb840ba360 con 0x55db72c20dc0 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.770+0000 7fbba1467740 10 client.25589 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.770+0000 7fbba1467740 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.770+0000 7fbba1467740 20 client.25589 populate_metadata read hostname 'smithi026' 2024-06-13T07:02:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.770+0000 7fbba1467740 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db72c26920 0x55db72cbdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.770+0000 7fbba1467740 4 client.25589 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.770+0000 7fbba1467740 1 -- 172.21.15.26:0/4257078192 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db72b8bea0 con 0x55db72c26920 2024-06-13T07:02:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.771+0000 7fbb97400640 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db72c26920 0x55db72cbdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.771+0000 7fbb97400640 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db72c26920 0x55db72cbdab0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9d6144c448412d server_cookie=4dafa3025530bc82 in_seq=0 out_seq=0 2024-06-13T07:02:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.771+0000 7fbb81600640 10 client.25589 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.848+0000 7fbb81600640 1 -- 172.21.15.26:0/4257078192 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55db72b8bea0 con 0x55db72c26920 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.848+0000 7fbb81600640 10 client.25589 handle_command_reply: tid=0 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.848+0000 7fbba1467740 1 client.25589 shutdown 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 -- 172.21.15.26:0/4257078192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 msgr2=0x55db72c416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c416e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbb840a4950 tx=0x7fbb84003c00 comp rx=0 tx=0).stop 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 -- 172.21.15.26:0/4257078192 shutdown_connections 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db72c26920 0x55db72cbdab0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db72c20dc0 0x55db72c416e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db72c217b0 0x55db72c426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 --2- 172.21.15.26:0/4257078192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db72be8fb0 0x55db72c411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 -- 172.21.15.26:0/4257078192 >> 172.21.15.26:0/4257078192 conn(0x55db72c3bc60 msgr2=0x55db72c570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbb81600640 0 client.25589 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:02:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 -- 172.21.15.26:0/4257078192 shutdown_connections 2024-06-13T07:02:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 1 -- 172.21.15.26:0/4257078192 wait complete. 2024-06-13T07:02:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.849+0000 7fbba1467740 20 client.25589 trim_cache size 0 max 16384 2024-06-13T07:02:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:02:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:02:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 -- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbb7c0a2c40 msgr2=0x7fbb7c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbb7c0a2c40 0x7fbb7c0a5100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55db72c22010 tx=0x7fbb9000d040 comp rx=0 tx=0).stop 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 -- 172.21.15.26:0/3337125926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb98104bf0 msgr2=0x7fbb98195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb98104bf0 0x7fbb98195ec0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e9a0 tx=0x7fbb8400ee60 comp rx=0 tx=0).stop 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 -- 172.21.15.26:0/3337125926 shutdown_connections 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbb7c0a2c40 0x7fbb7c0a5100 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb98104bf0 0x7fbb98195ec0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb98107520 0x7fbb9819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 --2- 172.21.15.26:0/3337125926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb981022c0 0x7fbb98195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 -- 172.21.15.26:0/3337125926 >> 172.21.15.26:0/3337125926 conn(0x7fbb980fbc90 msgr2=0x7fbb98102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 -- 172.21.15.26:0/3337125926 shutdown_connections 2024-06-13T07:02:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:02:52.850+0000 7fbb9d000640 1 -- 172.21.15.26:0/3337125926 wait complete. 2024-06-13T07:02:53.331 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:02:53.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:02:53.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 51727 4.39 MB/sec warmup 67 sec latency 32.471 ms 2024-06-13T07:02:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:54.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:53 smithi136 ceph-mon[37229]: pgmap v581: 129 pgs: 129 active+clean; 37 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 988 KiB/s wr, 181 op/s 2024-06-13T07:02:54.229 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:53 smithi026 ceph-mon[32041]: pgmap v581: 129 pgs: 129 active+clean; 37 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 988 KiB/s wr, 181 op/s 2024-06-13T07:02:54.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:53 smithi135 ceph-mon[36276]: pgmap v581: 129 pgs: 129 active+clean; 37 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 988 KiB/s wr, 181 op/s 2024-06-13T07:02:54.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 53313 4.50 MB/sec warmup 68 sec latency 16.885 ms 2024-06-13T07:02:55.508 INFO:tasks.workunit.client.0.smithi026.stdout: 1 53805 4.46 MB/sec warmup 69 sec latency 18.848 ms 2024-06-13T07:02:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:55 smithi136 ceph-mon[37229]: pgmap v582: 129 pgs: 129 active+clean; 38 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1006 KiB/s wr, 155 op/s 2024-06-13T07:02:56.136 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:55 smithi026 ceph-mon[32041]: pgmap v582: 129 pgs: 129 active+clean; 38 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1006 KiB/s wr, 155 op/s 2024-06-13T07:02:56.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:55 smithi135 ceph-mon[36276]: pgmap v582: 129 pgs: 129 active+clean; 38 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1006 KiB/s wr, 155 op/s 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.486+0000 7fc1c5400700 1 -- 172.21.15.26:0/3110644429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1c010a680 msgr2=0x7fc1c010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.486+0000 7fc1c5400700 1 --2- 172.21.15.26:0/3110644429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1c010a680 0x7fc1c010aa60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009990 tx=0x7fc1b0009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.486+0000 7fc1c5400700 1 -- 172.21.15.26:0/3110644429 shutdown_connections 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.486+0000 7fc1c5400700 1 --2- 172.21.15.26:0/3110644429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1c010a680 0x7fc1c010aa60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.486+0000 7fc1c5400700 1 --2- 172.21.15.26:0/3110644429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c0105040 0x7fc1c0111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.486+0000 7fc1c5400700 1 --2- 172.21.15.26:0/3110644429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c0104680 0x7fc1c0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.486+0000 7fc1c5400700 1 -- 172.21.15.26:0/3110644429 >> 172.21.15.26:0/3110644429 conn(0x7fc1c0100170 msgr2=0x7fc1c0102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.487+0000 7fc1c5400700 1 -- 172.21.15.26:0/3110644429 shutdown_connections 2024-06-13T07:02:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.487+0000 7fc1c5400700 1 -- 172.21.15.26:0/3110644429 wait complete. 2024-06-13T07:02:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.488+0000 7fc1c5400700 1 Processor -- start 2024-06-13T07:02:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.488+0000 7fc1c5400700 1 -- start start 2024-06-13T07:02:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1c5400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c0104680 0x7fc1c019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1c5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c0105040 0x7fc1c019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1c5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1c010a680 0x7fc1c019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1c5400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01149d0 con 0x7fc1c0104680 2024-06-13T07:02:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1c5400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c0114850 con 0x7fc1c010a680 2024-06-13T07:02:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1c5400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c0114b50 con 0x7fc1c0105040 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1be800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c0104680 0x7fc1c019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1bf200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1c010a680 0x7fc1c019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1be800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c0104680 0x7fc1c019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54646/0 (socket says 172.21.15.26:54646) 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1be800700 1 -- 172.21.15.26:0/972116331 learned_addr learned my addr 172.21.15.26:0/972116331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.489+0000 7fc1bde00700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c0105040 0x7fc1c019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.490+0000 7fc1be800700 1 -- 172.21.15.26:0/972116331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c0105040 msgr2=0x7fc1c019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.490+0000 7fc1be800700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c0105040 0x7fc1c019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.490+0000 7fc1be800700 1 -- 172.21.15.26:0/972116331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1c010a680 msgr2=0x7fc1c019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.490+0000 7fc1be800700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1c010a680 0x7fc1c019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.490+0000 7fc1be800700 1 -- 172.21.15.26:0/972116331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c019fdd0 con 0x7fc1c0104680 2024-06-13T07:02:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.490+0000 7fc1be800700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c0104680 0x7fc1c019adc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000bae0 tx=0x7fc1b000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:56.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.491+0000 7fc1b7400700 1 -- 172.21.15.26:0/972116331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0026070 con 0x7fc1c0104680 2024-06-13T07:02:56.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.491+0000 7fc1b7400700 1 -- 172.21.15.26:0/972116331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b0003d70 con 0x7fc1c0104680 2024-06-13T07:02:56.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.491+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c01a0060 con 0x7fc1c0104680 2024-06-13T07:02:56.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.491+0000 7fc1b7400700 1 -- 172.21.15.26:0/972116331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0021410 con 0x7fc1c0104680 2024-06-13T07:02:56.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.491+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c01a7940 con 0x7fc1c0104680 2024-06-13T07:02:56.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.492+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c010cc10 con 0x7fc1c0104680 2024-06-13T07:02:56.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.495+0000 7fc1b7400700 1 -- 172.21.15.26:0/972116331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc1b0029070 con 0x7fc1c0104680 2024-06-13T07:02:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.496+0000 7fc1b7400700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc1a40a2c20 0x7fc1a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.497+0000 7fc1b7400700 1 -- 172.21.15.26:0/972116331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc1b00d01d0 con 0x7fc1c0104680 2024-06-13T07:02:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.497+0000 7fc1bde00700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc1a40a2c20 0x7fc1a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.498+0000 7fc1bde00700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc1a40a2c20 0x7fc1a40a50e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc1c01026e0 tx=0x7fc1ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.503+0000 7fc1b7400700 1 -- 172.21.15.26:0/972116331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc1b005d850 con 0x7fc1c0104680 2024-06-13T07:02:56.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 54819 4.47 MB/sec warmup 70 sec latency 41.174 ms 2024-06-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.840+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1c0069180 con 0x7fc1c0104680 2024-06-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.841+0000 7fc1b7400700 1 -- 172.21.15.26:0/972116331 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11875 (secure 0 0 0) 0x7fc1b0013070 con 0x7fc1c0104680 2024-06-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T07:02:20.565914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":117,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":119,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24511,24517,24506]},"id":1}]} 2024-06-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.844+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc1a40a2c20 msgr2=0x7fc1a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.844+0000 7fc1c5400700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc1a40a2c20 0x7fc1a40a50e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc1c01026e0 tx=0x7fc1ac009260 comp rx=0 tx=0).stop 2024-06-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.844+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c0104680 msgr2=0x7fc1c019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.844+0000 7fc1c5400700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c0104680 0x7fc1c019adc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000bae0 tx=0x7fc1b000bb10 comp rx=0 tx=0).stop 2024-06-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.845+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 shutdown_connections 2024-06-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.845+0000 7fc1c5400700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc1a40a2c20 0x7fc1a40a50e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.845+0000 7fc1c5400700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1c010a680 0x7fc1c019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.845+0000 7fc1c5400700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c0104680 0x7fc1c019adc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.845+0000 7fc1c5400700 1 --2- 172.21.15.26:0/972116331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c0105040 0x7fc1c019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.845+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 >> 172.21.15.26:0/972116331 conn(0x7fc1c0100170 msgr2=0x7fc1c01007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.845+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 shutdown_connections 2024-06-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:02:56.845+0000 7fc1c5400700 1 -- 172.21.15.26:0/972116331 wait complete. 2024-06-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 32 2024-06-13T07:02:57.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:56 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/972116331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:02:57.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:56 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/972116331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:02:57.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:56 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/972116331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:02:57.508 INFO:tasks.workunit.client.0.smithi026.stdout: 1 56231 4.51 MB/sec warmup 71 sec latency 20.383 ms 2024-06-13T07:02:57.988 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:02:58.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:57 smithi136 ceph-mon[37229]: pgmap v583: 129 pgs: 129 active+clean; 39 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1003 KiB/s wr, 165 op/s 2024-06-13T07:02:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:57 smithi026 ceph-mon[32041]: pgmap v583: 129 pgs: 129 active+clean; 39 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1003 KiB/s wr, 165 op/s 2024-06-13T07:02:58.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:57 smithi135 ceph-mon[36276]: pgmap v583: 129 pgs: 129 active+clean; 39 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1003 KiB/s wr, 165 op/s 2024-06-13T07:02:58.488 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:58.507 INFO:tasks.workunit.client.0.smithi026.stdout: 1 57168 4.54 MB/sec warmup 72 sec latency 71.724 ms 2024-06-13T07:02:59.508 INFO:tasks.workunit.client.0.smithi026.stdout: 1 57958 4.51 MB/sec warmup 73 sec latency 43.029 ms 2024-06-13T07:03:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:02:59 smithi026 ceph-mon[32041]: pgmap v584: 129 pgs: 129 active+clean; 37 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-06-13T07:03:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:02:59 smithi135 ceph-mon[36276]: pgmap v584: 129 pgs: 129 active+clean; 37 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-06-13T07:03:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:02:59 smithi136 ceph-mon[37229]: pgmap v584: 129 pgs: 129 active+clean; 37 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-06-13T07:03:00.508 INFO:tasks.workunit.client.0.smithi026.stdout: 1 59230 4.51 MB/sec warmup 74 sec latency 44.940 ms 2024-06-13T07:03:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 -- 172.21.15.26:0/4274841648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 msgr2=0x7fc4b410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/4274841648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 0x7fc4b410e690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009990 tx=0x7fc49c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 -- 172.21.15.26:0/4274841648 shutdown_connections 2024-06-13T07:03:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/4274841648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b4074ab0 0x7fc4b4074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/4274841648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b410ed00 0x7fc4b4111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/4274841648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 0x7fc4b410e690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 -- 172.21.15.26:0/4274841648 >> 172.21.15.26:0/4274841648 conn(0x7fc4b40fde80 msgr2=0x7fc4b41002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 -- 172.21.15.26:0/4274841648 shutdown_connections 2024-06-13T07:03:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.325+0000 7fc4b3e00700 1 -- 172.21.15.26:0/4274841648 wait complete. 2024-06-13T07:03:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.326+0000 7fc4b3e00700 1 Processor -- start 2024-06-13T07:03:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.326+0000 7fc4b3e00700 1 -- start start 2024-06-13T07:03:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4b3e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4074ab0 0x7fc4b419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4b3e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 0x7fc4b419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4b3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b410ed00 0x7fc4b419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4b3e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4114a30 con 0x7fc4b4074ab0 2024-06-13T07:03:01.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4b3e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41148b0 con 0x7fc4b410ed00 2024-06-13T07:03:01.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4b3e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4114bb0 con 0x7fc4b4075460 2024-06-13T07:03:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4aa000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 0x7fc4b419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4aa000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 0x7fc4b419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37558/0 (socket says 172.21.15.26:37558) 2024-06-13T07:03:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4aa000700 1 -- 172.21.15.26:0/1080121685 learned_addr learned my addr 172.21.15.26:0/1080121685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4b3400700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b410ed00 0x7fc4b419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.327+0000 7fc4b2a00700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4074ab0 0x7fc4b419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4b3400700 1 -- 172.21.15.26:0/1080121685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 msgr2=0x7fc4b419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4b3400700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 0x7fc4b419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4b3400700 1 -- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4074ab0 msgr2=0x7fc4b419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4b3400700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4074ab0 0x7fc4b419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4b3400700 1 -- 172.21.15.26:0/1080121685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b419fe80 con 0x7fc4b410ed00 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4b3400700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b410ed00 0x7fc4b419f740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00e900 tx=0x7fc4ac00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4abe00700 1 -- 172.21.15.26:0/1080121685 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac004220 con 0x7fc4b410ed00 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4abe00700 1 -- 172.21.15.26:0/1080121685 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac0043c0 con 0x7fc4b410ed00 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4abe00700 1 -- 172.21.15.26:0/1080121685 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac010430 con 0x7fc4b410ed00 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.328+0000 7fc4b3e00700 1 -- 172.21.15.26:0/1080121685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b41a0170 con 0x7fc4b410ed00 2024-06-13T07:03:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.329+0000 7fc4b3e00700 1 -- 172.21.15.26:0/1080121685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b41a79f0 con 0x7fc4b410ed00 2024-06-13T07:03:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.332+0000 7fc4abe00700 1 -- 172.21.15.26:0/1080121685 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc4ac00c780 con 0x7fc4b410ed00 2024-06-13T07:03:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.334+0000 7fc4c41fab80 1 Processor -- start 2024-06-13T07:03:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.334+0000 7fc4c41fab80 1 -- start start 2024-06-13T07:03:01.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4abe00700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc4980a2ca0 0x7fc4980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce91140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce9bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281ce9e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56281cda0a10 con 0x56281ce9c530 2024-06-13T07:03:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56281cde3b00 con 0x56281ce91680 2024-06-13T07:03:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56281cd9e8f0 con 0x56281ce90d60 2024-06-13T07:03:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4abe00700 1 -- 172.21.15.26:0/1080121685 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc4ac0cf560 con 0x7fc4b410ed00 2024-06-13T07:03:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4b2a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce91140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4b2a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce91140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37572/0 (socket says 172.21.15.26:37572) 2024-06-13T07:03:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4b2a00700 1 -- 172.21.15.26:0/1972580393 learned_addr learned my addr 172.21.15.26:0/1972580393 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4aa000700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc4980a2ca0 0x7fc4980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.335+0000 7fc4aa000700 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce9bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.336+0000 7fc4b3400700 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281ce9e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.336+0000 7fc4aa000700 1 -- 172.21.15.26:0/1972580393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 msgr2=0x56281ce91140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.336+0000 7fc4aa000700 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce91140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.336+0000 7fc4aa000700 1 -- 172.21.15.26:0/1972580393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 msgr2=0x56281ce9e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.336+0000 7fc4aa000700 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281ce9e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.336+0000 7fc4aa000700 1 -- 172.21.15.26:0/1972580393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281cd8bf90 con 0x56281ce91680 2024-06-13T07:03:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.337+0000 7fc4aa000700 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce9bff0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc49c006010 tx=0x7fc49c00bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b168acafd8bb5c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.337+0000 7fc4aa000700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc4980a2ca0 0x7fc4980a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0098e0 tx=0x7fc49c025000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.337+0000 7fc4a3e00700 1 -- 172.21.15.26:0/1972580393 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c003960 con 0x56281ce91680 2024-06-13T07:03:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.337+0000 7fc4a3e00700 1 -- 172.21.15.26:0/1972580393 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49c028460 con 0x56281ce91680 2024-06-13T07:03:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.337+0000 7fc4a3e00700 1 -- 172.21.15.26:0/1972580393 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00f680 con 0x56281ce91680 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 -- 172.21.15.26:0/1972580393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 msgr2=0x56281ce9bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce9bff0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc49c006010 tx=0x7fc49c00bd60 comp rx=0 tx=0).stop 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 -- 172.21.15.26:0/1972580393 shutdown_connections 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281ce9e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce91140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/1972580393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce9bff0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 -- 172.21.15.26:0/1972580393 >> 172.21.15.26:0/1972580393 conn(0x56281cdbe170 msgr2=0x56281ce8d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 -- 172.21.15.26:0/1972580393 shutdown_connections 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.338+0000 7fc4c41fab80 1 -- 172.21.15.26:0/1972580393 wait complete. 2024-06-13T07:03:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.339+0000 7fc4c41fab80 1 -- start start 2024-06-13T07:03:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.339+0000 7fc4c41fab80 1 Processor -- start 2024-06-13T07:03:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281ce995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56281cd7ff60 con 0x56281ce9c530 2024-06-13T07:03:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56281cddfac0 con 0x56281ce91680 2024-06-13T07:03:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56281ce9ef60 con 0x56281ce90d60 2024-06-13T07:03:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4c41fab80 10 client.0 _ll_register_callbacks cb 0x56281cd79380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4b2a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4b2a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce9bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37582/0 (socket says 172.21.15.26:37582) 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4b2a00700 1 -- 172.21.15.26:0/2451469193 learned_addr learned my addr 172.21.15.26:0/2451469193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.340+0000 7fc4aa000700 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.341+0000 7fc4b2a00700 1 -- 172.21.15.26:0/2451469193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 msgr2=0x56281ce990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.341+0000 7fc4b2a00700 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.341+0000 7fc4b2a00700 1 -- 172.21.15.26:0/2451469193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 msgr2=0x56281ce995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.341+0000 7fc4b2a00700 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281ce995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.341+0000 7fc4b2a00700 1 -- 172.21.15.26:0/2451469193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56281cd328d0 con 0x56281ce90d60 2024-06-13T07:03:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.341+0000 7fc4b2a00700 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce9bdb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4004a30 tx=0x7fc4a4014650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8133ecb2eb24132b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.341+0000 7fc4a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.341+0000 7fc4a0c00700 1 -- 172.21.15.26:0/2451469193 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4020630 con 0x56281ce90d60 2024-06-13T07:03:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.342+0000 7fc4a0c00700 1 -- 172.21.15.26:0/2451469193 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc4a40207d0 con 0x56281ce90d60 2024-06-13T07:03:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.342+0000 7fc4c41fab80 1 -- 172.21.15.26:0/2451469193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49c028460 con 0x56281ce90d60 2024-06-13T07:03:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.342+0000 7fc4a0c00700 1 -- 172.21.15.26:0/2451469193 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4a4029050 con 0x56281ce90d60 2024-06-13T07:03:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.342+0000 7fc4c41fab80 10 client.25654 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.342+0000 7fc4c41fab80 1 -- 172.21.15.26:0/2451469193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56281cea97c0 con 0x56281ce90d60 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.342+0000 7fc4a0c00700 1 -- 172.21.15.26:0/2451469193 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49c028460 con 0x56281ce90d60 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.343+0000 7fc4a0c00700 1 -- 172.21.15.26:0/2451469193 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fc4a4016b70 con 0x56281ce90d60 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.343+0000 7fc4c41fab80 10 client.25654 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.343+0000 7fc4c41fab80 10 client.25654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.343+0000 7fc4c41fab80 20 client.25654 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.343+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x56281ceabe00 0x56281ceae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.343+0000 7fc4c41fab80 4 client.25654 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.343+0000 7fc4c41fab80 1 -- 172.21.15.26:0/2451469193 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56281cdf1000 con 0x56281ceabe00 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.343+0000 7fc4aa000700 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x56281ceabe00 0x56281ceae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.344+0000 7fc4aa000700 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x56281ceabe00 0x56281ceae250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae1319fc17d43697 server_cookie=151f94c0c4401a1c in_seq=0 out_seq=0 2024-06-13T07:03:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.344+0000 7fc4a0c00700 10 client.25654 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T07:03:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.345+0000 7fc4a0c00700 1 -- 172.21.15.26:0/2451469193 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56281cdf1000 con 0x56281ceabe00 2024-06-13T07:03:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.345+0000 7fc4a0c00700 10 client.25654 handle_command_reply: tid=0 2024-06-13T07:03:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.345+0000 7fc4c41fab80 1 client.25654 shutdown 2024-06-13T07:03:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.345+0000 7fc4c41fab80 1 -- 172.21.15.26:0/2451469193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 msgr2=0x56281ce9bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.345+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce9bdb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4004a30 tx=0x7fc4a4014650 comp rx=0 tx=0).stop 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 1 -- 172.21.15.26:0/2451469193 shutdown_connections 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x56281ceabe00 0x56281ceae250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281ce995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce90d60 0x56281ce9bdb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/2451469193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 1 -- 172.21.15.26:0/2451469193 >> 172.21.15.26:0/2451469193 conn(0x56281cdbe170 msgr2=0x56281ce8ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4a0c00700 0 client.25654 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 1 -- 172.21.15.26:0/2451469193 shutdown_connections 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 1 -- 172.21.15.26:0/2451469193 wait complete. 2024-06-13T07:03:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.346+0000 7fc4c41fab80 20 client.25654 trim_cache size 0 max 16384 2024-06-13T07:03:01.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.365+0000 7fc4c41fab80 1 Processor -- start 2024-06-13T07:03:01.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.365+0000 7fc4c41fab80 1 -- start start 2024-06-13T07:03:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.365+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce90d60 0x56281cea8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.365+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce979a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.365+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce9c530 0x56281ce97ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.365+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b2d50 con 0x56281ce90d60 2024-06-13T07:03:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.365+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b05b0 con 0x56281ce91680 2024-06-13T07:03:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.365+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41ae400 con 0x56281ce9c530 2024-06-13T07:03:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4b2a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce90d60 0x56281cea8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4aa000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4b2a00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce90d60 0x56281cea8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54690/0 (socket says 172.21.15.26:54690) 2024-06-13T07:03:01.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4aa000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50364/0 (socket says 172.21.15.26:50364) 2024-06-13T07:03:01.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4b2a00700 1 -- 172.21.15.26:0/346060832 learned_addr learned my addr 172.21.15.26:0/346060832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:01.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4b3400700 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce9c530 0x56281ce97ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4aa000700 1 -- 172.21.15.26:0/346060832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce9c530 msgr2=0x56281ce97ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:01.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4aa000700 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce9c530 0x56281ce97ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4aa000700 1 -- 172.21.15.26:0/346060832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce90d60 msgr2=0x56281cea8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4aa000700 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce90d60 0x56281cea8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4aa000700 1 -- 172.21.15.26:0/346060832 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281ce91390 con 0x56281ce91680 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4b3400700 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce9c530 0x56281ce97ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4b2a00700 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce90d60 0x56281cea8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.366+0000 7fc4aa000700 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce979a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc49c028c70 tx=0x7fc49c028e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=852dd01970bd8806 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.367+0000 7fc4a3400700 1 -- 172.21.15.26:0/346060832 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c042070 con 0x56281ce91680 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.367+0000 7fc4a3400700 1 -- 172.21.15.26:0/346060832 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49c00be30 con 0x56281ce91680 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.367+0000 7fc4a3400700 1 -- 172.21.15.26:0/346060832 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c03d2e0 con 0x56281ce91680 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 -- 172.21.15.26:0/346060832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 msgr2=0x56281ce979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce979a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc49c028c70 tx=0x7fc49c028e70 comp rx=0 tx=0).stop 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 -- 172.21.15.26:0/346060832 shutdown_connections 2024-06-13T07:03:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce9c530 0x56281ce97ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce90d60 0x56281cea8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/346060832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce91680 0x56281ce979a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 -- 172.21.15.26:0/346060832 >> 172.21.15.26:0/346060832 conn(0x56281cdbe170 msgr2=0x56281ce8cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 -- 172.21.15.26:0/346060832 shutdown_connections 2024-06-13T07:03:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.368+0000 7fc4c41fab80 1 -- 172.21.15.26:0/346060832 wait complete. 2024-06-13T07:03:01.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 1 -- start start 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 1 Processor -- start 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce90d60 0x56281cea1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce91680 0x56281cea1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281cf0eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41aea70 con 0x56281ce9c530 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41afee0 con 0x56281ce90d60 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41af140 con 0x56281ce91680 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.369+0000 7fc4c41fab80 10 client.0 _ll_register_callbacks cb 0x56281cd79380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce90d60 0x56281cea1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce90d60 0x56281cea1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50378/0 (socket says 172.21.15.26:50378) 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b2a00700 1 -- 172.21.15.26:0/323803440 learned_addr learned my addr 172.21.15.26:0/323803440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4aa000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce91680 0x56281cea1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b3400700 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281cf0eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b2a00700 1 -- 172.21.15.26:0/323803440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce91680 msgr2=0x56281cea1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b2a00700 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce91680 0x56281cea1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b2a00700 1 -- 172.21.15.26:0/323803440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 msgr2=0x56281cf0eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b2a00700 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281cf0eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.370+0000 7fc4b2a00700 1 -- 172.21.15.26:0/323803440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56281cf0f130 con 0x56281ce90d60 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.371+0000 7fc4aa000700 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce91680 0x56281cea1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.371+0000 7fc4b2a00700 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce90d60 0x56281cea1900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009550 tx=0x7fc4a4009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f505ad413b5066e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.371+0000 7fc4a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:03:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.371+0000 7fc4a0c00700 1 -- 172.21.15.26:0/323803440 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4033070 con 0x56281ce90d60 2024-06-13T07:03:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.371+0000 7fc4c41fab80 1 -- 172.21.15.26:0/323803440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49c00be30 con 0x56281ce90d60 2024-06-13T07:03:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.371+0000 7fc4a0c00700 1 -- 172.21.15.26:0/323803440 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc4a4003a00 con 0x56281ce90d60 2024-06-13T07:03:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4a0c00700 1 -- 172.21.15.26:0/323803440 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4a4003c20 con 0x56281ce90d60 2024-06-13T07:03:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4c41fab80 10 client.25613 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4c41fab80 1 -- 172.21.15.26:0/323803440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56281cf164c0 con 0x56281ce90d60 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4a0c00700 1 -- 172.21.15.26:0/323803440 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49c00be30 con 0x56281ce90d60 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4a0c00700 1 -- 172.21.15.26:0/323803440 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fc4a402b400 con 0x56281ce90d60 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4c41fab80 10 client.25613 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4c41fab80 10 client.25613 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4c41fab80 20 client.25613 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x56281cf16bc0 0x56281cf18fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4c41fab80 4 client.25613 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.372+0000 7fc4c41fab80 1 -- 172.21.15.26:0/323803440 --> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56281cdeb580 con 0x56281cf16bc0 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.373+0000 7fc4aa000700 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x56281cf16bc0 0x56281cf18fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4aa000700 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x56281cf16bc0 0x56281cf18fe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8da6feeb66f797c server_cookie=48eebe094042db02 in_seq=0 out_seq=0 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4a0c00700 10 client.25613 ms_handle_connect on v2:172.21.15.135:6838/1905980530 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4a0c00700 1 -- 172.21.15.26:0/323803440 <== mds.2 v2:172.21.15.135:6838/1905980530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6043 (crc 0 0 0) 0x56281cdeb580 con 0x56281cf16bc0 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4a0c00700 10 client.25613 handle_command_reply: tid=0 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 client.25613 shutdown 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 -- 172.21.15.26:0/323803440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce90d60 msgr2=0x56281cea1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce90d60 0x56281cea1900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009550 tx=0x7fc4a4009bd0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 -- 172.21.15.26:0/323803440 shutdown_connections 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.135:6838/1905980530,v1:172.21.15.135:6839/1905980530] conn(0x56281cf16bc0 0x56281cf18fe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56281ce90d60 0x56281cea1900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281ce9c530 0x56281cf0eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 --2- 172.21.15.26:0/323803440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56281ce91680 0x56281cea1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4c41fab80 1 -- 172.21.15.26:0/323803440 >> 172.21.15.26:0/323803440 conn(0x56281cdbe170 msgr2=0x56281ce98c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:01.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.374+0000 7fc4a0c00700 0 client.25613 ms_handle_reset on v2:172.21.15.135:6838/1905980530 2024-06-13T07:03:01.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.375+0000 7fc4c41fab80 1 -- 172.21.15.26:0/323803440 shutdown_connections 2024-06-13T07:03:01.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.375+0000 7fc4c41fab80 1 -- 172.21.15.26:0/323803440 wait complete. 2024-06-13T07:03:01.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.375+0000 7fc4c41fab80 20 client.25613 trim_cache size 0 max 16384 2024-06-13T07:03:01.377 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 38801065, 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628339, 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1" 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.378 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 449605864, 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628332, 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04" 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 555714115, 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628838, 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/CHANGED.DB" 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1081715645, 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628843, 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.MB" 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1366808144, 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628331, 2024-06-13T07:03:01.380 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX" 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1414278848, 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628347, 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV" 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1469477869, 2024-06-13T07:03:01.381 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628345, 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03" 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1592486186, 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628336, 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0" 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.382 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1594152973, 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628340, 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL" 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1879428922, 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628338, 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1" 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2123497955, 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628335, 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX" 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2494968839, 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628330, 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB" 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.384 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2674821089, 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628343, 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX" 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2741091717, 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628341, 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB" 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.385 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2782723332, 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628337, 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0" 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2939340827, 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628844, 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.DB" 2024-06-13T07:03:01.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3098618049, 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628342, 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB" 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3132777122, 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628302, 2024-06-13T07:03:01.387 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS" 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3293433281, 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628346, 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL" 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3313458462, 2024-06-13T07:03:01.388 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628301, 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS" 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3587143827, 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628348, 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL" 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3621784038, 2024-06-13T07:03:01.389 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628836, 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.VAL" 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3766301338, 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628334, 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB" 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3932598800, 2024-06-13T07:03:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628333, 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04" 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3952965238, 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628344, 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03" 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:01.391 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4153425059, 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628349, 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL" 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 -- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc4980a2ca0 msgr2=0x7fc4980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc4980a2ca0 0x7fc4980a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0098e0 tx=0x7fc49c025000 comp rx=0 tx=0).stop 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 -- 172.21.15.26:0/1080121685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b410ed00 msgr2=0x7fc4b419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b410ed00 0x7fc4b419f740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00e900 tx=0x7fc4ac00ec10 comp rx=0 tx=0).stop 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 -- 172.21.15.26:0/1080121685 shutdown_connections 2024-06-13T07:03:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc4980a2ca0 0x7fc4980a5160 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b410ed00 0x7fc4b419f740 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4074ab0 0x7fc4b419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 --2- 172.21.15.26:0/1080121685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4b4075460 0x7fc4b419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:01.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 -- 172.21.15.26:0/1080121685 >> 172.21.15.26:0/1080121685 conn(0x7fc4b40fde80 msgr2=0x7fc4b40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:01.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 -- 172.21.15.26:0/1080121685 shutdown_connections 2024-06-13T07:03:01.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:01.376+0000 7fc4b3e00700 1 -- 172.21.15.26:0/1080121685 wait complete. 2024-06-13T07:03:01.508 INFO:tasks.workunit.client.0.smithi026.stdout: 1 60564 4.61 MB/sec warmup 75 sec latency 19.238 ms 2024-06-13T07:03:02.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:01 smithi026 ceph-mon[32041]: pgmap v585: 129 pgs: 129 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-06-13T07:03:02.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:01 smithi135 ceph-mon[36276]: pgmap v585: 129 pgs: 129 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-06-13T07:03:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:01 smithi136 ceph-mon[37229]: pgmap v585: 129 pgs: 129 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-06-13T07:03:02.508 INFO:tasks.workunit.client.0.smithi026.stdout: 1 61017 4.56 MB/sec warmup 76 sec latency 28.757 ms 2024-06-13T07:03:03.509 INFO:tasks.workunit.client.0.smithi026.stdout: 1 61483 4.52 MB/sec warmup 77 sec latency 343.322 ms 2024-06-13T07:03:04.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:03 smithi026 ceph-mon[32041]: pgmap v586: 129 pgs: 129 active+clean; 38 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 176 op/s 2024-06-13T07:03:04.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:03:04.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:03 smithi135 ceph-mon[36276]: pgmap v586: 129 pgs: 129 active+clean; 38 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 176 op/s 2024-06-13T07:03:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:03 smithi136 ceph-mon[37229]: pgmap v586: 129 pgs: 129 active+clean; 38 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 176 op/s 2024-06-13T07:03:04.508 INFO:tasks.workunit.client.0.smithi026.stdout: 1 62785 4.53 MB/sec warmup 78 sec latency 189.020 ms 2024-06-13T07:03:04.649 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:05.509 INFO:tasks.workunit.client.0.smithi026.stdout: 1 64130 4.62 MB/sec warmup 79 sec latency 21.880 ms 2024-06-13T07:03:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 -- 172.21.15.26:0/1476292842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 msgr2=0x7fbacc101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 --2- 172.21.15.26:0/1476292842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 0x7fbacc101f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009900 tx=0x7fbab802a410 comp rx=0 tx=0).stop 2024-06-13T07:03:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 -- 172.21.15.26:0/1476292842 shutdown_connections 2024-06-13T07:03:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 --2- 172.21.15.26:0/1476292842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 0x7fbacc101f70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 --2- 172.21.15.26:0/1476292842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc1024b0 0x7fbacc108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 --2- 172.21.15.26:0/1476292842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbacc100900 0x7fbacc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 -- 172.21.15.26:0/1476292842 >> 172.21.15.26:0/1476292842 conn(0x7fbacc0fbe70 msgr2=0x7fbacc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 -- 172.21.15.26:0/1476292842 shutdown_connections 2024-06-13T07:03:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.003+0000 7fbad2c00640 1 -- 172.21.15.26:0/1476292842 wait complete. 2024-06-13T07:03:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.004+0000 7fbad2c00640 1 Processor -- start 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbad2c00640 1 -- start start 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbad2c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc100900 0x7fbacc197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbad2c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 0x7fbacc198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbad2c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbacc1024b0 0x7fbacc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbad2c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc10c600 con 0x7fbacc1024b0 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbad2c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc10c480 con 0x7fbacc101af0 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbad2c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc10c780 con 0x7fbacc100900 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc100900 0x7fbacc197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc100900 0x7fbacc197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58792/0 (socket says 172.21.15.26:58792) 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbacbe00640 1 -- 172.21.15.26:0/2541032392 learned_addr learned my addr 172.21.15.26:0/2541032392 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.005+0000 7fbacb400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 0x7fbacc198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.006+0000 7fbad0c00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbacc1024b0 0x7fbacc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.006+0000 7fbacbe00640 1 -- 172.21.15.26:0/2541032392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 msgr2=0x7fbacc198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.006+0000 7fbacbe00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 0x7fbacc198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.006+0000 7fbacbe00640 1 -- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbacc1024b0 msgr2=0x7fbacc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.006+0000 7fbacbe00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbacc1024b0 0x7fbacc19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.006+0000 7fbacbe00640 1 -- 172.21.15.26:0/2541032392 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc19f950 con 0x7fbacc100900 2024-06-13T07:03:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.006+0000 7fbacb400640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 0x7fbacc198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.006+0000 7fbacbe00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc100900 0x7fbacc197d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e9c0 tx=0x7fbabc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.007+0000 7fbac8c00640 1 -- 172.21.15.26:0/2541032392 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00cd80 con 0x7fbacc100900 2024-06-13T07:03:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.007+0000 7fbac8c00640 1 -- 172.21.15.26:0/2541032392 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbabc0035f0 con 0x7fbacc100900 2024-06-13T07:03:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.007+0000 7fbac8c00640 1 -- 172.21.15.26:0/2541032392 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc010690 con 0x7fbacc100900 2024-06-13T07:03:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.007+0000 7fbad2c00640 1 -- 172.21.15.26:0/2541032392 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc19fc40 con 0x7fbacc100900 2024-06-13T07:03:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.007+0000 7fbad2c00640 1 -- 172.21.15.26:0/2541032392 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc1a0190 con 0x7fbacc100900 2024-06-13T07:03:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.010+0000 7fbac8c00640 1 -- 172.21.15.26:0/2541032392 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbabc0037a0 con 0x7fbacc100900 2024-06-13T07:03:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.010+0000 7fbad5978740 1 Processor -- start 2024-06-13T07:03:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.010+0000 7fbad5978740 1 -- start start 2024-06-13T07:03:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad5978740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e232e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad5978740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad5978740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e2335bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad5978740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5603e2291420 con 0x5603e232dc00 2024-06-13T07:03:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad5978740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603e2238c50 con 0x5603e2333760 2024-06-13T07:03:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad5978740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5603e228d3e0 con 0x5603e232e5f0 2024-06-13T07:03:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e232e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e232e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54718/0 (socket says 172.21.15.26:54718) 2024-06-13T07:03:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbad0c00640 1 -- 172.21.15.26:0/1304448844 learned_addr learned my addr 172.21.15.26:0/1304448844 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.011+0000 7fbacb400640 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e2335bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbad0c00640 1 -- 172.21.15.26:0/1304448844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 msgr2=0x5603e23330f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbad0c00640 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23330f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbad0c00640 1 -- 172.21.15.26:0/1304448844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 msgr2=0x5603e2335bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbad0c00640 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e2335bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbad0c00640 1 -- 172.21.15.26:0/1304448844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e1fa5210 con 0x5603e232dc00 2024-06-13T07:03:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbacb400640 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e2335bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbacbe00640 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbac8c00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbab00a2cb0 0x7fbab00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbad0c00640 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e232e020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009f60 tx=0x7fbac4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50844070f370c80 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbac8c00640 1 -- 172.21.15.26:0/2541032392 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbabc0cf940 con 0x7fbacc100900 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbacbe00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbab00a2cb0 0x7fbab00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbac1200640 1 -- 172.21.15.26:0/1304448844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac401a070 con 0x5603e232dc00 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbac1200640 1 -- 172.21.15.26:0/1304448844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac40158f0 con 0x5603e232dc00 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.012+0000 7fbac1200640 1 -- 172.21.15.26:0/1304448844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac400a880 con 0x5603e232dc00 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbad5978740 1 -- 172.21.15.26:0/1304448844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 msgr2=0x5603e232e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbacbe00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbab00a2cb0 0x7fbab00a5170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbabc09bce0 tx=0x7fbabc0d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbad5978740 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e232e020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009f60 tx=0x7fbac4009f90 comp rx=0 tx=0).stop 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbad5978740 1 -- 172.21.15.26:0/1304448844 shutdown_connections 2024-06-13T07:03:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbad5978740 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e2335bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbad5978740 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbad5978740 1 --2- 172.21.15.26:0/1304448844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e232e020 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbad5978740 1 -- 172.21.15.26:0/1304448844 >> 172.21.15.26:0/1304448844 conn(0x5603e22f5fb0 msgr2=0x5603e2329680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.013+0000 7fbad5978740 1 -- 172.21.15.26:0/1304448844 shutdown_connections 2024-06-13T07:03:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.014+0000 7fbad5978740 1 -- 172.21.15.26:0/1304448844 wait complete. 2024-06-13T07:03:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.014+0000 7fbad5978740 1 -- start start 2024-06-13T07:03:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.014+0000 7fbad5978740 1 Processor -- start 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad5978740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e2337f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad5978740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad5978740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e233f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad5978740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5603e2336710 con 0x5603e232dc00 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad5978740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603e224c0b0 con 0x5603e2333760 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad5978740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5603e2336890 con 0x5603e232e5f0 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad5978740 10 client.0 _ll_register_callbacks cb 0x5603e2237f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbacb400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbacb400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23384a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58808/0 (socket says 172.21.15.26:58808) 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbacb400640 1 -- 172.21.15.26:0/1646635202 learned_addr learned my addr 172.21.15.26:0/1646635202 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad0c00640 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e2337f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad0c00640 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e233f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbacb400640 1 -- 172.21.15.26:0/1646635202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 msgr2=0x5603e233f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbacb400640 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e233f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbacb400640 1 -- 172.21.15.26:0/1646635202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 msgr2=0x5603e2337f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbacb400640 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e2337f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbacb400640 1 -- 172.21.15.26:0/1646635202 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e2071620 con 0x5603e232e5f0 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.015+0000 7fbad0c00640 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e233f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.016+0000 7fbacb400640 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23384a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbab802a930 tx=0x7fbab8031950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56eef0e4e65830f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.016+0000 7fbab5600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.016+0000 7fbab5600640 1 -- 172.21.15.26:0/1646635202 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8035040 con 0x5603e232e5f0 2024-06-13T07:03:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.016+0000 7fbad5978740 1 -- 172.21.15.26:0/1646635202 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac40158f0 con 0x5603e232e5f0 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.016+0000 7fbab5600640 1 -- 172.21.15.26:0/1646635202 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbab8004240 con 0x5603e232e5f0 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.016+0000 7fbab5600640 1 -- 172.21.15.26:0/1646635202 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbab804e050 con 0x5603e232e5f0 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.016+0000 7fbad5978740 10 client.25669 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.016+0000 7fbad5978740 1 -- 172.21.15.26:0/1646635202 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603e2342910 con 0x5603e232e5f0 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbab5600640 1 -- 172.21.15.26:0/1646635202 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac40158f0 con 0x5603e232e5f0 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbab5600640 1 -- 172.21.15.26:0/1646635202 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fbab8004bb0 con 0x5603e232e5f0 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbad5978740 10 client.25669 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbad5978740 10 client.25669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbad5978740 20 client.25669 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbad5978740 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5603e2343fc0 0x5603e2346420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbad5978740 4 client.25669 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbad5978740 1 -- 172.21.15.26:0/1646635202 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603e229eb50 con 0x5603e2343fc0 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.017+0000 7fbad0c00640 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5603e2343fc0 0x5603e2346420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.018+0000 7fbad0c00640 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5603e2343fc0 0x5603e2346420 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e459bcd46d94cda5 server_cookie=e52e916913e3459 in_seq=0 out_seq=0 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.018+0000 7fbab5600640 10 client.25669 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbab5600640 1 -- 172.21.15.26:0/1646635202 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5603e229eb50 con 0x5603e2343fc0 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbab5600640 10 client.25669 handle_command_reply: tid=0 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 client.25669 shutdown 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 -- 172.21.15.26:0/1646635202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 msgr2=0x5603e23384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23384a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbab802a930 tx=0x7fbab8031950 comp rx=0 tx=0).stop 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 -- 172.21.15.26:0/1646635202 shutdown_connections 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5603e2343fc0 0x5603e2346420 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e233f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e23384a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbab5600640 0 client.25669 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 --2- 172.21.15.26:0/1646635202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e2337f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 -- 172.21.15.26:0/1646635202 >> 172.21.15.26:0/1646635202 conn(0x5603e22f5fb0 msgr2=0x5603e2328840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 -- 172.21.15.26:0/1646635202 shutdown_connections 2024-06-13T07:03:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 1 -- 172.21.15.26:0/1646635202 wait complete. 2024-06-13T07:03:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.019+0000 7fbad5978740 20 client.25669 trim_cache size 0 max 16384 2024-06-13T07:03:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.038+0000 7fbad5978740 1 Processor -- start 2024-06-13T07:03:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbad5978740 1 -- start start 2024-06-13T07:03:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbad5978740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e23286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbad5978740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e234ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbad5978740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e234b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbad5978740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc1ad0c0 con 0x5603e232dc00 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbad5978740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc1aa960 con 0x5603e2333760 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbad5978740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc1a87b0 con 0x5603e232e5f0 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbacb400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e234ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.039+0000 7fbad0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e23286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbacb400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e234ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58810/0 (socket says 172.21.15.26:58810) 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbad0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e23286c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54724/0 (socket says 172.21.15.26:54724) 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbacb400640 1 -- 172.21.15.26:0/4256782879 learned_addr learned my addr 172.21.15.26:0/4256782879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbad0c00640 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e234b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbad0c00640 1 -- 172.21.15.26:0/4256782879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 msgr2=0x5603e234ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbad0c00640 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e234ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbad0c00640 1 -- 172.21.15.26:0/4256782879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 msgr2=0x5603e234b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbad0c00640 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e234b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbad0c00640 1 -- 172.21.15.26:0/4256782879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e232ee80 con 0x5603e232dc00 2024-06-13T07:03:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbacb400640 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e234ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.040+0000 7fbad0c00640 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e23286c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbac400c320 tx=0x7fbac4004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a059cbca4d4c473 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.041+0000 7fbab7e00640 1 -- 172.21.15.26:0/4256782879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac401a040 con 0x5603e232dc00 2024-06-13T07:03:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.041+0000 7fbab7e00640 1 -- 172.21.15.26:0/4256782879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4015670 con 0x5603e232dc00 2024-06-13T07:03:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.041+0000 7fbab7e00640 1 -- 172.21.15.26:0/4256782879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4004380 con 0x5603e232dc00 2024-06-13T07:03:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 -- 172.21.15.26:0/4256782879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 msgr2=0x5603e23286c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e23286c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbac400c320 tx=0x7fbac4004a60 comp rx=0 tx=0).stop 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 -- 172.21.15.26:0/4256782879 shutdown_connections 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e2333760 0x5603e234b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e232e5f0 0x5603e234ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 --2- 172.21.15.26:0/4256782879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232dc00 0x5603e23286c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 -- 172.21.15.26:0/4256782879 >> 172.21.15.26:0/4256782879 conn(0x5603e22f5fb0 msgr2=0x5603e2327fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 -- 172.21.15.26:0/4256782879 shutdown_connections 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 -- 172.21.15.26:0/4256782879 wait complete. 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 -- start start 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.042+0000 7fbad5978740 1 Processor -- start 2024-06-13T07:03:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad5978740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e232dc00 0x5603e23aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad5978740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232e5f0 0x5603e23aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad5978740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e2333760 0x5603e23b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad5978740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc1a8e20 con 0x5603e232e5f0 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad5978740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc1aa290 con 0x5603e232dc00 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad5978740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc1a94f0 con 0x5603e2333760 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad5978740 10 client.0 _ll_register_callbacks cb 0x7fbab8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e232dc00 0x5603e23aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e232dc00 0x5603e23aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39460/0 (socket says 172.21.15.26:39460) 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad0c00640 1 -- 172.21.15.26:0/3675227120 learned_addr learned my addr 172.21.15.26:0/3675227120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbacb400640 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232e5f0 0x5603e23aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.043+0000 7fbad0c00640 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e2333760 0x5603e23b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbad0c00640 1 -- 172.21.15.26:0/3675227120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e2333760 msgr2=0x5603e23b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbad0c00640 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e2333760 0x5603e23b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbad0c00640 1 -- 172.21.15.26:0/3675227120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232e5f0 msgr2=0x5603e23aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbad0c00640 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232e5f0 0x5603e23aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbad0c00640 1 -- 172.21.15.26:0/3675227120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e23b1cb0 con 0x5603e232dc00 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbad0c00640 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e232dc00 0x5603e23aa1b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009f00 tx=0x7fbac4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad53ea2c8b193558 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbab5600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbab5600640 1 -- 172.21.15.26:0/3675227120 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4033040 con 0x5603e232dc00 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbab5600640 1 -- 172.21.15.26:0/3675227120 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbac4009ac0 con 0x5603e232dc00 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbad5978740 1 -- 172.21.15.26:0/3675227120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac4015670 con 0x5603e232dc00 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.044+0000 7fbab5600640 1 -- 172.21.15.26:0/3675227120 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbac401a040 con 0x5603e232dc00 2024-06-13T07:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbad5978740 10 client.25628 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbad5978740 1 -- 172.21.15.26:0/3675227120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603e23b2fc0 con 0x5603e232dc00 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbab5600640 1 -- 172.21.15.26:0/3675227120 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4015670 con 0x5603e232dc00 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbab5600640 1 -- 172.21.15.26:0/3675227120 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fbac4020040 con 0x5603e232dc00 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbad5978740 10 client.25628 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbad5978740 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbad5978740 20 client.25628 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbad5978740 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5603e23b37d0 0x5603e23b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbad5978740 4 client.25628 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.045+0000 7fbad5978740 1 -- 172.21.15.26:0/3675227120 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603e2298ea0 con 0x5603e23b37d0 2024-06-13T07:03:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.046+0000 7fbacb400640 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5603e23b37d0 0x5603e23b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.046+0000 7fbacb400640 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5603e23b37d0 0x5603e23b5bf0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c919020e6d4b6275 server_cookie=6326c52e001d373f in_seq=0 out_seq=0 2024-06-13T07:03:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.046+0000 7fbab5600640 10 client.25628 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbab5600640 1 -- 172.21.15.26:0/3675227120 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5603e2298ea0 con 0x5603e23b37d0 2024-06-13T07:03:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbab5600640 10 client.25628 handle_command_reply: tid=0 2024-06-13T07:03:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 client.25628 shutdown 2024-06-13T07:03:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 -- 172.21.15.26:0/3675227120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e232dc00 msgr2=0x5603e23aa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e232dc00 0x5603e23aa1b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009f00 tx=0x7fbac4004ad0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 -- 172.21.15.26:0/3675227120 shutdown_connections 2024-06-13T07:03:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5603e23b37d0 0x5603e23b5bf0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603e2333760 0x5603e23b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603e232e5f0 0x5603e23aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 --2- 172.21.15.26:0/3675227120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603e232dc00 0x5603e23aa1b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbab5600640 0 client.25628 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 -- 172.21.15.26:0/3675227120 >> 172.21.15.26:0/3675227120 conn(0x5603e22f5fb0 msgr2=0x5603e232c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 -- 172.21.15.26:0/3675227120 shutdown_connections 2024-06-13T07:03:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.106+0000 7fbad5978740 1 -- 172.21.15.26:0/3675227120 wait complete. 2024-06-13T07:03:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.107+0000 7fbad5978740 20 client.25628 trim_cache size 0 max 16384 2024-06-13T07:03:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:03:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:03:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:03:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.107+0000 7fbad2c00640 1 -- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbab00a2cb0 msgr2=0x7fbab00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.107+0000 7fbad2c00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbab00a2cb0 0x7fbab00a5170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbabc09bce0 tx=0x7fbabc0d5530 comp rx=0 tx=0).stop 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.107+0000 7fbad2c00640 1 -- 172.21.15.26:0/2541032392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc100900 msgr2=0x7fbacc197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.107+0000 7fbad2c00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc100900 0x7fbacc197d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e9c0 tx=0x7fbabc00ee80 comp rx=0 tx=0).stop 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.108+0000 7fbad2c00640 1 -- 172.21.15.26:0/2541032392 shutdown_connections 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.108+0000 7fbad2c00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbab00a2cb0 0x7fbab00a5170 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.108+0000 7fbad2c00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbacc101af0 0x7fbacc198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.108+0000 7fbad2c00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbacc1024b0 0x7fbacc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.108+0000 7fbad2c00640 1 --2- 172.21.15.26:0/2541032392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc100900 0x7fbacc197d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.108+0000 7fbad2c00640 1 -- 172.21.15.26:0/2541032392 >> 172.21.15.26:0/2541032392 conn(0x7fbacc0fbe70 msgr2=0x7fbacc069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.108+0000 7fbad2c00640 1 -- 172.21.15.26:0/2541032392 shutdown_connections 2024-06-13T07:03:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:06.108+0000 7fbad2c00640 1 -- 172.21.15.26:0/2541032392 wait complete. 2024-06-13T07:03:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:05 smithi026 ceph-mon[32041]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-13T07:03:06.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:05 smithi135 ceph-mon[36276]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-13T07:03:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:05 smithi136 ceph-mon[37229]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-13T07:03:06.509 INFO:tasks.workunit.client.0.smithi026.stdout: 1 64737 4.57 MB/sec warmup 80 sec latency 30.006 ms 2024-06-13T07:03:07.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.436+0000 7fd977200700 1 -- 172.21.15.26:0/4142559159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd970109390 msgr2=0x7fd970115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.436+0000 7fd977200700 1 --2- 172.21.15.26:0/4142559159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd970109390 0x7fd970115830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b0a0 tx=0x7fd96c00b3b0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.436+0000 7fd977200700 1 -- 172.21.15.26:0/4142559159 shutdown_connections 2024-06-13T07:03:07.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.436+0000 7fd977200700 1 --2- 172.21.15.26:0/4142559159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd970109390 0x7fd970115830 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.436+0000 7fd977200700 1 --2- 172.21.15.26:0/4142559159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97010d8a0 0x7fd97010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.436+0000 7fd977200700 1 --2- 172.21.15.26:0/4142559159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9701089d0 0x7fd970108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.436+0000 7fd977200700 1 -- 172.21.15.26:0/4142559159 >> 172.21.15.26:0/4142559159 conn(0x7fd970078220 msgr2=0x7fd9701072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:07.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.437+0000 7fd977200700 1 -- 172.21.15.26:0/4142559159 shutdown_connections 2024-06-13T07:03:07.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.437+0000 7fd977200700 1 -- 172.21.15.26:0/4142559159 wait complete. 2024-06-13T07:03:07.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.439+0000 7fd977200700 1 Processor -- start 2024-06-13T07:03:07.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.439+0000 7fd977200700 1 -- start start 2024-06-13T07:03:07.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.439+0000 7fd977200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9701089d0 0x7fd97019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.439+0000 7fd977200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970109390 0x7fd97019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.439+0000 7fd977200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97010d8a0 0x7fd9701a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.439+0000 7fd977200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd970118cd0 con 0x7fd97010d8a0 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.439+0000 7fd977200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd970118b50 con 0x7fd9701089d0 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.439+0000 7fd977200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd970118e50 con 0x7fd970109390 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd976800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97010d8a0 0x7fd9701a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd975400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970109390 0x7fd97019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd976800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97010d8a0 0x7fd9701a39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54750/0 (socket says 172.21.15.26:54750) 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd975400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970109390 0x7fd97019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58842/0 (socket says 172.21.15.26:58842) 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd975e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9701089d0 0x7fd97019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd975400700 1 -- 172.21.15.26:0/55318314 learned_addr learned my addr 172.21.15.26:0/55318314 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd976800700 1 -- 172.21.15.26:0/55318314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970109390 msgr2=0x7fd97019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd976800700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970109390 0x7fd97019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd976800700 1 -- 172.21.15.26:0/55318314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9701089d0 msgr2=0x7fd97019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd976800700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9701089d0 0x7fd97019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd976800700 1 -- 172.21.15.26:0/55318314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9701a40e0 con 0x7fd97010d8a0 2024-06-13T07:03:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.440+0000 7fd975e00700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9701089d0 0x7fd97019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.441+0000 7fd976800700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97010d8a0 0x7fd9701a39a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b0a0 tx=0x7fd96c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.441+0000 7fd96aa00700 1 -- 172.21.15.26:0/55318314 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c021070 con 0x7fd97010d8a0 2024-06-13T07:03:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.441+0000 7fd96aa00700 1 -- 172.21.15.26:0/55318314 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96c007b80 con 0x7fd97010d8a0 2024-06-13T07:03:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.441+0000 7fd96aa00700 1 -- 172.21.15.26:0/55318314 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c005040 con 0x7fd97010d8a0 2024-06-13T07:03:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.441+0000 7fd977200700 1 -- 172.21.15.26:0/55318314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9701a4370 con 0x7fd97010d8a0 2024-06-13T07:03:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.441+0000 7fd977200700 1 -- 172.21.15.26:0/55318314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9701abc50 con 0x7fd97010d8a0 2024-06-13T07:03:07.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.446+0000 7fd96aa00700 1 -- 172.21.15.26:0/55318314 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd96c024070 con 0x7fd97010d8a0 2024-06-13T07:03:07.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.447+0000 7fd98314cb80 1 Processor -- start 2024-06-13T07:03:07.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.447+0000 7fd98314cb80 1 -- start start 2024-06-13T07:03:07.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.447+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff404420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.447+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff404de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.447+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.447+0000 7fd98314cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555aff311a10 con 0x555aff40d570 2024-06-13T07:03:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.447+0000 7fd98314cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555aff354b00 con 0x555aff37ee40 2024-06-13T07:03:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.447+0000 7fd98314cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555aff30f8f0 con 0x555aff404960 2024-06-13T07:03:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff404de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff404de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58854/0 (socket says 172.21.15.26:58854) 2024-06-13T07:03:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975e00700 1 -- 172.21.15.26:0/116571357 learned_addr learned my addr 172.21.15.26:0/116571357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd96aa00700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9580a2ca0 0x7fd9580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975400700 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff404420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975400700 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd96aa00700 1 -- 172.21.15.26:0/55318314 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd96c00f020 con 0x7fd97010d8a0 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975e00700 1 -- 172.21.15.26:0/116571357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 msgr2=0x555aff404420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975e00700 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff404420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975e00700 1 -- 172.21.15.26:0/116571357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 msgr2=0x555aff40f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975e00700 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975e00700 1 -- 172.21.15.26:0/116571357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aff2fcf90 con 0x555aff404960 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.448+0000 7fd975400700 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.449+0000 7fd976800700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9580a2ca0 0x7fd9580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.449+0000 7fd975e00700 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff404de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9640115f0 tx=0x7fd964011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c48b82d23448a0cc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.449+0000 7fd95f400700 1 -- 172.21.15.26:0/116571357 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96401e070 con 0x555aff404960 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.450+0000 7fd95f400700 1 -- 172.21.15.26:0/116571357 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd964019b10 con 0x555aff404960 2024-06-13T07:03:07.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.450+0000 7fd95f400700 1 -- 172.21.15.26:0/116571357 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96400eb60 con 0x555aff404960 2024-06-13T07:03:07.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.450+0000 7fd98314cb80 1 -- 172.21.15.26:0/116571357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 msgr2=0x555aff404de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.450+0000 7fd976800700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9580a2ca0 0x7fd9580a5160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006100 tx=0x7fd96c0099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:07.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.450+0000 7fd98314cb80 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff404de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9640115f0 tx=0x7fd964011900 comp rx=0 tx=0).stop 2024-06-13T07:03:07.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.451+0000 7fd98314cb80 1 -- 172.21.15.26:0/116571357 shutdown_connections 2024-06-13T07:03:07.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.451+0000 7fd98314cb80 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff404420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.451+0000 7fd98314cb80 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.451+0000 7fd98314cb80 1 --2- 172.21.15.26:0/116571357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff404de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.451+0000 7fd98314cb80 1 -- 172.21.15.26:0/116571357 >> 172.21.15.26:0/116571357 conn(0x555aff32f170 msgr2=0x555aff3fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.451+0000 7fd98314cb80 1 -- 172.21.15.26:0/116571357 shutdown_connections 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.451+0000 7fd98314cb80 1 -- 172.21.15.26:0/116571357 wait complete. 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.451+0000 7fd98314cb80 1 -- start start 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.452+0000 7fd98314cb80 1 Processor -- start 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.452+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff40cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.452+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff40a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.452+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.452+0000 7fd98314cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555aff2f0f60 con 0x555aff40d570 2024-06-13T07:03:07.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.452+0000 7fd98314cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555aff350ac0 con 0x555aff37ee40 2024-06-13T07:03:07.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.452+0000 7fd98314cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555aff410180 con 0x555aff404960 2024-06-13T07:03:07.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd98314cb80 10 client.0 _ll_register_callbacks cb 0x555aff2ee720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:07.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff40cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff40cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39480/0 (socket says 172.21.15.26:39480) 2024-06-13T07:03:07.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975400700 1 -- 172.21.15.26:0/2022560354 learned_addr learned my addr 172.21.15.26:0/2022560354 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:07.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975e00700 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff40a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975400700 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975e00700 1 -- 172.21.15.26:0/2022560354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 msgr2=0x555aff40cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975e00700 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff40cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975e00700 1 -- 172.21.15.26:0/2022560354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 msgr2=0x555aff40a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975e00700 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975e00700 1 -- 172.21.15.26:0/2022560354 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aff2a38d0 con 0x555aff404960 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.453+0000 7fd975400700 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff40cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.454+0000 7fd975e00700 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff40a030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd9640115c0 tx=0x7fd96400e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7c590f49861065 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.454+0000 7fd953e00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.454+0000 7fd953e00700 1 -- 172.21.15.26:0/2022560354 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964027020 con 0x555aff404960 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.454+0000 7fd98314cb80 1 -- 172.21.15.26:0/2022560354 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd964019b10 con 0x555aff404960 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.454+0000 7fd953e00700 1 -- 172.21.15.26:0/2022560354 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd964019730 con 0x555aff404960 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd953e00700 1 -- 172.21.15.26:0/2022560354 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd964032050 con 0x555aff404960 2024-06-13T07:03:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd98314cb80 10 client.25687 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd98314cb80 1 -- 172.21.15.26:0/2022560354 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555aff41a900 con 0x555aff404960 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd953e00700 1 -- 172.21.15.26:0/2022560354 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd964019b10 con 0x555aff404960 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd953e00700 1 -- 172.21.15.26:0/2022560354 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fd964004750 con 0x555aff404960 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd98314cb80 10 client.25687 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd98314cb80 10 client.25687 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd98314cb80 20 client.25687 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd98314cb80 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x555aff41cf40 0x555aff41f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd98314cb80 4 client.25687 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd98314cb80 1 -- 172.21.15.26:0/2022560354 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aff362000 con 0x555aff41cf40 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.455+0000 7fd975400700 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x555aff41cf40 0x555aff41f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.456+0000 7fd975400700 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x555aff41cf40 0x555aff41f330 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f6495256b67abf8 server_cookie=4c622a3c4fff44f in_seq=0 out_seq=0 2024-06-13T07:03:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.456+0000 7fd953e00700 10 client.25687 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T07:03:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd953e00700 1 -- 172.21.15.26:0/2022560354 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555aff362000 con 0x555aff41cf40 2024-06-13T07:03:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd953e00700 10 client.25687 handle_command_reply: tid=0 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 client.25687 shutdown 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 -- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 msgr2=0x555aff40a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff40a030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd9640115c0 tx=0x7fd96400e630 comp rx=0 tx=0).stop 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 -- 172.21.15.26:0/2022560354 shutdown_connections 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x555aff41cf40 0x555aff41f330 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff40cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff40d570 0x555aff40a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd953e00700 0 client.25687 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T07:03:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 --2- 172.21.15.26:0/2022560354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff404960 0x555aff40a030 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.457+0000 7fd98314cb80 1 -- 172.21.15.26:0/2022560354 >> 172.21.15.26:0/2022560354 conn(0x555aff32f170 msgr2=0x555aff3fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.458+0000 7fd98314cb80 1 -- 172.21.15.26:0/2022560354 shutdown_connections 2024-06-13T07:03:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.458+0000 7fd98314cb80 1 -- 172.21.15.26:0/2022560354 wait complete. 2024-06-13T07:03:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.458+0000 7fd98314cb80 20 client.25687 trim_cache size 0 max 16384 2024-06-13T07:03:07.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.477+0000 7fd98314cb80 1 Processor -- start 2024-06-13T07:03:07.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.478+0000 7fd98314cb80 1 -- start start 2024-06-13T07:03:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.478+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 0x555aff403080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.478+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff37ee40 0x555aff4091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.478+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff404960 0x555aff40bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.478+0000 7fd98314cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701b71f0 con 0x555aff37ee40 2024-06-13T07:03:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.478+0000 7fd98314cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701b4a50 con 0x555aff404960 2024-06-13T07:03:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.478+0000 7fd98314cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701b28a0 con 0x555aff32f170 2024-06-13T07:03:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.478+0000 7fd975e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff37ee40 0x555aff4091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff37ee40 0x555aff4091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54782/0 (socket says 172.21.15.26:54782) 2024-06-13T07:03:07.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975e00700 1 -- 172.21.15.26:0/3503430423 learned_addr learned my addr 172.21.15.26:0/3503430423 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:07.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975400700 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff404960 0x555aff40bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975400700 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 0x555aff403080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975e00700 1 -- 172.21.15.26:0/3503430423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 msgr2=0x555aff403080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975e00700 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 0x555aff403080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975e00700 1 -- 172.21.15.26:0/3503430423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff404960 msgr2=0x555aff40bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975e00700 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff404960 0x555aff40bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975e00700 1 -- 172.21.15.26:0/3503430423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aff3ff2f0 con 0x555aff37ee40 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975400700 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff404960 0x555aff40bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.479+0000 7fd975e00700 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff37ee40 0x555aff4091b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd964011510 tx=0x7fd964011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef149d67a8853dca server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.480+0000 7fd95ea00700 1 -- 172.21.15.26:0/3503430423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964027070 con 0x555aff37ee40 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.480+0000 7fd95ea00700 1 -- 172.21.15.26:0/3503430423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd964005520 con 0x555aff37ee40 2024-06-13T07:03:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.480+0000 7fd95ea00700 1 -- 172.21.15.26:0/3503430423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96401e040 con 0x555aff37ee40 2024-06-13T07:03:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 -- 172.21.15.26:0/3503430423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff37ee40 msgr2=0x555aff4091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff37ee40 0x555aff4091b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd964011510 tx=0x7fd964011900 comp rx=0 tx=0).stop 2024-06-13T07:03:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 -- 172.21.15.26:0/3503430423 shutdown_connections 2024-06-13T07:03:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff404960 0x555aff40bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 0x555aff403080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 --2- 172.21.15.26:0/3503430423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff37ee40 0x555aff4091b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 -- 172.21.15.26:0/3503430423 >> 172.21.15.26:0/3503430423 conn(0x555aff41f600 msgr2=0x555aff40b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 -- 172.21.15.26:0/3503430423 shutdown_connections 2024-06-13T07:03:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.481+0000 7fd98314cb80 1 -- 172.21.15.26:0/3503430423 wait complete. 2024-06-13T07:03:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.482+0000 7fd98314cb80 1 -- start start 2024-06-13T07:03:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.482+0000 7fd98314cb80 1 Processor -- start 2024-06-13T07:03:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 0x555aff410530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff410a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd98314cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff404960 0x555aff485a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd98314cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701b2f10 con 0x555aff404960 2024-06-13T07:03:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd98314cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701b4380 con 0x555aff37ee40 2024-06-13T07:03:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd98314cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701b35e0 con 0x555aff32f170 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd98314cb80 10 client.0 _ll_register_callbacks cb 0x555aff2ee720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff410a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff410a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39502/0 (socket says 172.21.15.26:39502) 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975e00700 1 -- 172.21.15.26:0/5891256 learned_addr learned my addr 172.21.15.26:0/5891256 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975400700 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 0x555aff410530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975400700 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff404960 0x555aff485a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975e00700 1 -- 172.21.15.26:0/5891256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 msgr2=0x555aff410530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975e00700 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 0x555aff410530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975e00700 1 -- 172.21.15.26:0/5891256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff404960 msgr2=0x555aff485a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975e00700 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff404960 0x555aff485a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975e00700 1 -- 172.21.15.26:0/5891256 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aff485fa0 con 0x555aff37ee40 2024-06-13T07:03:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.483+0000 7fd975400700 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff404960 0x555aff485a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:07.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.484+0000 7fd975e00700 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff410a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd964014040 tx=0x7fd96402b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42cbae758927d994 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:07.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.484+0000 7fd953e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:03:07.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.484+0000 7fd953e00700 1 -- 172.21.15.26:0/5891256 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96401e070 con 0x555aff37ee40 2024-06-13T07:03:07.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.484+0000 7fd953e00700 1 -- 172.21.15.26:0/5891256 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd964019430 con 0x555aff37ee40 2024-06-13T07:03:07.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.484+0000 7fd98314cb80 1 -- 172.21.15.26:0/5891256 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd964005520 con 0x555aff37ee40 2024-06-13T07:03:07.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.484+0000 7fd953e00700 1 -- 172.21.15.26:0/5891256 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd964027070 con 0x555aff37ee40 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.484+0000 7fd98314cb80 10 client.25646 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.484+0000 7fd98314cb80 1 -- 172.21.15.26:0/5891256 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555aff487150 con 0x555aff37ee40 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.485+0000 7fd953e00700 1 -- 172.21.15.26:0/5891256 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd964005520 con 0x555aff37ee40 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.485+0000 7fd953e00700 1 -- 172.21.15.26:0/5891256 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fd964003680 con 0x555aff37ee40 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.485+0000 7fd98314cb80 10 client.25646 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.485+0000 7fd98314cb80 10 client.25646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.485+0000 7fd98314cb80 20 client.25646 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.485+0000 7fd98314cb80 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x555aff40d570 0x555aff489700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.485+0000 7fd98314cb80 4 client.25646 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.485+0000 7fd98314cb80 1 -- 172.21.15.26:0/5891256 --> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555aff35c580 con 0x555aff40d570 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.486+0000 7fd975400700 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x555aff40d570 0x555aff489700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.486+0000 7fd975400700 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x555aff40d570 0x555aff489700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e40f9903e7d2c194 server_cookie=721100b3ac47cba2 in_seq=0 out_seq=0 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.486+0000 7fd953e00700 10 client.25646 ms_handle_connect on v2:172.21.15.136:6838/2531149732 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.486+0000 7fd953e00700 1 -- 172.21.15.26:0/5891256 <== mds.1 v2:172.21.15.136:6838/2531149732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+678 (crc 0 0 0) 0x555aff35c580 con 0x555aff40d570 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.486+0000 7fd953e00700 10 client.25646 handle_command_reply: tid=0 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.486+0000 7fd98314cb80 1 client.25646 shutdown 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 -- 172.21.15.26:0/5891256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 msgr2=0x555aff410a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff410a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd964014040 tx=0x7fd96402b650 comp rx=0 tx=0).stop 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 -- 172.21.15.26:0/5891256 shutdown_connections 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.136:6838/2531149732,v1:172.21.15.136:6839/2531149732] conn(0x555aff40d570 0x555aff489700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aff404960 0x555aff485a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aff32f170 0x555aff410530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd953e00700 0 client.25646 ms_handle_reset on v2:172.21.15.136:6838/2531149732 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 --2- 172.21.15.26:0/5891256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aff37ee40 0x555aff410a70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 -- 172.21.15.26:0/5891256 >> 172.21.15.26:0/5891256 conn(0x555aff41f600 msgr2=0x555aff409030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 -- 172.21.15.26:0/5891256 shutdown_connections 2024-06-13T07:03:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.487+0000 7fd98314cb80 1 -- 172.21.15.26:0/5891256 wait complete. 2024-06-13T07:03:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.488+0000 7fd98314cb80 20 client.25646 trim_cache size 0 max 16384 2024-06-13T07:03:07.489 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 113134873, 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628327, 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC" 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2336066654, 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628328, 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC" 2024-06-13T07:03:07.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:03:07.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:03:07.491 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-06-13T07:03:07.491 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3358686177, 2024-06-13T07:03:07.491 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628326, 2024-06-13T07:03:07.491 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS" 2024-06-13T07:03:07.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:03:07.491 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-06-13T07:03:07.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.488+0000 7fd977200700 1 -- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9580a2ca0 msgr2=0x7fd9580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.488+0000 7fd977200700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9580a2ca0 0x7fd9580a5160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006100 tx=0x7fd96c0099e0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.488+0000 7fd977200700 1 -- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97010d8a0 msgr2=0x7fd9701a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.488+0000 7fd977200700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97010d8a0 0x7fd9701a39a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b0a0 tx=0x7fd96c004d00 comp rx=0 tx=0).stop 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.489+0000 7fd977200700 1 -- 172.21.15.26:0/55318314 shutdown_connections 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.489+0000 7fd977200700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd9580a2ca0 0x7fd9580a5160 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.489+0000 7fd977200700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9701089d0 0x7fd97019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.489+0000 7fd977200700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97010d8a0 0x7fd9701a39a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.489+0000 7fd977200700 1 --2- 172.21.15.26:0/55318314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970109390 0x7fd97019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.489+0000 7fd977200700 1 -- 172.21.15.26:0/55318314 >> 172.21.15.26:0/55318314 conn(0x7fd970078220 msgr2=0x7fd9700787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.489+0000 7fd977200700 1 -- 172.21.15.26:0/55318314 shutdown_connections 2024-06-13T07:03:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:07.489+0000 7fd977200700 1 -- 172.21.15.26:0/55318314 wait complete. 2024-06-13T07:03:07.510 INFO:tasks.workunit.client.0.smithi026.stdout: 1 66260 4.59 MB/sec warmup 81 sec latency 26.181 ms 2024-06-13T07:03:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:07 smithi026 ceph-mon[32041]: pgmap v588: 129 pgs: 129 active+clean; 40 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-06-13T07:03:08.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:07 smithi135 ceph-mon[36276]: pgmap v588: 129 pgs: 129 active+clean; 40 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-06-13T07:03:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:07 smithi136 ceph-mon[37229]: pgmap v588: 129 pgs: 129 active+clean; 40 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-06-13T07:03:08.510 INFO:tasks.workunit.client.0.smithi026.stdout: 1 67479 4.67 MB/sec warmup 82 sec latency 40.003 ms 2024-06-13T07:03:08.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:03:09.141 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:09.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:08 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:03:09.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:08 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:03:09.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:08 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:03:09.510 INFO:tasks.workunit.client.0.smithi026.stdout: 1 68211 4.63 MB/sec warmup 83 sec latency 29.716 ms 2024-06-13T07:03:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:09 smithi026 ceph-mon[32041]: pgmap v589: 129 pgs: 129 active+clean; 40 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-06-13T07:03:10.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:09 smithi135 ceph-mon[36276]: pgmap v589: 129 pgs: 129 active+clean; 40 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-06-13T07:03:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:09 smithi136 ceph-mon[37229]: pgmap v589: 129 pgs: 129 active+clean; 40 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-06-13T07:03:10.510 INFO:tasks.workunit.client.0.smithi026.stdout: 1 69797 4.66 MB/sec warmup 84 sec latency 24.491 ms 2024-06-13T07:03:11.124 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.124 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.124 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.125 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.125 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:03:11.125 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:03:11.125 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:03:11.125 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:03:11.125 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:03:11.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:03:11.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:03:11.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:03:11.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:03:11.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:03:11.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:03:11.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:03:11.510 INFO:tasks.workunit.client.0.smithi026.stdout: 1 70968 4.73 MB/sec warmup 85 sec latency 20.858 ms 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 -- 172.21.15.26:0/3581058132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945810ca00 msgr2=0x7f945810eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 --2- 172.21.15.26:0/3581058132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945810ca00 0x7f945810eec0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f94500098f0 tx=0x7f9450009c00 comp rx=0 tx=0).stop 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 -- 172.21.15.26:0/3581058132 shutdown_connections 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 --2- 172.21.15.26:0/3581058132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945810ca00 0x7f945810eec0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 --2- 172.21.15.26:0/3581058132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94581035d0 0x7f94581039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 --2- 172.21.15.26:0/3581058132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9458103ef0 0x7f945810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 -- 172.21.15.26:0/3581058132 >> 172.21.15.26:0/3581058132 conn(0x7f94580fce20 msgr2=0x7f94580ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 -- 172.21.15.26:0/3581058132 shutdown_connections 2024-06-13T07:03:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.690+0000 7f945de00700 1 -- 172.21.15.26:0/3581058132 wait complete. 2024-06-13T07:03:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.691+0000 7f945de00700 1 Processor -- start 2024-06-13T07:03:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.691+0000 7f945de00700 1 -- start start 2024-06-13T07:03:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f945de00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581035d0 0x7f9458104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f945de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9458103ef0 0x7f9458104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f945de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945810ca00 0x7f9458104ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f945de00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94581127a0 con 0x7f94581035d0 2024-06-13T07:03:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f945de00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9458112620 con 0x7f945810ca00 2024-06-13T07:03:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f945de00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9458112920 con 0x7f9458103ef0 2024-06-13T07:03:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f944fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9458103ef0 0x7f9458104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f944fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9458103ef0 0x7f9458104990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58904/0 (socket says 172.21.15.26:58904) 2024-06-13T07:03:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f945ca00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581035d0 0x7f9458104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f944fe00700 1 -- 172.21.15.26:0/633344458 learned_addr learned my addr 172.21.15.26:0/633344458 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.692+0000 7f945d400700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945810ca00 0x7f9458104ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f944fe00700 1 -- 172.21.15.26:0/633344458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945810ca00 msgr2=0x7f9458104ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f944fe00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945810ca00 0x7f9458104ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f944fe00700 1 -- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581035d0 msgr2=0x7f9458104450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f944fe00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581035d0 0x7f9458104450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f944fe00700 1 -- 172.21.15.26:0/633344458 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94581ab520 con 0x7f9458103ef0 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f944fe00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9458103ef0 0x7f9458104990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f944400e9f0 tx=0x7f944400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f9456000700 1 -- 172.21.15.26:0/633344458 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444004a30 con 0x7f9458103ef0 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f9456000700 1 -- 172.21.15.26:0/633344458 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9444005500 con 0x7f9458103ef0 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f9456000700 1 -- 172.21.15.26:0/633344458 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944400cdc0 con 0x7f9458103ef0 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.693+0000 7f945de00700 1 -- 172.21.15.26:0/633344458 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94581ab810 con 0x7f9458103ef0 2024-06-13T07:03:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.694+0000 7f945de00700 1 -- 172.21.15.26:0/633344458 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94581abd50 con 0x7f9458103ef0 2024-06-13T07:03:11.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.697+0000 7f9456000700 1 -- 172.21.15.26:0/633344458 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9444005000 con 0x7f9458103ef0 2024-06-13T07:03:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.698+0000 7f9456000700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f943c0a2ca0 0x7f943c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.699+0000 7f9456000700 1 -- 172.21.15.26:0/633344458 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94440cf470 con 0x7f9458103ef0 2024-06-13T07:03:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.699+0000 7f945ca00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f943c0a2ca0 0x7f943c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.699+0000 7f9469ccab80 1 Processor -- start 2024-06-13T07:03:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.699+0000 7f9469ccab80 1 -- start start 2024-06-13T07:03:11.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17df9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e03ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17e06920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f9469ccab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fe17d08a10 con 0x55fe17df9680 2024-06-13T07:03:11.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f9469ccab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fe17d4bb00 con 0x55fe17df8d60 2024-06-13T07:03:11.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f9469ccab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fe17d068f0 con 0x55fe17e04530 2024-06-13T07:03:11.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f945ca00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f943c0a2ca0 0x7f943c0a5160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f944800f540 tx=0x7f944800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:11.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f944fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e03ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f944fe00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e03ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54830/0 (socket says 172.21.15.26:54830) 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f944fe00700 1 -- 172.21.15.26:0/3770843822 learned_addr learned my addr 172.21.15.26:0/3770843822 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f945ca00700 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17df9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.700+0000 7f945d400700 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17e06920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.701+0000 7f944fe00700 1 -- 172.21.15.26:0/3770843822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 msgr2=0x55fe17e06920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.701+0000 7f944fe00700 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17e06920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.701+0000 7f944fe00700 1 -- 172.21.15.26:0/3770843822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 msgr2=0x55fe17df9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.701+0000 7f944fe00700 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17df9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.701+0000 7f944fe00700 1 -- 172.21.15.26:0/3770843822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe17cf3f90 con 0x55fe17df9680 2024-06-13T07:03:11.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.701+0000 7f944fe00700 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e03ff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9444097ce0 tx=0x7f9444004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d39c8c304723e2f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:11.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.702+0000 7f944e000700 1 -- 172.21.15.26:0/3770843822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440b4070 con 0x55fe17df9680 2024-06-13T07:03:11.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.702+0000 7f944e000700 1 -- 172.21.15.26:0/3770843822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944409bf80 con 0x55fe17df9680 2024-06-13T07:03:11.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.702+0000 7f944e000700 1 -- 172.21.15.26:0/3770843822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444016e80 con 0x55fe17df9680 2024-06-13T07:03:11.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 -- 172.21.15.26:0/3770843822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 msgr2=0x55fe17e03ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e03ff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9444097ce0 tx=0x7f9444004f60 comp rx=0 tx=0).stop 2024-06-13T07:03:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 -- 172.21.15.26:0/3770843822 shutdown_connections 2024-06-13T07:03:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17df9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17e06920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 --2- 172.21.15.26:0/3770843822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e03ff0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 -- 172.21.15.26:0/3770843822 >> 172.21.15.26:0/3770843822 conn(0x55fe17d26170 msgr2=0x55fe17df5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 -- 172.21.15.26:0/3770843822 shutdown_connections 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.703+0000 7f9469ccab80 1 -- 172.21.15.26:0/3770843822 wait complete. 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.704+0000 7f9469ccab80 1 -- start start 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.704+0000 7f9469ccab80 1 Processor -- start 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e03d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df9680 0x55fe17e01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17e04530 0x55fe17e015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f9469ccab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fe17ce7f60 con 0x55fe17e04530 2024-06-13T07:03:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f9469ccab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fe17d47ac0 con 0x55fe17df9680 2024-06-13T07:03:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f9469ccab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fe17e06fc0 con 0x55fe17df8d60 2024-06-13T07:03:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f9469ccab80 10 client.0 _ll_register_callbacks cb 0x55fe17ce1380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:11.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945ca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e03d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945ca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e03d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58918/0 (socket says 172.21.15.26:58918) 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945ca00700 1 -- 172.21.15.26:0/346569203 learned_addr learned my addr 172.21.15.26:0/346569203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945d400700 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17e04530 0x55fe17e015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f944fe00700 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df9680 0x55fe17e01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945ca00700 1 -- 172.21.15.26:0/346569203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df9680 msgr2=0x55fe17e01090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945ca00700 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df9680 0x55fe17e01090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945ca00700 1 -- 172.21.15.26:0/346569203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17e04530 msgr2=0x55fe17e015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945ca00700 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17e04530 0x55fe17e015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.705+0000 7f945ca00700 1 -- 172.21.15.26:0/346569203 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe17c9a8d0 con 0x55fe17df8d60 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.706+0000 7f945ca00700 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e03d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9448010010 tx=0x7f9448004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e3afc3eec23950f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.706+0000 7f9442a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:11.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.706+0000 7f9442a00700 1 -- 172.21.15.26:0/346569203 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448027070 con 0x55fe17df8d60 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.706+0000 7f9442a00700 1 -- 172.21.15.26:0/346569203 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9448004db0 con 0x55fe17df8d60 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.706+0000 7f9469ccab80 1 -- 172.21.15.26:0/346569203 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944409bf80 con 0x55fe17df8d60 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9442a00700 1 -- 172.21.15.26:0/346569203 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9448032050 con 0x55fe17df8d60 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9469ccab80 10 client.25705 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9469ccab80 1 -- 172.21.15.26:0/346569203 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fe17e11810 con 0x55fe17df8d60 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9442a00700 1 -- 172.21.15.26:0/346569203 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944409bf80 con 0x55fe17df8d60 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9442a00700 1 -- 172.21.15.26:0/346569203 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f94480056f0 con 0x55fe17df8d60 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9469ccab80 10 client.25705 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9469ccab80 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9469ccab80 20 client.25705 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:11.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9469ccab80 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe17e13da0 0x55fe17e161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9469ccab80 4 client.25705 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:11.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.707+0000 7f9469ccab80 1 -- 172.21.15.26:0/346569203 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe17d59000 con 0x55fe17e13da0 2024-06-13T07:03:11.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.708+0000 7f944fe00700 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe17e13da0 0x55fe17e161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.708+0000 7f944fe00700 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe17e13da0 0x55fe17e161f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9731f36c4403497f server_cookie=8f0975332fb81d0 in_seq=0 out_seq=0 2024-06-13T07:03:11.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.708+0000 7f9442a00700 10 client.25705 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.709+0000 7f9442a00700 1 -- 172.21.15.26:0/346569203 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe17d59000 con 0x55fe17e13da0 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.709+0000 7f9442a00700 10 client.25705 handle_command_reply: tid=0 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.709+0000 7f9469ccab80 1 client.25705 shutdown 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 -- 172.21.15.26:0/346569203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 msgr2=0x55fe17e03d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e03d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9448010010 tx=0x7f9448004ad0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 -- 172.21.15.26:0/346569203 shutdown_connections 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe17e13da0 0x55fe17e161f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17e04530 0x55fe17e015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e03d90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 --2- 172.21.15.26:0/346569203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df9680 0x55fe17e01090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 -- 172.21.15.26:0/346569203 >> 172.21.15.26:0/346569203 conn(0x55fe17d26170 msgr2=0x55fe17df4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:11.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9442a00700 0 client.25705 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:11.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 -- 172.21.15.26:0/346569203 shutdown_connections 2024-06-13T07:03:11.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 1 -- 172.21.15.26:0/346569203 wait complete. 2024-06-13T07:03:11.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.710+0000 7f9469ccab80 20 client.25705 trim_cache size 0 max 16384 2024-06-13T07:03:11.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.730+0000 7f9469ccab80 1 Processor -- start 2024-06-13T07:03:11.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.730+0000 7f9469ccab80 1 -- start start 2024-06-13T07:03:11.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.730+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17e100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17df4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17df5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f9469ccab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94581b74d0 con 0x55fe17df9680 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f9469ccab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94581b4d30 con 0x55fe17df8d60 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f9469ccab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94581b2b80 con 0x55fe17e04530 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f945ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17e100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f945d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17df5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f945ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17e100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39544/0 (socket says 172.21.15.26:39544) 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f945d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17df5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58934/0 (socket says 172.21.15.26:58934) 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f945ca00700 1 -- 172.21.15.26:0/1159824900 learned_addr learned my addr 172.21.15.26:0/1159824900 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f944fe00700 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17df4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f945ca00700 1 -- 172.21.15.26:0/1159824900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 msgr2=0x55fe17df5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f945ca00700 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17df5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.731+0000 7f945ca00700 1 -- 172.21.15.26:0/1159824900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 msgr2=0x55fe17df4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.732+0000 7f945ca00700 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17df4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.732+0000 7f945ca00700 1 -- 172.21.15.26:0/1159824900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe17df6770 con 0x55fe17df8d60 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.732+0000 7f945d400700 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17df5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.732+0000 7f945ca00700 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17e100a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f944800af60 tx=0x7f944800af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1a15594e77663c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.732+0000 7f944d600700 1 -- 172.21.15.26:0/1159824900 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944803b070 con 0x55fe17df8d60 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.732+0000 7f944d600700 1 -- 172.21.15.26:0/1159824900 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9448023740 con 0x55fe17df8d60 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.732+0000 7f944d600700 1 -- 172.21.15.26:0/1159824900 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448027040 con 0x55fe17df8d60 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 -- 172.21.15.26:0/1159824900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 msgr2=0x55fe17e100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17e100a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f944800af60 tx=0x7f944800af90 comp rx=0 tx=0).stop 2024-06-13T07:03:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 -- 172.21.15.26:0/1159824900 shutdown_connections 2024-06-13T07:03:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17df8d60 0x55fe17e100a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17e04530 0x55fe17df5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1159824900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17df4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 -- 172.21.15.26:0/1159824900 >> 172.21.15.26:0/1159824900 conn(0x55fe17d26170 msgr2=0x55fe17df43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 -- 172.21.15.26:0/1159824900 shutdown_connections 2024-06-13T07:03:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.733+0000 7f9469ccab80 1 -- 172.21.15.26:0/1159824900 wait complete. 2024-06-13T07:03:11.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.734+0000 7f9469ccab80 1 -- start start 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.734+0000 7f9469ccab80 1 Processor -- start 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e0c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f9469ccab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17e04530 0x55fe17e0a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f9469ccab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94581b31f0 con 0x55fe17df9680 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f9469ccab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94581b4660 con 0x55fe17e04530 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f9469ccab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94581b38c0 con 0x55fe17df8d60 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f9469ccab80 10 client.0 _ll_register_callbacks cb 0x55fe17ce1380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945ca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e0c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945ca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e0c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58936/0 (socket says 172.21.15.26:58936) 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945ca00700 1 -- 172.21.15.26:0/1022836979 learned_addr learned my addr 172.21.15.26:0/1022836979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17e04530 0x55fe17e0a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f944fe00700 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945ca00700 1 -- 172.21.15.26:0/1022836979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17e04530 msgr2=0x55fe17e0a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945ca00700 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17e04530 0x55fe17e0a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945ca00700 1 -- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 msgr2=0x55fe17e09bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945ca00700 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e09bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f945ca00700 1 -- 172.21.15.26:0/1022836979 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe17e0a7d0 con 0x55fe17df8d60 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.735+0000 7f944fe00700 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e09bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.736+0000 7f945ca00700 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e0c820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9448010010 tx=0x7f9448017d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a165ffa759510e04 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.736+0000 7f9442a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.736+0000 7f9442a00700 1 -- 172.21.15.26:0/1022836979 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448046060 con 0x55fe17df8d60 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.736+0000 7f9469ccab80 1 -- 172.21.15.26:0/1022836979 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9448023740 con 0x55fe17df8d60 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.736+0000 7f9442a00700 1 -- 172.21.15.26:0/1022836979 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9448023540 con 0x55fe17df8d60 2024-06-13T07:03:11.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.736+0000 7f9442a00700 1 -- 172.21.15.26:0/1022836979 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f944804d050 con 0x55fe17df8d60 2024-06-13T07:03:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.736+0000 7f9469ccab80 10 client.25711 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.736+0000 7f9469ccab80 1 -- 172.21.15.26:0/1022836979 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fe17e7ea00 con 0x55fe17df8d60 2024-06-13T07:03:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f9442a00700 1 -- 172.21.15.26:0/1022836979 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9448023740 con 0x55fe17df8d60 2024-06-13T07:03:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f9442a00700 1 -- 172.21.15.26:0/1022836979 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f9448015810 con 0x55fe17df8d60 2024-06-13T07:03:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f9469ccab80 10 client.25711 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f9469ccab80 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f9469ccab80 20 client.25711 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe17e7f050 0x55fe17e81470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f9469ccab80 4 client.25711 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f9469ccab80 1 -- 172.21.15.26:0/1022836979 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe17d53580 con 0x55fe17e7f050 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.737+0000 7f944fe00700 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe17e7f050 0x55fe17e81470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.738+0000 7f9442a00700 10 client.25711 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.738+0000 7f944fe00700 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe17e7f050 0x55fe17e81470 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef293d473c7ddb1 server_cookie=bd0e254b632f3970 in_seq=0 out_seq=0 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.738+0000 7f9442a00700 1 -- 172.21.15.26:0/1022836979 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe17d53580 con 0x55fe17e7f050 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.738+0000 7f9442a00700 10 client.25711 handle_command_reply: tid=0 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.738+0000 7f9469ccab80 1 client.25711 shutdown 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 -- 172.21.15.26:0/1022836979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 msgr2=0x55fe17e0c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e0c820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9448010010 tx=0x7f9448017d90 comp rx=0 tx=0).stop 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 -- 172.21.15.26:0/1022836979 shutdown_connections 2024-06-13T07:03:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe17e7f050 0x55fe17e81470 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe17e04530 0x55fe17e0a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe17df8d60 0x55fe17e0c820 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 --2- 172.21.15.26:0/1022836979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe17df9680 0x55fe17e09bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9442a00700 0 client.25711 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 -- 172.21.15.26:0/1022836979 >> 172.21.15.26:0/1022836979 conn(0x55fe17d26170 msgr2=0x55fe17e0f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 -- 172.21.15.26:0/1022836979 shutdown_connections 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 1 -- 172.21.15.26:0/1022836979 wait complete. 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.739+0000 7f9469ccab80 20 client.25711 trim_cache size 0 max 16384 2024-06-13T07:03:11.741 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2024-06-13T07:03:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.740+0000 7f945de00700 1 -- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f943c0a2ca0 msgr2=0x7f943c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.740+0000 7f945de00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f943c0a2ca0 0x7f943c0a5160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f944800f540 tx=0x7f944800b040 comp rx=0 tx=0).stop 2024-06-13T07:03:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.740+0000 7f945de00700 1 -- 172.21.15.26:0/633344458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9458103ef0 msgr2=0x7f9458104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.740+0000 7f945de00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9458103ef0 0x7f9458104990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f944400e9f0 tx=0x7f944400ed00 comp rx=0 tx=0).stop 2024-06-13T07:03:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.741+0000 7f945de00700 1 -- 172.21.15.26:0/633344458 shutdown_connections 2024-06-13T07:03:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.741+0000 7f945de00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f943c0a2ca0 0x7f943c0a5160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.741+0000 7f945de00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945810ca00 0x7f9458104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.741+0000 7f945de00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581035d0 0x7f9458104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.741+0000 7f945de00700 1 --2- 172.21.15.26:0/633344458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9458103ef0 0x7f9458104990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.741+0000 7f945de00700 1 -- 172.21.15.26:0/633344458 >> 172.21.15.26:0/633344458 conn(0x7f94580fce20 msgr2=0x7f94580fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.741+0000 7f945de00700 1 -- 172.21.15.26:0/633344458 shutdown_connections 2024-06-13T07:03:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:03:11.741+0000 7f945de00700 1 -- 172.21.15.26:0/633344458 wait complete. 2024-06-13T07:03:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:12 smithi136 ceph-mon[37229]: pgmap v590: 129 pgs: 129 active+clean; 40 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:03:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:12 smithi026 ceph-mon[32041]: pgmap v590: 129 pgs: 129 active+clean; 40 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:03:12.510 INFO:tasks.workunit.client.0.smithi026.stdout: 1 71515 4.69 MB/sec warmup 86 sec latency 16.201 ms 2024-06-13T07:03:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:12 smithi135 ceph-mon[36276]: pgmap v590: 129 pgs: 129 active+clean; 40 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:03:12.801 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 38801065, 'ino': 1099511628339, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 449605864, 'ino': 1099511628332, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04'}, {'damage_type': 'backtrace', 'id': 555714115, 'ino': 1099511628838, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/CHANGED.DB'}, {'damage_type': 'backtrace', 'id': 1081715645, 'ino': 1099511628843, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.MB'}, {'damage_type': 'backtrace', 'id': 1366808144, 'ino': 1099511628331, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX'}, {'damage_type': 'backtrace', 'id': 1414278848, 'ino': 1099511628347, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 1469477869, 'ino': 1099511628345, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 1592486186, 'ino': 1099511628336, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0'}, {'damage_type': 'backtrace', 'id': 1594152973, 'ino': 1099511628340, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL'}, {'damage_type': 'backtrace', 'id': 1879428922, 'ino': 1099511628338, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 2123497955, 'ino': 1099511628335, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX'}, {'damage_type': 'backtrace', 'id': 2494968839, 'ino': 1099511628330, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB'}, {'damage_type': 'backtrace', 'id': 2674821089, 'ino': 1099511628343, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX'}, {'damage_type': 'backtrace', 'id': 2741091717, 'ino': 1099511628341, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 2782723332, 'ino': 1099511628337, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0'}, {'damage_type': 'backtrace', 'id': 2939340827, 'ino': 1099511628844, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ANSWER.DB'}, {'damage_type': 'backtrace', 'id': 3098618049, 'ino': 1099511628342, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB'}, {'damage_type': 'backtrace', 'id': 3132777122, 'ino': 1099511628302, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 3293433281, 'ino': 1099511628346, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 3313458462, 'ino': 1099511628301, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 3587143827, 'ino': 1099511628348, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 3621784038, 'ino': 1099511628836, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.VAL'}, {'damage_type': 'backtrace', 'id': 3766301338, 'ino': 1099511628334, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB'}, {'damage_type': 'backtrace', 'id': 3932598800, 'ino': 1099511628333, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04'}, {'damage_type': 'backtrace', 'id': 3952965238, 'ino': 1099511628344, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 4153425059, 'ino': 1099511628349, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL'}] 2024-06-13T07:03:12.801 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 113134873, 'ino': 1099511628327, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 2336066654, 'ino': 1099511628328, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 3358686177, 'ino': 1099511628326, 'path': '/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS'}] 2024-06-13T07:03:12.802 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-06-13T07:03:13.510 INFO:tasks.workunit.client.0.smithi026.stdout: 1 72451 4.69 MB/sec warmup 87 sec latency 40.329 ms 2024-06-13T07:03:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:14 smithi136 ceph-mon[37229]: pgmap v591: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:03:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:14 smithi026 ceph-mon[32041]: pgmap v591: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:03:14.510 INFO:tasks.workunit.client.0.smithi026.stdout: 1 73447 4.67 MB/sec warmup 88 sec latency 211.077 ms 2024-06-13T07:03:14.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:14 smithi135 ceph-mon[36276]: pgmap v591: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:03:15.511 INFO:tasks.workunit.client.0.smithi026.stdout: 1 73447 4.61 MB/sec warmup 89 sec latency 1211.244 ms 2024-06-13T07:03:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:16 smithi136 ceph-mon[37229]: pgmap v592: 129 pgs: 129 active+clean; 42 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-06-13T07:03:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:16 smithi026 ceph-mon[32041]: pgmap v592: 129 pgs: 129 active+clean; 42 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-06-13T07:03:16.511 INFO:tasks.workunit.client.0.smithi026.stdout: 1 73447 4.56 MB/sec warmup 90 sec latency 2211.909 ms 2024-06-13T07:03:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:16 smithi135 ceph-mon[36276]: pgmap v592: 129 pgs: 129 active+clean; 42 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-06-13T07:03:17.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 73487 4.52 MB/sec warmup 91 sec latency 3126.860 ms 2024-06-13T07:03:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:18 smithi026 ceph-mon[32041]: pgmap v593: 129 pgs: 129 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-06-13T07:03:18.511 INFO:tasks.workunit.client.0.smithi026.stdout: 1 74658 4.58 MB/sec warmup 92 sec latency 22.494 ms 2024-06-13T07:03:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:18 smithi135 ceph-mon[36276]: pgmap v593: 129 pgs: 129 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-06-13T07:03:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:18 smithi136 ceph-mon[37229]: pgmap v593: 129 pgs: 129 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-06-13T07:03:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 -- 172.21.15.26:0/394393309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94180743a0 msgr2=0x7f9418074800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 --2- 172.21.15.26:0/394393309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94180743a0 0x7f9418074800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f940402a480 comp rx=0 tx=0).stop 2024-06-13T07:03:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 -- 172.21.15.26:0/394393309 shutdown_connections 2024-06-13T07:03:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 --2- 172.21.15.26:0/394393309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94180743a0 0x7f9418074800 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 --2- 172.21.15.26:0/394393309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94181096f0 0x7f941810bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 --2- 172.21.15.26:0/394393309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9418075980 0x7f9418073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 -- 172.21.15.26:0/394393309 >> 172.21.15.26:0/394393309 conn(0x7f94180fcdd0 msgr2=0x7f94180ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 -- 172.21.15.26:0/394393309 shutdown_connections 2024-06-13T07:03:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.276+0000 7f941d000640 1 -- 172.21.15.26:0/394393309 wait complete. 2024-06-13T07:03:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.277+0000 7f941d000640 1 Processor -- start 2024-06-13T07:03:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.277+0000 7f941d000640 1 -- start start 2024-06-13T07:03:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f941d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94180743a0 0x7f9418199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f941d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9418075980 0x7f941819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f941d000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94181096f0 0x7f94181a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f941d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f941810f730 con 0x7f9418075980 2024-06-13T07:03:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f941d000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f941810f5b0 con 0x7f94181096f0 2024-06-13T07:03:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f941d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f941810f8b0 con 0x7f94180743a0 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f9417400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94180743a0 0x7f9418199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f9417400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94180743a0 0x7f9418199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:46560/0 (socket says 172.21.15.26:46560) 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f9417400640 1 -- 172.21.15.26:0/3881146462 learned_addr learned my addr 172.21.15.26:0/3881146462 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f9416a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9418075980 0x7f941819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.278+0000 7f9417e00640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94181096f0 0x7f94181a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f9417400640 1 -- 172.21.15.26:0/3881146462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94181096f0 msgr2=0x7f94181a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f9417400640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94181096f0 0x7f94181a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f9417400640 1 -- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9418075980 msgr2=0x7f941819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f9417400640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9418075980 0x7f941819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f9417400640 1 -- 172.21.15.26:0/3881146462 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94181a1960 con 0x7f94180743a0 2024-06-13T07:03:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f9417400640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94180743a0 0x7f9418199d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f940fe00640 1 -- 172.21.15.26:0/3881146462 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940800cd80 con 0x7f94180743a0 2024-06-13T07:03:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f940fe00640 1 -- 172.21.15.26:0/3881146462 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9408005500 con 0x7f94180743a0 2024-06-13T07:03:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f940fe00640 1 -- 172.21.15.26:0/3881146462 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408010690 con 0x7f94180743a0 2024-06-13T07:03:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f941d000640 1 -- 172.21.15.26:0/3881146462 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94181a1c50 con 0x7f94180743a0 2024-06-13T07:03:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.279+0000 7f941d000640 1 -- 172.21.15.26:0/3881146462 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181a2120 con 0x7f94180743a0 2024-06-13T07:03:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.283+0000 7f940fe00640 1 -- 172.21.15.26:0/3881146462 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94080046c0 con 0x7f94180743a0 2024-06-13T07:03:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.283+0000 7f94215dc740 1 Processor -- start 2024-06-13T07:03:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.283+0000 7f94215dc740 1 -- start start 2024-06-13T07:03:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f94215dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f94215dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97de2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f94215dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f94215dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5644b973c420 con 0x5644b97d8dc0 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f94215dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5644b96e3c50 con 0x5644b97d97b0 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f94215dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5644b97383e0 con 0x5644b97de920 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9417e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9417400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97de2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9417e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46402/0 (socket says 172.21.15.26:46402) 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9417e00640 1 -- 172.21.15.26:0/3664347869 learned_addr learned my addr 172.21.15.26:0/3664347869 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9416a00640 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9416a00640 1 -- 172.21.15.26:0/3664347869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 msgr2=0x5644b97de2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9416a00640 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97de2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9416a00640 1 -- 172.21.15.26:0/3664347869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 msgr2=0x5644b97d91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.284+0000 7f9416a00640 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f9416a00640 1 -- 172.21.15.26:0/3664347869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b9450210 con 0x5644b97de920 2024-06-13T07:03:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f9417e00640 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f9416a00640 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e0db0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f940402a9a0 tx=0x7f9404005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9667091633ac01b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f940fe00640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f93fc0a2c00 0x7f93fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f940cc00640 1 -- 172.21.15.26:0/3664347869 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404035040 con 0x5644b97de920 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f940cc00640 1 -- 172.21.15.26:0/3664347869 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9404005520 con 0x5644b97de920 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f940fe00640 1 -- 172.21.15.26:0/3881146462 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94080cf810 con 0x7f94180743a0 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f940cc00640 1 -- 172.21.15.26:0/3664347869 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94040316b0 con 0x5644b97de920 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.285+0000 7f9417e00640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f93fc0a2c00 0x7f93fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f94215dc740 1 -- 172.21.15.26:0/3664347869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 msgr2=0x5644b97e0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f94215dc740 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e0db0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f940402a9a0 tx=0x7f9404005c00 comp rx=0 tx=0).stop 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f9417e00640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f93fc0a2c00 0x7f93fc0a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5644b97da010 tx=0x7f9410009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f94215dc740 1 -- 172.21.15.26:0/3664347869 shutdown_connections 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f94215dc740 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97de2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f94215dc740 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e0db0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f94215dc740 1 --2- 172.21.15.26:0/3664347869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f94215dc740 1 -- 172.21.15.26:0/3664347869 >> 172.21.15.26:0/3664347869 conn(0x5644b97a0fb0 msgr2=0x5644b97d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.286+0000 7f94215dc740 1 -- 172.21.15.26:0/3664347869 shutdown_connections 2024-06-13T07:03:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.287+0000 7f94215dc740 1 -- 172.21.15.26:0/3664347869 wait complete. 2024-06-13T07:03:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.287+0000 7f94215dc740 1 -- start start 2024-06-13T07:03:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.287+0000 7f94215dc740 1 Processor -- start 2024-06-13T07:03:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f94215dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97d8dc0 0x5644b97e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f94215dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d97b0 0x5644b97e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f94215dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97de920 0x5644b97ea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f94215dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5644b97e1870 con 0x5644b97d97b0 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f94215dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5644b96f70b0 con 0x5644b97de920 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f94215dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5644b97e19f0 con 0x5644b97d8dc0 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f94215dc740 10 client.0 _ll_register_callbacks cb 0x5644b96e2f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f9417400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97d8dc0 0x5644b97e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f9417400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97d8dc0 0x5644b97e30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:46578/0 (socket says 172.21.15.26:46578) 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f9417400640 1 -- 172.21.15.26:0/3509793989 learned_addr learned my addr 172.21.15.26:0/3509793989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f9417e00640 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97de920 0x5644b97ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.288+0000 7f9416a00640 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d97b0 0x5644b97e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9417400640 1 -- 172.21.15.26:0/3509793989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97de920 msgr2=0x5644b97ea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9417400640 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97de920 0x5644b97ea670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9417400640 1 -- 172.21.15.26:0/3509793989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d97b0 msgr2=0x5644b97e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9417400640 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d97b0 0x5644b97e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9417400640 1 -- 172.21.15.26:0/3509793989 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b951c620 con 0x5644b97d8dc0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9417400640 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97d8dc0 0x5644b97e30f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f94080985f0 tx=0x7f9408097f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a0d02ae9049588 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9401600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9401600640 1 -- 172.21.15.26:0/3509793989 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940809c320 con 0x5644b97d8dc0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f9401600640 1 -- 172.21.15.26:0/3509793989 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94080a4400 con 0x5644b97d8dc0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.289+0000 7f94215dc740 1 -- 172.21.15.26:0/3509793989 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9404005520 con 0x5644b97d8dc0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f9401600640 1 -- 172.21.15.26:0/3509793989 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f940809bc20 con 0x5644b97d8dc0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f94215dc740 10 client.25723 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f94215dc740 1 -- 172.21.15.26:0/3509793989 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644b97edaa0 con 0x5644b97d8dc0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f9401600640 1 -- 172.21.15.26:0/3509793989 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9404005520 con 0x5644b97d8dc0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f9401600640 1 -- 172.21.15.26:0/3509793989 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f94080ba2a0 con 0x5644b97d8dc0 2024-06-13T07:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f94215dc740 10 client.25723 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f94215dc740 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f94215dc740 20 client.25723 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f94215dc740 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5644b97ef190 0x5644b97f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f94215dc740 4 client.25723 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f94215dc740 1 -- 172.21.15.26:0/3509793989 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b9749b50 con 0x5644b97ef190 2024-06-13T07:03:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.290+0000 7f9416a00640 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5644b97ef190 0x5644b97f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.291+0000 7f9401600640 10 client.25723 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.291+0000 7f9416a00640 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5644b97ef190 0x5644b97f1600 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8d7450576b9102 server_cookie=ccbd6fde9f049451 in_seq=0 out_seq=0 2024-06-13T07:03:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f9401600640 1 -- 172.21.15.26:0/3509793989 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5644b9749b50 con 0x5644b97ef190 2024-06-13T07:03:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f9401600640 10 client.25723 handle_command_reply: tid=0 2024-06-13T07:03:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 client.25723 shutdown 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 -- 172.21.15.26:0/3509793989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97d8dc0 msgr2=0x5644b97e30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97d8dc0 0x5644b97e30f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f94080985f0 tx=0x7f9408097f80 comp rx=0 tx=0).stop 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 -- 172.21.15.26:0/3509793989 shutdown_connections 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5644b97ef190 0x5644b97f1600 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97de920 0x5644b97ea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d97b0 0x5644b97e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 --2- 172.21.15.26:0/3509793989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97d8dc0 0x5644b97e30f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f9401600640 0 client.25723 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.292+0000 7f94215dc740 1 -- 172.21.15.26:0/3509793989 >> 172.21.15.26:0/3509793989 conn(0x5644b97a0fb0 msgr2=0x5644b97d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.293+0000 7f94215dc740 1 -- 172.21.15.26:0/3509793989 shutdown_connections 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.293+0000 7f94215dc740 1 -- 172.21.15.26:0/3509793989 wait complete. 2024-06-13T07:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.293+0000 7f94215dc740 20 client.25723 trim_cache size 0 max 16384 2024-06-13T07:03:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.312+0000 7f94215dc740 1 Processor -- start 2024-06-13T07:03:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.312+0000 7f94215dc740 1 -- start start 2024-06-13T07:03:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.312+0000 7f94215dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.312+0000 7f94215dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97ea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.312+0000 7f94215dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.312+0000 7f94215dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94181aefd0 con 0x5644b97d8dc0 2024-06-13T07:03:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.312+0000 7f94215dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ac870 con 0x5644b97d97b0 2024-06-13T07:03:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.312+0000 7f94215dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94181aa6c0 con 0x5644b97de920 2024-06-13T07:03:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9417e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9416a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9417e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:46584/0 (socket says 172.21.15.26:46584) 2024-06-13T07:03:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9417e00640 1 -- 172.21.15.26:0/3499796935 learned_addr learned my addr 172.21.15.26:0/3499796935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9417400640 1 --2- 172.21.15.26:0/3499796935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9416a00640 1 -- 172.21.15.26:0/3499796935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 msgr2=0x5644b97e2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9416a00640 1 --2- 172.21.15.26:0/3499796935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9416a00640 1 -- 172.21.15.26:0/3499796935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 msgr2=0x5644b97d7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9416a00640 1 --2- 172.21.15.26:0/3499796935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.313+0000 7f9416a00640 1 -- 172.21.15.26:0/3499796935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b97e1b20 con 0x5644b97d97b0 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.314+0000 7f9416a00640 1 --2- 172.21.15.26:0/3499796935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97ea630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f9404000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7267ca582ec7794 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.314+0000 7f9403e00640 1 -- 172.21.15.26:0/3499796935 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404035040 con 0x5644b97d97b0 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.314+0000 7f9403e00640 1 -- 172.21.15.26:0/3499796935 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9404004300 con 0x5644b97d97b0 2024-06-13T07:03:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.314+0000 7f9403e00640 1 -- 172.21.15.26:0/3499796935 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404003a70 con 0x5644b97d97b0 2024-06-13T07:03:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 -- 172.21.15.26:0/3499796935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 msgr2=0x5644b97ea630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 --2- 172.21.15.26:0/3499796935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97ea630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f9404000bc0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 -- 172.21.15.26:0/3499796935 shutdown_connections 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 --2- 172.21.15.26:0/3499796935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97ea630 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 --2- 172.21.15.26:0/3499796935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 --2- 172.21.15.26:0/3499796935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97d7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 -- 172.21.15.26:0/3499796935 >> 172.21.15.26:0/3499796935 conn(0x5644b97a0fb0 msgr2=0x5644b97d7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 -- 172.21.15.26:0/3499796935 shutdown_connections 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 -- 172.21.15.26:0/3499796935 wait complete. 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 -- start start 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.315+0000 7f94215dc740 1 Processor -- start 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f94215dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f94215dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f94215dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f94215dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94181aad30 con 0x5644b97d8dc0 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f94215dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ac1a0 con 0x5644b97d97b0 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f94215dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ab400 con 0x5644b97de920 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f94215dc740 10 client.0 _ll_register_callbacks cb 0x7f94080d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f9417400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f9417e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f9417400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46430/0 (socket says 172.21.15.26:46430) 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f9417400640 1 -- 172.21.15.26:0/759747350 learned_addr learned my addr 172.21.15.26:0/759747350 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.316+0000 7f9416a00640 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9417400640 1 -- 172.21.15.26:0/759747350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 msgr2=0x5644b97e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9417400640 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9417400640 1 -- 172.21.15.26:0/759747350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 msgr2=0x5644b97e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9417400640 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9417400640 1 -- 172.21.15.26:0/759747350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b97e9220 con 0x5644b97d8dc0 2024-06-13T07:03:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9417e00640 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9416a00640 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9417400640 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97e7780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f940800c6a0 tx=0x7f94080cf6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d715d81ad9e9b5eb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9401600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:03:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9401600640 1 -- 172.21.15.26:0/759747350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94080a9060 con 0x5644b97d8dc0 2024-06-13T07:03:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f9401600640 1 -- 172.21.15.26:0/759747350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f940809b270 con 0x5644b97d8dc0 2024-06-13T07:03:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.317+0000 7f94215dc740 1 -- 172.21.15.26:0/759747350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9404004300 con 0x5644b97d8dc0 2024-06-13T07:03:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f9401600640 1 -- 172.21.15.26:0/759747350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f940809b760 con 0x5644b97d8dc0 2024-06-13T07:03:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f94215dc740 10 client.15591 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f94215dc740 1 -- 172.21.15.26:0/759747350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644b97e9220 con 0x5644b97d8dc0 2024-06-13T07:03:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f9401600640 1 -- 172.21.15.26:0/759747350 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9404004300 con 0x5644b97d8dc0 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f9401600640 1 -- 172.21.15.26:0/759747350 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f9408097970 con 0x5644b97d8dc0 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f94215dc740 10 client.15591 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f94215dc740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f94215dc740 20 client.15591 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f94215dc740 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5644b9859fa0 0x5644b985c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f94215dc740 4 client.15591 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.318+0000 7f94215dc740 1 -- 172.21.15.26:0/759747350 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644b9743ea0 con 0x5644b9859fa0 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.319+0000 7f9416a00640 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5644b9859fa0 0x5644b985c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.319+0000 7f9416a00640 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5644b9859fa0 0x5644b985c390 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed9cfbb00468431 server_cookie=3a15111191d3a26f in_seq=0 out_seq=0 2024-06-13T07:03:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.319+0000 7f9401600640 10 client.15591 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f9401600640 1 -- 172.21.15.26:0/759747350 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5644b9743ea0 con 0x5644b9859fa0 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f9401600640 10 client.15591 handle_command_reply: tid=0 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 client.15591 shutdown 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 -- 172.21.15.26:0/759747350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 msgr2=0x5644b97e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97e7780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f940800c6a0 tx=0x7f94080cf6b0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 -- 172.21.15.26:0/759747350 shutdown_connections 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5644b9859fa0 0x5644b985c390 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644b97d97b0 0x5644b97e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644b97de920 0x5644b97e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 --2- 172.21.15.26:0/759747350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b97d8dc0 0x5644b97e7780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f9401600640 0 client.15591 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 -- 172.21.15.26:0/759747350 >> 172.21.15.26:0/759747350 conn(0x5644b97a0fb0 msgr2=0x5644b97e2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 -- 172.21.15.26:0/759747350 shutdown_connections 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.383+0000 7f94215dc740 1 -- 172.21.15.26:0/759747350 wait complete. 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.384+0000 7f94215dc740 20 client.15591 trim_cache size 0 max 16384 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:03:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.384+0000 7f941d000640 1 -- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f93fc0a2c00 msgr2=0x7f93fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.384+0000 7f941d000640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f93fc0a2c00 0x7f93fc0a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5644b97da010 tx=0x7f9410009500 comp rx=0 tx=0).stop 2024-06-13T07:03:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.384+0000 7f941d000640 1 -- 172.21.15.26:0/3881146462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94180743a0 msgr2=0x7f9418199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.384+0000 7f941d000640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94180743a0 0x7f9418199d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800edf0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.385+0000 7f941d000640 1 -- 172.21.15.26:0/3881146462 shutdown_connections 2024-06-13T07:03:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.385+0000 7f941d000640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f93fc0a2c00 0x7f93fc0a50c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.385+0000 7f941d000640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94181096f0 0x7f94181a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.385+0000 7f941d000640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9418075980 0x7f941819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.385+0000 7f941d000640 1 --2- 172.21.15.26:0/3881146462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94180743a0 0x7f9418199d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.385+0000 7f941d000640 1 -- 172.21.15.26:0/3881146462 >> 172.21.15.26:0/3881146462 conn(0x7f94180fcdd0 msgr2=0x7f9418105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.385+0000 7f941d000640 1 -- 172.21.15.26:0/3881146462 shutdown_connections 2024-06-13T07:03:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:19.385+0000 7f941d000640 1 -- 172.21.15.26:0/3881146462 wait complete. 2024-06-13T07:03:19.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 75132 4.54 MB/sec warmup 93 sec latency 44.041 ms 2024-06-13T07:03:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:20 smithi026 ceph-mon[32041]: pgmap v594: 129 pgs: 129 active+clean; 42 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-06-13T07:03:20.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 76052 4.55 MB/sec warmup 94 sec latency 34.156 ms 2024-06-13T07:03:20.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:20 smithi135 ceph-mon[36276]: pgmap v594: 129 pgs: 129 active+clean; 42 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-06-13T07:03:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:20 smithi136 ceph-mon[37229]: pgmap v594: 129 pgs: 129 active+clean; 42 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-06-13T07:03:21.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 77119 4.53 MB/sec warmup 95 sec latency 21.285 ms 2024-06-13T07:03:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:22 smithi026 ceph-mon[32041]: pgmap v595: 129 pgs: 129 active+clean; 38 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-13T07:03:22.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 78290 4.60 MB/sec warmup 96 sec latency 21.773 ms 2024-06-13T07:03:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:22 smithi135 ceph-mon[36276]: pgmap v595: 129 pgs: 129 active+clean; 38 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-13T07:03:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:22 smithi136 ceph-mon[37229]: pgmap v595: 129 pgs: 129 active+clean; 38 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-13T07:03:23.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 78654 4.55 MB/sec warmup 97 sec latency 22.322 ms 2024-06-13T07:03:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:24 smithi026 ceph-mon[32041]: pgmap v596: 129 pgs: 129 active+clean; 39 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:03:24.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 79538 4.56 MB/sec warmup 98 sec latency 52.599 ms 2024-06-13T07:03:24.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:24 smithi135 ceph-mon[36276]: pgmap v596: 129 pgs: 129 active+clean; 39 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:03:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:24 smithi136 ceph-mon[37229]: pgmap v596: 129 pgs: 129 active+clean; 39 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:03:25.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 80587 4.54 MB/sec warmup 99 sec latency 29.954 ms 2024-06-13T07:03:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:26 smithi026 ceph-mon[32041]: pgmap v597: 129 pgs: 129 active+clean; 39 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-06-13T07:03:26.512 INFO:tasks.workunit.client.0.smithi026.stdout: 1 81748 4.59 MB/sec warmup 100 sec latency 25.434 ms 2024-06-13T07:03:26.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:26 smithi135 ceph-mon[36276]: pgmap v597: 129 pgs: 129 active+clean; 39 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-06-13T07:03:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:26 smithi136 ceph-mon[37229]: pgmap v597: 129 pgs: 129 active+clean; 39 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-06-13T07:03:27.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 82169 4.56 MB/sec warmup 101 sec latency 20.773 ms 2024-06-13T07:03:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:28 smithi026 ceph-mon[32041]: pgmap v598: 129 pgs: 129 active+clean; 41 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-13T07:03:28.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 82919 4.54 MB/sec warmup 102 sec latency 26.293 ms 2024-06-13T07:03:28.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:28 smithi135 ceph-mon[36276]: pgmap v598: 129 pgs: 129 active+clean; 41 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-13T07:03:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:28 smithi136 ceph-mon[37229]: pgmap v598: 129 pgs: 129 active+clean; 41 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-13T07:03:29.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 84126 4.55 MB/sec warmup 103 sec latency 45.176 ms 2024-06-13T07:03:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:30 smithi026 ceph-mon[32041]: pgmap v599: 129 pgs: 129 active+clean; 40 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-13T07:03:30.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 85400 4.61 MB/sec warmup 104 sec latency 27.752 ms 2024-06-13T07:03:30.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:30 smithi135 ceph-mon[36276]: pgmap v599: 129 pgs: 129 active+clean; 40 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-13T07:03:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:30 smithi136 ceph-mon[37229]: pgmap v599: 129 pgs: 129 active+clean; 40 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-13T07:03:31.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 85862 4.57 MB/sec warmup 105 sec latency 21.966 ms 2024-06-13T07:03:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:32 smithi026 ceph-mon[32041]: pgmap v600: 129 pgs: 129 active+clean; 41 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-06-13T07:03:32.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 86753 4.58 MB/sec warmup 106 sec latency 31.398 ms 2024-06-13T07:03:32.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:32 smithi135 ceph-mon[36276]: pgmap v600: 129 pgs: 129 active+clean; 41 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-06-13T07:03:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 -- 172.21.15.26:0/1020422468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374102330 msgr2=0x7fe374104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 --2- 172.21.15.26:0/1020422468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374102330 0x7fe374104720 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe364009900 tx=0x7fe36402a410 comp rx=0 tx=0).stop 2024-06-13T07:03:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 -- 172.21.15.26:0/1020422468 shutdown_connections 2024-06-13T07:03:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 --2- 172.21.15.26:0/1020422468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe374104c60 0x7fe374107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 --2- 172.21.15.26:0/1020422468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe374107590 0x7fe374109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 --2- 172.21.15.26:0/1020422468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374102330 0x7fe374104720 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 -- 172.21.15.26:0/1020422468 >> 172.21.15.26:0/1020422468 conn(0x7fe3740fbce0 msgr2=0x7fe3740fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 -- 172.21.15.26:0/1020422468 shutdown_connections 2024-06-13T07:03:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.553+0000 7fe373e00640 1 -- 172.21.15.26:0/1020422468 wait complete. 2024-06-13T07:03:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.554+0000 7fe373e00640 1 Processor -- start 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.554+0000 7fe373e00640 1 -- start start 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe373e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe374102330 0x7fe374199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe373e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374104c60 0x7fe37419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe373e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe374107590 0x7fe3741a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe373e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410d4c0 con 0x7fe374104c60 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe373e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410d340 con 0x7fe374102330 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe373e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410d640 con 0x7fe374107590 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe372a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe374102330 0x7fe374199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe372a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe374102330 0x7fe374199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39210/0 (socket says 172.21.15.26:39210) 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe372a00640 1 -- 172.21.15.26:0/2474197008 learned_addr learned my addr 172.21.15.26:0/2474197008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe373400640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe374107590 0x7fe3741a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.555+0000 7fe372000640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374104c60 0x7fe37419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe372a00640 1 -- 172.21.15.26:0/2474197008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe374107590 msgr2=0x7fe3741a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe372a00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe374107590 0x7fe3741a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe372a00640 1 -- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374104c60 msgr2=0x7fe37419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe372a00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374104c60 0x7fe37419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe372a00640 1 -- 172.21.15.26:0/2474197008 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3741a19c0 con 0x7fe374102330 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe372000640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374104c60 0x7fe37419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe372a00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe374102330 0x7fe374199db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe364009230 tx=0x7fe364031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe36b400640 1 -- 172.21.15.26:0/2474197008 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36403f030 con 0x7fe374102330 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe373e00640 1 -- 172.21.15.26:0/2474197008 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3741a1c50 con 0x7fe374102330 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe36b400640 1 -- 172.21.15.26:0/2474197008 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe364003ad0 con 0x7fe374102330 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.556+0000 7fe36b400640 1 -- 172.21.15.26:0/2474197008 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364035070 con 0x7fe374102330 2024-06-13T07:03:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.557+0000 7fe373e00640 1 -- 172.21.15.26:0/2474197008 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3741a2190 con 0x7fe374102330 2024-06-13T07:03:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.560+0000 7fe36b400640 1 -- 172.21.15.26:0/2474197008 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe364042030 con 0x7fe374102330 2024-06-13T07:03:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.560+0000 7fe37c3f4740 1 Processor -- start 2024-06-13T07:03:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.560+0000 7fe37c3f4740 1 -- start start 2024-06-13T07:03:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 0x555d4c1281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c1287b0 0x555d4c12d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c12d920 0x555d4c12fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555d4c08b420 con 0x555d4c127dc0 2024-06-13T07:03:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d4c032c50 con 0x555d4c1287b0 2024-06-13T07:03:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555d4c0873e0 con 0x555d4c12d920 2024-06-13T07:03:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe372a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c1287b0 0x555d4c12d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe372000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c12d920 0x555d4c12fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe372000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c12d920 0x555d4c12fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44080/0 (socket says 172.21.15.26:44080) 2024-06-13T07:03:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe372000640 1 -- 172.21.15.26:0/1518329155 learned_addr learned my addr 172.21.15.26:0/1518329155 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.561+0000 7fe373400640 1 --2- 172.21.15.26:0/1518329155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 0x555d4c1281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.562+0000 7fe372a00640 1 -- 172.21.15.26:0/1518329155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c12d920 msgr2=0x555d4c12fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.562+0000 7fe372a00640 1 --2- 172.21.15.26:0/1518329155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c12d920 0x555d4c12fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.562+0000 7fe372a00640 1 -- 172.21.15.26:0/1518329155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 msgr2=0x555d4c1281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.562+0000 7fe372a00640 1 --2- 172.21.15.26:0/1518329155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 0x555d4c1281e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.562+0000 7fe372a00640 1 -- 172.21.15.26:0/1518329155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4bd9f210 con 0x555d4c1287b0 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.562+0000 7fe372a00640 1 --2- 172.21.15.26:0/1518329155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c1287b0 0x555d4c12d2b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe3640ed100 tx=0x7fe3640f3c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8654241562cb8266 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.562+0000 7fe36b400640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3580a2b80 0x7fe3580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.562+0000 7fe35f400640 1 -- 172.21.15.26:0/1518329155 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3640b80d0 con 0x555d4c1287b0 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.563+0000 7fe373400640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3580a2b80 0x7fe3580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.563+0000 7fe36b400640 1 -- 172.21.15.26:0/2474197008 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe3640efab0 con 0x7fe374102330 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.563+0000 7fe35f400640 1 -- 172.21.15.26:0/1518329155 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3640bc5c0 con 0x555d4c1287b0 2024-06-13T07:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.563+0000 7fe35f400640 1 -- 172.21.15.26:0/1518329155 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3640bb310 con 0x555d4c1287b0 2024-06-13T07:03:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1518329155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c1287b0 msgr2=0x555d4c12d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe373400640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3580a2b80 0x7fe3580a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x555d4c129010 tx=0x7fe36c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1518329155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c1287b0 0x555d4c12d2b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe3640ed100 tx=0x7fe3640f3c20 comp rx=0 tx=0).stop 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1518329155 shutdown_connections 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1518329155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c1287b0 0x555d4c12d2b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1518329155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c12d920 0x555d4c12fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1518329155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 0x555d4c1281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1518329155 >> 172.21.15.26:0/1518329155 conn(0x555d4c0effb0 msgr2=0x555d4c123840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1518329155 shutdown_connections 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1518329155 wait complete. 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.564+0000 7fe37c3f4740 1 -- start start 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe37c3f4740 1 Processor -- start 2024-06-13T07:03:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c131de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c132320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 0x555d4c139360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555d4c130690 con 0x555d4c12d920 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d4c0460b0 con 0x555d4c127dc0 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555d4c130810 con 0x555d4c1287b0 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe37c3f4740 10 client.0 _ll_register_callbacks cb 0x555d4c031f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe372a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c131de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe372a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c131de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39226/0 (socket says 172.21.15.26:39226) 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe372000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c132320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe372a00640 1 -- 172.21.15.26:0/1157533117 learned_addr learned my addr 172.21.15.26:0/1157533117 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.565+0000 7fe373400640 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 0x555d4c139360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.566+0000 7fe372a00640 1 -- 172.21.15.26:0/1157533117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 msgr2=0x555d4c132320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.566+0000 7fe372a00640 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c132320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.566+0000 7fe372a00640 1 -- 172.21.15.26:0/1157533117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 msgr2=0x555d4c139360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.566+0000 7fe372a00640 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 0x555d4c139360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.566+0000 7fe372a00640 1 -- 172.21.15.26:0/1157533117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4be6b620 con 0x555d4c127dc0 2024-06-13T07:03:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.566+0000 7fe372a00640 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c131de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3640bbe50 tx=0x7fe3640efd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c75a990afc9fed3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.566+0000 7fe353e00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:03:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.566+0000 7fe353e00640 1 -- 172.21.15.26:0/1157533117 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364049040 con 0x555d4c127dc0 2024-06-13T07:03:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.567+0000 7fe353e00640 1 -- 172.21.15.26:0/1157533117 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe36403b4d0 con 0x555d4c127dc0 2024-06-13T07:03:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.567+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1157533117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3640bc5c0 con 0x555d4c127dc0 2024-06-13T07:03:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.568+0000 7fe353e00640 1 -- 172.21.15.26:0/1157533117 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe3640bb640 con 0x555d4c127dc0 2024-06-13T07:03:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.568+0000 7fe37c3f4740 10 client.25694 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.568+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1157533117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d4c130690 con 0x555d4c127dc0 2024-06-13T07:03:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.568+0000 7fe353e00640 1 -- 172.21.15.26:0/1157533117 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3640bc5c0 con 0x555d4c127dc0 2024-06-13T07:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.568+0000 7fe353e00640 1 -- 172.21.15.26:0/1157533117 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fe3640bb310 con 0x555d4c127dc0 2024-06-13T07:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.568+0000 7fe37c3f4740 10 client.25694 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.568+0000 7fe37c3f4740 10 client.25694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.569+0000 7fe37c3f4740 20 client.25694 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.569+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d4c13da60 0x555d4c13fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.569+0000 7fe37c3f4740 4 client.25694 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.569+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1157533117 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d4c098b50 con 0x555d4c13da60 2024-06-13T07:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.570+0000 7fe372000640 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d4c13da60 0x555d4c13fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.572+0000 7fe372000640 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d4c13da60 0x555d4c13fe50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81bc21fdc5c8dc65 server_cookie=f3708c0276318c11 in_seq=0 out_seq=0 2024-06-13T07:03:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.572+0000 7fe353e00640 10 client.25694 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.573+0000 7fe353e00640 1 -- 172.21.15.26:0/1157533117 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555d4c098b50 con 0x555d4c13da60 2024-06-13T07:03:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.573+0000 7fe353e00640 10 client.25694 handle_command_reply: tid=0 2024-06-13T07:03:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.573+0000 7fe37c3f4740 1 client.25694 shutdown 2024-06-13T07:03:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.573+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1157533117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 msgr2=0x555d4c131de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.573+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c131de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3640bbe50 tx=0x7fe3640efd30 comp rx=0 tx=0).stop 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1157533117 shutdown_connections 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d4c13da60 0x555d4c13fe50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 0x555d4c139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c132320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe353e00640 0 client.25694 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/1157533117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c131de0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1157533117 >> 172.21.15.26:0/1157533117 conn(0x555d4c0effb0 msgr2=0x555d4c1229a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1157533117 shutdown_connections 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 1 -- 172.21.15.26:0/1157533117 wait complete. 2024-06-13T07:03:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.574+0000 7fe37c3f4740 20 client.25694 trim_cache size 0 max 16384 2024-06-13T07:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.594+0000 7fe37c3f4740 1 Processor -- start 2024-06-13T07:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.594+0000 7fe37c3f4740 1 -- start start 2024-06-13T07:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.594+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c122820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.594+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c122d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.594+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 0x555d4c139320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.594+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741aef70 con 0x555d4c12d920 2024-06-13T07:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.594+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741ac810 con 0x555d4c127dc0 2024-06-13T07:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.594+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741aa660 con 0x555d4c1287b0 2024-06-13T07:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c122820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39228/0 (socket says 172.21.15.26:39228) 2024-06-13T07:03:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372a00640 1 -- 172.21.15.26:0/2355247628 learned_addr learned my addr 172.21.15.26:0/2355247628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe373400640 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 0x555d4c139320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372000640 1 -- 172.21.15.26:0/2355247628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 msgr2=0x555d4c122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372000640 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372000640 1 -- 172.21.15.26:0/2355247628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 msgr2=0x555d4c139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372000640 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 0x555d4c139320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372000640 1 -- 172.21.15.26:0/2355247628 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4c1404d0 con 0x555d4c1287b0 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372a00640 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.595+0000 7fe372000640 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c122d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x555d4c1406a0 tx=0x7fe360005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45d84c23dfb520c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.596+0000 7fe35ea00640 1 -- 172.21.15.26:0/2355247628 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360004220 con 0x555d4c1287b0 2024-06-13T07:03:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.596+0000 7fe35ea00640 1 -- 172.21.15.26:0/2355247628 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3600043c0 con 0x555d4c1287b0 2024-06-13T07:03:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.596+0000 7fe35ea00640 1 -- 172.21.15.26:0/2355247628 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360011ba0 con 0x555d4c1287b0 2024-06-13T07:03:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2355247628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 msgr2=0x555d4c122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c122d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x555d4c1406a0 tx=0x7fe360005f90 comp rx=0 tx=0).stop 2024-06-13T07:03:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2355247628 shutdown_connections 2024-06-13T07:03:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c12d920 0x555d4c139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c122d60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2355247628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c127dc0 0x555d4c122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2355247628 >> 172.21.15.26:0/2355247628 conn(0x555d4c0effb0 msgr2=0x555d4c122110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2355247628 shutdown_connections 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2355247628 wait complete. 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.597+0000 7fe37c3f4740 1 -- start start 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe37c3f4740 1 Processor -- start 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 0x555d4c1360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c136620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe37c3f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c12d920 0x555d4c1a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741aacd0 con 0x555d4c127dc0 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741ac140 con 0x555d4c12d920 2024-06-13T07:03:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe37c3f4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741ab3a0 con 0x555d4c1287b0 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe37c3f4740 10 client.0 _ll_register_callbacks cb 0x7fe36c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe372a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 0x555d4c1360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe373400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c12d920 0x555d4c1a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.598+0000 7fe372000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe372000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c136620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44088/0 (socket says 172.21.15.26:44088) 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe372000640 1 -- 172.21.15.26:0/2728154271 learned_addr learned my addr 172.21.15.26:0/2728154271 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe373400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c12d920 0x555d4c1a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39240/0 (socket says 172.21.15.26:39240) 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe372000640 1 -- 172.21.15.26:0/2728154271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c12d920 msgr2=0x555d4c1a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe372000640 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c12d920 0x555d4c1a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe372000640 1 -- 172.21.15.26:0/2728154271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 msgr2=0x555d4c1360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe372000640 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 0x555d4c1360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe372000640 1 -- 172.21.15.26:0/2728154271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4c1a6c60 con 0x555d4c1287b0 2024-06-13T07:03:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe372000640 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c136620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe360009e90 tx=0x7fe360005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=294e734415a9cb53 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.599+0000 7fe353e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe353e00640 1 -- 172.21.15.26:0/2728154271 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360034070 con 0x555d4c1287b0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2728154271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3600043c0 con 0x555d4c1287b0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe353e00640 1 -- 172.21.15.26:0/2728154271 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe360004020 con 0x555d4c1287b0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe353e00640 1 -- 172.21.15.26:0/2728154271 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe360040050 con 0x555d4c1287b0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe37c3f4740 10 client.25747 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2728154271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d4c1a7e60 con 0x555d4c1287b0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe353e00640 1 -- 172.21.15.26:0/2728154271 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3600043c0 con 0x555d4c1287b0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe353e00640 1 -- 172.21.15.26:0/2728154271 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fe360003b40 con 0x555d4c1287b0 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe37c3f4740 10 client.25747 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe37c3f4740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe37c3f4740 20 client.25747 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.600+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d4c1a8820 0x555d4c1aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.601+0000 7fe37c3f4740 4 client.25747 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.601+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2728154271 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d4c092ea0 con 0x555d4c1a8820 2024-06-13T07:03:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.601+0000 7fe372a00640 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d4c1a8820 0x555d4c1aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.601+0000 7fe372a00640 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d4c1a8820 0x555d4c1aac40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c074471121fd93d0 server_cookie=8a25cf6f48161c55 in_seq=0 out_seq=0 2024-06-13T07:03:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.601+0000 7fe353e00640 10 client.25747 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:32 smithi136 ceph-mon[37229]: pgmap v600: 129 pgs: 129 active+clean; 41 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-06-13T07:03:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.670+0000 7fe353e00640 1 -- 172.21.15.26:0/2728154271 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555d4c092ea0 con 0x555d4c1a8820 2024-06-13T07:03:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.670+0000 7fe353e00640 10 client.25747 handle_command_reply: tid=0 2024-06-13T07:03:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.670+0000 7fe37c3f4740 1 client.25747 shutdown 2024-06-13T07:03:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2728154271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 msgr2=0x555d4c136620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c136620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe360009e90 tx=0x7fe360005760 comp rx=0 tx=0).stop 2024-06-13T07:03:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2728154271 shutdown_connections 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d4c1a8820 0x555d4c1aac40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d4c12d920 0x555d4c1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe353e00640 0 client.25747 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d4c1287b0 0x555d4c136620 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 --2- 172.21.15.26:0/2728154271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d4c127dc0 0x555d4c1360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2728154271 >> 172.21.15.26:0/2728154271 conn(0x555d4c0effb0 msgr2=0x555d4c1229a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2728154271 shutdown_connections 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 1 -- 172.21.15.26:0/2728154271 wait complete. 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.671+0000 7fe37c3f4740 20 client.25747 trim_cache size 0 max 16384 2024-06-13T07:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 -- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3580a2b80 msgr2=0x7fe3580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3580a2b80 0x7fe3580a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x555d4c129010 tx=0x7fe36c009600 comp rx=0 tx=0).stop 2024-06-13T07:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 -- 172.21.15.26:0/2474197008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe374102330 msgr2=0x7fe374199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe374102330 0x7fe374199db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe364009230 tx=0x7fe364031b60 comp rx=0 tx=0).stop 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 -- 172.21.15.26:0/2474197008 shutdown_connections 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3580a2b80 0x7fe3580a5040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe374107590 0x7fe3741a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe374104c60 0x7fe37419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 --2- 172.21.15.26:0/2474197008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe374102330 0x7fe374199db0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 -- 172.21.15.26:0/2474197008 >> 172.21.15.26:0/2474197008 conn(0x7fe3740fbce0 msgr2=0x7fe374101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 -- 172.21.15.26:0/2474197008 shutdown_connections 2024-06-13T07:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:32.672+0000 7fe373e00640 1 -- 172.21.15.26:0/2474197008 wait complete. 2024-06-13T07:03:33.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 87962 4.59 MB/sec warmup 107 sec latency 66.896 ms 2024-06-13T07:03:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:34 smithi026 ceph-mon[32041]: pgmap v601: 129 pgs: 129 active+clean; 43 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:03:34.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 89018 4.62 MB/sec warmup 108 sec latency 41.733 ms 2024-06-13T07:03:34.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:34 smithi135 ceph-mon[36276]: pgmap v601: 129 pgs: 129 active+clean; 43 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:03:34.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:34 smithi136 ceph-mon[37229]: pgmap v601: 129 pgs: 129 active+clean; 43 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:03:35.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:35 smithi026 ceph-mon[32041]: pgmap v602: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-06-13T07:03:35.513 INFO:tasks.workunit.client.0.smithi026.stdout: 1 89470 4.58 MB/sec warmup 109 sec latency 42.237 ms 2024-06-13T07:03:35.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:35 smithi135 ceph-mon[36276]: pgmap v602: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-06-13T07:03:35.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:35 smithi136 ceph-mon[37229]: pgmap v602: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-06-13T07:03:36.514 INFO:tasks.workunit.client.0.smithi026.stdout: 1 90490 4.58 MB/sec warmup 110 sec latency 36.789 ms 2024-06-13T07:03:37.514 INFO:tasks.workunit.client.0.smithi026.stdout: 1 91625 4.60 MB/sec warmup 111 sec latency 26.762 ms 2024-06-13T07:03:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:37 smithi136 ceph-mon[37229]: pgmap v603: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-06-13T07:03:38.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:37 smithi026 ceph-mon[32041]: pgmap v603: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-06-13T07:03:38.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:37 smithi135 ceph-mon[36276]: pgmap v603: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-06-13T07:03:38.514 INFO:tasks.workunit.client.0.smithi026.stdout: 1 92523 4.62 MB/sec warmup 112 sec latency 23.515 ms 2024-06-13T07:03:39.514 INFO:tasks.workunit.client.0.smithi026.stdout: 1 92991 4.59 MB/sec warmup 113 sec latency 46.324 ms 2024-06-13T07:03:39.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:39 smithi136 ceph-mon[37229]: pgmap v604: 129 pgs: 129 active+clean; 44 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-06-13T07:03:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:39 smithi026 ceph-mon[32041]: pgmap v604: 129 pgs: 129 active+clean; 44 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-06-13T07:03:40.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:39 smithi135 ceph-mon[36276]: pgmap v604: 129 pgs: 129 active+clean; 44 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-06-13T07:03:40.514 INFO:tasks.workunit.client.0.smithi026.stdout: 1 93918 4.59 MB/sec warmup 114 sec latency 45.028 ms 2024-06-13T07:03:41.514 INFO:tasks.workunit.client.0.smithi026.stdout: 1 95169 4.61 MB/sec warmup 115 sec latency 33.723 ms 2024-06-13T07:03:41.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:41 smithi026 ceph-mon[32041]: pgmap v605: 129 pgs: 129 active+clean; 44 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-13T07:03:41.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:41 smithi136 ceph-mon[37229]: pgmap v605: 129 pgs: 129 active+clean; 44 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-13T07:03:42.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:41 smithi135 ceph-mon[36276]: pgmap v605: 129 pgs: 129 active+clean; 44 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-13T07:03:42.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 96082 4.63 MB/sec warmup 116 sec latency 20.199 ms 2024-06-13T07:03:43.514 INFO:tasks.workunit.client.0.smithi026.stdout: 1 96502 4.59 MB/sec warmup 117 sec latency 27.872 ms 2024-06-13T07:03:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:43 smithi136 ceph-mon[37229]: pgmap v606: 129 pgs: 129 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:03:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:43 smithi026 ceph-mon[32041]: pgmap v606: 129 pgs: 129 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:03:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:43 smithi135 ceph-mon[36276]: pgmap v606: 129 pgs: 129 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:03:44.515 INFO:tasks.workunit.client.0.smithi026.stdout: 1 97375 4.60 MB/sec warmup 118 sec latency 29.689 ms 2024-06-13T07:03:45.515 INFO:tasks.workunit.client.0.smithi026.stdout: 1 98568 4.61 MB/sec warmup 119 sec latency 37.555 ms 2024-06-13T07:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 -- 172.21.15.26:0/2957023897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0102480 msgr2=0x7ff7d0104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 --2- 172.21.15.26:0/2957023897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0102480 0x7ff7d0104870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009900 tx=0x7ff7c002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 -- 172.21.15.26:0/2957023897 shutdown_connections 2024-06-13T07:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 --2- 172.21.15.26:0/2957023897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d01076e0 0x7ff7d0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 --2- 172.21.15.26:0/2957023897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d0104db0 0x7ff7d01071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 --2- 172.21.15.26:0/2957023897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0102480 0x7ff7d0104870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 -- 172.21.15.26:0/2957023897 >> 172.21.15.26:0/2957023897 conn(0x7ff7d00fbe90 msgr2=0x7ff7d00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 -- 172.21.15.26:0/2957023897 shutdown_connections 2024-06-13T07:03:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.834+0000 7ff7d7000640 1 -- 172.21.15.26:0/2957023897 wait complete. 2024-06-13T07:03:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.835+0000 7ff7d7000640 1 Processor -- start 2024-06-13T07:03:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7d7000640 1 -- start start 2024-06-13T07:03:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7d7000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d0102480 0x7ff7d0199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7d7000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0104db0 0x7ff7d019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7d7000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d01076e0 0x7ff7d01a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7d7000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010d680 con 0x7ff7d0102480 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7d7000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010d500 con 0x7ff7d01076e0 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7d7000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010d800 con 0x7ff7d0104db0 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d0102480 0x7ff7d0199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7cf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0104db0 0x7ff7d019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d0102480 0x7ff7d0199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39536/0 (socket says 172.21.15.26:39536) 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7cf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0104db0 0x7ff7d019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54918/0 (socket says 172.21.15.26:54918) 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7cfe00640 1 -- 172.21.15.26:0/4026627467 learned_addr learned my addr 172.21.15.26:0/4026627467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.836+0000 7ff7d5000640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d01076e0 0x7ff7d01a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7cf400640 1 -- 172.21.15.26:0/4026627467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d01076e0 msgr2=0x7ff7d01a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7cf400640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d01076e0 0x7ff7d01a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7cf400640 1 -- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d0102480 msgr2=0x7ff7d0199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7cf400640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d0102480 0x7ff7d0199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7cf400640 1 -- 172.21.15.26:0/4026627467 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d01a1a50 con 0x7ff7d0104db0 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7cf400640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0104db0 0x7ff7d019a370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00e9c0 tx=0x7ff7bc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7ccc00640 1 -- 172.21.15.26:0/4026627467 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc00ccc0 con 0x7ff7d0104db0 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7ccc00640 1 -- 172.21.15.26:0/4026627467 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7bc010910 con 0x7ff7d0104db0 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7ccc00640 1 -- 172.21.15.26:0/4026627467 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0188c0 con 0x7ff7d0104db0 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.837+0000 7ff7d7000640 1 -- 172.21.15.26:0/4026627467 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d01a1d40 con 0x7ff7d0104db0 2024-06-13T07:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.838+0000 7ff7d7000640 1 -- 172.21.15.26:0/4026627467 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d01a21e0 con 0x7ff7d0104db0 2024-06-13T07:03:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.841+0000 7ff7ccc00640 1 -- 172.21.15.26:0/4026627467 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff7bc010ac0 con 0x7ff7d0104db0 2024-06-13T07:03:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.842+0000 7ff7d9e22740 1 Processor -- start 2024-06-13T07:03:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.842+0000 7ff7d9e22740 1 -- start start 2024-06-13T07:03:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.842+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c73020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.842+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c735f0 0x563581c780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.842+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c7abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.842+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563581bd6420 con 0x563581c78760 2024-06-13T07:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.842+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563581b7dc50 con 0x563581c735f0 2024-06-13T07:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.842+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563581bd23e0 con 0x563581c72c00 2024-06-13T07:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7d5000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c73020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7d5000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c73020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54924/0 (socket says 172.21.15.26:54924) 2024-06-13T07:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7d5000640 1 -- 172.21.15.26:0/2365292341 learned_addr learned my addr 172.21.15.26:0/2365292341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c735f0 0x563581c780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7cf400640 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c7abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7d5000640 1 -- 172.21.15.26:0/2365292341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c735f0 msgr2=0x563581c780f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7d5000640 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c735f0 0x563581c780f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7d5000640 1 -- 172.21.15.26:0/2365292341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 msgr2=0x563581c7abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7d5000640 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c7abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.843+0000 7ff7d5000640 1 -- 172.21.15.26:0/2365292341 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635818ea210 con 0x563581c72c00 2024-06-13T07:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.844+0000 7ff7d5000640 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c73020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009fd0 tx=0x7ff7c80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a45a91dcdad4b503 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.844+0000 7ff7ccc00640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7b40a2cb0 0x7ff7b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.844+0000 7ff7ccc00640 1 -- 172.21.15.26:0/4026627467 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff7bc0cf750 con 0x7ff7d0104db0 2024-06-13T07:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.844+0000 7ff7c5600640 1 -- 172.21.15.26:0/2365292341 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8004d20 con 0x563581c72c00 2024-06-13T07:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.844+0000 7ff7c5600640 1 -- 172.21.15.26:0/2365292341 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c8004050 con 0x563581c72c00 2024-06-13T07:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.844+0000 7ff7c5600640 1 -- 172.21.15.26:0/2365292341 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8005260 con 0x563581c72c00 2024-06-13T07:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.844+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7b40a2cb0 0x7ff7b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.845+0000 7ff7d9e22740 1 -- 172.21.15.26:0/2365292341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 msgr2=0x563581c73020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.845+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7b40a2cb0 0x7ff7b40a5170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0031440 tx=0x7ff7c00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.845+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c73020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009fd0 tx=0x7ff7c80118d0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.845+0000 7ff7d9e22740 1 -- 172.21.15.26:0/2365292341 shutdown_connections 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.845+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c735f0 0x563581c780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.845+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c7abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.845+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/2365292341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c73020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.845+0000 7ff7d9e22740 1 -- 172.21.15.26:0/2365292341 >> 172.21.15.26:0/2365292341 conn(0x563581c3afb0 msgr2=0x563581c6e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.846+0000 7ff7d9e22740 1 -- 172.21.15.26:0/2365292341 shutdown_connections 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.846+0000 7ff7d9e22740 1 -- 172.21.15.26:0/2365292341 wait complete. 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.846+0000 7ff7d9e22740 1 -- start start 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.846+0000 7ff7d9e22740 1 Processor -- start 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c7d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c7d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c84500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563581c7b710 con 0x563581c78760 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563581b910b0 con 0x563581c72c00 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563581c7b890 con 0x563581c735f0 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7d9e22740 10 client.0 _ll_register_callbacks cb 0x563581b7cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7cf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c7d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7cf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c7d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54932/0 (socket says 172.21.15.26:54932) 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7cf400640 1 -- 172.21.15.26:0/204206060 learned_addr learned my addr 172.21.15.26:0/204206060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7d5000640 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c84500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c7d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7cf400640 1 -- 172.21.15.26:0/204206060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 msgr2=0x563581c7d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7cf400640 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c7d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7cf400640 1 -- 172.21.15.26:0/204206060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 msgr2=0x563581c84500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.847+0000 7ff7cf400640 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c84500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7cf400640 1 -- 172.21.15.26:0/204206060 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635819b6620 con 0x563581c735f0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c7d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7cf400640 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c7d5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc09bba0 tx=0x7ff7bc09b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd5fa000ac9a8cc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7ba000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7ba000640 1 -- 172.21.15.26:0/204206060 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0a47d0 con 0x563581c735f0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7ba000640 1 -- 172.21.15.26:0/204206060 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff7bc0cfb40 con 0x563581c735f0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7d9e22740 1 -- 172.21.15.26:0/204206060 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c8004050 con 0x563581c735f0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7ba000640 1 -- 172.21.15.26:0/204206060 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff7bc0cfd40 con 0x563581c735f0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7d9e22740 10 client.25759 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.848+0000 7ff7d9e22740 1 -- 172.21.15.26:0/204206060 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563581c878e0 con 0x563581c735f0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7ba000640 1 -- 172.21.15.26:0/204206060 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c8004050 con 0x563581c735f0 2024-06-13T07:03:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7ba000640 1 -- 172.21.15.26:0/204206060 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7ff7bc097960 con 0x563581c735f0 2024-06-13T07:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7d9e22740 10 client.25759 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7d9e22740 10 client.25759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7d9e22740 20 client.25759 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563581c88f00 0x563581c8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7d9e22740 4 client.25759 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7d9e22740 1 -- 172.21.15.26:0/204206060 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563581be3b50 con 0x563581c88f00 2024-06-13T07:03:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.849+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563581c88f00 0x563581c8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.850+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563581c88f00 0x563581c8b350 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9cc8693de007526 server_cookie=24a9de3f85d565c3 in_seq=0 out_seq=0 2024-06-13T07:03:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.850+0000 7ff7ba000640 10 client.25759 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7ba000640 1 -- 172.21.15.26:0/204206060 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563581be3b50 con 0x563581c88f00 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7ba000640 10 client.25759 handle_command_reply: tid=0 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 client.25759 shutdown 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 -- 172.21.15.26:0/204206060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 msgr2=0x563581c7d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c7d5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc09bba0 tx=0x7ff7bc09b570 comp rx=0 tx=0).stop 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 -- 172.21.15.26:0/204206060 shutdown_connections 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563581c88f00 0x563581c8b350 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c84500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c7d5a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/204206060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c7d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7ba000640 0 client.25759 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 -- 172.21.15.26:0/204206060 >> 172.21.15.26:0/204206060 conn(0x563581c3afb0 msgr2=0x563581c6d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 -- 172.21.15.26:0/204206060 shutdown_connections 2024-06-13T07:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.851+0000 7ff7d9e22740 1 -- 172.21.15.26:0/204206060 wait complete. 2024-06-13T07:03:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.852+0000 7ff7d9e22740 20 client.25759 trim_cache size 0 max 16384 2024-06-13T07:03:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.872+0000 7ff7d9e22740 1 Processor -- start 2024-06-13T07:03:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.872+0000 7ff7d9e22740 1 -- start start 2024-06-13T07:03:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.872+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c71d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.872+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c735f0 0x563581c844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.872+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c78760 0x563581c8fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.872+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01af170 con 0x563581c735f0 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.872+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01aca10 con 0x563581c78760 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.872+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01aa860 con 0x563581c72c00 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c735f0 0x563581c844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c71d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7d5000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c78760 0x563581c8fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7d5000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c78760 0x563581c8fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55162/0 (socket says 172.21.15.26:55162) 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7d5000640 1 -- 172.21.15.26:0/10464614 learned_addr learned my addr 172.21.15.26:0/10464614 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c71d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54934/0 (socket says 172.21.15.26:54934) 2024-06-13T07:03:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cf400640 1 -- 172.21.15.26:0/10464614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 msgr2=0x563581c71d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cf400640 1 --2- 172.21.15.26:0/10464614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c71d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cf400640 1 -- 172.21.15.26:0/10464614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c78760 msgr2=0x563581c8fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cf400640 1 --2- 172.21.15.26:0/10464614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c78760 0x563581c8fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cf400640 1 -- 172.21.15.26:0/10464614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563581c77d10 con 0x563581c735f0 2024-06-13T07:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/10464614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c71d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.873+0000 7ff7cf400640 1 --2- 172.21.15.26:0/10464614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c735f0 0x563581c844c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc09b910 tx=0x7ff7bc09b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de38bf394645278e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.874+0000 7ff7c4c00640 1 -- 172.21.15.26:0/10464614 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc09aad0 con 0x563581c735f0 2024-06-13T07:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.874+0000 7ff7c4c00640 1 -- 172.21.15.26:0/10464614 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7bc09bc10 con 0x563581c735f0 2024-06-13T07:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.874+0000 7ff7c4c00640 1 -- 172.21.15.26:0/10464614 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc09b060 con 0x563581c735f0 2024-06-13T07:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 -- 172.21.15.26:0/10464614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c735f0 msgr2=0x563581c844c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/10464614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c735f0 0x563581c844c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc09b910 tx=0x7ff7bc09b6a0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 -- 172.21.15.26:0/10464614 shutdown_connections 2024-06-13T07:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/10464614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c78760 0x563581c8fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/10464614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c735f0 0x563581c844c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/10464614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c72c00 0x563581c71d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 -- 172.21.15.26:0/10464614 >> 172.21.15.26:0/10464614 conn(0x563581c3afb0 msgr2=0x563581c71650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 -- 172.21.15.26:0/10464614 shutdown_connections 2024-06-13T07:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 -- 172.21.15.26:0/10464614 wait complete. 2024-06-13T07:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.875+0000 7ff7d9e22740 1 -- start start 2024-06-13T07:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7d9e22740 1 Processor -- start 2024-06-13T07:03:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:45 smithi136 ceph-mon[37229]: pgmap v607: 129 pgs: 129 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-13T07:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c82160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7d9e22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c836b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01aaed0 con 0x563581c78760 2024-06-13T07:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01ac340 con 0x563581c72c00 2024-06-13T07:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7d9e22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01ab5a0 con 0x563581c735f0 2024-06-13T07:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7d9e22740 10 client.0 _ll_register_callbacks cb 0x7ff7bc00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.876+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c82160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c82160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55164/0 (socket says 172.21.15.26:55164) 2024-06-13T07:03:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cfe00640 1 -- 172.21.15.26:0/1120784630 learned_addr learned my addr 172.21.15.26:0/1120784630 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cf400640 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7d5000640 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c836b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cfe00640 1 -- 172.21.15.26:0/1120784630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 msgr2=0x563581c826a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c826a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cfe00640 1 -- 172.21.15.26:0/1120784630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 msgr2=0x563581c836b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c836b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cfe00640 1 -- 172.21.15.26:0/1120784630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563581c83ca0 con 0x563581c72c00 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.877+0000 7ff7cfe00640 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c82160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009a20 tx=0x7ff7c0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59b48c4b523438b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.878+0000 7ff7ba000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.878+0000 7ff7ba000640 1 -- 172.21.15.26:0/1120784630 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0035040 con 0x563581c72c00 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.878+0000 7ff7d9e22740 1 -- 172.21.15.26:0/1120784630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7bc09bc10 con 0x563581c72c00 2024-06-13T07:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.878+0000 7ff7ba000640 1 -- 172.21.15.26:0/1120784630 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff7c003d450 con 0x563581c72c00 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7ba000640 1 -- 172.21.15.26:0/1120784630 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff7c0003e50 con 0x563581c72c00 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7d9e22740 10 client.25718 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7d9e22740 1 -- 172.21.15.26:0/1120784630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563581cf4160 con 0x563581c72c00 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7ba000640 1 -- 172.21.15.26:0/1120784630 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7bc09bc10 con 0x563581c72c00 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7ba000640 1 -- 172.21.15.26:0/1120784630 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7ff7c005b810 con 0x563581c72c00 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7d9e22740 10 client.25718 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7d9e22740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7d9e22740 20 client.25718 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563581cf4b40 0x563581cf6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7d9e22740 4 client.25718 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.879+0000 7ff7d9e22740 1 -- 172.21.15.26:0/1120784630 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563581bddea0 con 0x563581cf4b40 2024-06-13T07:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.880+0000 7ff7cf400640 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563581cf4b40 0x563581cf6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.880+0000 7ff7cf400640 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563581cf4b40 0x563581cf6f60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714ff223aa05e53c server_cookie=61203db812e8ab0f in_seq=0 out_seq=0 2024-06-13T07:03:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.880+0000 7ff7ba000640 10 client.25718 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.931+0000 7ff7ba000640 1 -- 172.21.15.26:0/1120784630 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563581bddea0 con 0x563581cf4b40 2024-06-13T07:03:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.931+0000 7ff7ba000640 10 client.25718 handle_command_reply: tid=0 2024-06-13T07:03:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 client.25718 shutdown 2024-06-13T07:03:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 -- 172.21.15.26:0/1120784630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 msgr2=0x563581c82160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c82160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009a20 tx=0x7ff7c0009260 comp rx=0 tx=0).stop 2024-06-13T07:03:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 -- 172.21.15.26:0/1120784630 shutdown_connections 2024-06-13T07:03:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563581cf4b40 0x563581cf6f60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563581c78760 0x563581c836b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563581c735f0 0x563581c826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 --2- 172.21.15.26:0/1120784630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563581c72c00 0x563581c82160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7ba000640 0 client.25718 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 -- 172.21.15.26:0/1120784630 >> 172.21.15.26:0/1120784630 conn(0x563581c3afb0 msgr2=0x563581c6cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 -- 172.21.15.26:0/1120784630 shutdown_connections 2024-06-13T07:03:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 1 -- 172.21.15.26:0/1120784630 wait complete. 2024-06-13T07:03:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.932+0000 7ff7d9e22740 20 client.25718 trim_cache size 0 max 16384 2024-06-13T07:03:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:03:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:03:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:03:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 -- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7b40a2cb0 msgr2=0x7ff7b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7b40a2cb0 0x7ff7b40a5170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0031440 tx=0x7ff7c00315b0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 -- 172.21.15.26:0/4026627467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0104db0 msgr2=0x7ff7d019a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0104db0 0x7ff7d019a370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00e9c0 tx=0x7ff7bc00ee80 comp rx=0 tx=0).stop 2024-06-13T07:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 -- 172.21.15.26:0/4026627467 shutdown_connections 2024-06-13T07:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff7b40a2cb0 0x7ff7b40a5170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d01076e0 0x7ff7d01a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d0104db0 0x7ff7d019a370 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 --2- 172.21.15.26:0/4026627467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d0102480 0x7ff7d0199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.933+0000 7ff7d7000640 1 -- 172.21.15.26:0/4026627467 >> 172.21.15.26:0/4026627467 conn(0x7ff7d00fbe90 msgr2=0x7ff7d0101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.934+0000 7ff7d7000640 1 -- 172.21.15.26:0/4026627467 shutdown_connections 2024-06-13T07:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:45.934+0000 7ff7d7000640 1 -- 172.21.15.26:0/4026627467 wait complete. 2024-06-13T07:03:46.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:45 smithi026 ceph-mon[32041]: pgmap v607: 129 pgs: 129 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-13T07:03:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:45 smithi135 ceph-mon[36276]: pgmap v607: 129 pgs: 129 active+clean; 47 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-13T07:03:47.515 INFO:tasks.workunit.client.0.smithi026.stdout: 1 100040 0.52 MB/sec execute 1 sec latency 34.723 ms 2024-06-13T07:03:47.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:47 smithi136 ceph-mon[37229]: pgmap v608: 129 pgs: 129 active+clean; 46 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:03:48.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:47 smithi026 ceph-mon[32041]: pgmap v608: 129 pgs: 129 active+clean; 46 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:03:48.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:47 smithi135 ceph-mon[36276]: pgmap v608: 129 pgs: 129 active+clean; 46 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:03:48.515 INFO:tasks.workunit.client.0.smithi026.stdout: 1 100919 2.73 MB/sec execute 2 sec latency 22.016 ms 2024-06-13T07:03:49.515 INFO:tasks.workunit.client.0.smithi026.stdout: 1 101957 2.79 MB/sec execute 3 sec latency 44.727 ms 2024-06-13T07:03:49.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:49 smithi136 ceph-mon[37229]: pgmap v609: 129 pgs: 129 active+clean; 47 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:03:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:49 smithi026 ceph-mon[32041]: pgmap v609: 129 pgs: 129 active+clean; 47 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:03:50.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:49 smithi135 ceph-mon[36276]: pgmap v609: 129 pgs: 129 active+clean; 47 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:03:50.515 INFO:tasks.workunit.client.0.smithi026.stdout: 1 103121 4.78 MB/sec execute 4 sec latency 28.811 ms 2024-06-13T07:03:51.515 INFO:tasks.workunit.client.0.smithi026.stdout: 1 103491 3.89 MB/sec execute 5 sec latency 22.636 ms 2024-06-13T07:03:51.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:51 smithi136 ceph-mon[37229]: pgmap v610: 129 pgs: 129 active+clean; 47 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:03:52.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:51 smithi026 ceph-mon[32041]: pgmap v610: 129 pgs: 129 active+clean; 47 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:03:52.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:51 smithi135 ceph-mon[36276]: pgmap v610: 129 pgs: 129 active+clean; 47 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:03:52.516 INFO:tasks.workunit.client.0.smithi026.stdout: 1 104362 4.08 MB/sec execute 6 sec latency 35.429 ms 2024-06-13T07:03:53.516 INFO:tasks.workunit.client.0.smithi026.stdout: 1 105378 3.87 MB/sec execute 7 sec latency 22.395 ms 2024-06-13T07:03:53.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:53 smithi136 ceph-mon[37229]: pgmap v611: 129 pgs: 129 active+clean; 44 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:03:54.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:53 smithi026 ceph-mon[32041]: pgmap v611: 129 pgs: 129 active+clean; 44 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:03:54.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:53 smithi135 ceph-mon[36276]: pgmap v611: 129 pgs: 129 active+clean; 44 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:03:54.516 INFO:tasks.workunit.client.0.smithi026.stdout: 1 106465 4.61 MB/sec execute 8 sec latency 22.156 ms 2024-06-13T07:03:55.516 INFO:tasks.workunit.client.0.smithi026.stdout: 1 106959 4.28 MB/sec execute 9 sec latency 51.931 ms 2024-06-13T07:03:55.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:55 smithi136 ceph-mon[37229]: pgmap v612: 129 pgs: 129 active+clean; 45 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:03:56.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:55 smithi026 ceph-mon[32041]: pgmap v612: 129 pgs: 129 active+clean; 45 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:03:56.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:55 smithi135 ceph-mon[36276]: pgmap v612: 129 pgs: 129 active+clean; 45 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:03:56.516 INFO:tasks.workunit.client.0.smithi026.stdout: 1 107701 4.07 MB/sec execute 10 sec latency 25.961 ms 2024-06-13T07:03:57.516 INFO:tasks.workunit.client.0.smithi026.stdout: 1 108845 4.14 MB/sec execute 11 sec latency 33.816 ms 2024-06-13T07:03:58.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:57 smithi026 ceph-mon[32041]: pgmap v613: 129 pgs: 129 active+clean; 45 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-13T07:03:58.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:57 smithi135 ceph-mon[36276]: pgmap v613: 129 pgs: 129 active+clean; 45 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-13T07:03:58.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:57 smithi136 ceph-mon[37229]: pgmap v613: 129 pgs: 129 active+clean; 45 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-13T07:03:58.516 INFO:tasks.workunit.client.0.smithi026.stdout: 1 109809 4.42 MB/sec execute 12 sec latency 21.573 ms 2024-06-13T07:03:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3063686578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc04076360 msgr2=0x7fcc04076780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3063686578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc04076360 0x7fcc04076780 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009900 tx=0x7fcbf802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3063686578 shutdown_connections 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3063686578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc04077f40 0x7fcc040787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3063686578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc040775a0 0x7fcc04077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3063686578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc04076360 0x7fcc04076780 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3063686578 >> 172.21.15.26:0/3063686578 conn(0x7fcc040fefd0 msgr2=0x7fcc04101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3063686578 shutdown_connections 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.091+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3063686578 wait complete. 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.092+0000 7fcc0ac00640 1 Processor -- start 2024-06-13T07:03:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc0ac00640 1 -- start start 2024-06-13T07:03:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc0ac00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04076360 0x7fcc0419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc0ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040775a0 0x7fcc0419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc0ac00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc04077f40 0x7fcc041a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc0ac00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04113af0 con 0x7fcc04076360 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc0ac00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04113970 con 0x7fcc04077f40 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc0ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04113c70 con 0x7fcc040775a0 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc09800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04076360 0x7fcc0419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc09800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04076360 0x7fcc0419e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52446/0 (socket says 172.21.15.26:52446) 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc09800640 1 -- 172.21.15.26:0/3832407138 learned_addr learned my addr 172.21.15.26:0/3832407138 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc08e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040775a0 0x7fcc0419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.093+0000 7fcc0a200640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc04077f40 0x7fcc041a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc08e00640 1 -- 172.21.15.26:0/3832407138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc04077f40 msgr2=0x7fcc041a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc08e00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc04077f40 0x7fcc041a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc08e00640 1 -- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04076360 msgr2=0x7fcc0419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc08e00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04076360 0x7fcc0419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc08e00640 1 -- 172.21.15.26:0/3832407138 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc041a5d50 con 0x7fcc040775a0 2024-06-13T07:03:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc09800640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04076360 0x7fcc0419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc08e00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040775a0 0x7fcc0419e670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000e9f0 tx=0x7fcbf000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc02000640 1 -- 172.21.15.26:0/3832407138 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000ccc0 con 0x7fcc040775a0 2024-06-13T07:03:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc02000640 1 -- 172.21.15.26:0/3832407138 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf0010910 con 0x7fcc040775a0 2024-06-13T07:03:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc02000640 1 -- 172.21.15.26:0/3832407138 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0018960 con 0x7fcc040775a0 2024-06-13T07:03:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.094+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3832407138 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc041a5fe0 con 0x7fcc040775a0 2024-06-13T07:03:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.095+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3832407138 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc041a64b0 con 0x7fcc040775a0 2024-06-13T07:03:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.098+0000 7fcc02000640 1 -- 172.21.15.26:0/3832407138 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbf0010ac0 con 0x7fcc040775a0 2024-06-13T07:03:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.099+0000 7fcc0ef93740 1 Processor -- start 2024-06-13T07:03:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.099+0000 7fcc0ef93740 1 -- start start 2024-06-13T07:03:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.099+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a2991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 0x561c5a29e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a2a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561c5a1fc420 con 0x561c5a298dc0 2024-06-13T07:03:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561c5a1a3c50 con 0x561c5a29e920 2024-06-13T07:03:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561c5a1f83e0 con 0x561c5a2997b0 2024-06-13T07:03:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc0a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a2991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc0a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a2991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52452/0 (socket says 172.21.15.26:52452) 2024-06-13T07:03:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc0a200640 1 -- 172.21.15.26:0/3551980447 learned_addr learned my addr 172.21.15.26:0/3551980447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.100+0000 7fcc08e00640 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a2a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc0a200640 1 -- 172.21.15.26:0/3551980447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 msgr2=0x561c5a29e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc0a200640 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 0x561c5a29e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc0a200640 1 -- 172.21.15.26:0/3551980447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 msgr2=0x561c5a2a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc0a200640 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a2a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc0a200640 1 -- 172.21.15.26:0/3551980447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c59f10210 con 0x561c5a298dc0 2024-06-13T07:03:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc08e00640 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a2a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc02000640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbec0a2c00 0x7fcbec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc02000640 1 -- 172.21.15.26:0/3832407138 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcbf00cf770 con 0x7fcc040775a0 2024-06-13T07:03:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc0a200640 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a2991e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009fd0 tx=0x7fcbfc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cb6422b90bbf9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.101+0000 7fcc09800640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbec0a2c00 0x7fcbec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.102+0000 7fcbf6a00640 1 -- 172.21.15.26:0/3551980447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc004d20 con 0x561c5a298dc0 2024-06-13T07:03:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.102+0000 7fcbf6a00640 1 -- 172.21.15.26:0/3551980447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc004050 con 0x561c5a298dc0 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.102+0000 7fcbf6a00640 1 -- 172.21.15.26:0/3551980447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc005260 con 0x561c5a298dc0 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3551980447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 msgr2=0x561c5a2991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a2991e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009fd0 tx=0x7fcbfc0118d0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc09800640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbec0a2c00 0x7fcbec0a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x561c5a29a220 tx=0x7fcbf8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3551980447 shutdown_connections 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a2a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 0x561c5a29e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/3551980447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a2991e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3551980447 >> 172.21.15.26:0/3551980447 conn(0x561c5a260fb0 msgr2=0x561c5a294840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3551980447 shutdown_connections 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.103+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3551980447 wait complete. 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 1 -- start start 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 1 Processor -- start 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2a3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561c5a2a18d0 con 0x561c5a29e920 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561c5a1b70b0 con 0x561c5a2997b0 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561c5a2a1a50 con 0x561c5a298dc0 2024-06-13T07:03:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.104+0000 7fcc0ef93740 10 client.0 _ll_register_callbacks cb 0x561c5a1a2f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc09800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc0a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc08e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc09800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2a3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59594/0 (socket says 172.21.15.26:59594) 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc09800640 1 -- 172.21.15.26:0/1710805507 learned_addr learned my addr 172.21.15.26:0/1710805507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc0a200640 1 -- 172.21.15.26:0/1710805507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 msgr2=0x561c5a2a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc0a200640 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc0a200640 1 -- 172.21.15.26:0/1710805507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 msgr2=0x561c5a2a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc0a200640 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc0a200640 1 -- 172.21.15.26:0/1710805507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c59fdc620 con 0x561c5a29e920 2024-06-13T07:03:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.105+0000 7fcc09800640 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.106+0000 7fcc0a200640 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2aa6d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc01b040 tx=0x7fcbfc004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5092e03e772d65 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.106+0000 7fcbeb400640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.106+0000 7fcbeb400640 1 -- 172.21.15.26:0/1710805507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc028020 con 0x561c5a29e920 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.106+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1710805507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbfc004050 con 0x561c5a29e920 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.106+0000 7fcbeb400640 1 -- 172.21.15.26:0/1710805507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcbfc004410 con 0x561c5a29e920 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.107+0000 7fcbeb400640 1 -- 172.21.15.26:0/1710805507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcbfc033050 con 0x561c5a29e920 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.107+0000 7fcc0ef93740 10 client.15639 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.107+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1710805507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c5a2ada90 con 0x561c5a29e920 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.107+0000 7fcbeb400640 1 -- 172.21.15.26:0/1710805507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc004050 con 0x561c5a29e920 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.107+0000 7fcbeb400640 1 -- 172.21.15.26:0/1710805507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fcbfc00f4e0 con 0x561c5a29e920 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.108+0000 7fcc0ef93740 10 client.15639 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.108+0000 7fcc0ef93740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.108+0000 7fcc0ef93740 20 client.15639 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.108+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561c5a2af080 0x561c5a2b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.108+0000 7fcc0ef93740 4 client.15639 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.108+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1710805507 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c5a209b50 con 0x561c5a2af080 2024-06-13T07:03:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.108+0000 7fcc09800640 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561c5a2af080 0x561c5a2b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.109+0000 7fcc09800640 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561c5a2af080 0x561c5a2b14a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a844433192207a99 server_cookie=1782aea22b23fc3e in_seq=0 out_seq=0 2024-06-13T07:03:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.109+0000 7fcbeb400640 10 client.15639 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcbeb400640 1 -- 172.21.15.26:0/1710805507 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561c5a209b50 con 0x561c5a2af080 2024-06-13T07:03:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcbeb400640 10 client.15639 handle_command_reply: tid=0 2024-06-13T07:03:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 client.15639 shutdown 2024-06-13T07:03:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1710805507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 msgr2=0x561c5a2aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2aa6d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc01b040 tx=0x7fcbfc004d60 comp rx=0 tx=0).stop 2024-06-13T07:03:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1710805507 shutdown_connections 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561c5a2af080 0x561c5a2b14a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2aa6d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1710805507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcbeb400640 0 client.15639 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1710805507 >> 172.21.15.26:0/1710805507 conn(0x561c5a260fb0 msgr2=0x561c5a2939a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1710805507 shutdown_connections 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1710805507 wait complete. 2024-06-13T07:03:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.110+0000 7fcc0ef93740 20 client.15639 trim_cache size 0 max 16384 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.130+0000 7fcc0ef93740 1 Processor -- start 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.130+0000 7fcc0ef93740 1 -- start start 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.130+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a294a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.130+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 0x561c5a2aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.130+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a297810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.130+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041b35d0 con 0x561c5a298dc0 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041b0e70 con 0x561c5a29e920 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041aecc0 con 0x561c5a2997b0 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc09800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a294a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc0a200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a297810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc09800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a294a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52484/0 (socket says 172.21.15.26:52484) 2024-06-13T07:03:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc09800640 1 -- 172.21.15.26:0/3698840809 learned_addr learned my addr 172.21.15.26:0/3698840809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc08e00640 1 --2- 172.21.15.26:0/3698840809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 0x561c5a2aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc0a200640 1 -- 172.21.15.26:0/3698840809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 msgr2=0x561c5a2aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc0a200640 1 --2- 172.21.15.26:0/3698840809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 0x561c5a2aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc0a200640 1 -- 172.21.15.26:0/3698840809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 msgr2=0x561c5a294a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc0a200640 1 --2- 172.21.15.26:0/3698840809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a294a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.131+0000 7fcc0a200640 1 -- 172.21.15.26:0/3698840809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c5a290000 con 0x561c5a29e920 2024-06-13T07:03:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.132+0000 7fcc0a200640 1 --2- 172.21.15.26:0/3698840809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a297810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc006010 tx=0x7fcbfc005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417f08e6f047cfe8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.132+0000 7fcbf6000640 1 -- 172.21.15.26:0/3698840809 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc028040 con 0x561c5a29e920 2024-06-13T07:03:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.132+0000 7fcbf6000640 1 -- 172.21.15.26:0/3698840809 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc011dd0 con 0x561c5a29e920 2024-06-13T07:03:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.132+0000 7fcbf6000640 1 -- 172.21.15.26:0/3698840809 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc019690 con 0x561c5a29e920 2024-06-13T07:03:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3698840809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 msgr2=0x561c5a297810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/3698840809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a297810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc006010 tx=0x7fcbfc005cf0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3698840809 shutdown_connections 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/3698840809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a29e920 0x561c5a297810 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/3698840809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a2997b0 0x561c5a2aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/3698840809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a298dc0 0x561c5a294a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3698840809 >> 172.21.15.26:0/3698840809 conn(0x561c5a260fb0 msgr2=0x561c5a294320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3698840809 shutdown_connections 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.133+0000 7fcc0ef93740 1 -- 172.21.15.26:0/3698840809 wait complete. 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 1 -- start start 2024-06-13T07:03:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 1 Processor -- start 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041af330 con 0x561c5a29e920 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041b07a0 con 0x561c5a2997b0 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041afa00 con 0x561c5a298dc0 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.134+0000 7fcc0ef93740 10 client.0 _ll_register_callbacks cb 0x7fcbf00d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2aa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59610/0 (socket says 172.21.15.26:59610) 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 -- 172.21.15.26:0/1522463701 learned_addr learned my addr 172.21.15.26:0/1522463701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc0a200640 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc08e00640 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 -- 172.21.15.26:0/1522463701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 msgr2=0x561c5a2a6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 -- 172.21.15.26:0/1522463701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 msgr2=0x561c5a2a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 -- 172.21.15.26:0/1522463701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c5a317d10 con 0x561c5a298dc0 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc08e00640 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcc09800640 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2aa440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80096c0 tx=0x7fcbf803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f50e5e293071566 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.135+0000 7fcbeb400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcbeb400640 1 -- 172.21.15.26:0/1522463701 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8035040 con 0x561c5a298dc0 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcbeb400640 1 -- 172.21.15.26:0/1522463701 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcbf8003770 con 0x561c5a298dc0 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1522463701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbfc011dd0 con 0x561c5a298dc0 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcbeb400640 1 -- 172.21.15.26:0/1522463701 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcbf8003970 con 0x561c5a298dc0 2024-06-13T07:03:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 10 client.25780 fetch_fsmap learned FSMap version 32 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1522463701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c5a318f40 con 0x561c5a298dc0 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcbeb400640 1 -- 172.21.15.26:0/1522463701 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc011dd0 con 0x561c5a298dc0 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcbeb400640 1 -- 172.21.15.26:0/1522463701 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fcbf803cc90 con 0x561c5a298dc0 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 10 client.25780 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 20 client.25780 populate_metadata read hostname 'smithi026' 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561c5a319930 0x561c5a31bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 4 client.25780 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.136+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1522463701 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c5a203ea0 con 0x561c5a319930 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.137+0000 7fcc08e00640 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561c5a319930 0x561c5a31bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.137+0000 7fcc08e00640 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561c5a319930 0x561c5a31bd50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97a37cc67e5362e server_cookie=c3644bb44dbe56f1 in_seq=0 out_seq=0 2024-06-13T07:03:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.137+0000 7fcbeb400640 10 client.25780 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.197+0000 7fcbeb400640 1 -- 172.21.15.26:0/1522463701 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561c5a203ea0 con 0x561c5a319930 2024-06-13T07:03:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.197+0000 7fcbeb400640 10 client.25780 handle_command_reply: tid=0 2024-06-13T07:03:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.197+0000 7fcc0ef93740 1 client.25780 shutdown 2024-06-13T07:03:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1522463701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 msgr2=0x561c5a2aa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2aa440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80096c0 tx=0x7fcbf803d000 comp rx=0 tx=0).stop 2024-06-13T07:03:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1522463701 shutdown_connections 2024-06-13T07:03:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561c5a319930 0x561c5a31bd50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c5a2997b0 0x561c5a2a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c5a29e920 0x561c5a2a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 --2- 172.21.15.26:0/1522463701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c5a298dc0 0x561c5a2aa440 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcbeb400640 0 client.25780 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1522463701 >> 172.21.15.26:0/1522463701 conn(0x561c5a260fb0 msgr2=0x561c5a2942c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1522463701 shutdown_connections 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 1 -- 172.21.15.26:0/1522463701 wait complete. 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.198+0000 7fcc0ef93740 20 client.25780 trim_cache size 0 max 16384 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:03:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:03:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:03:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbec0a2c00 msgr2=0x7fcbec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbec0a2c00 0x7fcbec0a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x561c5a29a220 tx=0x7fcbf8005e50 comp rx=0 tx=0).stop 2024-06-13T07:03:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3832407138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040775a0 msgr2=0x7fcc0419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040775a0 0x7fcc0419e670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000e9f0 tx=0x7fcbf000eeb0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3832407138 shutdown_connections 2024-06-13T07:03:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbec0a2c00 0x7fcbec0a50c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc04077f40 0x7fcc041a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040775a0 0x7fcc0419e670 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 --2- 172.21.15.26:0/3832407138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04076360 0x7fcc0419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.199+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3832407138 >> 172.21.15.26:0/3832407138 conn(0x7fcc040fefd0 msgr2=0x7fcc04100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.200+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3832407138 shutdown_connections 2024-06-13T07:03:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:03:59.200+0000 7fcc0ac00640 1 -- 172.21.15.26:0/3832407138 wait complete. 2024-06-13T07:03:59.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 110424 4.43 MB/sec execute 13 sec latency 44.185 ms 2024-06-13T07:04:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:03:59 smithi026 ceph-mon[32041]: pgmap v614: 129 pgs: 129 active+clean; 46 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-13T07:04:00.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:03:59 smithi135 ceph-mon[36276]: pgmap v614: 129 pgs: 129 active+clean; 46 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-13T07:04:00.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:03:59 smithi136 ceph-mon[37229]: pgmap v614: 129 pgs: 129 active+clean; 46 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-13T07:04:00.516 INFO:tasks.workunit.client.0.smithi026.stdout: 1 111142 4.26 MB/sec execute 14 sec latency 50.523 ms 2024-06-13T07:04:01.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 112335 4.30 MB/sec execute 15 sec latency 33.891 ms 2024-06-13T07:04:02.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:01 smithi026 ceph-mon[32041]: pgmap v615: 129 pgs: 129 active+clean; 47 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-13T07:04:02.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:01 smithi135 ceph-mon[36276]: pgmap v615: 129 pgs: 129 active+clean; 47 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-13T07:04:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:01 smithi136 ceph-mon[37229]: pgmap v615: 129 pgs: 129 active+clean; 47 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-13T07:04:02.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 113205 4.44 MB/sec execute 16 sec latency 28.097 ms 2024-06-13T07:04:03.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 113914 4.52 MB/sec execute 17 sec latency 33.051 ms 2024-06-13T07:04:04.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:03 smithi026 ceph-mon[32041]: pgmap v616: 129 pgs: 129 active+clean; 48 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:04:04.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:03 smithi135 ceph-mon[36276]: pgmap v616: 129 pgs: 129 active+clean; 48 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:04:04.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:03 smithi136 ceph-mon[37229]: pgmap v616: 129 pgs: 129 active+clean; 48 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:04:04.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 114417 4.30 MB/sec execute 18 sec latency 32.332 ms 2024-06-13T07:04:05.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 115511 4.33 MB/sec execute 19 sec latency 86.921 ms 2024-06-13T07:04:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:05 smithi026 ceph-mon[32041]: pgmap v617: 129 pgs: 129 active+clean; 48 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-13T07:04:06.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:05 smithi135 ceph-mon[36276]: pgmap v617: 129 pgs: 129 active+clean; 48 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-13T07:04:06.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:05 smithi136 ceph-mon[37229]: pgmap v617: 129 pgs: 129 active+clean; 48 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-13T07:04:06.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 116565 4.47 MB/sec execute 20 sec latency 32.752 ms 2024-06-13T07:04:07.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 117451 4.57 MB/sec execute 21 sec latency 22.064 ms 2024-06-13T07:04:08.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:07 smithi026 ceph-mon[32041]: pgmap v618: 129 pgs: 129 active+clean; 48 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:04:08.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:07 smithi135 ceph-mon[36276]: pgmap v618: 129 pgs: 129 active+clean; 48 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:04:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:07 smithi136 ceph-mon[37229]: pgmap v618: 129 pgs: 129 active+clean; 48 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:04:08.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 117973 4.39 MB/sec execute 22 sec latency 26.171 ms 2024-06-13T07:04:09.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 119069 4.41 MB/sec execute 23 sec latency 31.632 ms 2024-06-13T07:04:10.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:09 smithi026 ceph-mon[32041]: pgmap v619: 129 pgs: 129 active+clean; 50 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:04:10.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:09 smithi135 ceph-mon[36276]: pgmap v619: 129 pgs: 129 active+clean; 50 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:04:10.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:09 smithi136 ceph-mon[37229]: pgmap v619: 129 pgs: 129 active+clean; 50 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:04:10.517 INFO:tasks.workunit.client.0.smithi026.stdout: 1 119426 4.28 MB/sec execute 24 sec latency 847.842 ms 2024-06-13T07:04:11.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:10 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:04:11.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:10 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:04:11.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:10 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:04:11.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 119426 4.11 MB/sec execute 25 sec latency 1847.900 ms 2024-06-13T07:04:12.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: pgmap v620: 129 pgs: 129 active+clean; 50 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:04:12.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:04:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:11 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: pgmap v620: 129 pgs: 129 active+clean; 50 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:04:12.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:11 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: pgmap v620: 129 pgs: 129 active+clean; 50 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.130 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.131 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:04:12.131 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:04:12.131 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:11 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:04:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 -- 172.21.15.26:0/3950948845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3840102a50 msgr2=0x7f3840102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 --2- 172.21.15.26:0/3950948845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3840102a50 0x7f3840102ed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3828009900 tx=0x7f382802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 -- 172.21.15.26:0/3950948845 shutdown_connections 2024-06-13T07:04:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 --2- 172.21.15.26:0/3950948845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3840103410 0x7f3840109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 --2- 172.21.15.26:0/3950948845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3840102a50 0x7f3840102ed0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 --2- 172.21.15.26:0/3950948845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3840101860 0x7f3840101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 -- 172.21.15.26:0/3950948845 >> 172.21.15.26:0/3950948845 conn(0x7f38400fcdd0 msgr2=0x7f38400ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 -- 172.21.15.26:0/3950948845 shutdown_connections 2024-06-13T07:04:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.348+0000 7f383fe00640 1 -- 172.21.15.26:0/3950948845 wait complete. 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.349+0000 7f383fe00640 1 Processor -- start 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.349+0000 7f383fe00640 1 -- start start 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.349+0000 7f383fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3840101860 0x7f3840075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.349+0000 7f383fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3840102a50 0x7f3840073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.349+0000 7f383fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3840103410 0x7f38400742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.349+0000 7f383fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f384010d530 con 0x7f3840102a50 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.349+0000 7f383fe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f384010d3b0 con 0x7f3840103410 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.349+0000 7f383fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f384010d6b0 con 0x7f3840101860 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3840102a50 0x7f3840073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3840102a50 0x7f3840073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38044/0 (socket says 172.21.15.26:38044) 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383e000640 1 -- 172.21.15.26:0/4251798807 learned_addr learned my addr 172.21.15.26:0/4251798807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383f400640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3840103410 0x7f38400742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383ea00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3840101860 0x7f3840075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383e000640 1 -- 172.21.15.26:0/4251798807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3840101860 msgr2=0x7f3840075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383e000640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3840101860 0x7f3840075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383e000640 1 -- 172.21.15.26:0/4251798807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3840103410 msgr2=0x7f38400742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383e000640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3840103410 0x7f38400742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.350+0000 7f383e000640 1 -- 172.21.15.26:0/4251798807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3840074b60 con 0x7f3840102a50 2024-06-13T07:04:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.351+0000 7f383e000640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3840102a50 0x7f3840073d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3828031440 tx=0x7f3828032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.351+0000 7f3837400640 1 -- 172.21.15.26:0/4251798807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382803f030 con 0x7f3840102a50 2024-06-13T07:04:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.351+0000 7f383fe00640 1 -- 172.21.15.26:0/4251798807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38401a1c30 con 0x7f3840102a50 2024-06-13T07:04:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.351+0000 7f3837400640 1 -- 172.21.15.26:0/4251798807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3828003ce0 con 0x7f3840102a50 2024-06-13T07:04:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.351+0000 7f3837400640 1 -- 172.21.15.26:0/4251798807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828035070 con 0x7f3840102a50 2024-06-13T07:04:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.351+0000 7f383fe00640 1 -- 172.21.15.26:0/4251798807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38401a21d0 con 0x7f3840102a50 2024-06-13T07:04:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.354+0000 7f38485ea740 1 Processor -- start 2024-06-13T07:04:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.354+0000 7f3837400640 1 -- 172.21.15.26:0/4251798807 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3828042030 con 0x7f3840102a50 2024-06-13T07:04:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f38485ea740 1 -- start start 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f38485ea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3491dc0 0x5647b34921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f38485ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b34927b0 0x5647b34972b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f38485ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3497920 0x5647b3499db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f38485ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5647b33f5420 con 0x5647b34927b0 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f38485ea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647b339cc50 con 0x5647b3491dc0 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f38485ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5647b33f13e0 con 0x5647b3497920 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f383f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3491dc0 0x5647b34921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f383e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3497920 0x5647b3499db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f383ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b34927b0 0x5647b34972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f383f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3491dc0 0x5647b34921e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54916/0 (socket says 172.21.15.26:54916) 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.355+0000 7f383f400640 1 -- 172.21.15.26:0/1353980472 learned_addr learned my addr 172.21.15.26:0/1353980472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f383f400640 1 -- 172.21.15.26:0/1353980472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3497920 msgr2=0x5647b3499db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f383f400640 1 --2- 172.21.15.26:0/1353980472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3497920 0x5647b3499db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f383f400640 1 -- 172.21.15.26:0/1353980472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b34927b0 msgr2=0x5647b34972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f383f400640 1 --2- 172.21.15.26:0/1353980472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b34927b0 0x5647b34972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f383f400640 1 -- 172.21.15.26:0/1353980472 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b3109210 con 0x5647b3491dc0 2024-06-13T07:04:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f3837400640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f38240a2b80 0x7f38240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f383f400640 1 --2- 172.21.15.26:0/1353980472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3491dc0 0x5647b34921e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f383800b6b0 tx=0x7f383800bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71fa3b757ec40d9f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f3837400640 1 -- 172.21.15.26:0/4251798807 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f38280ef9d0 con 0x7f3840102a50 2024-06-13T07:04:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f382fe00640 1 -- 172.21.15.26:0/1353980472 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838003bd0 con 0x5647b3491dc0 2024-06-13T07:04:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f383ea00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f38240a2b80 0x7f38240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f382fe00640 1 -- 172.21.15.26:0/1353980472 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838015070 con 0x5647b3491dc0 2024-06-13T07:04:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.356+0000 7f382fe00640 1 -- 172.21.15.26:0/1353980472 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38380116a0 con 0x5647b3491dc0 2024-06-13T07:04:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.357+0000 7f38485ea740 1 -- 172.21.15.26:0/1353980472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3491dc0 msgr2=0x5647b34921e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.357+0000 7f38485ea740 1 --2- 172.21.15.26:0/1353980472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3491dc0 0x5647b34921e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f383800b6b0 tx=0x7f383800bb70 comp rx=0 tx=0).stop 2024-06-13T07:04:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.357+0000 7f383ea00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f38240a2b80 0x7f38240a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5647b3492bb0 tx=0x7f3830009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.357+0000 7f38485ea740 1 -- 172.21.15.26:0/1353980472 shutdown_connections 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.357+0000 7f38485ea740 1 --2- 172.21.15.26:0/1353980472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3497920 0x5647b3499db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.357+0000 7f38485ea740 1 --2- 172.21.15.26:0/1353980472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b34927b0 0x5647b34972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.357+0000 7f38485ea740 1 --2- 172.21.15.26:0/1353980472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3491dc0 0x5647b34921e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.357+0000 7f38485ea740 1 -- 172.21.15.26:0/1353980472 >> 172.21.15.26:0/1353980472 conn(0x5647b3459fb0 msgr2=0x5647b348d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.358+0000 7f38485ea740 1 -- 172.21.15.26:0/1353980472 shutdown_connections 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.358+0000 7f38485ea740 1 -- 172.21.15.26:0/1353980472 wait complete. 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.358+0000 7f38485ea740 1 -- start start 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.358+0000 7f38485ea740 1 Processor -- start 2024-06-13T07:04:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f38485ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b349c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f38485ea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b349c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f38485ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f38485ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5647b349a8d0 con 0x5647b3497920 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f38485ea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647b33b00b0 con 0x5647b34927b0 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f38485ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5647b349aa50 con 0x5647b3491dc0 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f38485ea740 10 client.0 _ll_register_callbacks cb 0x5647b339bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f383e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b349c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f383f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f383ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b349c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f383e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b349c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54924/0 (socket says 172.21.15.26:54924) 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f383e000640 1 -- 172.21.15.26:0/3715789341 learned_addr learned my addr 172.21.15.26:0/3715789341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f383e000640 1 -- 172.21.15.26:0/3715789341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 msgr2=0x5647b349c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.359+0000 7f383e000640 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b349c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f383e000640 1 -- 172.21.15.26:0/3715789341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 msgr2=0x5647b34a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f383e000640 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f383e000640 1 -- 172.21.15.26:0/3715789341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b31d5620 con 0x5647b34927b0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f383e000640 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b349c720 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38280bbe20 tx=0x7f38280f3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=227620cd68ef08e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f382cc00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f382cc00640 1 -- 172.21.15.26:0/3715789341 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828049070 con 0x5647b34927b0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f382cc00640 1 -- 172.21.15.26:0/3715789341 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f38280b8500 con 0x5647b34927b0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f38485ea740 1 -- 172.21.15.26:0/3715789341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3838015070 con 0x5647b34927b0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f382cc00640 1 -- 172.21.15.26:0/3715789341 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f38280b8130 con 0x5647b34927b0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f38485ea740 10 client.25748 fetch_fsmap learned FSMap version 32 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.360+0000 7f38485ea740 1 -- 172.21.15.26:0/3715789341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647b34a6ad0 con 0x5647b34927b0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f382cc00640 1 -- 172.21.15.26:0/3715789341 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838015070 con 0x5647b34927b0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f382cc00640 1 -- 172.21.15.26:0/3715789341 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f38280c45c0 con 0x5647b34927b0 2024-06-13T07:04:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f38485ea740 10 client.25748 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:04:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f38485ea740 10 client.25748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:04:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f38485ea740 20 client.25748 populate_metadata read hostname 'smithi026' 2024-06-13T07:04:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f38485ea740 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5647b34a82b0 0x5647b34aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f38485ea740 4 client.25748 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f38485ea740 1 -- 172.21.15.26:0/3715789341 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647b3402b50 con 0x5647b34a82b0 2024-06-13T07:04:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.361+0000 7f383ea00640 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5647b34a82b0 0x5647b34aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.362+0000 7f383ea00640 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5647b34a82b0 0x5647b34aa6f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89893ba1fb317b73 server_cookie=97c715d63fbb7abc in_seq=0 out_seq=0 2024-06-13T07:04:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.362+0000 7f382cc00640 10 client.25748 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f382cc00640 1 -- 172.21.15.26:0/3715789341 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5647b3402b50 con 0x5647b34a82b0 2024-06-13T07:04:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f382cc00640 10 client.25748 handle_command_reply: tid=0 2024-06-13T07:04:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 client.25748 shutdown 2024-06-13T07:04:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 -- 172.21.15.26:0/3715789341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 msgr2=0x5647b349c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b349c720 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38280bbe20 tx=0x7f38280f3040 comp rx=0 tx=0).stop 2024-06-13T07:04:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 -- 172.21.15.26:0/3715789341 shutdown_connections 2024-06-13T07:04:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5647b34a82b0 0x5647b34aa6f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b349c720 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 --2- 172.21.15.26:0/3715789341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b349c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f382cc00640 0 client.25748 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 -- 172.21.15.26:0/3715789341 >> 172.21.15.26:0/3715789341 conn(0x5647b3459fb0 msgr2=0x5647b348c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 -- 172.21.15.26:0/3715789341 shutdown_connections 2024-06-13T07:04:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.363+0000 7f38485ea740 1 -- 172.21.15.26:0/3715789341 wait complete. 2024-06-13T07:04:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.364+0000 7f38485ea740 20 client.25748 trim_cache size 0 max 16384 2024-06-13T07:04:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.381+0000 7f38485ea740 1 Processor -- start 2024-06-13T07:04:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.381+0000 7f38485ea740 1 -- start start 2024-06-13T07:04:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.381+0000 7f38485ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3491dc0 0x5647b348d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.381+0000 7f38485ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b34927b0 0x5647b34a3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.381+0000 7f38485ea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3497920 0x5647b3490500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.381+0000 7f38485ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38401af0d0 con 0x5647b3491dc0 2024-06-13T07:04:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.381+0000 7f38485ea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f38401ac970 con 0x5647b3497920 2024-06-13T07:04:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.381+0000 7f38485ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f38401aa7c0 con 0x5647b34927b0 2024-06-13T07:04:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3491dc0 0x5647b348d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3497920 0x5647b3490500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3491dc0 0x5647b348d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38068/0 (socket says 172.21.15.26:38068) 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383ea00640 1 -- 172.21.15.26:0/4087770881 learned_addr learned my addr 172.21.15.26:0/4087770881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3497920 0x5647b3490500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54928/0 (socket says 172.21.15.26:54928) 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383e000640 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b34927b0 0x5647b34a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383f400640 1 -- 172.21.15.26:0/4087770881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b34927b0 msgr2=0x5647b34a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383f400640 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b34927b0 0x5647b34a3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383f400640 1 -- 172.21.15.26:0/4087770881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3491dc0 msgr2=0x5647b348d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383f400640 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3491dc0 0x5647b348d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383f400640 1 -- 172.21.15.26:0/4087770881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b3496e20 con 0x5647b3497920 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.382+0000 7f383ea00640 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3491dc0 0x5647b348d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.383+0000 7f383f400640 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3497920 0x5647b3490500 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3838000c00 tx=0x7f3838005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5bd4379223fd7b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.383+0000 7f382f400640 1 -- 172.21.15.26:0/4087770881 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838003bd0 con 0x5647b3497920 2024-06-13T07:04:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.383+0000 7f382f400640 1 -- 172.21.15.26:0/4087770881 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838015070 con 0x5647b3497920 2024-06-13T07:04:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.383+0000 7f382f400640 1 -- 172.21.15.26:0/4087770881 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838011440 con 0x5647b3497920 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 -- 172.21.15.26:0/4087770881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3497920 msgr2=0x5647b3490500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3497920 0x5647b3490500 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3838000c00 tx=0x7f3838005870 comp rx=0 tx=0).stop 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 -- 172.21.15.26:0/4087770881 shutdown_connections 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b3497920 0x5647b3490500 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b34927b0 0x5647b34a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 --2- 172.21.15.26:0/4087770881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3491dc0 0x5647b348d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 -- 172.21.15.26:0/4087770881 >> 172.21.15.26:0/4087770881 conn(0x5647b3459fb0 msgr2=0x5647b348d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 -- 172.21.15.26:0/4087770881 shutdown_connections 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 -- 172.21.15.26:0/4087770881 wait complete. 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 -- start start 2024-06-13T07:04:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.384+0000 7f38485ea740 1 Processor -- start 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f38485ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b34a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f38485ea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b34a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f38485ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f38485ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38401aae30 con 0x5647b3497920 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f38485ea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f38401ac2a0 con 0x5647b34927b0 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f38485ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f38401ab500 con 0x5647b3491dc0 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f38485ea740 10 client.0 _ll_register_callbacks cb 0x7f38280f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f383ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b34a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f383ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b34a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55134/0 (socket says 172.21.15.26:55134) 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f383ea00640 1 -- 172.21.15.26:0/2713130833 learned_addr learned my addr 172.21.15.26:0/2713130833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f383e000640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b34a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.385+0000 7f383f400640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f383ea00640 1 -- 172.21.15.26:0/2713130833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 msgr2=0x5647b34a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f383ea00640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b34a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f383ea00640 1 -- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 msgr2=0x5647b34a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f383ea00640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f383ea00640 1 -- 172.21.15.26:0/2713130833 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b34a2c50 con 0x5647b3491dc0 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f383f400640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f383e000640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b34a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f383ea00640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b34a11d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f38401028b0 tx=0x7f3830004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8564002fbb02b099 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f382cc00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f382cc00640 1 -- 172.21.15.26:0/2713130833 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383000fd40 con 0x5647b3491dc0 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f38485ea740 1 -- 172.21.15.26:0/2713130833 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3838015070 con 0x5647b3491dc0 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.386+0000 7f382cc00640 1 -- 172.21.15.26:0/2713130833 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f3830004b20 con 0x5647b3491dc0 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f382cc00640 1 -- 172.21.15.26:0/2713130833 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3830004d20 con 0x5647b3491dc0 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f38485ea740 10 client.25798 fetch_fsmap learned FSMap version 32 2024-06-13T07:04:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f38485ea740 1 -- 172.21.15.26:0/2713130833 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647b3513270 con 0x5647b3491dc0 2024-06-13T07:04:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f382cc00640 1 -- 172.21.15.26:0/2713130833 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838015070 con 0x5647b3491dc0 2024-06-13T07:04:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f382cc00640 1 -- 172.21.15.26:0/2713130833 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f383002a4a0 con 0x5647b3491dc0 2024-06-13T07:04:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f38485ea740 10 client.25798 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:04:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f38485ea740 10 client.25798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:04:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f38485ea740 20 client.25798 populate_metadata read hostname 'smithi026' 2024-06-13T07:04:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f38485ea740 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5647b3513ca0 0x5647b35160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f38485ea740 4 client.25798 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:04:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.387+0000 7f38485ea740 1 -- 172.21.15.26:0/2713130833 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647b33fcea0 con 0x5647b3513ca0 2024-06-13T07:04:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.388+0000 7f383e000640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5647b3513ca0 0x5647b35160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.388+0000 7f383e000640 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5647b3513ca0 0x5647b35160c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a689183ccb0aaf00 server_cookie=4b4ea61d03be2f3a in_seq=0 out_seq=0 2024-06-13T07:04:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.388+0000 7f382cc00640 10 client.25798 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.466+0000 7f382cc00640 1 -- 172.21.15.26:0/2713130833 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5647b33fcea0 con 0x5647b3513ca0 2024-06-13T07:04:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.466+0000 7f382cc00640 10 client.25798 handle_command_reply: tid=0 2024-06-13T07:04:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.466+0000 7f38485ea740 1 client.25798 shutdown 2024-06-13T07:04:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 -- 172.21.15.26:0/2713130833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 msgr2=0x5647b34a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b34a11d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f38401028b0 tx=0x7f3830004a40 comp rx=0 tx=0).stop 2024-06-13T07:04:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 -- 172.21.15.26:0/2713130833 shutdown_connections 2024-06-13T07:04:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5647b3513ca0 0x5647b35160c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647b34927b0 0x5647b34a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647b3497920 0x5647b34a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 --2- 172.21.15.26:0/2713130833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647b3491dc0 0x5647b34a11d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f382cc00640 0 client.25798 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 -- 172.21.15.26:0/2713130833 >> 172.21.15.26:0/2713130833 conn(0x5647b3459fb0 msgr2=0x5647b348c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 -- 172.21.15.26:0/2713130833 shutdown_connections 2024-06-13T07:04:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 1 -- 172.21.15.26:0/2713130833 wait complete. 2024-06-13T07:04:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.467+0000 7f38485ea740 20 client.25798 trim_cache size 0 max 16384 2024-06-13T07:04:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:04:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:04:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:04:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.468+0000 7f383fe00640 1 -- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f38240a2b80 msgr2=0x7f38240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.468+0000 7f383fe00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f38240a2b80 0x7f38240a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5647b3492bb0 tx=0x7f3830009500 comp rx=0 tx=0).stop 2024-06-13T07:04:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.468+0000 7f383fe00640 1 -- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3840102a50 msgr2=0x7f3840073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.468+0000 7f383fe00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3840102a50 0x7f3840073d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3828031440 tx=0x7f3828032ce0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.469+0000 7f383fe00640 1 -- 172.21.15.26:0/4251798807 shutdown_connections 2024-06-13T07:04:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.469+0000 7f383fe00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f38240a2b80 0x7f38240a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.469+0000 7f383fe00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3840103410 0x7f38400742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.469+0000 7f383fe00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3840102a50 0x7f3840073d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.469+0000 7f383fe00640 1 --2- 172.21.15.26:0/4251798807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3840101860 0x7f3840075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.469+0000 7f383fe00640 1 -- 172.21.15.26:0/4251798807 >> 172.21.15.26:0/4251798807 conn(0x7f38400fcdd0 msgr2=0x7f3840105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.469+0000 7f383fe00640 1 -- 172.21.15.26:0/4251798807 shutdown_connections 2024-06-13T07:04:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:12.469+0000 7f383fe00640 1 -- 172.21.15.26:0/4251798807 wait complete. 2024-06-13T07:04:12.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 119493 3.96 MB/sec execute 26 sec latency 2725.123 ms 2024-06-13T07:04:13.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:04:13.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:04:13.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:04:13.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:04:13.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:04:13.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:04:13.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:04:13.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:04:13.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:04:13.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 119520 3.82 MB/sec execute 27 sec latency 979.736 ms 2024-06-13T07:04:14.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:13 smithi135 ceph-mon[36276]: pgmap v621: 129 pgs: 129 active+clean; 51 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:04:14.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:13 smithi136 ceph-mon[37229]: pgmap v621: 129 pgs: 129 active+clean; 51 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:04:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:13 smithi026 ceph-mon[32041]: pgmap v621: 129 pgs: 129 active+clean; 51 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:04:14.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 119581 3.70 MB/sec execute 28 sec latency 1758.399 ms 2024-06-13T07:04:15.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 120617 3.91 MB/sec execute 29 sec latency 125.965 ms 2024-06-13T07:04:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:15 smithi135 ceph-mon[36276]: pgmap v622: 129 pgs: 129 active+clean; 49 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-13T07:04:16.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:15 smithi136 ceph-mon[37229]: pgmap v622: 129 pgs: 129 active+clean; 49 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-13T07:04:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:15 smithi026 ceph-mon[32041]: pgmap v622: 129 pgs: 129 active+clean; 49 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-13T07:04:16.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 120617 3.78 MB/sec execute 30 sec latency 1081.633 ms 2024-06-13T07:04:17.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 120706 3.66 MB/sec execute 31 sec latency 1998.164 ms 2024-06-13T07:04:18.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:17 smithi136 ceph-mon[37229]: pgmap v623: 129 pgs: 129 active+clean; 49 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-06-13T07:04:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:17 smithi026 ceph-mon[32041]: pgmap v623: 129 pgs: 129 active+clean; 49 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-06-13T07:04:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:17 smithi135 ceph-mon[36276]: pgmap v623: 129 pgs: 129 active+clean; 49 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-06-13T07:04:18.518 INFO:tasks.workunit.client.0.smithi026.stdout: 1 120774 3.55 MB/sec execute 32 sec latency 864.144 ms 2024-06-13T07:04:19.519 INFO:tasks.workunit.client.0.smithi026.stdout: 1 120793 3.45 MB/sec execute 33 sec latency 1636.782 ms 2024-06-13T07:04:20.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:19 smithi136 ceph-mon[37229]: pgmap v624: 129 pgs: 129 active+clean; 47 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-13T07:04:20.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:19 smithi026 ceph-mon[32041]: pgmap v624: 129 pgs: 129 active+clean; 47 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-13T07:04:20.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:19 smithi135 ceph-mon[36276]: pgmap v624: 129 pgs: 129 active+clean; 47 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-13T07:04:20.519 INFO:tasks.workunit.client.0.smithi026.stdout: 1 121191 3.38 MB/sec execute 34 sec latency 212.074 ms 2024-06-13T07:04:21.519 INFO:tasks.workunit.client.0.smithi026.stdout: 1 122043 3.43 MB/sec execute 35 sec latency 70.993 ms 2024-06-13T07:04:22.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:21 smithi136 ceph-mon[37229]: pgmap v625: 129 pgs: 129 active+clean; 46 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 723 KiB/s wr, 89 op/s 2024-06-13T07:04:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:21 smithi026 ceph-mon[32041]: pgmap v625: 129 pgs: 129 active+clean; 46 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 723 KiB/s wr, 89 op/s 2024-06-13T07:04:22.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:21 smithi135 ceph-mon[36276]: pgmap v625: 129 pgs: 129 active+clean; 46 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 723 KiB/s wr, 89 op/s 2024-06-13T07:04:22.519 INFO:tasks.workunit.client.0.smithi026.stdout: 1 123111 3.41 MB/sec execute 36 sec latency 27.061 ms 2024-06-13T07:04:23.520 INFO:tasks.workunit.client.0.smithi026.stdout: 1 124007 3.50 MB/sec execute 37 sec latency 25.750 ms 2024-06-13T07:04:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:23 smithi136 ceph-mon[37229]: pgmap v626: 129 pgs: 129 active+clean; 47 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 983 KiB/s wr, 99 op/s 2024-06-13T07:04:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:23 smithi026 ceph-mon[32041]: pgmap v626: 129 pgs: 129 active+clean; 47 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 983 KiB/s wr, 99 op/s 2024-06-13T07:04:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:23 smithi135 ceph-mon[36276]: pgmap v626: 129 pgs: 129 active+clean; 47 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 983 KiB/s wr, 99 op/s 2024-06-13T07:04:24.519 INFO:tasks.workunit.client.0.smithi026.stdout: 1 124576 3.53 MB/sec execute 38 sec latency 23.375 ms 2024-06-13T07:04:25.519 INFO:tasks.workunit.client.0.smithi026.stdout: 1 125251 3.49 MB/sec execute 39 sec latency 67.692 ms 2024-06-13T07:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 -- 172.21.15.26:0/1496264815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0109be0 msgr2=0x7f94c0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 --2- 172.21.15.26:0/1496264815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0109be0 0x7f94c0110470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009ca0 tx=0x7f94b802a8c0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 -- 172.21.15.26:0/1496264815 shutdown_connections 2024-06-13T07:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 --2- 172.21.15.26:0/1496264815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0109be0 0x7f94c0110470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 --2- 172.21.15.26:0/1496264815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0109220 0x7f94c01096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 --2- 172.21.15.26:0/1496264815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c0108030 0x7f94c0108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 -- 172.21.15.26:0/1496264815 >> 172.21.15.26:0/1496264815 conn(0x7f94c0076ff0 msgr2=0x7f94c0079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 -- 172.21.15.26:0/1496264815 shutdown_connections 2024-06-13T07:04:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.623+0000 7f94c5c00640 1 -- 172.21.15.26:0/1496264815 wait complete. 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.624+0000 7f94c5c00640 1 Processor -- start 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94c5c00640 1 -- start start 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94c5c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0108030 0x7f94c019e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94c5c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0109220 0x7f94c019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94c5c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c0109be0 0x7f94c01a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94c5c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0113c80 con 0x7f94c0109220 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94c5c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0113b00 con 0x7f94c0108030 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94c5c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0113e00 con 0x7f94c0109be0 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94be000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0109220 0x7f94c019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94bea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0108030 0x7f94c019e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c0109be0 0x7f94c01a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94bea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0108030 0x7f94c019e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55530/0 (socket says 172.21.15.26:55530) 2024-06-13T07:04:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c0109be0 0x7f94c01a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50044/0 (socket says 172.21.15.26:50044) 2024-06-13T07:04:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94bf400640 1 -- 172.21.15.26:0/3710802433 learned_addr learned my addr 172.21.15.26:0/3710802433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.625+0000 7f94be000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0109220 0x7f94c019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40850/0 (socket says 172.21.15.26:40850) 2024-06-13T07:04:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.626+0000 7f94bea00640 1 -- 172.21.15.26:0/3710802433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c0109be0 msgr2=0x7f94c01a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.626+0000 7f94bea00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c0109be0 0x7f94c01a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.626+0000 7f94bea00640 1 -- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0109220 msgr2=0x7f94c019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.626+0000 7f94bea00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0109220 0x7f94c019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.626+0000 7f94bea00640 1 -- 172.21.15.26:0/3710802433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c01a5e30 con 0x7f94c0108030 2024-06-13T07:04:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.626+0000 7f94bea00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0108030 0x7f94c019e210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e9c0 tx=0x7f94b000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.626+0000 7f94b7400640 1 -- 172.21.15.26:0/3710802433 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b000ccc0 con 0x7f94c0108030 2024-06-13T07:04:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.627+0000 7f94b7400640 1 -- 172.21.15.26:0/3710802433 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b00035f0 con 0x7f94c0108030 2024-06-13T07:04:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.627+0000 7f94b7400640 1 -- 172.21.15.26:0/3710802433 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b000ce60 con 0x7f94c0108030 2024-06-13T07:04:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.627+0000 7f94c5c00640 1 -- 172.21.15.26:0/3710802433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c01a6120 con 0x7f94c0108030 2024-06-13T07:04:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.627+0000 7f94c5c00640 1 -- 172.21.15.26:0/3710802433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c01a66c0 con 0x7f94c0108030 2024-06-13T07:04:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94c8975740 1 Processor -- start 2024-06-13T07:04:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94b7400640 1 -- 172.21.15.26:0/3710802433 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94b00037a0 con 0x7f94c0108030 2024-06-13T07:04:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94c8975740 1 -- start start 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94c8975740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2a1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94c8975740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a15f0 0x55b0af2a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94c8975740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94c8975740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b0af204420 con 0x55b0af2a0c00 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94c8975740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b0af1abc50 con 0x55b0af2a15f0 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.631+0000 7f94c8975740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b0af2003e0 con 0x55b0af2a6760 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94bea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a15f0 0x55b0af2a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94bea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a15f0 0x55b0af2a60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55532/0 (socket says 172.21.15.26:55532) 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94bea00640 1 -- 172.21.15.26:0/2267185349 learned_addr learned my addr 172.21.15.26:0/2267185349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94be000640 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94bf400640 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94be000640 1 -- 172.21.15.26:0/2267185349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a15f0 msgr2=0x55b0af2a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94be000640 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a15f0 0x55b0af2a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94be000640 1 -- 172.21.15.26:0/2267185349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 msgr2=0x55b0af2a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94be000640 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94be000640 1 -- 172.21.15.26:0/2267185349 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0aef18210 con 0x55b0af2a6760 2024-06-13T07:04:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.632+0000 7f94bea00640 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a15f0 0x55b0af2a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.633+0000 7f94be000640 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2a8bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009bf0 tx=0x7f94ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f692e2d470b004d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.633+0000 7f94b7400640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94a40a2cb0 0x7f94a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.633+0000 7f94abe00640 1 -- 172.21.15.26:0/2267185349 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac009da0 con 0x55b0af2a6760 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.633+0000 7f94b7400640 1 -- 172.21.15.26:0/3710802433 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94b00cf740 con 0x7f94c0108030 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.633+0000 7f94abe00640 1 -- 172.21.15.26:0/2267185349 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac005020 con 0x55b0af2a6760 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.633+0000 7f94abe00640 1 -- 172.21.15.26:0/2267185349 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac004b90 con 0x55b0af2a6760 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.634+0000 7f94bf400640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94a40a2cb0 0x7f94a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.634+0000 7f94c8975740 1 -- 172.21.15.26:0/2267185349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 msgr2=0x55b0af2a8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.634+0000 7f94c8975740 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2a8bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009bf0 tx=0x7f94ac011810 comp rx=0 tx=0).stop 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.634+0000 7f94c8975740 1 -- 172.21.15.26:0/2267185349 shutdown_connections 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.634+0000 7f94c8975740 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a15f0 0x55b0af2a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.634+0000 7f94c8975740 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2a8bf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.634+0000 7f94c8975740 1 --2- 172.21.15.26:0/2267185349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.634+0000 7f94c8975740 1 -- 172.21.15.26:0/2267185349 >> 172.21.15.26:0/2267185349 conn(0x55b0af268fb0 msgr2=0x55b0af29c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.635+0000 7f94c8975740 1 -- 172.21.15.26:0/2267185349 shutdown_connections 2024-06-13T07:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.635+0000 7f94bf400640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94a40a2cb0 0x7f94a40a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94b8005e30 tx=0x7f94b803d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.635+0000 7f94c8975740 1 -- 172.21.15.26:0/2267185349 wait complete. 2024-06-13T07:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.635+0000 7f94c8975740 1 -- start start 2024-06-13T07:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94c8975740 1 Processor -- start 2024-06-13T07:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94c8975740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a0c00 0x55b0af2aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94c8975740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a15f0 0x55b0af2ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94c8975740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94c8975740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b0af2a9710 con 0x55b0af2a15f0 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94c8975740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b0af1bf0b0 con 0x55b0af2a0c00 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94c8975740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b0af2a9890 con 0x55b0af2a6760 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94c8975740 10 client.0 _ll_register_callbacks cb 0x55b0af1aaf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94be000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a15f0 0x55b0af2ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94be000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a15f0 0x55b0af2ab4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40878/0 (socket says 172.21.15.26:40878) 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94be000640 1 -- 172.21.15.26:0/1939684528 learned_addr learned my addr 172.21.15.26:0/1939684528 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.636+0000 7f94bea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a0c00 0x55b0af2aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94bf400640 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94bea00640 1 -- 172.21.15.26:0/1939684528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 msgr2=0x55b0af2b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94bea00640 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94bea00640 1 -- 172.21.15.26:0/1939684528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a15f0 msgr2=0x55b0af2ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94bea00640 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a15f0 0x55b0af2ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94bea00640 1 -- 172.21.15.26:0/1939684528 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0aefe4620 con 0x55b0af2a0c00 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94be000640 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a15f0 0x55b0af2ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94bea00640 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a0c00 0x55b0af2aaf90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f94b009bb40 tx=0x7f94b0097730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a7a8764dab5359a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.637+0000 7f94a8c00640 1 -- 172.21.15.26:0/1939684528 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b009b6f0 con 0x55b0af2a0c00 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94a8c00640 1 -- 172.21.15.26:0/1939684528 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94b00cf400 con 0x55b0af2a0c00 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94c8975740 1 -- 172.21.15.26:0/1939684528 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94ac005020 con 0x55b0af2a0c00 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94a8c00640 1 -- 172.21.15.26:0/1939684528 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f94b009b510 con 0x55b0af2a0c00 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94c8975740 10 client.25766 fetch_fsmap learned FSMap version 33 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94c8975740 1 -- 172.21.15.26:0/1939684528 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b0af2b5880 con 0x55b0af2a0c00 2024-06-13T07:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94a8c00640 1 -- 172.21.15.26:0/1939684528 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac005020 con 0x55b0af2a0c00 2024-06-13T07:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94a8c00640 1 -- 172.21.15.26:0/1939684528 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f94b00a4390 con 0x55b0af2a0c00 2024-06-13T07:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94c8975740 10 client.25766 fetch_fsmap finished waiting for FSMap version 33 2024-06-13T07:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94c8975740 10 client.25766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.638+0000 7f94c8975740 20 client.25766 populate_metadata read hostname 'smithi026' 2024-06-13T07:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.639+0000 7f94c8975740 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b0af2b6fb0 0x55b0af2b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.639+0000 7f94c8975740 4 client.25766 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.639+0000 7f94c8975740 1 -- 172.21.15.26:0/1939684528 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0af211b50 con 0x55b0af2b6fb0 2024-06-13T07:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.639+0000 7f94be000640 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b0af2b6fb0 0x55b0af2b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.640+0000 7f94be000640 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b0af2b6fb0 0x55b0af2b9400 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0e806927ea9d9f server_cookie=4657b0ff34ecf576 in_seq=0 out_seq=0 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.640+0000 7f94a8c00640 10 client.25766 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.642+0000 7f94a8c00640 1 -- 172.21.15.26:0/1939684528 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b0af211b50 con 0x55b0af2b6fb0 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.642+0000 7f94a8c00640 10 client.25766 handle_command_reply: tid=0 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.642+0000 7f94c8975740 1 client.25766 shutdown 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 -- 172.21.15.26:0/1939684528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a0c00 msgr2=0x55b0af2aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a0c00 0x55b0af2aaf90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f94b009bb40 tx=0x7f94b0097730 comp rx=0 tx=0).stop 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 -- 172.21.15.26:0/1939684528 shutdown_connections 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b0af2b6fb0 0x55b0af2b9400 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a6760 0x55b0af2b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a15f0 0x55b0af2ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94a8c00640 0 client.25766 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 --2- 172.21.15.26:0/1939684528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a0c00 0x55b0af2aaf90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 -- 172.21.15.26:0/1939684528 >> 172.21.15.26:0/1939684528 conn(0x55b0af268fb0 msgr2=0x55b0af29b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 -- 172.21.15.26:0/1939684528 shutdown_connections 2024-06-13T07:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 1 -- 172.21.15.26:0/1939684528 wait complete. 2024-06-13T07:04:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.643+0000 7f94c8975740 20 client.25766 trim_cache size 0 max 16384 2024-06-13T07:04:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.665+0000 7f94c8975740 1 Processor -- start 2024-06-13T07:04:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.665+0000 7f94c8975740 1 -- start start 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94c8975740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af29c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94c8975740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94c8975740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 0x55b0af29f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94c8975740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b35e0 con 0x55b0af2a0c00 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94c8975740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b0e80 con 0x55b0af2a6760 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94c8975740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01aecd0 con 0x55b0af2a15f0 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94bea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af29c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94be000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94be000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50060/0 (socket says 172.21.15.26:50060) 2024-06-13T07:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94be000640 1 -- 172.21.15.26:0/2630560750 learned_addr learned my addr 172.21.15.26:0/2630560750 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.666+0000 7f94bf400640 1 --2- 172.21.15.26:0/2630560750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 0x55b0af29f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94be000640 1 -- 172.21.15.26:0/2630560750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 msgr2=0x55b0af29f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94be000640 1 --2- 172.21.15.26:0/2630560750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 0x55b0af29f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94be000640 1 -- 172.21.15.26:0/2630560750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 msgr2=0x55b0af29c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94be000640 1 --2- 172.21.15.26:0/2630560750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af29c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94be000640 1 -- 172.21.15.26:0/2630560750 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0af2a19d0 con 0x55b0af2a15f0 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94be000640 1 --2- 172.21.15.26:0/2630560750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2b2460 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f94ac011d10 tx=0x7f94ac009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f93f8d73952e3f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94ab400640 1 -- 172.21.15.26:0/2630560750 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0097a0 con 0x55b0af2a15f0 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94ab400640 1 -- 172.21.15.26:0/2630560750 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac003ad0 con 0x55b0af2a15f0 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.667+0000 7f94ab400640 1 -- 172.21.15.26:0/2630560750 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac02b950 con 0x55b0af2a15f0 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.668+0000 7f94c8975740 1 -- 172.21.15.26:0/2630560750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 msgr2=0x55b0af2b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.668+0000 7f94c8975740 1 --2- 172.21.15.26:0/2630560750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2b2460 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f94ac011d10 tx=0x7f94ac009680 comp rx=0 tx=0).stop 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.668+0000 7f94c8975740 1 -- 172.21.15.26:0/2630560750 shutdown_connections 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.668+0000 7f94c8975740 1 --2- 172.21.15.26:0/2630560750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 0x55b0af29f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.668+0000 7f94c8975740 1 --2- 172.21.15.26:0/2630560750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2b2460 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.668+0000 7f94c8975740 1 --2- 172.21.15.26:0/2630560750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af29c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.668+0000 7f94c8975740 1 -- 172.21.15.26:0/2630560750 >> 172.21.15.26:0/2630560750 conn(0x55b0af268fb0 msgr2=0x55b0af2a2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.668+0000 7f94c8975740 1 -- 172.21.15.26:0/2630560750 shutdown_connections 2024-06-13T07:04:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.669+0000 7f94c8975740 1 -- 172.21.15.26:0/2630560750 wait complete. 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.669+0000 7f94c8975740 1 -- start start 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.669+0000 7f94c8975740 1 Processor -- start 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94c8975740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94c8975740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94c8975740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 0x55b0af2af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94c8975740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01af340 con 0x55b0af2a0c00 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94c8975740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b07b0 con 0x55b0af2a6760 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94c8975740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01afa10 con 0x55b0af2a15f0 2024-06-13T07:04:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94c8975740 10 client.0 _ll_register_callbacks cb 0x7f94b00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94be000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94be000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50062/0 (socket says 172.21.15.26:50062) 2024-06-13T07:04:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94be000640 1 -- 172.21.15.26:0/3320326715 learned_addr learned my addr 172.21.15.26:0/3320326715 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94bea00640 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.670+0000 7f94bf400640 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 0x55b0af2af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.671+0000 7f94be000640 1 -- 172.21.15.26:0/3320326715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 msgr2=0x55b0af2af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.671+0000 7f94be000640 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 0x55b0af2af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.671+0000 7f94be000640 1 -- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 msgr2=0x55b0af2b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.671+0000 7f94be000640 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.671+0000 7f94be000640 1 -- 172.21.15.26:0/3320326715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0af31fcd0 con 0x55b0af2a15f0 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.671+0000 7f94bea00640 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.671+0000 7f94be000640 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2aed40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94ac011850 tx=0x7f94ac005620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740d2236c44fc82f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.671+0000 7f94a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94a8c00640 1 -- 172.21.15.26:0/3320326715 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac03b060 con 0x55b0af2a15f0 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94c8975740 1 -- 172.21.15.26:0/3320326715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94ac003ad0 con 0x55b0af2a15f0 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94a8c00640 1 -- 172.21.15.26:0/3320326715 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94ac016040 con 0x55b0af2a15f0 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94a8c00640 1 -- 172.21.15.26:0/3320326715 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f94ac0035f0 con 0x55b0af2a15f0 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94c8975740 10 client.25816 fetch_fsmap learned FSMap version 33 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94c8975740 1 -- 172.21.15.26:0/3320326715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b0af320e80 con 0x55b0af2a15f0 2024-06-13T07:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94a8c00640 1 -- 172.21.15.26:0/3320326715 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac003ad0 con 0x55b0af2a15f0 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94a8c00640 1 -- 172.21.15.26:0/3320326715 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f94ac0245b0 con 0x55b0af2a15f0 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94c8975740 10 client.25816 fetch_fsmap finished waiting for FSMap version 33 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.672+0000 7f94c8975740 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.673+0000 7f94c8975740 20 client.25816 populate_metadata read hostname 'smithi026' 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.673+0000 7f94c8975740 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b0af321630 0x55b0af323a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.673+0000 7f94c8975740 4 client.25816 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.673+0000 7f94c8975740 1 -- 172.21.15.26:0/3320326715 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0af20bea0 con 0x55b0af321630 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.673+0000 7f94bea00640 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b0af321630 0x55b0af323a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.673+0000 7f94bea00640 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b0af321630 0x55b0af323a20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae4306d6ac0622a server_cookie=e81f6aef2cd0f652 in_seq=0 out_seq=0 2024-06-13T07:04:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.673+0000 7f94a8c00640 10 client.25816 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94a8c00640 1 -- 172.21.15.26:0/3320326715 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b0af20bea0 con 0x55b0af321630 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94a8c00640 10 client.25816 handle_command_reply: tid=0 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 client.25816 shutdown 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 -- 172.21.15.26:0/3320326715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 msgr2=0x55b0af2aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2aed40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94ac011850 tx=0x7f94ac005620 comp rx=0 tx=0).stop 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 -- 172.21.15.26:0/3320326715 shutdown_connections 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b0af321630 0x55b0af323a20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0af2a6760 0x55b0af2af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0af2a15f0 0x55b0af2aed40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94a8c00640 0 client.25816 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 --2- 172.21.15.26:0/3320326715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0af2a0c00 0x55b0af2b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 -- 172.21.15.26:0/3320326715 >> 172.21.15.26:0/3320326715 conn(0x55b0af268fb0 msgr2=0x55b0af2a2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 -- 172.21.15.26:0/3320326715 shutdown_connections 2024-06-13T07:04:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 1 -- 172.21.15.26:0/3320326715 wait complete. 2024-06-13T07:04:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.750+0000 7f94c8975740 20 client.25816 trim_cache size 0 max 16384 2024-06-13T07:04:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:04:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:04:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 -- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94a40a2cb0 msgr2=0x7f94a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94a40a2cb0 0x7f94a40a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94b8005e30 tx=0x7f94b803d040 comp rx=0 tx=0).stop 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 -- 172.21.15.26:0/3710802433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0108030 msgr2=0x7f94c019e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0108030 0x7f94c019e210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e9c0 tx=0x7f94b000ee80 comp rx=0 tx=0).stop 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 -- 172.21.15.26:0/3710802433 shutdown_connections 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94a40a2cb0 0x7f94a40a5170 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c0109be0 0x7f94c01a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0109220 0x7f94c019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 --2- 172.21.15.26:0/3710802433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c0108030 0x7f94c019e210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 -- 172.21.15.26:0/3710802433 >> 172.21.15.26:0/3710802433 conn(0x7f94c0076ff0 msgr2=0x7f94c010c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 -- 172.21.15.26:0/3710802433 shutdown_connections 2024-06-13T07:04:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:25.751+0000 7f94c5c00640 1 -- 172.21.15.26:0/3710802433 wait complete. 2024-06-13T07:04:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:25 smithi136 ceph-mon[37229]: pgmap v627: 129 pgs: 129 active+clean; 47 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-06-13T07:04:26.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:25 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:26.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:25 smithi026 ceph-mon[32041]: pgmap v627: 129 pgs: 129 active+clean; 47 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-06-13T07:04:26.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:25 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:26.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:25 smithi135 ceph-mon[36276]: pgmap v627: 129 pgs: 129 active+clean; 47 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-06-13T07:04:26.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:25 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:26.519 INFO:tasks.workunit.client.0.smithi026.stdout: 1 126588 3.53 MB/sec execute 40 sec latency 29.034 ms 2024-06-13T07:04:27.520 INFO:tasks.workunit.client.0.smithi026.stdout: 1 127451 3.62 MB/sec execute 41 sec latency 22.890 ms 2024-06-13T07:04:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:27 smithi136 ceph-mon[37229]: pgmap v628: 129 pgs: 129 active+clean; 48 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-06-13T07:04:28.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:27 smithi026 ceph-mon[32041]: pgmap v628: 129 pgs: 129 active+clean; 48 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-06-13T07:04:28.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:27 smithi135 ceph-mon[36276]: pgmap v628: 129 pgs: 129 active+clean; 48 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-06-13T07:04:28.520 INFO:tasks.workunit.client.0.smithi026.stdout: 1 128091 3.65 MB/sec execute 42 sec latency 24.371 ms 2024-06-13T07:04:29.520 INFO:tasks.workunit.client.0.smithi026.stdout: 1 128481 3.57 MB/sec execute 43 sec latency 33.530 ms 2024-06-13T07:04:30.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:29 smithi136 ceph-mon[37229]: pgmap v629: 129 pgs: 129 active+clean; 48 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-06-13T07:04:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:29 smithi026 ceph-mon[32041]: pgmap v629: 129 pgs: 129 active+clean; 48 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-06-13T07:04:30.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:29 smithi135 ceph-mon[36276]: pgmap v629: 129 pgs: 129 active+clean; 48 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-06-13T07:04:30.520 INFO:tasks.workunit.client.0.smithi026.stdout: 1 129308 3.60 MB/sec execute 44 sec latency 49.321 ms 2024-06-13T07:04:31.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:30 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:31.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:30 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:31.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:30 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:31.520 INFO:tasks.workunit.client.0.smithi026.stdout: 1 130491 3.65 MB/sec execute 45 sec latency 31.007 ms 2024-06-13T07:04:32.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:31 smithi136 ceph-mon[37229]: pgmap v630: 129 pgs: 129 active+clean; 48 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-13T07:04:32.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:31 smithi026 ceph-mon[32041]: pgmap v630: 129 pgs: 129 active+clean; 48 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-13T07:04:32.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:31 smithi135 ceph-mon[36276]: pgmap v630: 129 pgs: 129 active+clean; 48 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-13T07:04:32.520 INFO:tasks.workunit.client.0.smithi026.stdout: 1 131472 3.74 MB/sec execute 46 sec latency 23.655 ms 2024-06-13T07:04:33.520 INFO:tasks.workunit.client.0.smithi026.stdout: 1 131745 3.67 MB/sec execute 47 sec latency 24.188 ms 2024-06-13T07:04:34.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:33 smithi136 ceph-mon[37229]: pgmap v631: 129 pgs: 129 active+clean; 50 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-13T07:04:34.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:33 smithi026 ceph-mon[32041]: pgmap v631: 129 pgs: 129 active+clean; 50 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-13T07:04:34.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:33 smithi135 ceph-mon[36276]: pgmap v631: 129 pgs: 129 active+clean; 50 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-13T07:04:34.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 132380 3.63 MB/sec execute 48 sec latency 36.928 ms 2024-06-13T07:04:35.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 133479 3.64 MB/sec execute 49 sec latency 41.190 ms 2024-06-13T07:04:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:35 smithi136 ceph-mon[37229]: pgmap v632: 129 pgs: 129 active+clean; 50 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-13T07:04:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:35 smithi026 ceph-mon[32041]: pgmap v632: 129 pgs: 129 active+clean; 50 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-13T07:04:36.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:35 smithi135 ceph-mon[36276]: pgmap v632: 129 pgs: 129 active+clean; 50 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-13T07:04:36.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 134625 3.74 MB/sec execute 50 sec latency 66.103 ms 2024-06-13T07:04:37.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 135206 3.75 MB/sec execute 51 sec latency 24.695 ms 2024-06-13T07:04:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:37 smithi136 ceph-mon[37229]: pgmap v633: 129 pgs: 129 active+clean; 50 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:04:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:37 smithi026 ceph-mon[32041]: pgmap v633: 129 pgs: 129 active+clean; 50 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:04:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:37 smithi135 ceph-mon[36276]: pgmap v633: 129 pgs: 129 active+clean; 50 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:04:38.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 135704 3.70 MB/sec execute 52 sec latency 38.976 ms 2024-06-13T07:04:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.918+0000 7fb9dd400640 1 -- 172.21.15.26:0/2391252021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81000f0 msgr2=0x7fb9d8100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.918+0000 7fb9dd400640 1 --2- 172.21.15.26:0/2391252021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81000f0 0x7fb9d8100510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009ca0 tx=0x7fb9c802a8c0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.918+0000 7fb9dd400640 1 -- 172.21.15.26:0/2391252021 shutdown_connections 2024-06-13T07:04:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.918+0000 7fb9dd400640 1 --2- 172.21.15.26:0/2391252021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81013f0 0x7fb9d8101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.918+0000 7fb9dd400640 1 --2- 172.21.15.26:0/2391252021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d8100a50 0x7fb9d8100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.918+0000 7fb9dd400640 1 --2- 172.21.15.26:0/2391252021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81000f0 0x7fb9d8100510 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.918+0000 7fb9dd400640 1 -- 172.21.15.26:0/2391252021 >> 172.21.15.26:0/2391252021 conn(0x7fb9d80fbc90 msgr2=0x7fb9d80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.918+0000 7fb9dd400640 1 -- 172.21.15.26:0/2391252021 shutdown_connections 2024-06-13T07:04:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.919+0000 7fb9dd400640 1 -- 172.21.15.26:0/2391252021 wait complete. 2024-06-13T07:04:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.920+0000 7fb9dd400640 1 Processor -- start 2024-06-13T07:04:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.920+0000 7fb9dd400640 1 -- start start 2024-06-13T07:04:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81000f0 0x7fb9d8105c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d8100a50 0x7fb9d81061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81013f0 0x7fb9d81027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dd400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810f680 con 0x7fb9d8100a50 2024-06-13T07:04:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dd400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810f500 con 0x7fb9d81000f0 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dd400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810f800 con 0x7fb9d81013f0 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d8100a50 0x7fb9d81061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81013f0 0x7fb9d81027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81000f0 0x7fb9d8105c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81013f0 0x7fb9d81027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54392/0 (socket says 172.21.15.26:54392) 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81000f0 0x7fb9d8105c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37412/0 (socket says 172.21.15.26:37412) 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.921+0000 7fb9dca00640 1 -- 172.21.15.26:0/426822791 learned_addr learned my addr 172.21.15.26:0/426822791 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9dca00640 1 -- 172.21.15.26:0/426822791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81000f0 msgr2=0x7fb9d8105c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9dca00640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81000f0 0x7fb9d8105c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9dca00640 1 -- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d8100a50 msgr2=0x7fb9d81061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9dca00640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d8100a50 0x7fb9d81061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9dca00640 1 -- 172.21.15.26:0/426822791 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d81030a0 con 0x7fb9d81013f0 2024-06-13T07:04:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9dca00640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81013f0 0x7fb9d81027e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e970 tx=0x7fb9d000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9d4c00640 1 -- 172.21.15.26:0/426822791 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000ccc0 con 0x7fb9d81013f0 2024-06-13T07:04:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9d4c00640 1 -- 172.21.15.26:0/426822791 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d000ce60 con 0x7fb9d81013f0 2024-06-13T07:04:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9d4c00640 1 -- 172.21.15.26:0/426822791 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0010690 con 0x7fb9d81013f0 2024-06-13T07:04:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9dd400640 1 -- 172.21.15.26:0/426822791 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d81a6170 con 0x7fb9d81013f0 2024-06-13T07:04:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.922+0000 7fb9dd400640 1 -- 172.21.15.26:0/426822791 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d81a65d0 con 0x7fb9d81013f0 2024-06-13T07:04:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.926+0000 7fb9d4c00640 1 -- 172.21.15.26:0/426822791 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb9d00046c0 con 0x7fb9d81013f0 2024-06-13T07:04:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.926+0000 7fb9e1866740 1 Processor -- start 2024-06-13T07:04:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9e1866740 1 -- start start 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf1741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf1792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf17bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9e1866740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55deaf0d7420 con 0x55deaf1747b0 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9e1866740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55deaf07ec50 con 0x55deaf173dc0 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9e1866740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55deaf0d33e0 con 0x55deaf179920 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf1792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf17bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.927+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf1792b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46482/0 (socket says 172.21.15.26:46482) 2024-06-13T07:04:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d7e00640 1 -- 172.21.15.26:0/4152000572 learned_addr learned my addr 172.21.15.26:0/4152000572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d7400640 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf1741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d7e00640 1 -- 172.21.15.26:0/4152000572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 msgr2=0x55deaf17bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf17bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d7e00640 1 -- 172.21.15.26:0/4152000572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 msgr2=0x55deaf1741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf1741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d7e00640 1 -- 172.21.15.26:0/4152000572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deaedeb210 con 0x55deaf1747b0 2024-06-13T07:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d7400640 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf1741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.928+0000 7fb9d4c00640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb9bc0a2c00 0x7fb9bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.929+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf1792b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8035000 tx=0x7fb9c8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32db4838e9933ad server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.929+0000 7fb9d4c00640 1 -- 172.21.15.26:0/426822791 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb9d00cf810 con 0x7fb9d81013f0 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.929+0000 7fb9d7400640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb9bc0a2c00 0x7fb9bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.929+0000 7fb9cd200640 1 -- 172.21.15.26:0/4152000572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8049070 con 0x55deaf1747b0 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.929+0000 7fb9cd200640 1 -- 172.21.15.26:0/4152000572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c8036040 con 0x55deaf1747b0 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.929+0000 7fb9cd200640 1 -- 172.21.15.26:0/4152000572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c802fb90 con 0x55deaf1747b0 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.930+0000 7fb9e1866740 1 -- 172.21.15.26:0/4152000572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 msgr2=0x55deaf1792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.930+0000 7fb9e1866740 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf1792b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8035000 tx=0x7fb9c8009360 comp rx=0 tx=0).stop 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.930+0000 7fb9d7400640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb9bc0a2c00 0x7fb9bc0a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55deaf175010 tx=0x7fb9c4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.930+0000 7fb9e1866740 1 -- 172.21.15.26:0/4152000572 shutdown_connections 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.930+0000 7fb9e1866740 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf17bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.930+0000 7fb9e1866740 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf1792b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.931+0000 7fb9e1866740 1 --2- 172.21.15.26:0/4152000572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf1741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.931+0000 7fb9e1866740 1 -- 172.21.15.26:0/4152000572 >> 172.21.15.26:0/4152000572 conn(0x55deaf13bfb0 msgr2=0x55deaf16f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.931+0000 7fb9e1866740 1 -- 172.21.15.26:0/4152000572 shutdown_connections 2024-06-13T07:04:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.931+0000 7fb9e1866740 1 -- 172.21.15.26:0/4152000572 wait complete. 2024-06-13T07:04:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.931+0000 7fb9e1866740 1 -- start start 2024-06-13T07:04:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9e1866740 1 Processor -- start 2024-06-13T07:04:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf17def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf17e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf185470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9e1866740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55deaf17c6f0 con 0x55deaf1747b0 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9e1866740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55deaf0920b0 con 0x55deaf173dc0 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9e1866740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55deaf17c870 con 0x55deaf179920 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9e1866740 10 client.0 _ll_register_callbacks cb 0x55deaf07df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf17e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf17def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf17e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46492/0 (socket says 172.21.15.26:46492) 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.932+0000 7fb9d7400640 1 -- 172.21.15.26:0/1869034702 learned_addr learned my addr 172.21.15.26:0/1869034702 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9dca00640 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf185470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9d7400640 1 -- 172.21.15.26:0/1869034702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 msgr2=0x55deaf185470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9d7400640 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf185470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9d7400640 1 -- 172.21.15.26:0/1869034702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 msgr2=0x55deaf17def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9d7400640 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf17def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9d7400640 1 -- 172.21.15.26:0/1869034702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deaeeb7620 con 0x55deaf1747b0 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf17def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9dca00640 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf185470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9d7400640 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf17e430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8103bf0 tx=0x7fb9c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f311488a1b47970 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.933+0000 7fb9c1600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.934+0000 7fb9c1600640 1 -- 172.21.15.26:0/1869034702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4010e60 con 0x55deaf1747b0 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.934+0000 7fb9c1600640 1 -- 172.21.15.26:0/1869034702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb9c4020450 con 0x55deaf1747b0 2024-06-13T07:04:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.934+0000 7fb9e1866740 1 -- 172.21.15.26:0/1869034702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c8036040 con 0x55deaf1747b0 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.934+0000 7fb9c1600640 1 -- 172.21.15.26:0/1869034702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9c4031050 con 0x55deaf1747b0 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.934+0000 7fb9e1866740 10 client.15690 fetch_fsmap learned FSMap version 34 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.934+0000 7fb9e1866740 1 -- 172.21.15.26:0/1869034702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55deaf1888b0 con 0x55deaf1747b0 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.935+0000 7fb9c1600640 1 -- 172.21.15.26:0/1869034702 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c8036040 con 0x55deaf1747b0 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.935+0000 7fb9c1600640 1 -- 172.21.15.26:0/1869034702 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fb9c401d040 con 0x55deaf1747b0 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.935+0000 7fb9e1866740 10 client.15690 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.935+0000 7fb9e1866740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.935+0000 7fb9e1866740 20 client.15690 populate_metadata read hostname 'smithi026' 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.935+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55deaf189ff0 0x55deaf18c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.935+0000 7fb9e1866740 4 client.15690 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.935+0000 7fb9e1866740 1 -- 172.21.15.26:0/1869034702 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deaf0e4b50 con 0x55deaf189ff0 2024-06-13T07:04:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.936+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55deaf189ff0 0x55deaf18c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.936+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55deaf189ff0 0x55deaf18c460 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617827b92910d2b3 server_cookie=f2aa15d0ae57cad1 in_seq=0 out_seq=0 2024-06-13T07:04:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.936+0000 7fb9c1600640 10 client.15690 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9c1600640 1 -- 172.21.15.26:0/1869034702 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55deaf0e4b50 con 0x55deaf189ff0 2024-06-13T07:04:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9c1600640 10 client.15690 handle_command_reply: tid=0 2024-06-13T07:04:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 client.15690 shutdown 2024-06-13T07:04:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 -- 172.21.15.26:0/1869034702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 msgr2=0x55deaf17e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf17e430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8103bf0 tx=0x7fb9c4004730 comp rx=0 tx=0).stop 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 -- 172.21.15.26:0/1869034702 shutdown_connections 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55deaf189ff0 0x55deaf18c460 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf179920 0x55deaf185470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf1747b0 0x55deaf17e430 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1869034702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf17def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 -- 172.21.15.26:0/1869034702 >> 172.21.15.26:0/1869034702 conn(0x55deaf13bfb0 msgr2=0x55deaf16e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9c1600640 0 client.15690 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.939+0000 7fb9e1866740 1 -- 172.21.15.26:0/1869034702 shutdown_connections 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.940+0000 7fb9e1866740 1 -- 172.21.15.26:0/1869034702 wait complete. 2024-06-13T07:04:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.940+0000 7fb9e1866740 20 client.15690 trim_cache size 0 max 16384 2024-06-13T07:04:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.960+0000 7fb9e1866740 1 Processor -- start 2024-06-13T07:04:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.960+0000 7fb9e1866740 1 -- start start 2024-06-13T07:04:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.960+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf16f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.960+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf185430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.960+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf179920 0x55deaf175a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.960+0000 7fb9e1866740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b34b0 con 0x55deaf179920 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.960+0000 7fb9e1866740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b0d50 con 0x55deaf173dc0 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.960+0000 7fb9e1866740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81aeba0 con 0x55deaf1747b0 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf185430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf185430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54422/0 (socket says 172.21.15.26:54422) 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7400640 1 -- 172.21.15.26:0/2953946933 learned_addr learned my addr 172.21.15.26:0/2953946933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9dca00640 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf179920 0x55deaf175a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf16f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7400640 1 -- 172.21.15.26:0/2953946933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 msgr2=0x55deaf16f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7400640 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf16f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7400640 1 -- 172.21.15.26:0/2953946933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf179920 msgr2=0x55deaf175a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7400640 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf179920 0x55deaf175a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7400640 1 -- 172.21.15.26:0/2953946933 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deaf175040 con 0x55deaf1747b0 2024-06-13T07:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf16f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.961+0000 7fb9dca00640 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf179920 0x55deaf175a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.962+0000 7fb9d7400640 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf185430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4012fd0 tx=0x7fb9c400ea30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25fdf5a6f01e218b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.962+0000 7fb9c3e00640 1 -- 172.21.15.26:0/2953946933 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4039070 con 0x55deaf1747b0 2024-06-13T07:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.962+0000 7fb9c3e00640 1 -- 172.21.15.26:0/2953946933 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c401ea70 con 0x55deaf1747b0 2024-06-13T07:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.962+0000 7fb9c3e00640 1 -- 172.21.15.26:0/2953946933 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4020930 con 0x55deaf1747b0 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 -- 172.21.15.26:0/2953946933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 msgr2=0x55deaf185430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf185430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4012fd0 tx=0x7fb9c400ea30 comp rx=0 tx=0).stop 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 -- 172.21.15.26:0/2953946933 shutdown_connections 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf179920 0x55deaf175a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf185430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 --2- 172.21.15.26:0/2953946933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf173dc0 0x55deaf16f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 -- 172.21.15.26:0/2953946933 >> 172.21.15.26:0/2953946933 conn(0x55deaf13bfb0 msgr2=0x55deaf179100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 -- 172.21.15.26:0/2953946933 shutdown_connections 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.963+0000 7fb9e1866740 1 -- 172.21.15.26:0/2953946933 wait complete. 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.964+0000 7fb9e1866740 1 -- start start 2024-06-13T07:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.964+0000 7fb9e1866740 1 Processor -- start 2024-06-13T07:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf173dc0 0x55deaf1820f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf182630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9e1866740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf179920 0x55deaf1f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9e1866740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81af210 con 0x55deaf173dc0 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9e1866740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b0680 con 0x55deaf179920 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9e1866740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81af8e0 con 0x55deaf1747b0 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9e1866740 10 client.0 _ll_register_callbacks cb 0x7fb9c400ecd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf173dc0 0x55deaf1820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9dca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf179920 0x55deaf1f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf173dc0 0x55deaf1820f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46512/0 (socket says 172.21.15.26:46512) 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9d7e00640 1 -- 172.21.15.26:0/1551375253 learned_addr learned my addr 172.21.15.26:0/1551375253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.965+0000 7fb9d7400640 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf182630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9d7400640 1 -- 172.21.15.26:0/1551375253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf179920 msgr2=0x55deaf1f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9d7400640 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf179920 0x55deaf1f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9d7400640 1 -- 172.21.15.26:0/1551375253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf173dc0 msgr2=0x55deaf1820f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9d7400640 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf173dc0 0x55deaf1820f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9d7400640 1 -- 172.21.15.26:0/1551375253 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deaf1856b0 con 0x55deaf1747b0 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf173dc0 0x55deaf1820f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9dca00640 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf179920 0x55deaf1f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9d7400640 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf182630 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55deaf174f80 tx=0x7fb9c401e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bda91d165cdbfe0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9c1600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9c1600640 1 -- 172.21.15.26:0/1551375253 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4039040 con 0x55deaf1747b0 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9c1600640 1 -- 172.21.15.26:0/1551375253 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb9c4005040 con 0x55deaf1747b0 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.966+0000 7fb9e1866740 1 -- 172.21.15.26:0/1551375253 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c401ea70 con 0x55deaf1747b0 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9c1600640 1 -- 172.21.15.26:0/1551375253 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9c404c050 con 0x55deaf1747b0 2024-06-13T07:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9e1866740 10 client.25834 fetch_fsmap learned FSMap version 34 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9e1866740 1 -- 172.21.15.26:0/1551375253 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55deaf1f4270 con 0x55deaf1747b0 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9c1600640 1 -- 172.21.15.26:0/1551375253 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c401ea70 con 0x55deaf1747b0 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9c1600640 1 -- 172.21.15.26:0/1551375253 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fb9c400f040 con 0x55deaf1747b0 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9e1866740 10 client.25834 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9e1866740 10 client.25834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9e1866740 20 client.25834 populate_metadata read hostname 'smithi026' 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55deaf1f4b80 0x55deaf1f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9e1866740 4 client.25834 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:04:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.967+0000 7fb9e1866740 1 -- 172.21.15.26:0/1551375253 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55deaf0deea0 con 0x55deaf1f4b80 2024-06-13T07:04:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.968+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55deaf1f4b80 0x55deaf1f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.968+0000 7fb9d7e00640 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55deaf1f4b80 0x55deaf1f6fa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ed1f5bf59ae6ae server_cookie=9739ad43ba584852 in_seq=0 out_seq=0 2024-06-13T07:04:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:38.968+0000 7fb9c1600640 10 client.25834 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.058+0000 7fb9c1600640 1 -- 172.21.15.26:0/1551375253 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55deaf0deea0 con 0x55deaf1f4b80 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.058+0000 7fb9c1600640 10 client.25834 handle_command_reply: tid=0 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 client.25834 shutdown 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 -- 172.21.15.26:0/1551375253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 msgr2=0x55deaf182630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf182630 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55deaf174f80 tx=0x7fb9c401e710 comp rx=0 tx=0).stop 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 -- 172.21.15.26:0/1551375253 shutdown_connections 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55deaf1f4b80 0x55deaf1f6fa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55deaf179920 0x55deaf1f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55deaf1747b0 0x55deaf182630 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 --2- 172.21.15.26:0/1551375253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55deaf173dc0 0x55deaf1820f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9c1600640 0 client.25834 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 -- 172.21.15.26:0/1551375253 >> 172.21.15.26:0/1551375253 conn(0x55deaf13bfb0 msgr2=0x55deaf178cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 -- 172.21.15.26:0/1551375253 shutdown_connections 2024-06-13T07:04:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 1 -- 172.21.15.26:0/1551375253 wait complete. 2024-06-13T07:04:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.059+0000 7fb9e1866740 20 client.25834 trim_cache size 0 max 16384 2024-06-13T07:04:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:04:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:04:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:04:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 -- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb9bc0a2c00 msgr2=0x7fb9bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb9bc0a2c00 0x7fb9bc0a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55deaf175010 tx=0x7fb9c4005850 comp rx=0 tx=0).stop 2024-06-13T07:04:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 -- 172.21.15.26:0/426822791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81013f0 msgr2=0x7fb9d81027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81013f0 0x7fb9d81027e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e970 tx=0x7fb9d000ed70 comp rx=0 tx=0).stop 2024-06-13T07:04:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 -- 172.21.15.26:0/426822791 shutdown_connections 2024-06-13T07:04:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb9bc0a2c00 0x7fb9bc0a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9d81013f0 0x7fb9d81027e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d8100a50 0x7fb9d81061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 --2- 172.21.15.26:0/426822791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81000f0 0x7fb9d8105c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 -- 172.21.15.26:0/426822791 >> 172.21.15.26:0/426822791 conn(0x7fb9d80fbc90 msgr2=0x7fb9d80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.060+0000 7fb9dd400640 1 -- 172.21.15.26:0/426822791 shutdown_connections 2024-06-13T07:04:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:39.061+0000 7fb9dd400640 1 -- 172.21.15.26:0/426822791 wait complete. 2024-06-13T07:04:39.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 136837 3.72 MB/sec execute 53 sec latency 41.309 ms 2024-06-13T07:04:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:39 smithi136 ceph-mon[37229]: pgmap v634: 129 pgs: 129 active+clean; 51 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-13T07:04:40.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:39 smithi026 ceph-mon[32041]: pgmap v634: 129 pgs: 129 active+clean; 51 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-13T07:04:40.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:39 smithi135 ceph-mon[36276]: pgmap v634: 129 pgs: 129 active+clean; 51 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-13T07:04:40.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 137749 3.78 MB/sec execute 54 sec latency 24.573 ms 2024-06-13T07:04:41.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 138693 3.82 MB/sec execute 55 sec latency 38.887 ms 2024-06-13T07:04:42.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:41 smithi136 ceph-mon[37229]: pgmap v635: 129 pgs: 129 active+clean; 51 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-06-13T07:04:42.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:41 smithi026 ceph-mon[32041]: pgmap v635: 129 pgs: 129 active+clean; 51 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-06-13T07:04:42.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:41 smithi135 ceph-mon[36276]: pgmap v635: 129 pgs: 129 active+clean; 51 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-06-13T07:04:42.521 INFO:tasks.workunit.client.0.smithi026.stdout: 1 139102 3.77 MB/sec execute 56 sec latency 34.906 ms 2024-06-13T07:04:43.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 140062 3.79 MB/sec execute 57 sec latency 24.086 ms 2024-06-13T07:04:44.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:43 smithi026 ceph-mon[32041]: pgmap v636: 129 pgs: 129 active+clean; 53 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:04:44.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:43 smithi135 ceph-mon[36276]: pgmap v636: 129 pgs: 129 active+clean; 53 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:04:44.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:43 smithi136 ceph-mon[37229]: pgmap v636: 129 pgs: 129 active+clean; 53 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:04:44.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 141297 3.85 MB/sec execute 58 sec latency 22.269 ms 2024-06-13T07:04:45.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 142206 3.89 MB/sec execute 59 sec latency 24.678 ms 2024-06-13T07:04:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:45 smithi026 ceph-mon[32041]: pgmap v637: 129 pgs: 129 active+clean; 53 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-06-13T07:04:46.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:45 smithi135 ceph-mon[36276]: pgmap v637: 129 pgs: 129 active+clean; 53 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-06-13T07:04:46.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:45 smithi136 ceph-mon[37229]: pgmap v637: 129 pgs: 129 active+clean; 53 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-06-13T07:04:46.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 142713 3.84 MB/sec execute 60 sec latency 42.634 ms 2024-06-13T07:04:47.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 143826 3.86 MB/sec execute 61 sec latency 25.863 ms 2024-06-13T07:04:48.178 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:47 smithi136 ceph-mon[37229]: pgmap v638: 129 pgs: 129 active+clean; 53 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-13T07:04:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:47 smithi026 ceph-mon[32041]: pgmap v638: 129 pgs: 129 active+clean; 53 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-13T07:04:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:47 smithi135 ceph-mon[36276]: pgmap v638: 129 pgs: 129 active+clean; 53 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-13T07:04:48.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 144919 3.91 MB/sec execute 62 sec latency 25.646 ms 2024-06-13T07:04:49.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 145795 3.95 MB/sec execute 63 sec latency 20.627 ms 2024-06-13T07:04:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:49 smithi026 ceph-mon[32041]: pgmap v639: 129 pgs: 129 active+clean; 54 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:04:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:49 smithi135 ceph-mon[36276]: pgmap v639: 129 pgs: 129 active+clean; 54 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:04:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:49 smithi136 ceph-mon[37229]: pgmap v639: 129 pgs: 129 active+clean; 54 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:04:50.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 146240 3.89 MB/sec execute 64 sec latency 45.739 ms 2024-06-13T07:04:51.522 INFO:tasks.workunit.client.0.smithi026.stdout: 1 147286 3.91 MB/sec execute 65 sec latency 24.292 ms 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 -- 172.21.15.26:0/527867333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26281022d0 msgr2=0x7f2628108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 --2- 172.21.15.26:0/527867333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26281022d0 0x7f2628108b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ca0 tx=0x7f262002a8b0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 -- 172.21.15.26:0/527867333 shutdown_connections 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 --2- 172.21.15.26:0/527867333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2628101910 0x7f2628101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 --2- 172.21.15.26:0/527867333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26281022d0 0x7f2628108b60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 --2- 172.21.15.26:0/527867333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628100720 0x7f2628100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 -- 172.21.15.26:0/527867333 >> 172.21.15.26:0/527867333 conn(0x7f26280fbc90 msgr2=0x7f26280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 -- 172.21.15.26:0/527867333 shutdown_connections 2024-06-13T07:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.225+0000 7f262d200640 1 -- 172.21.15.26:0/527867333 wait complete. 2024-06-13T07:04:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.226+0000 7f262d200640 1 Processor -- start 2024-06-13T07:04:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.226+0000 7f262d200640 1 -- start start 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f262d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628100720 0x7f262810b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f262d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628101910 0x7f26281096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f262d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26281022d0 0x7f2628109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f262d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f262810d4c0 con 0x7f2628101910 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f262d200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f262810d340 con 0x7f26281022d0 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f262d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f262810d640 con 0x7f2628100720 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f2627400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628100720 0x7f262810b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f2627400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628100720 0x7f262810b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41252/0 (socket says 172.21.15.26:41252) 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f2627400640 1 -- 172.21.15.26:0/1490495307 learned_addr learned my addr 172.21.15.26:0/1490495307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f2627e00640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26281022d0 0x7f2628109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.227+0000 7f2626a00640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628101910 0x7f26281096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f2627400640 1 -- 172.21.15.26:0/1490495307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26281022d0 msgr2=0x7f2628109c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f2627400640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26281022d0 0x7f2628109c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f2627400640 1 -- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628101910 msgr2=0x7f26281096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f2627400640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628101910 0x7f26281096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f2627400640 1 -- 172.21.15.26:0/1490495307 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262810a4b0 con 0x7f2628100720 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f2627400640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628100720 0x7f262810b0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f261800e9c0 tx=0x7f261800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f261fe00640 1 -- 172.21.15.26:0/1490495307 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261800cd80 con 0x7f2628100720 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f261fe00640 1 -- 172.21.15.26:0/1490495307 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26180035f0 con 0x7f2628100720 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f262d200640 1 -- 172.21.15.26:0/1490495307 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26281a1c40 con 0x7f2628100720 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.228+0000 7f261fe00640 1 -- 172.21.15.26:0/1490495307 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618010690 con 0x7f2628100720 2024-06-13T07:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.229+0000 7f262d200640 1 -- 172.21.15.26:0/1490495307 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281a2190 con 0x7f2628100720 2024-06-13T07:04:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.233+0000 7f26315f9740 1 Processor -- start 2024-06-13T07:04:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.234+0000 7f261fe00640 1 -- 172.21.15.26:0/1490495307 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26180037a0 con 0x7f2628100720 2024-06-13T07:04:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.235+0000 7f26315f9740 1 -- start start 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.235+0000 7f26315f9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcedc0 0x556725fcf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.235+0000 7f26315f9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fd42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.235+0000 7f26315f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fd4920 0x556725fd6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.235+0000 7f26315f9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556725f32420 con 0x556725fcf7b0 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.235+0000 7f26315f9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556725ed9c50 con 0x556725fcedc0 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.235+0000 7f26315f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556725f2e3e0 con 0x556725fd4920 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.235+0000 7f261fe00640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f260c0a2c00 0x7f260c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2627400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fd42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2626a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fd4920 0x556725fd6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2627e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcedc0 0x556725fcf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2626a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fd4920 0x556725fd6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41260/0 (socket says 172.21.15.26:41260) 2024-06-13T07:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2626a00640 1 -- 172.21.15.26:0/3314977847 learned_addr learned my addr 172.21.15.26:0/3314977847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f261fe00640 1 -- 172.21.15.26:0/1490495307 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f26180cf7f0 con 0x7f2628100720 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2626a00640 1 -- 172.21.15.26:0/3314977847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcedc0 msgr2=0x556725fcf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2626a00640 1 --2- 172.21.15.26:0/3314977847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcedc0 0x556725fcf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2627e00640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f260c0a2c00 0x7f260c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2626a00640 1 -- 172.21.15.26:0/3314977847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 msgr2=0x556725fd42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2626a00640 1 --2- 172.21.15.26:0/3314977847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fd42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.236+0000 7f2626a00640 1 -- 172.21.15.26:0/3314977847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556725c46210 con 0x556725fd4920 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.237+0000 7f2626a00640 1 --2- 172.21.15.26:0/3314977847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fd4920 0x556725fd6db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2614009ca0 tx=0x7f261400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475075db4992720d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.237+0000 7f261cc00640 1 -- 172.21.15.26:0/3314977847 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614026070 con 0x556725fd4920 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.237+0000 7f2627e00640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f260c0a2c00 0x7f260c0a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x556725fd0010 tx=0x7f262002a8b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.237+0000 7f261cc00640 1 -- 172.21.15.26:0/3314977847 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2614009e50 con 0x556725fd4920 2024-06-13T07:04:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.237+0000 7f261cc00640 1 -- 172.21.15.26:0/3314977847 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614020730 con 0x556725fd4920 2024-06-13T07:04:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 -- 172.21.15.26:0/3314977847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fd4920 msgr2=0x556725fd6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 --2- 172.21.15.26:0/3314977847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fd4920 0x556725fd6db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2614009ca0 tx=0x7f261400f790 comp rx=0 tx=0).stop 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 -- 172.21.15.26:0/3314977847 shutdown_connections 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 --2- 172.21.15.26:0/3314977847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fd4920 0x556725fd6db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 --2- 172.21.15.26:0/3314977847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fd42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 --2- 172.21.15.26:0/3314977847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcedc0 0x556725fcf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 -- 172.21.15.26:0/3314977847 >> 172.21.15.26:0/3314977847 conn(0x556725f96fb0 msgr2=0x556725fca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 -- 172.21.15.26:0/3314977847 shutdown_connections 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.238+0000 7f26315f9740 1 -- 172.21.15.26:0/3314977847 wait complete. 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.239+0000 7f26315f9740 1 -- start start 2024-06-13T07:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.239+0000 7f26315f9740 1 Processor -- start 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f26315f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fd90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f26315f9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 0x556725fd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f26315f9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fd4920 0x556725fe0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f26315f9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556725fd78d0 con 0x556725fd4920 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f26315f9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556725eed0b0 con 0x556725fcf7b0 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f26315f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556725fd7a50 con 0x556725fcedc0 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f26315f9740 10 client.0 _ll_register_callbacks cb 0x556725ed8f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f2627400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fd90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f2627400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fd90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41264/0 (socket says 172.21.15.26:41264) 2024-06-13T07:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f2627400640 1 -- 172.21.15.26:0/4048042328 learned_addr learned my addr 172.21.15.26:0/4048042328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f2626a00640 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 0x556725fd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f2627e00640 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fd4920 0x556725fe0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.240+0000 7f2627400640 1 -- 172.21.15.26:0/4048042328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 msgr2=0x556725fd9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2627400640 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 0x556725fd9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2627400640 1 -- 172.21.15.26:0/4048042328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fd4920 msgr2=0x556725fe0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2627400640 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fd4920 0x556725fe0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2627400640 1 -- 172.21.15.26:0/4048042328 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556725d12620 con 0x556725fcedc0 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2627400640 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fd90c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2618098630 tx=0x7f261806abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5fc50ecb87bc32 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2611600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2611600640 1 -- 172.21.15.26:0/4048042328 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261809bc60 con 0x556725fcedc0 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2611600640 1 -- 172.21.15.26:0/4048042328 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f261806bb80 con 0x556725fcedc0 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f26315f9740 1 -- 172.21.15.26:0/4048042328 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2614009e50 con 0x556725fcedc0 2024-06-13T07:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f2611600640 1 -- 172.21.15.26:0/4048042328 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f26180a0010 con 0x556725fcedc0 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f26315f9740 10 client.25846 fetch_fsmap learned FSMap version 34 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.241+0000 7f26315f9740 1 -- 172.21.15.26:0/4048042328 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556725fe3a80 con 0x556725fcedc0 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f2611600640 1 -- 172.21.15.26:0/4048042328 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2614009e50 con 0x556725fcedc0 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f2611600640 1 -- 172.21.15.26:0/4048042328 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f261809ab20 con 0x556725fcedc0 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f26315f9740 10 client.25846 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f26315f9740 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f26315f9740 20 client.25846 populate_metadata read hostname 'smithi026' 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f26315f9740 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556725fe5150 0x556725fe75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f26315f9740 4 client.25846 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f26315f9740 1 -- 172.21.15.26:0/4048042328 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556725f3fb50 con 0x556725fe5150 2024-06-13T07:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.242+0000 7f2626a00640 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556725fe5150 0x556725fe75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.243+0000 7f2626a00640 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556725fe5150 0x556725fe75e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331460f9b1846c50 server_cookie=50b0ae3b3c3be76b in_seq=0 out_seq=0 2024-06-13T07:04:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.243+0000 7f2611600640 10 client.25846 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f2611600640 1 -- 172.21.15.26:0/4048042328 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556725f3fb50 con 0x556725fe5150 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f2611600640 10 client.25846 handle_command_reply: tid=0 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 client.25846 shutdown 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 -- 172.21.15.26:0/4048042328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 msgr2=0x556725fd90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fd90c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2618098630 tx=0x7f261806abe0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 -- 172.21.15.26:0/4048042328 shutdown_connections 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556725fe5150 0x556725fe75e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 0x556725fd9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fd4920 0x556725fe0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 --2- 172.21.15.26:0/4048042328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fd90c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 -- 172.21.15.26:0/4048042328 >> 172.21.15.26:0/4048042328 conn(0x556725f96fb0 msgr2=0x556725fc99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f2611600640 0 client.25846 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 -- 172.21.15.26:0/4048042328 shutdown_connections 2024-06-13T07:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.244+0000 7f26315f9740 1 -- 172.21.15.26:0/4048042328 wait complete. 2024-06-13T07:04:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.245+0000 7f26315f9740 20 client.25846 trim_cache size 0 max 16384 2024-06-13T07:04:52.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:51 smithi026 ceph-mon[32041]: pgmap v640: 129 pgs: 129 active+clean; 55 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1014 KiB/s wr, 166 op/s 2024-06-13T07:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.265+0000 7f26315f9740 1 Processor -- start 2024-06-13T07:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.265+0000 7f26315f9740 1 -- start start 2024-06-13T07:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f26315f9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725f96fb0 0x556725fd7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f26315f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fe0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f26315f9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 0x556725fcdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f26315f9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26281aacc0 con 0x556725f96fb0 2024-06-13T07:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f26315f9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26281a8560 con 0x556725fcf7b0 2024-06-13T07:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f26315f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f26281a63b0 con 0x556725fcedc0 2024-06-13T07:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2626a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fe0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2627400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725f96fb0 0x556725fd7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2626a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fe0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41266/0 (socket says 172.21.15.26:41266) 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2626a00640 1 -- 172.21.15.26:0/3044889620 learned_addr learned my addr 172.21.15.26:0/3044889620 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2627e00640 1 --2- 172.21.15.26:0/3044889620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 0x556725fcdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2626a00640 1 -- 172.21.15.26:0/3044889620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 msgr2=0x556725fcdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2626a00640 1 --2- 172.21.15.26:0/3044889620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 0x556725fcdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2626a00640 1 -- 172.21.15.26:0/3044889620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725f96fb0 msgr2=0x556725fd7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2626a00640 1 --2- 172.21.15.26:0/3044889620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725f96fb0 0x556725fd7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.266+0000 7f2626a00640 1 -- 172.21.15.26:0/3044889620 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556725fcfb90 con 0x556725fcedc0 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.267+0000 7f2626a00640 1 --2- 172.21.15.26:0/3044889620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fe0600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f261401b010 tx=0x7f2614005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=255cfefe635e590b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.267+0000 7f2613e00640 1 -- 172.21.15.26:0/3044889620 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261403c070 con 0x556725fcedc0 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.267+0000 7f2613e00640 1 -- 172.21.15.26:0/3044889620 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2614009800 con 0x556725fcedc0 2024-06-13T07:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.267+0000 7f2613e00640 1 -- 172.21.15.26:0/3044889620 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614026040 con 0x556725fcedc0 2024-06-13T07:04:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 -- 172.21.15.26:0/3044889620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 msgr2=0x556725fe0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 --2- 172.21.15.26:0/3044889620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fe0600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f261401b010 tx=0x7f2614005d30 comp rx=0 tx=0).stop 2024-06-13T07:04:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 -- 172.21.15.26:0/3044889620 shutdown_connections 2024-06-13T07:04:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 --2- 172.21.15.26:0/3044889620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725fcf7b0 0x556725fcdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 --2- 172.21.15.26:0/3044889620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fe0600 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 --2- 172.21.15.26:0/3044889620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725f96fb0 0x556725fd7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 -- 172.21.15.26:0/3044889620 >> 172.21.15.26:0/3044889620 conn(0x556725fd4460 msgr2=0x556725fcd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 -- 172.21.15.26:0/3044889620 shutdown_connections 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.268+0000 7f26315f9740 1 -- 172.21.15.26:0/3044889620 wait complete. 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 1 -- start start 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 1 Processor -- start 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725f96fb0 0x556725fdd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fdecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26281a6a20 con 0x556725fcf7b0 2024-06-13T07:04:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26281a7e90 con 0x556725f96fb0 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f26281a70f0 con 0x556725fcedc0 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.269+0000 7f26315f9740 10 client.0 _ll_register_callbacks cb 0x7f2618096680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2627400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725f96fb0 0x556725fdd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fddcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41276/0 (socket says 172.21.15.26:41276) 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 -- 172.21.15.26:0/1840826524 learned_addr learned my addr 172.21.15.26:0/1840826524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2627e00640 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fdecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 -- 172.21.15.26:0/1840826524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725f96fb0 msgr2=0x556725fdd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725f96fb0 0x556725fdd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 -- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 msgr2=0x556725fdecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fdecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 -- 172.21.15.26:0/1840826524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556725fdf2b0 con 0x556725fcedc0 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2627e00640 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fdecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2627400640 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725f96fb0 0x556725fdd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2626a00640 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fddcb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2614009c70 tx=0x7f2614009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f4b53605785404 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.270+0000 7f2611600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:04:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.271+0000 7f2611600640 1 -- 172.21.15.26:0/1840826524 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614026070 con 0x556725fcedc0 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.271+0000 7f26315f9740 1 -- 172.21.15.26:0/1840826524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2614009800 con 0x556725fcedc0 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.271+0000 7f2611600640 1 -- 172.21.15.26:0/1840826524 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2614005500 con 0x556725fcedc0 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.271+0000 7f2611600640 1 -- 172.21.15.26:0/1840826524 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f261403c040 con 0x556725fcedc0 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.271+0000 7f26315f9740 10 client.25852 fetch_fsmap learned FSMap version 34 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.271+0000 7f26315f9740 1 -- 172.21.15.26:0/1840826524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556725fe05c0 con 0x556725fcedc0 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.271+0000 7f2611600640 1 -- 172.21.15.26:0/1840826524 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2614009800 con 0x556725fcedc0 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.271+0000 7f2611600640 1 -- 172.21.15.26:0/1840826524 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f2614048030 con 0x556725fcedc0 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f26315f9740 10 client.25852 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f26315f9740 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f26315f9740 20 client.25852 populate_metadata read hostname 'smithi026' 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f26315f9740 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556725fd4920 0x5567260521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f26315f9740 4 client.25852 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:04:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f26315f9740 1 -- 172.21.15.26:0/1840826524 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556725f39ea0 con 0x556725fd4920 2024-06-13T07:04:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f2627400640 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556725fd4920 0x5567260521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f2611600640 10 client.25852 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.272+0000 7f2627400640 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556725fd4920 0x5567260521c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0205606d584880 server_cookie=fef049d42740ec91 in_seq=0 out_seq=0 2024-06-13T07:04:52.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:51 smithi135 ceph-mon[36276]: pgmap v640: 129 pgs: 129 active+clean; 55 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1014 KiB/s wr, 166 op/s 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.348+0000 7f2611600640 1 -- 172.21.15.26:0/1840826524 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556725f39ea0 con 0x556725fd4920 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.348+0000 7f2611600640 10 client.25852 handle_command_reply: tid=0 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 client.25852 shutdown 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 -- 172.21.15.26:0/1840826524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 msgr2=0x556725fddcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fddcb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2614009c70 tx=0x7f2614009e50 comp rx=0 tx=0).stop 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 -- 172.21.15.26:0/1840826524 shutdown_connections 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556725fd4920 0x5567260521c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556725fcf7b0 0x556725fdecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556725fcedc0 0x556725fddcb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 --2- 172.21.15.26:0/1840826524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556725f96fb0 0x556725fdd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 -- 172.21.15.26:0/1840826524 >> 172.21.15.26:0/1840826524 conn(0x556725fd4460 msgr2=0x556725fc9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f2611600640 0 client.25852 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:04:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 -- 172.21.15.26:0/1840826524 shutdown_connections 2024-06-13T07:04:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.349+0000 7f26315f9740 1 -- 172.21.15.26:0/1840826524 wait complete. 2024-06-13T07:04:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.350+0000 7f26315f9740 20 client.25852 trim_cache size 0 max 16384 2024-06-13T07:04:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:04:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:04:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:04:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 -- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f260c0a2c00 msgr2=0x7f260c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f260c0a2c00 0x7f260c0a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x556725fd0010 tx=0x7f262002a8b0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 -- 172.21.15.26:0/1490495307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628100720 msgr2=0x7f262810b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628100720 0x7f262810b0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f261800e9c0 tx=0x7f261800ee80 comp rx=0 tx=0).stop 2024-06-13T07:04:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 -- 172.21.15.26:0/1490495307 shutdown_connections 2024-06-13T07:04:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f260c0a2c00 0x7f260c0a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26281022d0 0x7f2628109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2628101910 0x7f26281096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 --2- 172.21.15.26:0/1490495307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628100720 0x7f262810b0a0 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f261800e9c0 tx=0x7f261800ee80 comp rx=0 tx=0).stop 2024-06-13T07:04:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.351+0000 7f262d200640 1 -- 172.21.15.26:0/1490495307 >> 172.21.15.26:0/1490495307 conn(0x7f26280fbc90 msgr2=0x7f2628104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.352+0000 7f262d200640 1 -- 172.21.15.26:0/1490495307 shutdown_connections 2024-06-13T07:04:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:04:52.352+0000 7f262d200640 1 -- 172.21.15.26:0/1490495307 wait complete. 2024-06-13T07:04:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:51 smithi136 ceph-mon[37229]: pgmap v640: 129 pgs: 129 active+clean; 55 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1014 KiB/s wr, 166 op/s 2024-06-13T07:04:52.523 INFO:tasks.workunit.client.0.smithi026.stdout: 1 148788 3.99 MB/sec execute 66 sec latency 49.272 ms 2024-06-13T07:04:53.523 INFO:tasks.workunit.client.0.smithi026.stdout: 1 149376 4.00 MB/sec execute 67 sec latency 21.035 ms 2024-06-13T07:04:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:53 smithi026 ceph-mon[32041]: pgmap v641: 129 pgs: 129 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-13T07:04:54.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:53 smithi135 ceph-mon[36276]: pgmap v641: 129 pgs: 129 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-13T07:04:54.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:53 smithi136 ceph-mon[37229]: pgmap v641: 129 pgs: 129 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-13T07:04:54.523 INFO:tasks.workunit.client.0.smithi026.stdout: 1 149913 3.95 MB/sec execute 68 sec latency 23.255 ms 2024-06-13T07:04:55.523 INFO:tasks.workunit.client.0.smithi026.stdout: 1 150705 3.96 MB/sec execute 69 sec latency 26.644 ms 2024-06-13T07:04:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:55 smithi026 ceph-mon[32041]: pgmap v642: 129 pgs: 129 active+clean; 56 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 891 KiB/s wr, 134 op/s 2024-06-13T07:04:56.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:55 smithi135 ceph-mon[36276]: pgmap v642: 129 pgs: 129 active+clean; 56 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 891 KiB/s wr, 134 op/s 2024-06-13T07:04:56.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:55 smithi136 ceph-mon[37229]: pgmap v642: 129 pgs: 129 active+clean; 56 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 891 KiB/s wr, 134 op/s 2024-06-13T07:04:56.523 INFO:tasks.workunit.client.0.smithi026.stdout: 1 151972 4.01 MB/sec execute 70 sec latency 53.624 ms 2024-06-13T07:04:57.523 INFO:tasks.workunit.client.0.smithi026.stdout: 1 152882 4.04 MB/sec execute 71 sec latency 24.856 ms 2024-06-13T07:04:58.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:57 smithi026 ceph-mon[32041]: pgmap v643: 129 pgs: 129 active+clean; 53 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 878 KiB/s wr, 153 op/s 2024-06-13T07:04:58.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:57 smithi135 ceph-mon[36276]: pgmap v643: 129 pgs: 129 active+clean; 53 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 878 KiB/s wr, 153 op/s 2024-06-13T07:04:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:57 smithi136 ceph-mon[37229]: pgmap v643: 129 pgs: 129 active+clean; 53 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 878 KiB/s wr, 153 op/s 2024-06-13T07:04:58.523 INFO:tasks.workunit.client.0.smithi026.stdout: 1 153377 3.99 MB/sec execute 72 sec latency 27.280 ms 2024-06-13T07:04:59.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 154440 4.01 MB/sec execute 73 sec latency 28.633 ms 2024-06-13T07:05:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:04:59 smithi026 ceph-mon[32041]: pgmap v644: 129 pgs: 129 active+clean; 53 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 976 KiB/s wr, 159 op/s 2024-06-13T07:05:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:04:59 smithi135 ceph-mon[36276]: pgmap v644: 129 pgs: 129 active+clean; 53 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 976 KiB/s wr, 159 op/s 2024-06-13T07:05:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:04:59 smithi136 ceph-mon[37229]: pgmap v644: 129 pgs: 129 active+clean; 53 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 976 KiB/s wr, 159 op/s 2024-06-13T07:05:00.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 155546 4.05 MB/sec execute 74 sec latency 22.631 ms 2024-06-13T07:05:01.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 156412 4.08 MB/sec execute 75 sec latency 27.988 ms 2024-06-13T07:05:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:01 smithi026 ceph-mon[32041]: pgmap v645: 129 pgs: 129 active+clean; 55 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1004 KiB/s wr, 166 op/s 2024-06-13T07:05:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:01 smithi135 ceph-mon[36276]: pgmap v645: 129 pgs: 129 active+clean; 55 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1004 KiB/s wr, 166 op/s 2024-06-13T07:05:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:01 smithi136 ceph-mon[37229]: pgmap v645: 129 pgs: 129 active+clean; 55 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1004 KiB/s wr, 166 op/s 2024-06-13T07:05:02.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 156912 4.04 MB/sec execute 76 sec latency 27.628 ms 2024-06-13T07:05:03.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 158234 4.05 MB/sec execute 77 sec latency 19.955 ms 2024-06-13T07:05:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:03 smithi026 ceph-mon[32041]: pgmap v646: 129 pgs: 129 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-13T07:05:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:03 smithi135 ceph-mon[36276]: pgmap v646: 129 pgs: 129 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-13T07:05:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:03 smithi136 ceph-mon[37229]: pgmap v646: 129 pgs: 129 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-13T07:05:04.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 159105 4.09 MB/sec execute 78 sec latency 22.612 ms 2024-06-13T07:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 -- 172.21.15.26:0/999443311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701000f0 msgr2=0x7f0d70100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 --2- 172.21.15.26:0/999443311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701000f0 0x7f0d70100510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009900 tx=0x7f0d6402a410 comp rx=0 tx=0).stop 2024-06-13T07:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 -- 172.21.15.26:0/999443311 shutdown_connections 2024-06-13T07:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 --2- 172.21.15.26:0/999443311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d70100a50 0x7f0d70100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 --2- 172.21.15.26:0/999443311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d701013f0 0x7f0d70101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 --2- 172.21.15.26:0/999443311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701000f0 0x7f0d70100510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 -- 172.21.15.26:0/999443311 >> 172.21.15.26:0/999443311 conn(0x7f0d700fbc90 msgr2=0x7f0d700fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 -- 172.21.15.26:0/999443311 shutdown_connections 2024-06-13T07:05:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.511+0000 7f0d76000640 1 -- 172.21.15.26:0/999443311 wait complete. 2024-06-13T07:05:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.512+0000 7f0d76000640 1 Processor -- start 2024-06-13T07:05:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.512+0000 7f0d76000640 1 -- start start 2024-06-13T07:05:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d76000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d701000f0 0x7f0d70199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d76000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100a50 0x7f0d7019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d76000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701013f0 0x7f0d701a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d76000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010f660 con 0x7f0d70100a50 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d76000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010f4e0 con 0x7f0d701000f0 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d76000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010f7e0 con 0x7f0d701013f0 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d701000f0 0x7f0d70199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d75600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701013f0 0x7f0d701a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100a50 0x7f0d7019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d701000f0 0x7f0d70199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:33184/0 (socket says 172.21.15.26:33184) 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.513+0000 7f0d74c00640 1 -- 172.21.15.26:0/2579034680 learned_addr learned my addr 172.21.15.26:0/2579034680 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d75600640 1 -- 172.21.15.26:0/2579034680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d701000f0 msgr2=0x7f0d70199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d75600640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d701000f0 0x7f0d70199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d75600640 1 -- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100a50 msgr2=0x7f0d7019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d75600640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100a50 0x7f0d7019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d75600640 1 -- 172.21.15.26:0/2579034680 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d701a1960 con 0x7f0d701013f0 2024-06-13T07:05:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d75600640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701013f0 0x7f0d701a12c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009fd0 tx=0x7f0d6800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d6d600640 1 -- 172.21.15.26:0/2579034680 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800ce20 con 0x7f0d701013f0 2024-06-13T07:05:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d6d600640 1 -- 172.21.15.26:0/2579034680 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d68005020 con 0x7f0d701013f0 2024-06-13T07:05:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d6d600640 1 -- 172.21.15.26:0/2579034680 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68010720 con 0x7f0d701013f0 2024-06-13T07:05:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.514+0000 7f0d76000640 1 -- 172.21.15.26:0/2579034680 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d701a1c50 con 0x7f0d701013f0 2024-06-13T07:05:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.515+0000 7f0d76000640 1 -- 172.21.15.26:0/2579034680 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701a2150 con 0x7f0d701013f0 2024-06-13T07:05:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d7a483740 1 Processor -- start 2024-06-13T07:05:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d6d600640 1 -- 172.21.15.26:0/2579034680 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d68005540 con 0x7f0d701013f0 2024-06-13T07:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d7a483740 1 -- start start 2024-06-13T07:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35cdc0 0x55e3cc35d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc364db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d7a483740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e3cc2c0420 con 0x55e3cc35cdc0 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d7a483740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e3cc267c50 con 0x55e3cc362920 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.518+0000 7f0d7a483740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e3cc2bc3e0 con 0x55e3cc35d7b0 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35cdc0 0x55e3cc35d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35cdc0 0x55e3cc35d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44370/0 (socket says 172.21.15.26:44370) 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d6fe00640 1 -- 172.21.15.26:0/3773041379 learned_addr learned my addr 172.21.15.26:0/3773041379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d6fe00640 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc364db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d74c00640 1 -- 172.21.15.26:0/3773041379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 msgr2=0x55e3cc364db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d74c00640 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc364db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d74c00640 1 -- 172.21.15.26:0/3773041379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35cdc0 msgr2=0x55e3cc35d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d74c00640 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35cdc0 0x55e3cc35d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d74c00640 1 -- 172.21.15.26:0/3773041379 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3cbfd4210 con 0x55e3cc35d7b0 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.519+0000 7f0d6fe00640 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc364db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.520+0000 7f0d74c00640 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3622b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d7019b220 tx=0x7f0d64005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7c59dbddefb81 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.520+0000 7f0d6d600640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0d580a2c00 0x7f0d580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.520+0000 7f0d5dc00640 1 -- 172.21.15.26:0/3773041379 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64040070 con 0x55e3cc35d7b0 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.520+0000 7f0d6d600640 1 -- 172.21.15.26:0/2579034680 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0d680cfba0 con 0x7f0d701013f0 2024-06-13T07:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.520+0000 7f0d5dc00640 1 -- 172.21.15.26:0/3773041379 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d64005030 con 0x55e3cc35d7b0 2024-06-13T07:05:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.520+0000 7f0d5dc00640 1 -- 172.21.15.26:0/3773041379 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64035040 con 0x55e3cc35d7b0 2024-06-13T07:05:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.520+0000 7f0d6fe00640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0d580a2c00 0x7f0d580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.520+0000 7f0d7a483740 1 -- 172.21.15.26:0/3773041379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35d7b0 msgr2=0x55e3cc3622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3622b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d7019b220 tx=0x7f0d64005dd0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 -- 172.21.15.26:0/3773041379 shutdown_connections 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d6fe00640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0d580a2c00 0x7f0d580a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55e3cc35e010 tx=0x7f0d60015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc364db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3622b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3773041379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35cdc0 0x55e3cc35d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 -- 172.21.15.26:0/3773041379 >> 172.21.15.26:0/3773041379 conn(0x55e3cc324fb0 msgr2=0x55e3cc358840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 -- 172.21.15.26:0/3773041379 shutdown_connections 2024-06-13T07:05:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 -- 172.21.15.26:0/3773041379 wait complete. 2024-06-13T07:05:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.521+0000 7f0d7a483740 1 -- start start 2024-06-13T07:05:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d7a483740 1 Processor -- start 2024-06-13T07:05:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc367060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc36e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d7a483740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e3cc365870 con 0x55e3cc35d7b0 2024-06-13T07:05:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d7a483740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e3cc27b0b0 con 0x55e3cc362920 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d7a483740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e3cc3659f0 con 0x55e3cc35cdc0 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d7a483740 10 client.0 _ll_register_callbacks cb 0x55e3cc266f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc367060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d75600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc36e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc367060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39442/0 (socket says 172.21.15.26:39442) 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d74c00640 1 -- 172.21.15.26:0/1647600159 learned_addr learned my addr 172.21.15.26:0/1647600159 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.522+0000 7f0d6fe00640 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d75600640 1 -- 172.21.15.26:0/1647600159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 msgr2=0x55e3cc367060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d75600640 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc367060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d75600640 1 -- 172.21.15.26:0/1647600159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 msgr2=0x55e3cc3675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d75600640 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d75600640 1 -- 172.21.15.26:0/1647600159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3cc0a0620 con 0x55e3cc362920 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d74c00640 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc367060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d75600640 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc36e5e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d6809bcb0 tx=0x7f0d6809c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63d789ef27e4dabe server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d52000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d52000640 1 -- 172.21.15.26:0/1647600159 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680a4990 con 0x55e3cc362920 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d52000640 1 -- 172.21.15.26:0/1647600159 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0d6809b400 con 0x55e3cc362920 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.523+0000 7f0d7a483740 1 -- 172.21.15.26:0/1647600159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d64005030 con 0x55e3cc362920 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d52000640 1 -- 172.21.15.26:0/1647600159 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d680a0360 con 0x55e3cc362920 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d7a483740 10 client.25820 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d7a483740 1 -- 172.21.15.26:0/1647600159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e3cc371a10 con 0x55e3cc362920 2024-06-13T07:05:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d52000640 1 -- 172.21.15.26:0/1647600159 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d64005030 con 0x55e3cc362920 2024-06-13T07:05:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d52000640 1 -- 172.21.15.26:0/1647600159 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f0d68096a90 con 0x55e3cc362920 2024-06-13T07:05:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d7a483740 10 client.25820 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d7a483740 10 client.25820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d7a483740 20 client.25820 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e3cc3730f0 0x55e3cc375520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d7a483740 4 client.25820 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.524+0000 7f0d7a483740 1 -- 172.21.15.26:0/1647600159 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3cc2cdb50 con 0x55e3cc3730f0 2024-06-13T07:05:05.531 INFO:tasks.workunit.client.0.smithi026.stdout: 1 159959 4.11 MB/sec execute 79 sec latency 51.050 ms 2024-06-13T07:05:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.525+0000 7f0d74c00640 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e3cc3730f0 0x55e3cc375520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.525+0000 7f0d74c00640 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e3cc3730f0 0x55e3cc375520 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c72bb118ee4b873 server_cookie=d6c51cf4f2cf2a9c in_seq=0 out_seq=0 2024-06-13T07:05:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.525+0000 7f0d52000640 10 client.25820 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.526+0000 7f0d52000640 1 -- 172.21.15.26:0/1647600159 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e3cc2cdb50 con 0x55e3cc3730f0 2024-06-13T07:05:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.526+0000 7f0d52000640 10 client.25820 handle_command_reply: tid=0 2024-06-13T07:05:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.526+0000 7f0d7a483740 1 client.25820 shutdown 2024-06-13T07:05:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.526+0000 7f0d7a483740 1 -- 172.21.15.26:0/1647600159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 msgr2=0x55e3cc36e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc36e5e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d6809bcb0 tx=0x7f0d6809c6b0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 -- 172.21.15.26:0/1647600159 shutdown_connections 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e3cc3730f0 0x55e3cc375520 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc362920 0x55e3cc36e5e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1647600159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc367060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 -- 172.21.15.26:0/1647600159 >> 172.21.15.26:0/1647600159 conn(0x55e3cc324fb0 msgr2=0x55e3cc3579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d52000640 0 client.25820 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 -- 172.21.15.26:0/1647600159 shutdown_connections 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 1 -- 172.21.15.26:0/1647600159 wait complete. 2024-06-13T07:05:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.527+0000 7f0d7a483740 20 client.25820 trim_cache size 0 max 16384 2024-06-13T07:05:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.546+0000 7f0d7a483740 1 Processor -- start 2024-06-13T07:05:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.546+0000 7f0d7a483740 1 -- start start 2024-06-13T07:05:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.546+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc324fb0 0x55e3cc35ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.546+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.546+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 0x55e3cc35f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.546+0000 7f0d7a483740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701aefe0 con 0x55e3cc35d7b0 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.546+0000 7f0d7a483740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701ac880 con 0x55e3cc35cdc0 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.546+0000 7f0d7a483740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701aa6d0 con 0x55e3cc324fb0 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc324fb0 0x55e3cc35ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:33220/0 (socket says 172.21.15.26:33220) 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc324fb0 0x55e3cc35ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39456/0 (socket says 172.21.15.26:39456) 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d6fe00640 1 -- 172.21.15.26:0/1504410478 learned_addr learned my addr 172.21.15.26:0/1504410478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d75600640 1 --2- 172.21.15.26:0/1504410478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 0x55e3cc35f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d74c00640 1 -- 172.21.15.26:0/1504410478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35cdc0 msgr2=0x55e3cc36e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d74c00640 1 --2- 172.21.15.26:0/1504410478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d74c00640 1 -- 172.21.15.26:0/1504410478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 msgr2=0x55e3cc35f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d74c00640 1 --2- 172.21.15.26:0/1504410478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 0x55e3cc35f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d74c00640 1 -- 172.21.15.26:0/1504410478 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3cc365b20 con 0x55e3cc324fb0 2024-06-13T07:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.547+0000 7f0d74c00640 1 --2- 172.21.15.26:0/1504410478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc324fb0 0x55e3cc35ead0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0d6402a450 tx=0x7f0d640315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1a61b4fda24c10 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.548+0000 7f0d5d200640 1 -- 172.21.15.26:0/1504410478 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64040050 con 0x55e3cc324fb0 2024-06-13T07:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.548+0000 7f0d5d200640 1 -- 172.21.15.26:0/1504410478 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d64005510 con 0x55e3cc324fb0 2024-06-13T07:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.548+0000 7f0d5d200640 1 -- 172.21.15.26:0/1504410478 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64035040 con 0x55e3cc324fb0 2024-06-13T07:05:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.548+0000 7f0d7a483740 1 -- 172.21.15.26:0/1504410478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc324fb0 msgr2=0x55e3cc35ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.548+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1504410478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc324fb0 0x55e3cc35ead0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0d6402a450 tx=0x7f0d640315b0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 -- 172.21.15.26:0/1504410478 shutdown_connections 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1504410478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1504410478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc35d7b0 0x55e3cc35f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 --2- 172.21.15.26:0/1504410478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc324fb0 0x55e3cc35ead0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 -- 172.21.15.26:0/1504410478 >> 172.21.15.26:0/1504410478 conn(0x55e3cc362460 msgr2=0x55e3cc367060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 -- 172.21.15.26:0/1504410478 shutdown_connections 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 -- 172.21.15.26:0/1504410478 wait complete. 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 -- start start 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.549+0000 7f0d7a483740 1 Processor -- start 2024-06-13T07:05:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc324fb0 0x55e3cc36b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d7a483740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3dbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d7a483740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701aad40 con 0x55e3cc324fb0 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d7a483740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701ac1b0 con 0x55e3cc35d7b0 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d7a483740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701ab410 con 0x55e3cc35cdc0 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d7a483740 10 client.0 _ll_register_callbacks cb 0x7f0d68005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39472/0 (socket says 172.21.15.26:39472) 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d75600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3dbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d6fe00640 1 -- 172.21.15.26:0/3599092358 learned_addr learned my addr 172.21.15.26:0/3599092358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d74c00640 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc324fb0 0x55e3cc36b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d6fe00640 1 -- 172.21.15.26:0/3599092358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35d7b0 msgr2=0x55e3cc3dbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d6fe00640 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3dbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d6fe00640 1 -- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc324fb0 msgr2=0x55e3cc36b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d6fe00640 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc324fb0 0x55e3cc36b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d6fe00640 1 -- 172.21.15.26:0/3599092358 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3cc36be80 con 0x55e3cc35cdc0 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d75600640 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.550+0000 7f0d74c00640 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc324fb0 0x55e3cc36b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.551+0000 7f0d6fe00640 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36b810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009e70 tx=0x7f0d60005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a291e21424de49ba server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.551+0000 7f0d52000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.551+0000 7f0d52000640 1 -- 172.21.15.26:0/3599092358 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d600105f0 con 0x55e3cc35cdc0 2024-06-13T07:05:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.551+0000 7f0d7a483740 1 -- 172.21.15.26:0/3599092358 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d64005510 con 0x55e3cc35cdc0 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.551+0000 7f0d52000640 1 -- 172.21.15.26:0/3599092358 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0d60010790 con 0x55e3cc35cdc0 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.551+0000 7f0d52000640 1 -- 172.21.15.26:0/3599092358 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d60010990 con 0x55e3cc35cdc0 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.551+0000 7f0d7a483740 10 client.25870 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.551+0000 7f0d7a483740 1 -- 172.21.15.26:0/3599092358 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e3cc3dd530 con 0x55e3cc35cdc0 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d52000640 1 -- 172.21.15.26:0/3599092358 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d64005510 con 0x55e3cc35cdc0 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d52000640 1 -- 172.21.15.26:0/3599092358 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f0d60005020 con 0x55e3cc35cdc0 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d7a483740 10 client.25870 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d7a483740 10 client.25870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d7a483740 20 client.25870 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e3cc362920 0x55e3cc3dff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d7a483740 4 client.25870 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d7a483740 1 -- 172.21.15.26:0/3599092358 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3cc2c7ea0 con 0x55e3cc362920 2024-06-13T07:05:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.552+0000 7f0d74c00640 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e3cc362920 0x55e3cc3dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.553+0000 7f0d74c00640 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e3cc362920 0x55e3cc3dff80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=defcbfabcb27e0f0 server_cookie=558b8d78a3bcecad in_seq=0 out_seq=0 2024-06-13T07:05:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.553+0000 7f0d52000640 10 client.25870 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.636+0000 7f0d52000640 1 -- 172.21.15.26:0/3599092358 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e3cc2c7ea0 con 0x55e3cc362920 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.636+0000 7f0d52000640 10 client.25870 handle_command_reply: tid=0 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.636+0000 7f0d7a483740 1 client.25870 shutdown 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 -- 172.21.15.26:0/3599092358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 msgr2=0x55e3cc36b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36b810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009e70 tx=0x7f0d60005f90 comp rx=0 tx=0).stop 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 -- 172.21.15.26:0/3599092358 shutdown_connections 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e3cc362920 0x55e3cc3dff80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e3cc35d7b0 0x55e3cc3dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e3cc35cdc0 0x55e3cc36b810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 --2- 172.21.15.26:0/3599092358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3cc324fb0 0x55e3cc36b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d52000640 0 client.25870 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 -- 172.21.15.26:0/3599092358 >> 172.21.15.26:0/3599092358 conn(0x55e3cc362460 msgr2=0x55e3cc367060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 -- 172.21.15.26:0/3599092358 shutdown_connections 2024-06-13T07:05:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 1 -- 172.21.15.26:0/3599092358 wait complete. 2024-06-13T07:05:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.637+0000 7f0d7a483740 20 client.25870 trim_cache size 0 max 16384 2024-06-13T07:05:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:05:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:05:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:05:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 -- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0d580a2c00 msgr2=0x7f0d580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0d580a2c00 0x7f0d580a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55e3cc35e010 tx=0x7f0d60015040 comp rx=0 tx=0).stop 2024-06-13T07:05:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 -- 172.21.15.26:0/2579034680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701013f0 msgr2=0x7f0d701a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701013f0 0x7f0d701a12c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009fd0 tx=0x7f0d6800ef10 comp rx=0 tx=0).stop 2024-06-13T07:05:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 -- 172.21.15.26:0/2579034680 shutdown_connections 2024-06-13T07:05:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0d580a2c00 0x7f0d580a50c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d701013f0 0x7f0d701a12c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100a50 0x7f0d7019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 --2- 172.21.15.26:0/2579034680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d701000f0 0x7f0d70199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 -- 172.21.15.26:0/2579034680 >> 172.21.15.26:0/2579034680 conn(0x7f0d700fbc90 msgr2=0x7f0d700fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 -- 172.21.15.26:0/2579034680 shutdown_connections 2024-06-13T07:05:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:05.638+0000 7f0d76000640 1 -- 172.21.15.26:0/2579034680 wait complete. 2024-06-13T07:05:06.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:05 smithi026 ceph-mon[32041]: pgmap v647: 129 pgs: 129 active+clean; 57 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:05:06.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:05 smithi135 ceph-mon[36276]: pgmap v647: 129 pgs: 129 active+clean; 57 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:05:06.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:05 smithi136 ceph-mon[37229]: pgmap v647: 129 pgs: 129 active+clean; 57 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:05:06.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 160552 4.08 MB/sec execute 80 sec latency 42.657 ms 2024-06-13T07:05:07.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 162017 4.10 MB/sec execute 81 sec latency 26.381 ms 2024-06-13T07:05:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:07 smithi026 ceph-mon[32041]: pgmap v648: 129 pgs: 129 active+clean; 57 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-06-13T07:05:08.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:07 smithi135 ceph-mon[36276]: pgmap v648: 129 pgs: 129 active+clean; 57 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-06-13T07:05:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:07 smithi136 ceph-mon[37229]: pgmap v648: 129 pgs: 129 active+clean; 57 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-06-13T07:05:08.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 163137 4.18 MB/sec execute 82 sec latency 20.706 ms 2024-06-13T07:05:09.524 INFO:tasks.workunit.client.0.smithi026.stdout: 1 163650 4.15 MB/sec execute 83 sec latency 23.810 ms 2024-06-13T07:05:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:09 smithi026 ceph-mon[32041]: pgmap v649: 129 pgs: 129 active+clean; 57 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:05:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:09 smithi135 ceph-mon[36276]: pgmap v649: 129 pgs: 129 active+clean; 57 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:05:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:09 smithi136 ceph-mon[37229]: pgmap v649: 129 pgs: 129 active+clean; 57 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:05:10.525 INFO:tasks.workunit.client.0.smithi026.stdout: 1 164390 4.13 MB/sec execute 84 sec latency 32.794 ms 2024-06-13T07:05:11.525 INFO:tasks.workunit.client.0.smithi026.stdout: 1 165661 4.14 MB/sec execute 85 sec latency 54.388 ms 2024-06-13T07:05:12.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:11 smithi026 ceph-mon[32041]: pgmap v650: 129 pgs: 129 active+clean; 58 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-06-13T07:05:12.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:11 smithi135 ceph-mon[36276]: pgmap v650: 129 pgs: 129 active+clean; 58 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-06-13T07:05:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:11 smithi136 ceph-mon[37229]: pgmap v650: 129 pgs: 129 active+clean; 58 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-06-13T07:05:12.525 INFO:tasks.workunit.client.0.smithi026.stdout: 1 166995 4.22 MB/sec execute 86 sec latency 25.603 ms 2024-06-13T07:05:13.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:12 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:05:13.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:12 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:05:13.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:12 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:05:13.525 INFO:tasks.workunit.client.0.smithi026.stdout: 1 167635 4.19 MB/sec execute 87 sec latency 29.994 ms 2024-06-13T07:05:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:13 smithi026 ceph-mon[32041]: pgmap v651: 129 pgs: 129 active+clean; 58 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:05:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:05:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:05:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:05:14.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:13 smithi135 ceph-mon[36276]: pgmap v651: 129 pgs: 129 active+clean; 58 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:05:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:05:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:05:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:05:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:13 smithi136 ceph-mon[37229]: pgmap v651: 129 pgs: 129 active+clean; 58 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:05:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:05:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:05:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:05:14.525 INFO:tasks.workunit.client.0.smithi026.stdout: 1 168648 4.19 MB/sec execute 88 sec latency 24.672 ms 2024-06-13T07:05:15.525 INFO:tasks.workunit.client.0.smithi026.stdout: 1 169991 4.25 MB/sec execute 89 sec latency 27.568 ms 2024-06-13T07:05:16.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:15 smithi026 ceph-mon[32041]: pgmap v652: 129 pgs: 129 active+clean; 58 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:05:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:15 smithi135 ceph-mon[36276]: pgmap v652: 129 pgs: 129 active+clean; 58 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:05:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:15 smithi136 ceph-mon[37229]: pgmap v652: 129 pgs: 129 active+clean; 58 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:05:16.526 INFO:tasks.workunit.client.0.smithi026.stdout: 1 170665 4.25 MB/sec execute 90 sec latency 51.190 ms 2024-06-13T07:05:17.525 INFO:tasks.workunit.client.0.smithi026.stdout: 1 171456 4.23 MB/sec execute 91 sec latency 28.952 ms 2024-06-13T07:05:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:17 smithi026 ceph-mon[32041]: pgmap v653: 129 pgs: 129 active+clean; 59 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-13T07:05:18.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:17 smithi135 ceph-mon[36276]: pgmap v653: 129 pgs: 129 active+clean; 59 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-13T07:05:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:17 smithi136 ceph-mon[37229]: pgmap v653: 129 pgs: 129 active+clean; 59 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-13T07:05:18.526 INFO:tasks.workunit.client.0.smithi026.stdout: 1 172711 4.24 MB/sec execute 92 sec latency 19.722 ms 2024-06-13T07:05:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.803+0000 7fcf66e00640 1 -- 172.21.15.26:0/1892651886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60104e10 msgr2=0x7fcf60105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.803+0000 7fcf66e00640 1 --2- 172.21.15.26:0/1892651886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60104e10 0x7fcf60105290 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009900 tx=0x7fcf4802a410 comp rx=0 tx=0).stop 2024-06-13T07:05:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.803+0000 7fcf66e00640 1 -- 172.21.15.26:0/1892651886 shutdown_connections 2024-06-13T07:05:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.803+0000 7fcf66e00640 1 --2- 172.21.15.26:0/1892651886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf601057d0 0x7fcf6010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.803+0000 7fcf66e00640 1 --2- 172.21.15.26:0/1892651886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60104e10 0x7fcf60105290 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.803+0000 7fcf66e00640 1 --2- 172.21.15.26:0/1892651886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60103c20 0x7fcf60104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.803+0000 7fcf66e00640 1 -- 172.21.15.26:0/1892651886 >> 172.21.15.26:0/1892651886 conn(0x7fcf600ff190 msgr2=0x7fcf601015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.804+0000 7fcf66e00640 1 -- 172.21.15.26:0/1892651886 shutdown_connections 2024-06-13T07:05:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.804+0000 7fcf66e00640 1 -- 172.21.15.26:0/1892651886 wait complete. 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.805+0000 7fcf66e00640 1 Processor -- start 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.805+0000 7fcf66e00640 1 -- start start 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf66e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60103c20 0x7fcf60079750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf66e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60104e10 0x7fcf60079c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf66e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf601057d0 0x7fcf600762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf66e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010f8d0 con 0x7fcf60104e10 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf66e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010f750 con 0x7fcf601057d0 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf66e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010fa50 con 0x7fcf60103c20 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf5fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60103c20 0x7fcf60079750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf57400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60104e10 0x7fcf60079c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf5fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60103c20 0x7fcf60079750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43116/0 (socket says 172.21.15.26:43116) 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf5fe00640 1 -- 172.21.15.26:0/3641366212 learned_addr learned my addr 172.21.15.26:0/3641366212 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf64e00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf601057d0 0x7fcf600762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf5fe00640 1 -- 172.21.15.26:0/3641366212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf601057d0 msgr2=0x7fcf600762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf5fe00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf601057d0 0x7fcf600762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf5fe00640 1 -- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60104e10 msgr2=0x7fcf60079c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf5fe00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60104e10 0x7fcf60079c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf5fe00640 1 -- 172.21.15.26:0/3641366212 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf60076b80 con 0x7fcf60103c20 2024-06-13T07:05:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.806+0000 7fcf64e00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf601057d0 0x7fcf600762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.807+0000 7fcf5fe00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60103c20 0x7fcf60079750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9c0 tx=0x7fcf5000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.807+0000 7fcf5d600640 1 -- 172.21.15.26:0/3641366212 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000ccc0 con 0x7fcf60103c20 2024-06-13T07:05:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.807+0000 7fcf5d600640 1 -- 172.21.15.26:0/3641366212 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf500035f0 con 0x7fcf60103c20 2024-06-13T07:05:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.807+0000 7fcf5d600640 1 -- 172.21.15.26:0/3641366212 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000ce60 con 0x7fcf60103c20 2024-06-13T07:05:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.807+0000 7fcf66e00640 1 -- 172.21.15.26:0/3641366212 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601a62e0 con 0x7fcf60103c20 2024-06-13T07:05:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.807+0000 7fcf66e00640 1 -- 172.21.15.26:0/3641366212 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a6740 con 0x7fcf60103c20 2024-06-13T07:05:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf69da5740 1 Processor -- start 2024-06-13T07:05:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf5d600640 1 -- 172.21.15.26:0/3641366212 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf500037a0 con 0x7fcf60103c20 2024-06-13T07:05:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf69da5740 1 -- start start 2024-06-13T07:05:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5b9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5be0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf69da5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5608fc51c420 con 0x5608fc5be760 2024-06-13T07:05:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf69da5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5608fc4c3c50 con 0x5608fc5b8c00 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.811+0000 7fcf69da5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5608fc5183e0 con 0x5608fc5b95f0 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf64e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf64e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5b9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55938/0 (socket says 172.21.15.26:55938) 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf64e00640 1 -- 172.21.15.26:0/1459725201 learned_addr learned my addr 172.21.15.26:0/1459725201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf5fe00640 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5be0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf57400640 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf64e00640 1 -- 172.21.15.26:0/1459725201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 msgr2=0x5608fc5be0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf64e00640 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5be0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf64e00640 1 -- 172.21.15.26:0/1459725201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 msgr2=0x5608fc5c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf64e00640 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5c0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.812+0000 7fcf64e00640 1 -- 172.21.15.26:0/1459725201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608fc230210 con 0x5608fc5b8c00 2024-06-13T07:05:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.813+0000 7fcf64e00640 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5b9020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf58011810 tx=0x7fcf58011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a034307e03e92ff1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.813+0000 7fcf55600640 1 -- 172.21.15.26:0/1459725201 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58011310 con 0x5608fc5b8c00 2024-06-13T07:05:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.813+0000 7fcf55600640 1 -- 172.21.15.26:0/1459725201 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5800f9a0 con 0x5608fc5b8c00 2024-06-13T07:05:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.813+0000 7fcf55600640 1 -- 172.21.15.26:0/1459725201 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58005730 con 0x5608fc5b8c00 2024-06-13T07:05:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.813+0000 7fcf5d600640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcf440a2cb0 0x7fcf440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.813+0000 7fcf5d600640 1 -- 172.21.15.26:0/3641366212 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcf500cf750 con 0x7fcf60103c20 2024-06-13T07:05:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.813+0000 7fcf5fe00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcf440a2cb0 0x7fcf440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 -- 172.21.15.26:0/1459725201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 msgr2=0x5608fc5b9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5b9020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf58011810 tx=0x7fcf58011cd0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 -- 172.21.15.26:0/1459725201 shutdown_connections 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5be0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 --2- 172.21.15.26:0/1459725201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5b9020 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 -- 172.21.15.26:0/1459725201 >> 172.21.15.26:0/1459725201 conn(0x5608fc580fb0 msgr2=0x5608fc5b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 -- 172.21.15.26:0/1459725201 shutdown_connections 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 -- 172.21.15.26:0/1459725201 wait complete. 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf5fe00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcf440a2cb0 0x7fcf440a5170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcf500cce00 tx=0x7fcf500d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.814+0000 7fcf69da5740 1 -- start start 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.815+0000 7fcf69da5740 1 Processor -- start 2024-06-13T07:05:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.815+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.815+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5c3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.815+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.815+0000 7fcf69da5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5608fc5c1710 con 0x5608fc5be760 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.815+0000 7fcf69da5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5608fc4d70b0 con 0x5608fc5b8c00 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.815+0000 7fcf69da5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5608fc5c1890 con 0x5608fc5b95f0 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.815+0000 7fcf69da5740 10 client.0 _ll_register_callbacks cb 0x5608fc4c2f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5c3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43138/0 (socket says 172.21.15.26:43138) 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 -- 172.21.15.26:0/2983311049 learned_addr learned my addr 172.21.15.26:0/2983311049 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf64e00640 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf64e00640 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 -- 172.21.15.26:0/2983311049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 msgr2=0x5608fc5c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 -- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 msgr2=0x5608fc5ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 -- 172.21.15.26:0/2983311049 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608fc2fc620 con 0x5608fc5b95f0 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf64e00640 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf57400640 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5c3470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf48000c00 tx=0x7fcf480042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458f29465d820e2e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.816+0000 7fcf4e000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.817+0000 7fcf4e000640 1 -- 172.21.15.26:0/2983311049 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48035040 con 0x5608fc5b95f0 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.817+0000 7fcf4e000640 1 -- 172.21.15.26:0/2983311049 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcf48005450 con 0x5608fc5b95f0 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.817+0000 7fcf69da5740 1 -- 172.21.15.26:0/2983311049 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5800f9a0 con 0x5608fc5b95f0 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.817+0000 7fcf4e000640 1 -- 172.21.15.26:0/2983311049 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcf48005650 con 0x5608fc5b95f0 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.817+0000 7fcf69da5740 10 client.25882 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.817+0000 7fcf69da5740 1 -- 172.21.15.26:0/2983311049 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5608fc5cd8e0 con 0x5608fc5b95f0 2024-06-13T07:05:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.817+0000 7fcf4e000640 1 -- 172.21.15.26:0/2983311049 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5800f9a0 con 0x5608fc5b95f0 2024-06-13T07:05:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.818+0000 7fcf4e000640 1 -- 172.21.15.26:0/2983311049 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fcf48031b30 con 0x5608fc5b95f0 2024-06-13T07:05:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.818+0000 7fcf69da5740 10 client.25882 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.818+0000 7fcf69da5740 10 client.25882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.818+0000 7fcf69da5740 20 client.25882 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.818+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc5cf020 0x5608fc5d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.818+0000 7fcf69da5740 4 client.25882 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.818+0000 7fcf69da5740 1 -- 172.21.15.26:0/2983311049 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608fc529b50 con 0x5608fc5cf020 2024-06-13T07:05:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.818+0000 7fcf64e00640 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc5cf020 0x5608fc5d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.819+0000 7fcf64e00640 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc5cf020 0x5608fc5d1450 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82cf7d82fd36a297 server_cookie=998e25f6a086c774 in_seq=0 out_seq=0 2024-06-13T07:05:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.819+0000 7fcf4e000640 10 client.25882 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.820+0000 7fcf4e000640 1 -- 172.21.15.26:0/2983311049 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5608fc529b50 con 0x5608fc5cf020 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.820+0000 7fcf4e000640 10 client.25882 handle_command_reply: tid=0 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.820+0000 7fcf69da5740 1 client.25882 shutdown 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.820+0000 7fcf69da5740 1 -- 172.21.15.26:0/2983311049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 msgr2=0x5608fc5c3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.820+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5c3470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf48000c00 tx=0x7fcf480042e0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 1 -- 172.21.15.26:0/2983311049 shutdown_connections 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc5cf020 0x5608fc5d1450 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5be760 0x5608fc5ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5c3470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2983311049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 1 -- 172.21.15.26:0/2983311049 >> 172.21.15.26:0/2983311049 conn(0x5608fc580fb0 msgr2=0x5608fc5b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf4e000640 0 client.25882 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 1 -- 172.21.15.26:0/2983311049 shutdown_connections 2024-06-13T07:05:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 1 -- 172.21.15.26:0/2983311049 wait complete. 2024-06-13T07:05:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.821+0000 7fcf69da5740 20 client.25882 trim_cache size 0 max 16384 2024-06-13T07:05:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.840+0000 7fcf69da5740 1 Processor -- start 2024-06-13T07:05:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.840+0000 7fcf69da5740 1 -- start start 2024-06-13T07:05:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc580fb0 0x5608fc5ba8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf69da5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601b35e0 con 0x5608fc580fb0 2024-06-13T07:05:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf69da5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601b0e80 con 0x5608fc5b8c00 2024-06-13T07:05:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf69da5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601aecd0 con 0x5608fc5b95f0 2024-06-13T07:05:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf64e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc580fb0 0x5608fc5ba8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf57400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf64e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc580fb0 0x5608fc5ba8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33138/0 (socket says 172.21.15.26:33138) 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf64e00640 1 -- 172.21.15.26:0/2262519475 learned_addr learned my addr 172.21.15.26:0/2262519475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.841+0000 7fcf64e00640 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf57400640 1 -- 172.21.15.26:0/2262519475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 msgr2=0x5608fc5ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf57400640 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf57400640 1 -- 172.21.15.26:0/2262519475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc580fb0 msgr2=0x5608fc5ba8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf57400640 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc580fb0 0x5608fc5ba8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf57400640 1 -- 172.21.15.26:0/2262519475 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608fc5c3f20 con 0x5608fc5b8c00 2024-06-13T07:05:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf64e00640 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf57400640 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5bae10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009900 tx=0x7fcf480097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d8c4cc295d9cfa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf54c00640 1 -- 172.21.15.26:0/2262519475 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48057070 con 0x5608fc5b8c00 2024-06-13T07:05:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf54c00640 1 -- 172.21.15.26:0/2262519475 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf48040070 con 0x5608fc5b8c00 2024-06-13T07:05:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.842+0000 7fcf54c00640 1 -- 172.21.15.26:0/2262519475 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48035070 con 0x5608fc5b8c00 2024-06-13T07:05:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 -- 172.21.15.26:0/2262519475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 msgr2=0x5608fc5bae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5bae10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009900 tx=0x7fcf480097a0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 -- 172.21.15.26:0/2262519475 shutdown_connections 2024-06-13T07:05:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5bae10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc5b95f0 0x5608fc5ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 --2- 172.21.15.26:0/2262519475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc580fb0 0x5608fc5ba8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 -- 172.21.15.26:0/2262519475 >> 172.21.15.26:0/2262519475 conn(0x5608fc5be260 msgr2=0x5608fc5c2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 -- 172.21.15.26:0/2262519475 shutdown_connections 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.843+0000 7fcf69da5740 1 -- 172.21.15.26:0/2262519475 wait complete. 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.844+0000 7fcf69da5740 1 -- start start 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.844+0000 7fcf69da5740 1 Processor -- start 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.844+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc580fb0 0x5608fc5c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.844+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.844+0000 7fcf69da5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5b95f0 0x5608fc637d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.844+0000 7fcf69da5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601af340 con 0x5608fc5b95f0 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.844+0000 7fcf69da5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601b07b0 con 0x5608fc5b8c00 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.844+0000 7fcf69da5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601afa10 con 0x5608fc580fb0 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf69da5740 10 client.0 _ll_register_callbacks cb 0x7fcf580093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf57400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf57400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55966/0 (socket says 172.21.15.26:55966) 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf57400640 1 -- 172.21.15.26:0/3367799565 learned_addr learned my addr 172.21.15.26:0/3367799565 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf64e00640 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc580fb0 0x5608fc5c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf64e00640 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5b95f0 0x5608fc637d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf57400640 1 -- 172.21.15.26:0/3367799565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc580fb0 msgr2=0x5608fc5c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf57400640 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc580fb0 0x5608fc5c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf57400640 1 -- 172.21.15.26:0/3367799565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5b95f0 msgr2=0x5608fc637d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf57400640 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5b95f0 0x5608fc637d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.845+0000 7fcf57400640 1 -- 172.21.15.26:0/3367799565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608fc5c7e50 con 0x5608fc5b8c00 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf57400640 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c77b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf48000c00 tx=0x7fcf48031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58135fe35fac8334 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf4e000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf4e000640 1 -- 172.21.15.26:0/3367799565 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48035050 con 0x5608fc5b8c00 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf4e000640 1 -- 172.21.15.26:0/3367799565 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcf480318b0 con 0x5608fc5b8c00 2024-06-13T07:05:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf69da5740 1 -- 172.21.15.26:0/3367799565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf48040070 con 0x5608fc5b8c00 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf4e000640 1 -- 172.21.15.26:0/3367799565 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcf48057040 con 0x5608fc5b8c00 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf69da5740 10 client.25844 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf69da5740 1 -- 172.21.15.26:0/3367799565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5608fc6393a0 con 0x5608fc5b8c00 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.846+0000 7fcf4e000640 1 -- 172.21.15.26:0/3367799565 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf48040070 con 0x5608fc5b8c00 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf4e000640 1 -- 172.21.15.26:0/3367799565 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fcf4805a070 con 0x5608fc5b8c00 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf69da5740 10 client.25844 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf69da5740 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf69da5740 20 client.25844 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf69da5740 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc5be760 0x5608fc63bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf69da5740 4 client.25844 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf69da5740 1 -- 172.21.15.26:0/3367799565 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608fc523ea0 con 0x5608fc5be760 2024-06-13T07:05:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf64e00640 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc5be760 0x5608fc63bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.847+0000 7fcf64e00640 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc5be760 0x5608fc63bcd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e0fa3af516d7d9 server_cookie=cd1ed30790e9398f in_seq=0 out_seq=0 2024-06-13T07:05:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.848+0000 7fcf4e000640 10 client.25844 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf4e000640 1 -- 172.21.15.26:0/3367799565 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5608fc523ea0 con 0x5608fc5be760 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf4e000640 10 client.25844 handle_command_reply: tid=0 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 client.25844 shutdown 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 -- 172.21.15.26:0/3367799565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 msgr2=0x5608fc5c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c77b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf48000c00 tx=0x7fcf48031400 comp rx=0 tx=0).stop 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 -- 172.21.15.26:0/3367799565 shutdown_connections 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc5be760 0x5608fc63bcd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc5b8c00 0x5608fc5c77b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc5b95f0 0x5608fc637d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf4e000640 0 client.25844 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 --2- 172.21.15.26:0/3367799565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc580fb0 0x5608fc5c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 -- 172.21.15.26:0/3367799565 >> 172.21.15.26:0/3367799565 conn(0x5608fc5be260 msgr2=0x5608fc5b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 -- 172.21.15.26:0/3367799565 shutdown_connections 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 1 -- 172.21.15.26:0/3367799565 wait complete. 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.932+0000 7fcf69da5740 20 client.25844 trim_cache size 0 max 16384 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:05:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:05:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 -- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcf440a2cb0 msgr2=0x7fcf440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcf440a2cb0 0x7fcf440a5170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcf500cce00 tx=0x7fcf500d5530 comp rx=0 tx=0).stop 2024-06-13T07:05:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 -- 172.21.15.26:0/3641366212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60103c20 msgr2=0x7fcf60079750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60103c20 0x7fcf60079750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9c0 tx=0x7fcf5000edc0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 -- 172.21.15.26:0/3641366212 shutdown_connections 2024-06-13T07:05:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcf440a2cb0 0x7fcf440a5170 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf601057d0 0x7fcf600762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60104e10 0x7fcf60079c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 --2- 172.21.15.26:0/3641366212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf60103c20 0x7fcf60079750 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 -- 172.21.15.26:0/3641366212 >> 172.21.15.26:0/3641366212 conn(0x7fcf600ff190 msgr2=0x7fcf601080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 -- 172.21.15.26:0/3641366212 shutdown_connections 2024-06-13T07:05:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:18.933+0000 7fcf66e00640 1 -- 172.21.15.26:0/3641366212 wait complete. 2024-06-13T07:05:19.526 INFO:tasks.workunit.client.0.smithi026.stdout: 1 173914 4.31 MB/sec execute 93 sec latency 44.942 ms 2024-06-13T07:05:20.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:19 smithi026 ceph-mon[32041]: pgmap v654: 129 pgs: 129 active+clean; 60 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-13T07:05:20.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:19 smithi135 ceph-mon[36276]: pgmap v654: 129 pgs: 129 active+clean; 60 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-13T07:05:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:19 smithi136 ceph-mon[37229]: pgmap v654: 129 pgs: 129 active+clean; 60 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-13T07:05:20.526 INFO:tasks.workunit.client.0.smithi026.stdout: 1 174380 4.27 MB/sec execute 94 sec latency 31.597 ms 2024-06-13T07:05:21.526 INFO:tasks.workunit.client.0.smithi026.stdout: 1 175310 4.28 MB/sec execute 95 sec latency 26.852 ms 2024-06-13T07:05:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:21 smithi026 ceph-mon[32041]: pgmap v655: 129 pgs: 129 active+clean; 57 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:05:22.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:21 smithi135 ceph-mon[36276]: pgmap v655: 129 pgs: 129 active+clean; 57 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:05:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:21 smithi136 ceph-mon[37229]: pgmap v655: 129 pgs: 129 active+clean; 57 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:05:22.526 INFO:tasks.workunit.client.0.smithi026.stdout: 1 176706 4.30 MB/sec execute 96 sec latency 51.607 ms 2024-06-13T07:05:23.526 INFO:tasks.workunit.client.0.smithi026.stdout: 1 177667 4.34 MB/sec execute 97 sec latency 18.376 ms 2024-06-13T07:05:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:24 smithi026 ceph-mon[32041]: pgmap v656: 129 pgs: 129 active+clean; 59 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:05:24.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:24 smithi135 ceph-mon[36276]: pgmap v656: 129 pgs: 129 active+clean; 59 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:05:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:24 smithi136 ceph-mon[37229]: pgmap v656: 129 pgs: 129 active+clean; 59 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:05:24.526 INFO:tasks.workunit.client.0.smithi026.stdout: 1 178175 4.30 MB/sec execute 98 sec latency 30.649 ms 2024-06-13T07:05:25.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 179506 4.31 MB/sec execute 99 sec latency 21.061 ms 2024-06-13T07:05:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:26 smithi135 ceph-mon[36276]: pgmap v657: 129 pgs: 129 active+clean; 59 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-06-13T07:05:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:26 smithi136 ceph-mon[37229]: pgmap v657: 129 pgs: 129 active+clean; 59 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-06-13T07:05:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:26 smithi026 ceph-mon[32041]: pgmap v657: 129 pgs: 129 active+clean; 59 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-06-13T07:05:26.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 180571 4.34 MB/sec execute 100 sec latency 20.766 ms 2024-06-13T07:05:27.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 181354 4.36 MB/sec execute 101 sec latency 34.261 ms 2024-06-13T07:05:28.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:28 smithi135 ceph-mon[36276]: pgmap v658: 129 pgs: 129 active+clean; 60 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2024-06-13T07:05:28.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:28 smithi136 ceph-mon[37229]: pgmap v658: 129 pgs: 129 active+clean; 60 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2024-06-13T07:05:28.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:28 smithi026 ceph-mon[32041]: pgmap v658: 129 pgs: 129 active+clean; 60 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2024-06-13T07:05:28.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 182134 4.33 MB/sec execute 102 sec latency 27.358 ms 2024-06-13T07:05:29.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 183354 4.34 MB/sec execute 103 sec latency 31.889 ms 2024-06-13T07:05:30.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:30 smithi135 ceph-mon[36276]: pgmap v659: 129 pgs: 129 active+clean; 60 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-06-13T07:05:30.304 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:30 smithi026 ceph-mon[32041]: pgmap v659: 129 pgs: 129 active+clean; 60 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-06-13T07:05:30.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:30 smithi136 ceph-mon[37229]: pgmap v659: 129 pgs: 129 active+clean; 60 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-06-13T07:05:30.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 184540 4.40 MB/sec execute 104 sec latency 20.137 ms 2024-06-13T07:05:31.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 185002 4.37 MB/sec execute 105 sec latency 26.668 ms 2024-06-13T07:05:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 -- 172.21.15.26:0/2404045381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8330109710 msgr2=0x7f833010bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 --2- 172.21.15.26:0/2404045381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8330109710 0x7f833010bba0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8328009900 tx=0x7f832802a480 comp rx=0 tx=0).stop 2024-06-13T07:05:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 -- 172.21.15.26:0/2404045381 shutdown_connections 2024-06-13T07:05:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 --2- 172.21.15.26:0/2404045381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8330109710 0x7f833010bba0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 --2- 172.21.15.26:0/2404045381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8330074380 0x7f83300747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 --2- 172.21.15.26:0/2404045381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8330075960 0x7f8330073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 -- 172.21.15.26:0/2404045381 >> 172.21.15.26:0/2404045381 conn(0x7f83300fcdd0 msgr2=0x7f83300ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 -- 172.21.15.26:0/2404045381 shutdown_connections 2024-06-13T07:05:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.098+0000 7f8335a00640 1 -- 172.21.15.26:0/2404045381 wait complete. 2024-06-13T07:05:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.099+0000 7f8335a00640 1 Processor -- start 2024-06-13T07:05:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8335a00640 1 -- start start 2024-06-13T07:05:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8335a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8330074380 0x7f8330199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8335a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8330075960 0x7f833019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8335a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8330109710 0x7f83301a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8335a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f833010f740 con 0x7f8330109710 2024-06-13T07:05:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8335a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f833010f5c0 con 0x7f8330074380 2024-06-13T07:05:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8335a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f833010f8c0 con 0x7f8330075960 2024-06-13T07:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f832fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8330074380 0x7f8330199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f832fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8330074380 0x7f8330199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:57288/0 (socket says 172.21.15.26:57288) 2024-06-13T07:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f832fe00640 1 -- 172.21.15.26:0/222290127 learned_addr learned my addr 172.21.15.26:0/222290127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8327400640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8330075960 0x7f833019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f8335000640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8330109710 0x7f83301a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f832fe00640 1 -- 172.21.15.26:0/222290127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8330075960 msgr2=0x7f833019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f832fe00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8330075960 0x7f833019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.100+0000 7f832fe00640 1 -- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8330109710 msgr2=0x7f83301a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.101+0000 7f832fe00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8330109710 0x7f83301a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.101+0000 7f832fe00640 1 -- 172.21.15.26:0/222290127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83301a1900 con 0x7f8330074380 2024-06-13T07:05:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.101+0000 7f8335000640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8330109710 0x7f83301a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.101+0000 7f832fe00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8330074380 0x7f8330199cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f832000eb90 tx=0x7f832000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.102+0000 7f832d600640 1 -- 172.21.15.26:0/222290127 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320004d20 con 0x7f8330074380 2024-06-13T07:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.102+0000 7f832d600640 1 -- 172.21.15.26:0/222290127 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832000ee90 con 0x7f8330074380 2024-06-13T07:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.102+0000 7f8335a00640 1 -- 172.21.15.26:0/222290127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83301a1bf0 con 0x7f8330074380 2024-06-13T07:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.102+0000 7f832d600640 1 -- 172.21.15.26:0/222290127 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320010830 con 0x7f8330074380 2024-06-13T07:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.102+0000 7f8335a00640 1 -- 172.21.15.26:0/222290127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83301a20f0 con 0x7f8330074380 2024-06-13T07:05:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.105+0000 7f8339e5b740 1 Processor -- start 2024-06-13T07:05:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.105+0000 7f8339e5b740 1 -- start start 2024-06-13T07:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe160dc0 0x55bcbe1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe1662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe166920 0x55bcbe168db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8339e5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bcbe0c4420 con 0x55bcbe1617b0 2024-06-13T07:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8339e5b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bcbe06bc50 con 0x55bcbe166920 2024-06-13T07:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8339e5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bcbe0c03e0 con 0x55bcbe160dc0 2024-06-13T07:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f832d600640 1 -- 172.21.15.26:0/222290127 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8320005020 con 0x7f8330074380 2024-06-13T07:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8335000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe160dc0 0x55bcbe1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8335000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe160dc0 0x55bcbe1611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41612/0 (socket says 172.21.15.26:41612) 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8335000640 1 -- 172.21.15.26:0/9053181 learned_addr learned my addr 172.21.15.26:0/9053181 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8327400640 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe166920 0x55bcbe168db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f832fe00640 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe1662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8335000640 1 -- 172.21.15.26:0/9053181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe166920 msgr2=0x55bcbe168db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8335000640 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe166920 0x55bcbe168db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8335000640 1 -- 172.21.15.26:0/9053181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 msgr2=0x55bcbe1662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8335000640 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe1662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8335000640 1 -- 172.21.15.26:0/9053181 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcbddd8210 con 0x55bcbe160dc0 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.106+0000 7f8327400640 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe166920 0x55bcbe168db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.107+0000 7f8335000640 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe160dc0 0x55bcbe1611e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8328031440 tx=0x7f83280040b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ae0ae0a7e20b0c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.107+0000 7f8325600640 1 -- 172.21.15.26:0/9053181 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328035030 con 0x55bcbe160dc0 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.107+0000 7f8325600640 1 -- 172.21.15.26:0/9053181 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83280044a0 con 0x55bcbe160dc0 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.107+0000 7f8325600640 1 -- 172.21.15.26:0/9053181 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280316b0 con 0x55bcbe160dc0 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.107+0000 7f832d600640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f83140a2b80 0x7f83140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f832d600640 1 -- 172.21.15.26:0/222290127 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f83200cfc10 con 0x7f8330074380 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 -- 172.21.15.26:0/9053181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe160dc0 msgr2=0x55bcbe1611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe160dc0 0x55bcbe1611e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8328031440 tx=0x7f83280040b0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f832fe00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f83140a2b80 0x7f83140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 -- 172.21.15.26:0/9053181 shutdown_connections 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe166920 0x55bcbe168db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe1662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 --2- 172.21.15.26:0/9053181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe160dc0 0x55bcbe1611e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 -- 172.21.15.26:0/9053181 >> 172.21.15.26:0/9053181 conn(0x55bcbe128fb0 msgr2=0x55bcbe15c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 -- 172.21.15.26:0/9053181 shutdown_connections 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.108+0000 7f8339e5b740 1 -- 172.21.15.26:0/9053181 wait complete. 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f832fe00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f83140a2b80 0x7f83140a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f83200983f0 tx=0x7f8320005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f8339e5b740 1 -- start start 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f8339e5b740 1 Processor -- start 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe16b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 0x55bcbe1726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f8339e5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bcbe169870 con 0x55bcbe1617b0 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f8339e5b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bcbe07f0b0 con 0x55bcbe160dc0 2024-06-13T07:05:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.109+0000 7f8339e5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bcbe1699f0 con 0x55bcbe166920 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8335000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8335000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:57314/0 (socket says 172.21.15.26:57314) 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8335000640 1 -- 172.21.15.26:0/2904176666 learned_addr learned my addr 172.21.15.26:0/2904176666 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8327400640 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe16b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8339e5b740 10 client.0 _ll_register_callbacks cb 0x55bcbe06af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8335000640 1 -- 172.21.15.26:0/2904176666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 msgr2=0x55bcbe1726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8335000640 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 0x55bcbe1726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8335000640 1 -- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 msgr2=0x55bcbe16b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8335000640 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe16b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8335000640 1 -- 172.21.15.26:0/2904176666 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcbdea4620 con 0x55bcbe160dc0 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.110+0000 7f8327400640 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe16b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.111+0000 7f8335000640 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8328009850 tx=0x7f8328005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19ffd8c1700a172 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.111+0000 7f831e000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:05:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.111+0000 7f831e000640 1 -- 172.21.15.26:0/2904176666 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328035040 con 0x55bcbe160dc0 2024-06-13T07:05:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.111+0000 7f8339e5b740 1 -- 172.21.15.26:0/2904176666 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83280044a0 con 0x55bcbe160dc0 2024-06-13T07:05:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.111+0000 7f831e000640 1 -- 172.21.15.26:0/2904176666 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f83280055c0 con 0x55bcbe160dc0 2024-06-13T07:05:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.111+0000 7f831e000640 1 -- 172.21.15.26:0/2904176666 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f83280316b0 con 0x55bcbe160dc0 2024-06-13T07:05:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.111+0000 7f8339e5b740 10 client.25856 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.111+0000 7f8339e5b740 1 -- 172.21.15.26:0/2904176666 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bcbe175a30 con 0x55bcbe160dc0 2024-06-13T07:05:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f831e000640 1 -- 172.21.15.26:0/2904176666 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83280044a0 con 0x55bcbe160dc0 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f831e000640 1 -- 172.21.15.26:0/2904176666 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f8328003ca0 con 0x55bcbe160dc0 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f8339e5b740 10 client.25856 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f8339e5b740 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f8339e5b740 20 client.25856 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f8339e5b740 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bcbe176d80 0x55bcbe179170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f8339e5b740 4 client.25856 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f8339e5b740 1 -- 172.21.15.26:0/2904176666 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcbe0d1b50 con 0x55bcbe176d80 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.112+0000 7f8327400640 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bcbe176d80 0x55bcbe179170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.113+0000 7f8327400640 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bcbe176d80 0x55bcbe179170 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce489faccab71318 server_cookie=bdda40ec105299cc in_seq=0 out_seq=0 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.113+0000 7f831e000640 10 client.25856 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f831e000640 1 -- 172.21.15.26:0/2904176666 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bcbe0d1b50 con 0x55bcbe176d80 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f831e000640 10 client.25856 handle_command_reply: tid=0 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 client.25856 shutdown 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 -- 172.21.15.26:0/2904176666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 msgr2=0x55bcbe16b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8328009850 tx=0x7f8328005040 comp rx=0 tx=0).stop 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 -- 172.21.15.26:0/2904176666 shutdown_connections 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bcbe176d80 0x55bcbe179170 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 0x55bcbe1726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe1617b0 0x55bcbe16b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 --2- 172.21.15.26:0/2904176666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 -- 172.21.15.26:0/2904176666 >> 172.21.15.26:0/2904176666 conn(0x55bcbe128fb0 msgr2=0x55bcbe15b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f831e000640 0 client.25856 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 -- 172.21.15.26:0/2904176666 shutdown_connections 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.114+0000 7f8339e5b740 1 -- 172.21.15.26:0/2904176666 wait complete. 2024-06-13T07:05:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.115+0000 7f8339e5b740 20 client.25856 trim_cache size 0 max 16384 2024-06-13T07:05:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.134+0000 7f8339e5b740 1 Processor -- start 2024-06-13T07:05:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.134+0000 7f8339e5b740 1 -- start start 2024-06-13T07:05:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.134+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.134+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe1617b0 0x55bcbe1726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.134+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe166920 0x55bcbe15c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.134+0000 7f8339e5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83301aeff0 con 0x55bcbe166920 2024-06-13T07:05:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.134+0000 7f8339e5b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83301ac890 con 0x55bcbe160dc0 2024-06-13T07:05:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.134+0000 7f8339e5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f83301aa6e0 con 0x55bcbe1617b0 2024-06-13T07:05:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe1617b0 0x55bcbe1726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8335000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe166920 0x55bcbe15c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe1617b0 0x55bcbe1726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41636/0 (socket says 172.21.15.26:41636) 2024-06-13T07:05:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 -- 172.21.15.26:0/193590435 learned_addr learned my addr 172.21.15.26:0/193590435 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8335000640 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 -- 172.21.15.26:0/193590435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 msgr2=0x55bcbe16b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 -- 172.21.15.26:0/193590435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe166920 msgr2=0x55bcbe15c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe166920 0x55bcbe15c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 -- 172.21.15.26:0/193590435 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcbe165e40 con 0x55bcbe1617b0 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8335000640 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe166920 0x55bcbe15c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.135+0000 7f8327400640 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe1617b0 0x55bcbe1726a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55bcbe1798e0 tx=0x7f8318005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41145f5c3ce958a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8324c00640 1 -- 172.21.15.26:0/193590435 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318012110 con 0x55bcbe1617b0 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8324c00640 1 -- 172.21.15.26:0/193590435 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318003ad0 con 0x55bcbe1617b0 2024-06-13T07:05:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8324c00640 1 -- 172.21.15.26:0/193590435 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318010960 con 0x55bcbe1617b0 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8339e5b740 1 -- 172.21.15.26:0/193590435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe1617b0 msgr2=0x55bcbe1726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8339e5b740 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe1617b0 0x55bcbe1726a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55bcbe1798e0 tx=0x7f8318005f90 comp rx=0 tx=0).stop 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8339e5b740 1 -- 172.21.15.26:0/193590435 shutdown_connections 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8339e5b740 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe166920 0x55bcbe15c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8339e5b740 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe1617b0 0x55bcbe1726a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.136+0000 7f8339e5b740 1 --2- 172.21.15.26:0/193590435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe160dc0 0x55bcbe16b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.137+0000 7f8339e5b740 1 -- 172.21.15.26:0/193590435 >> 172.21.15.26:0/193590435 conn(0x55bcbe128fb0 msgr2=0x55bcbe15f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.137+0000 7f8339e5b740 1 -- 172.21.15.26:0/193590435 shutdown_connections 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.137+0000 7f8339e5b740 1 -- 172.21.15.26:0/193590435 wait complete. 2024-06-13T07:05:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.137+0000 7f8339e5b740 1 -- start start 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.137+0000 7f8339e5b740 1 Processor -- start 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe160dc0 0x55bcbe16fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe1617b0 0x55bcbe170330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8339e5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 0x55bcbe171340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8339e5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83301aad50 con 0x55bcbe160dc0 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8339e5b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83301ac1c0 con 0x55bcbe1617b0 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8339e5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f83301ab420 con 0x55bcbe166920 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8339e5b740 10 client.0 _ll_register_callbacks cb 0x7f83200093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8335000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe160dc0 0x55bcbe16fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8335000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe160dc0 0x55bcbe16fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53608/0 (socket says 172.21.15.26:53608) 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8335000640 1 -- 172.21.15.26:0/3126732790 learned_addr learned my addr 172.21.15.26:0/3126732790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8327400640 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe1617b0 0x55bcbe170330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.138+0000 7f8335000640 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 0x55bcbe171340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f8335000640 1 -- 172.21.15.26:0/3126732790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 msgr2=0x55bcbe171340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f8335000640 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 0x55bcbe171340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f8335000640 1 -- 172.21.15.26:0/3126732790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe1617b0 msgr2=0x55bcbe170330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f8335000640 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe1617b0 0x55bcbe170330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f8335000640 1 -- 172.21.15.26:0/3126732790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcbe171930 con 0x55bcbe160dc0 2024-06-13T07:05:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f8327400640 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe1617b0 0x55bcbe170330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f8335000640 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe160dc0 0x55bcbe16fdf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8328009900 tx=0x7f83280049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e75c7101314722 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f831e000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:05:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f831e000640 1 -- 172.21.15.26:0/3126732790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328035040 con 0x55bcbe160dc0 2024-06-13T07:05:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f8339e5b740 1 -- 172.21.15.26:0/3126732790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8318003ad0 con 0x55bcbe160dc0 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.139+0000 7f831e000640 1 -- 172.21.15.26:0/3126732790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8328004240 con 0x55bcbe160dc0 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f831e000640 1 -- 172.21.15.26:0/3126732790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8328004440 con 0x55bcbe160dc0 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f8339e5b740 10 client.15765 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f8339e5b740 1 -- 172.21.15.26:0/3126732790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bcbe1e19a0 con 0x55bcbe160dc0 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f831e000640 1 -- 172.21.15.26:0/3126732790 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318003ad0 con 0x55bcbe160dc0 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f831e000640 1 -- 172.21.15.26:0/3126732790 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f8328003b30 con 0x55bcbe160dc0 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f8339e5b740 10 client.15765 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f8339e5b740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f8339e5b740 20 client.15765 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f8339e5b740 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bcbe1e2370 0x55bcbe1e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f8339e5b740 4 client.15765 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.140+0000 7f8339e5b740 1 -- 172.21.15.26:0/3126732790 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcbe0cbea0 con 0x55bcbe1e2370 2024-06-13T07:05:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.141+0000 7f8327400640 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bcbe1e2370 0x55bcbe1e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.142+0000 7f8327400640 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bcbe1e2370 0x55bcbe1e4790 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af3f5c1c148c462 server_cookie=7d0240f649e6b33a in_seq=0 out_seq=0 2024-06-13T07:05:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.142+0000 7f831e000640 10 client.15765 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f831e000640 1 -- 172.21.15.26:0/3126732790 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bcbe0cbea0 con 0x55bcbe1e2370 2024-06-13T07:05:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f831e000640 10 client.15765 handle_command_reply: tid=0 2024-06-13T07:05:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 client.15765 shutdown 2024-06-13T07:05:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 -- 172.21.15.26:0/3126732790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe160dc0 msgr2=0x55bcbe16fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe160dc0 0x55bcbe16fdf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8328009900 tx=0x7f83280049d0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 -- 172.21.15.26:0/3126732790 shutdown_connections 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bcbe1e2370 0x55bcbe1e4790 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcbe1617b0 0x55bcbe170330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bcbe166920 0x55bcbe171340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 --2- 172.21.15.26:0/3126732790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcbe160dc0 0x55bcbe16fdf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f8339e5b740 1 -- 172.21.15.26:0/3126732790 >> 172.21.15.26:0/3126732790 conn(0x55bcbe128fb0 msgr2=0x55bcbe15b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.218+0000 7f831e000640 0 client.15765 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.219+0000 7f8339e5b740 1 -- 172.21.15.26:0/3126732790 shutdown_connections 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.219+0000 7f8339e5b740 1 -- 172.21.15.26:0/3126732790 wait complete. 2024-06-13T07:05:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.219+0000 7f8339e5b740 20 client.15765 trim_cache size 0 max 16384 2024-06-13T07:05:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:05:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:05:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:05:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 -- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f83140a2b80 msgr2=0x7f83140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f83140a2b80 0x7f83140a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f83200983f0 tx=0x7f8320005960 comp rx=0 tx=0).stop 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 -- 172.21.15.26:0/222290127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8330074380 msgr2=0x7f8330199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8330074380 0x7f8330199cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f832000eb90 tx=0x7f832000c6a0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 -- 172.21.15.26:0/222290127 shutdown_connections 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f83140a2b80 0x7f83140a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8330109710 0x7f83301a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8330075960 0x7f833019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 --2- 172.21.15.26:0/222290127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8330074380 0x7f8330199cf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 -- 172.21.15.26:0/222290127 >> 172.21.15.26:0/222290127 conn(0x7f83300fcdd0 msgr2=0x7f83301056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 -- 172.21.15.26:0/222290127 shutdown_connections 2024-06-13T07:05:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:32.220+0000 7f8335a00640 1 -- 172.21.15.26:0/222290127 wait complete. 2024-06-13T07:05:32.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:32 smithi135 ceph-mon[36276]: pgmap v660: 129 pgs: 129 active+clean; 61 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-06-13T07:05:32.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:32 smithi136 ceph-mon[37229]: pgmap v660: 129 pgs: 129 active+clean; 61 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-06-13T07:05:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:32 smithi026 ceph-mon[32041]: pgmap v660: 129 pgs: 129 active+clean; 61 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-06-13T07:05:32.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 186035 4.38 MB/sec execute 106 sec latency 49.278 ms 2024-06-13T07:05:33.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 187393 4.41 MB/sec execute 107 sec latency 19.652 ms 2024-06-13T07:05:34.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:34 smithi135 ceph-mon[36276]: pgmap v661: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-13T07:05:34.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:34 smithi136 ceph-mon[37229]: pgmap v661: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-13T07:05:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:34 smithi026 ceph-mon[32041]: pgmap v661: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-13T07:05:34.527 INFO:tasks.workunit.client.0.smithi026.stdout: 1 188319 4.43 MB/sec execute 108 sec latency 23.283 ms 2024-06-13T07:05:35.528 INFO:tasks.workunit.client.0.smithi026.stdout: 1 188808 4.39 MB/sec execute 109 sec latency 44.456 ms 2024-06-13T07:05:35.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:35 smithi135 ceph-mon[36276]: pgmap v662: 129 pgs: 129 active+clean; 61 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:05:35.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:35 smithi136 ceph-mon[37229]: pgmap v662: 129 pgs: 129 active+clean; 61 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:05:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:35 smithi026 ceph-mon[32041]: pgmap v662: 129 pgs: 129 active+clean; 61 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:05:36.528 INFO:tasks.workunit.client.0.smithi026.stdout: 1 190024 4.40 MB/sec execute 110 sec latency 19.799 ms 2024-06-13T07:05:37.528 INFO:tasks.workunit.client.0.smithi026.stdout: 1 191305 4.44 MB/sec execute 111 sec latency 26.505 ms 2024-06-13T07:05:38.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:37 smithi026 ceph-mon[32041]: pgmap v663: 129 pgs: 129 active+clean; 59 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 206 op/s 2024-06-13T07:05:38.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:37 smithi135 ceph-mon[36276]: pgmap v663: 129 pgs: 129 active+clean; 59 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 206 op/s 2024-06-13T07:05:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:37 smithi136 ceph-mon[37229]: pgmap v663: 129 pgs: 129 active+clean; 59 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 206 op/s 2024-06-13T07:05:38.528 INFO:tasks.workunit.client.0.smithi026.stdout: 1 191981 4.44 MB/sec execute 112 sec latency 24.891 ms 2024-06-13T07:05:39.528 INFO:tasks.workunit.client.0.smithi026.stdout: 1 192682 4.42 MB/sec execute 113 sec latency 31.926 ms 2024-06-13T07:05:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:39 smithi026 ceph-mon[32041]: pgmap v664: 129 pgs: 129 active+clean; 59 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-06-13T07:05:40.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:39 smithi135 ceph-mon[36276]: pgmap v664: 129 pgs: 129 active+clean; 59 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-06-13T07:05:40.129 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:39 smithi136 ceph-mon[37229]: pgmap v664: 129 pgs: 129 active+clean; 59 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-06-13T07:05:40.528 INFO:tasks.workunit.client.0.smithi026.stdout: 1 193945 4.42 MB/sec execute 114 sec latency 24.744 ms 2024-06-13T07:05:41.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 194925 4.45 MB/sec execute 115 sec latency 22.256 ms 2024-06-13T07:05:42.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:41 smithi026 ceph-mon[32041]: pgmap v665: 129 pgs: 129 active+clean; 61 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:05:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:41 smithi135 ceph-mon[36276]: pgmap v665: 129 pgs: 129 active+clean; 61 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:05:42.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:41 smithi136 ceph-mon[37229]: pgmap v665: 129 pgs: 129 active+clean; 61 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:05:42.528 INFO:tasks.workunit.client.0.smithi026.stdout: 1 195543 4.45 MB/sec execute 116 sec latency 42.534 ms 2024-06-13T07:05:43.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 196478 4.46 MB/sec execute 117 sec latency 31.419 ms 2024-06-13T07:05:44.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:43 smithi026 ceph-mon[32041]: pgmap v666: 129 pgs: 129 active+clean; 62 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:05:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:43 smithi135 ceph-mon[36276]: pgmap v666: 129 pgs: 129 active+clean; 62 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:05:44.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:43 smithi136 ceph-mon[37229]: pgmap v666: 129 pgs: 129 active+clean; 62 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:05:44.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 197586 4.44 MB/sec execute 118 sec latency 26.159 ms 2024-06-13T07:05:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.372+0000 7fafc8c00640 1 -- 172.21.15.26:0/3533486968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4102320 msgr2=0x7fafc4104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.372+0000 7fafc8c00640 1 --2- 172.21.15.26:0/3533486968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4102320 0x7fafc4104710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009900 tx=0x7fafb402a410 comp rx=0 tx=0).stop 2024-06-13T07:05:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.372+0000 7fafc8c00640 1 -- 172.21.15.26:0/3533486968 shutdown_connections 2024-06-13T07:05:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.372+0000 7fafc8c00640 1 --2- 172.21.15.26:0/3533486968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc4107580 0x7fafc4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.372+0000 7fafc8c00640 1 --2- 172.21.15.26:0/3533486968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafc4104c50 0x7fafc4107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.372+0000 7fafc8c00640 1 --2- 172.21.15.26:0/3533486968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4102320 0x7fafc4104710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.372+0000 7fafc8c00640 1 -- 172.21.15.26:0/3533486968 >> 172.21.15.26:0/3533486968 conn(0x7fafc40fbc90 msgr2=0x7fafc40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.373+0000 7fafc8c00640 1 -- 172.21.15.26:0/3533486968 shutdown_connections 2024-06-13T07:05:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.373+0000 7fafc8c00640 1 -- 172.21.15.26:0/3533486968 wait complete. 2024-06-13T07:05:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.374+0000 7fafc8c00640 1 Processor -- start 2024-06-13T07:05:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.374+0000 7fafc8c00640 1 -- start start 2024-06-13T07:05:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafc4102320 0x7fafc4101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc8c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc4104c50 0x7fafc41002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc8c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4107580 0x7fafc4100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc8c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fafc410d490 con 0x7fafc4102320 2024-06-13T07:05:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc8c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fafc410d310 con 0x7fafc4104c50 2024-06-13T07:05:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc8c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fafc410d610 con 0x7fafc4107580 2024-06-13T07:05:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafc4102320 0x7fafc4101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc4104c50 0x7fafc41002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4107580 0x7fafc4100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc4104c50 0x7fafc41002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51574/0 (socket says 172.21.15.26:51574) 2024-06-13T07:05:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.375+0000 7fafc2a00640 1 -- 172.21.15.26:0/2386120807 learned_addr learned my addr 172.21.15.26:0/2386120807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafc2a00640 1 -- 172.21.15.26:0/2386120807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4107580 msgr2=0x7fafc4100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafc2a00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4107580 0x7fafc4100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafc2a00640 1 -- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafc4102320 msgr2=0x7fafc4101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafc2a00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafc4102320 0x7fafc4101c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafc2a00640 1 -- 172.21.15.26:0/2386120807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc41010a0 con 0x7fafc4104c50 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafc3400640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafc4102320 0x7fafc4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafc3e00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4107580 0x7fafc4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafc2a00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc4104c50 0x7fafc41002d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fafb000e9c0 tx=0x7fafb000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.376+0000 7fafbbe00640 1 -- 172.21.15.26:0/2386120807 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb000cd80 con 0x7fafc4104c50 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.377+0000 7fafbbe00640 1 -- 172.21.15.26:0/2386120807 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb0005500 con 0x7fafc4104c50 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.377+0000 7fafc8c00640 1 -- 172.21.15.26:0/2386120807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc41a1c50 con 0x7fafc4104c50 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.377+0000 7fafbbe00640 1 -- 172.21.15.26:0/2386120807 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0010690 con 0x7fafc4104c50 2024-06-13T07:05:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.377+0000 7fafc8c00640 1 -- 172.21.15.26:0/2386120807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc41a21a0 con 0x7fafc4104c50 2024-06-13T07:05:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.380+0000 7fafcd03d740 1 Processor -- start 2024-06-13T07:05:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.380+0000 7fafbbe00640 1 -- 172.21.15.26:0/2386120807 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fafb00046c0 con 0x7fafc4104c50 2024-06-13T07:05:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.380+0000 7fafcd03d740 1 -- start start 2024-06-13T07:05:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e7291edc0 0x561e7291f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e729242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e72924920 0x561e72926db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafcd03d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561e72882420 con 0x561e72924920 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafcd03d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e72829c50 con 0x561e7291edc0 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafcd03d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561e7287e3e0 con 0x561e7291f7b0 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafc3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e729242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafc3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e729242b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:47886/0 (socket says 172.21.15.26:47886) 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafc3400640 1 -- 172.21.15.26:0/571988215 learned_addr learned my addr 172.21.15.26:0/571988215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafc3e00640 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e7291edc0 0x561e7291f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.381+0000 7fafc2a00640 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e72924920 0x561e72926db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafc3400640 1 -- 172.21.15.26:0/571988215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e7291edc0 msgr2=0x561e7291f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafc3400640 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e7291edc0 0x561e7291f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafc3400640 1 -- 172.21.15.26:0/571988215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e72924920 msgr2=0x561e72926db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafc3400640 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e72924920 0x561e72926db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafc3400640 1 -- 172.21.15.26:0/571988215 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e72596210 con 0x561e7291f7b0 2024-06-13T07:05:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafc3e00640 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e7291edc0 0x561e7291f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafc3400640 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e729242b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fafb402a930 tx=0x7fafb4009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05f973fa60a9d15 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafbbe00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fafa80a2b80 0x7fafa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafafe00640 1 -- 172.21.15.26:0/571988215 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4040070 con 0x561e7291f7b0 2024-06-13T07:05:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.382+0000 7fafafe00640 1 -- 172.21.15.26:0/571988215 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb40319b0 con 0x561e7291f7b0 2024-06-13T07:05:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.383+0000 7fafafe00640 1 -- 172.21.15.26:0/571988215 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4035040 con 0x561e7291f7b0 2024-06-13T07:05:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.383+0000 7fafbbe00640 1 -- 172.21.15.26:0/2386120807 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fafb00cf870 con 0x7fafc4104c50 2024-06-13T07:05:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.383+0000 7fafc3e00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fafa80a2b80 0x7fafa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 -- 172.21.15.26:0/571988215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 msgr2=0x561e729242b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e729242b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fafb402a930 tx=0x7fafb4009680 comp rx=0 tx=0).stop 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafc3e00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fafa80a2b80 0x7fafa80a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561e72920010 tx=0x7fafbc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 -- 172.21.15.26:0/571988215 shutdown_connections 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e72924920 0x561e72926db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e729242b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 --2- 172.21.15.26:0/571988215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e7291edc0 0x561e7291f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 -- 172.21.15.26:0/571988215 >> 172.21.15.26:0/571988215 conn(0x561e728e6fb0 msgr2=0x561e7291a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 -- 172.21.15.26:0/571988215 shutdown_connections 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 -- 172.21.15.26:0/571988215 wait complete. 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.384+0000 7fafcd03d740 1 -- start start 2024-06-13T07:05:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.385+0000 7fafcd03d740 1 Processor -- start 2024-06-13T07:05:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.385+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e72929120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.385+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72929660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.385+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e729306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.385+0000 7fafcd03d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561e72927870 con 0x561e7291edc0 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.385+0000 7fafcd03d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e7283d0b0 con 0x561e72924920 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.385+0000 7fafcd03d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561e729279f0 con 0x561e7291f7b0 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafcd03d740 10 client.0 _ll_register_callbacks cb 0x561e72828f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72929660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e72929120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e729306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72929660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:47896/0 (socket says 172.21.15.26:47896) 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 -- 172.21.15.26:0/1474009240 learned_addr learned my addr 172.21.15.26:0/1474009240 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 -- 172.21.15.26:0/1474009240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 msgr2=0x561e729306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e729306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 -- 172.21.15.26:0/1474009240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 msgr2=0x561e72929120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e72929120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 -- 172.21.15.26:0/1474009240 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e72662620 con 0x561e7291f7b0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafc2a00640 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72929660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fafb009bc10 tx=0x7fafb009bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b811d8d2571f2e59 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.386+0000 7fafacc00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.387+0000 7fafacc00640 1 -- 172.21.15.26:0/1474009240 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00a4660 con 0x561e7291f7b0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.387+0000 7fafacc00640 1 -- 172.21.15.26:0/1474009240 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fafb009b0d0 con 0x561e7291f7b0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.387+0000 7fafcd03d740 1 -- 172.21.15.26:0/1474009240 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb40319b0 con 0x561e7291f7b0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.387+0000 7fafacc00640 1 -- 172.21.15.26:0/1474009240 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fafb009bc80 con 0x561e7291f7b0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.387+0000 7fafcd03d740 10 client.25915 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.387+0000 7fafcd03d740 1 -- 172.21.15.26:0/1474009240 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561e72933ad0 con 0x561e7291f7b0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.387+0000 7fafacc00640 1 -- 172.21.15.26:0/1474009240 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb40319b0 con 0x561e7291f7b0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.387+0000 7fafacc00640 1 -- 172.21.15.26:0/1474009240 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fafb009abd0 con 0x561e7291f7b0 2024-06-13T07:05:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.388+0000 7fafcd03d740 10 client.25915 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.388+0000 7fafcd03d740 10 client.25915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.388+0000 7fafcd03d740 20 client.25915 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.388+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e729351a0 0x561e729375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.388+0000 7fafcd03d740 4 client.25915 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.388+0000 7fafcd03d740 1 -- 172.21.15.26:0/1474009240 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e7288fb50 con 0x561e729351a0 2024-06-13T07:05:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.388+0000 7fafc3400640 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e729351a0 0x561e729375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.389+0000 7fafc3400640 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e729351a0 0x561e729375f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe5135b89233f77 server_cookie=ab0471cf35888f72 in_seq=0 out_seq=0 2024-06-13T07:05:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.389+0000 7fafacc00640 10 client.25915 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafacc00640 1 -- 172.21.15.26:0/1474009240 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e7288fb50 con 0x561e729351a0 2024-06-13T07:05:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafacc00640 10 client.25915 handle_command_reply: tid=0 2024-06-13T07:05:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 client.25915 shutdown 2024-06-13T07:05:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 -- 172.21.15.26:0/1474009240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 msgr2=0x561e72929660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72929660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fafb009bc10 tx=0x7fafb009bb50 comp rx=0 tx=0).stop 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 -- 172.21.15.26:0/1474009240 shutdown_connections 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e729351a0 0x561e729375f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e729306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72929660 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafacc00640 0 client.25915 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1474009240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e72929120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 -- 172.21.15.26:0/1474009240 >> 172.21.15.26:0/1474009240 conn(0x561e728e6fb0 msgr2=0x561e729199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 -- 172.21.15.26:0/1474009240 shutdown_connections 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.390+0000 7fafcd03d740 1 -- 172.21.15.26:0/1474009240 wait complete. 2024-06-13T07:05:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.391+0000 7fafcd03d740 20 client.25915 trim_cache size 0 max 16384 2024-06-13T07:05:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.409+0000 7fafcd03d740 1 Processor -- start 2024-06-13T07:05:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafcd03d740 1 -- start start 2024-06-13T07:05:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7291a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72930660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e72915830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafcd03d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fafc41b34d0 con 0x561e7291edc0 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafcd03d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fafc41b0d70 con 0x561e72924920 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafcd03d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fafc41aebc0 con 0x561e7291f7b0 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72930660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafc3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7291a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72930660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:47906/0 (socket says 172.21.15.26:47906) 2024-06-13T07:05:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.410+0000 7fafc2a00640 1 -- 172.21.15.26:0/375861454 learned_addr learned my addr 172.21.15.26:0/375861454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafc3e00640 1 --2- 172.21.15.26:0/375861454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e72915830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafc2a00640 1 -- 172.21.15.26:0/375861454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 msgr2=0x561e72915830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafc2a00640 1 --2- 172.21.15.26:0/375861454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e72915830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafc2a00640 1 -- 172.21.15.26:0/375861454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 msgr2=0x561e7291a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafc2a00640 1 --2- 172.21.15.26:0/375861454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7291a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafc2a00640 1 -- 172.21.15.26:0/375861454 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e72916000 con 0x561e7291f7b0 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafc2a00640 1 --2- 172.21.15.26:0/375861454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72930660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fafb0098650 tx=0x7fafb00d5220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ecd373b2ab3ab0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafaf400640 1 -- 172.21.15.26:0/375861454 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb009b5e0 con 0x561e7291f7b0 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafaf400640 1 -- 172.21.15.26:0/375861454 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb00a1540 con 0x561e7291f7b0 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.411+0000 7fafaf400640 1 -- 172.21.15.26:0/375861454 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb009b0d0 con 0x561e7291f7b0 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.412+0000 7fafcd03d740 1 -- 172.21.15.26:0/375861454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 msgr2=0x561e72930660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.412+0000 7fafcd03d740 1 --2- 172.21.15.26:0/375861454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72930660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fafb0098650 tx=0x7fafb00d5220 comp rx=0 tx=0).stop 2024-06-13T07:05:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.412+0000 7fafcd03d740 1 -- 172.21.15.26:0/375861454 shutdown_connections 2024-06-13T07:05:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.412+0000 7fafcd03d740 1 --2- 172.21.15.26:0/375861454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e72915830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.412+0000 7fafcd03d740 1 --2- 172.21.15.26:0/375861454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e72930660 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.412+0000 7fafcd03d740 1 --2- 172.21.15.26:0/375861454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7291a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.413+0000 7fafcd03d740 1 -- 172.21.15.26:0/375861454 >> 172.21.15.26:0/375861454 conn(0x561e728e6fb0 msgr2=0x561e7291a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.413+0000 7fafcd03d740 1 -- 172.21.15.26:0/375861454 shutdown_connections 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.413+0000 7fafcd03d740 1 -- 172.21.15.26:0/375861454 wait complete. 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.413+0000 7fafcd03d740 1 -- start start 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafcd03d740 1 Processor -- start 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7292d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e7292de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafcd03d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e7292ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafcd03d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fafc41af230 con 0x561e7291edc0 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafcd03d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fafc41b06a0 con 0x561e72924920 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafcd03d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fafc41af900 con 0x561e7291f7b0 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafcd03d740 10 client.0 _ll_register_callbacks cb 0x7fafb0009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e7292ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7292d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.414+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e7292de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e7292ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51588/0 (socket says 172.21.15.26:51588) 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 -- 172.21.15.26:0/1109672944 learned_addr learned my addr 172.21.15.26:0/1109672944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 -- 172.21.15.26:0/1109672944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 msgr2=0x561e7292de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e7292de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 -- 172.21.15.26:0/1109672944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 msgr2=0x561e7292d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7292d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 -- 172.21.15.26:0/1109672944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7292f450 con 0x561e72924920 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc2a00640 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e7292de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3400640 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7292d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafc3e00640 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e7292ee60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fafbc003970 tx=0x7fafbc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9919334826b25cd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.415+0000 7fafacc00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.416+0000 7fafacc00640 1 -- 172.21.15.26:0/1109672944 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc02e060 con 0x561e72924920 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.416+0000 7fafcd03d740 1 -- 172.21.15.26:0/1109672944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb00a1540 con 0x561e72924920 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.416+0000 7fafacc00640 1 -- 172.21.15.26:0/1109672944 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fafbc004740 con 0x561e72924920 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.416+0000 7fafacc00640 1 -- 172.21.15.26:0/1109672944 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fafbc035050 con 0x561e72924920 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.416+0000 7fafcd03d740 10 client.25880 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.416+0000 7fafcd03d740 1 -- 172.21.15.26:0/1109672944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561e72930820 con 0x561e72924920 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafacc00640 1 -- 172.21.15.26:0/1109672944 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb00a1540 con 0x561e72924920 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafacc00640 1 -- 172.21.15.26:0/1109672944 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fafbc01f500 con 0x561e72924920 2024-06-13T07:05:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafcd03d740 10 client.25880 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafcd03d740 10 client.25880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafcd03d740 20 client.25880 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e729a05c0 0x561e729a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafcd03d740 4 client.25880 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafcd03d740 1 -- 172.21.15.26:0/1109672944 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e72889ea0 con 0x561e729a05c0 2024-06-13T07:05:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.417+0000 7fafc3400640 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e729a05c0 0x561e729a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.418+0000 7fafc3400640 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e729a05c0 0x561e729a29e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b25c4b02cabb23 server_cookie=f24bfc7d6c2eff73 in_seq=0 out_seq=0 2024-06-13T07:05:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.418+0000 7fafacc00640 10 client.25880 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafacc00640 1 -- 172.21.15.26:0/1109672944 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561e72889ea0 con 0x561e729a05c0 2024-06-13T07:05:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafacc00640 10 client.25880 handle_command_reply: tid=0 2024-06-13T07:05:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 client.25880 shutdown 2024-06-13T07:05:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 -- 172.21.15.26:0/1109672944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 msgr2=0x561e7292ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e7292ee60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fafbc003970 tx=0x7fafbc004a40 comp rx=0 tx=0).stop 2024-06-13T07:05:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 -- 172.21.15.26:0/1109672944 shutdown_connections 2024-06-13T07:05:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e729a05c0 0x561e729a29e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e72924920 0x561e7292ee60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7291f7b0 0x561e7292de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 --2- 172.21.15.26:0/1109672944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e7291edc0 0x561e7292d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafacc00640 0 client.25880 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 -- 172.21.15.26:0/1109672944 >> 172.21.15.26:0/1109672944 conn(0x561e728e6fb0 msgr2=0x561e7291d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 -- 172.21.15.26:0/1109672944 shutdown_connections 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 1 -- 172.21.15.26:0/1109672944 wait complete. 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.515+0000 7fafcd03d740 20 client.25880 trim_cache size 0 max 16384 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:05:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:05:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 -- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fafa80a2b80 msgr2=0x7fafa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fafa80a2b80 0x7fafa80a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561e72920010 tx=0x7fafbc009500 comp rx=0 tx=0).stop 2024-06-13T07:05:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 -- 172.21.15.26:0/2386120807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc4104c50 msgr2=0x7fafc41002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc4104c50 0x7fafc41002d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fafb000e9c0 tx=0x7fafb000ee80 comp rx=0 tx=0).stop 2024-06-13T07:05:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 -- 172.21.15.26:0/2386120807 shutdown_connections 2024-06-13T07:05:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fafa80a2b80 0x7fafa80a5040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc4104c50 0x7fafc41002d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4107580 0x7fafc4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 --2- 172.21.15.26:0/2386120807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafc4102320 0x7fafc4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 -- 172.21.15.26:0/2386120807 >> 172.21.15.26:0/2386120807 conn(0x7fafc40fbc90 msgr2=0x7fafc4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 -- 172.21.15.26:0/2386120807 shutdown_connections 2024-06-13T07:05:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:45.516+0000 7fafc8c00640 1 -- 172.21.15.26:0/2386120807 wait complete. 2024-06-13T07:05:45.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 198792 4.49 MB/sec execute 119 sec latency 29.792 ms 2024-06-13T07:05:46.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:45 smithi026 ceph-mon[32041]: pgmap v667: 129 pgs: 129 active+clean; 61 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-13T07:05:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:45 smithi135 ceph-mon[36276]: pgmap v667: 129 pgs: 129 active+clean; 61 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-13T07:05:46.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:45 smithi136 ceph-mon[37229]: pgmap v667: 129 pgs: 129 active+clean; 61 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-13T07:05:46.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 199258 4.46 MB/sec execute 120 sec latency 44.669 ms 2024-06-13T07:05:47.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 200298 4.47 MB/sec execute 121 sec latency 23.027 ms 2024-06-13T07:05:48.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:47 smithi026 ceph-mon[32041]: pgmap v668: 129 pgs: 129 active+clean; 63 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:05:48.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:47 smithi135 ceph-mon[36276]: pgmap v668: 129 pgs: 129 active+clean; 63 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:05:48.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:47 smithi136 ceph-mon[37229]: pgmap v668: 129 pgs: 129 active+clean; 63 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:05:48.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 201630 4.49 MB/sec execute 122 sec latency 26.775 ms 2024-06-13T07:05:49.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 202555 4.51 MB/sec execute 123 sec latency 21.389 ms 2024-06-13T07:05:50.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:49 smithi026 ceph-mon[32041]: pgmap v669: 129 pgs: 129 active+clean; 63 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-13T07:05:50.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:49 smithi135 ceph-mon[36276]: pgmap v669: 129 pgs: 129 active+clean; 63 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-13T07:05:50.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:49 smithi136 ceph-mon[37229]: pgmap v669: 129 pgs: 129 active+clean; 63 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-13T07:05:50.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 203100 4.48 MB/sec execute 124 sec latency 28.175 ms 2024-06-13T07:05:51.529 INFO:tasks.workunit.client.0.smithi026.stdout: 1 204433 4.48 MB/sec execute 125 sec latency 22.795 ms 2024-06-13T07:05:52.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:51 smithi026 ceph-mon[32041]: pgmap v670: 129 pgs: 129 active+clean; 65 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-06-13T07:05:52.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:51 smithi135 ceph-mon[36276]: pgmap v670: 129 pgs: 129 active+clean; 65 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-06-13T07:05:52.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:51 smithi136 ceph-mon[37229]: pgmap v670: 129 pgs: 129 active+clean; 65 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-06-13T07:05:52.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 205083 4.50 MB/sec execute 126 sec latency 306.985 ms 2024-06-13T07:05:53.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 206093 4.52 MB/sec execute 127 sec latency 39.610 ms 2024-06-13T07:05:54.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:53 smithi026 ceph-mon[32041]: pgmap v671: 129 pgs: 129 active+clean; 65 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:05:54.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:53 smithi135 ceph-mon[36276]: pgmap v671: 129 pgs: 129 active+clean; 65 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:05:54.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:53 smithi136 ceph-mon[37229]: pgmap v671: 129 pgs: 129 active+clean; 65 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-13T07:05:54.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 206703 4.49 MB/sec execute 128 sec latency 21.016 ms 2024-06-13T07:05:55.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 207505 4.49 MB/sec execute 129 sec latency 28.810 ms 2024-06-13T07:05:56.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:55 smithi026 ceph-mon[32041]: pgmap v672: 129 pgs: 129 active+clean; 65 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:05:56.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:55 smithi135 ceph-mon[36276]: pgmap v672: 129 pgs: 129 active+clean; 65 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:05:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:55 smithi136 ceph-mon[37229]: pgmap v672: 129 pgs: 129 active+clean; 65 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:05:56.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 208725 4.51 MB/sec execute 130 sec latency 33.641 ms 2024-06-13T07:05:57.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 209546 4.52 MB/sec execute 131 sec latency 67.531 ms 2024-06-13T07:05:58.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:57 smithi026 ceph-mon[32041]: pgmap v673: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-06-13T07:05:58.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:57 smithi135 ceph-mon[36276]: pgmap v673: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-06-13T07:05:58.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:57 smithi136 ceph-mon[37229]: pgmap v673: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-06-13T07:05:58.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 210104 4.50 MB/sec execute 132 sec latency 21.003 ms 2024-06-13T07:05:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.674+0000 7f7aa6600640 1 -- 172.21.15.26:0/2840701184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa01098c0 msgr2=0x7f7aa010bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.674+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2840701184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa01098c0 0x7f7aa010bd50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009860 tx=0x7f7a9802a340 comp rx=0 tx=0).stop 2024-06-13T07:05:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.674+0000 7f7aa6600640 1 -- 172.21.15.26:0/2840701184 shutdown_connections 2024-06-13T07:05:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.674+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2840701184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa0074400 0x7f7aa0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.674+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2840701184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa01098c0 0x7f7aa010bd50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.674+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2840701184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00759e0 0x7f7aa0073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.674+0000 7f7aa6600640 1 -- 172.21.15.26:0/2840701184 >> 172.21.15.26:0/2840701184 conn(0x7f7aa00fcf90 msgr2=0x7f7aa00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.674+0000 7f7aa6600640 1 -- 172.21.15.26:0/2840701184 shutdown_connections 2024-06-13T07:05:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.675+0000 7f7aa6600640 1 -- 172.21.15.26:0/2840701184 wait complete. 2024-06-13T07:05:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.676+0000 7f7aa6600640 1 Processor -- start 2024-06-13T07:05:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.676+0000 7f7aa6600640 1 -- start start 2024-06-13T07:05:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.676+0000 7f7aa6600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0074400 0x7f7aa0199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.676+0000 7f7aa6600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00759e0 0x7f7aa019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.676+0000 7f7aa6600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa01098c0 0x7f7aa01a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.676+0000 7f7aa6600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa010f840 con 0x7f7aa00759e0 2024-06-13T07:05:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.676+0000 7f7aa6600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa010f6c0 con 0x7f7aa01098c0 2024-06-13T07:05:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.676+0000 7f7aa6600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa010f9c0 con 0x7f7aa0074400 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0074400 0x7f7aa0199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa01098c0 0x7f7aa01a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0074400 0x7f7aa0199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44702/0 (socket says 172.21.15.26:44702) 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 -- 172.21.15.26:0/2377039166 learned_addr learned my addr 172.21.15.26:0/2377039166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a96a00640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00759e0 0x7f7aa019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 -- 172.21.15.26:0/2377039166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa01098c0 msgr2=0x7f7aa01a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa01098c0 0x7f7aa01a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 -- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00759e0 msgr2=0x7f7aa019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00759e0 0x7f7aa019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 -- 172.21.15.26:0/2377039166 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa01a19f0 con 0x7f7aa0074400 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9fe00640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa01098c0 0x7f7aa01a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a96a00640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00759e0 0x7f7aa019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9f400640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0074400 0x7f7aa0199dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c420 tx=0x7f7a90009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.677+0000 7f7a9cc00640 1 -- 172.21.15.26:0/2377039166 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90003970 con 0x7f7aa0074400 2024-06-13T07:05:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.678+0000 7f7a9cc00640 1 -- 172.21.15.26:0/2377039166 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a90004020 con 0x7f7aa0074400 2024-06-13T07:05:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.678+0000 7f7a9cc00640 1 -- 172.21.15.26:0/2377039166 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000bad0 con 0x7f7aa0074400 2024-06-13T07:05:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.678+0000 7f7aa6600640 1 -- 172.21.15.26:0/2377039166 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa01a1ce0 con 0x7f7aa0074400 2024-06-13T07:05:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.678+0000 7f7aa6600640 1 -- 172.21.15.26:0/2377039166 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa01a21d0 con 0x7f7aa0074400 2024-06-13T07:05:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.681+0000 7f7a9cc00640 1 -- 172.21.15.26:0/2377039166 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a9001d020 con 0x7f7aa0074400 2024-06-13T07:05:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.681+0000 7f7aa95ce740 1 Processor -- start 2024-06-13T07:05:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.681+0000 7f7aa95ce740 1 -- start start 2024-06-13T07:05:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970844d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x5597084520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x559708454bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5597083b0420 con 0x55970844cc00 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559708357c50 con 0x559708452760 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5597083ac3e0 con 0x55970844d5f0 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7a9f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x5597084520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x559708454bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7a9fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970844d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x559708454bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37746/0 (socket says 172.21.15.26:37746) 2024-06-13T07:05:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.682+0000 7f7a96a00640 1 -- 172.21.15.26:0/1650138964 learned_addr learned my addr 172.21.15.26:0/1650138964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a96a00640 1 -- 172.21.15.26:0/1650138964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 msgr2=0x5597084520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a96a00640 1 --2- 172.21.15.26:0/1650138964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x5597084520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a96a00640 1 -- 172.21.15.26:0/1650138964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 msgr2=0x55970844d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a96a00640 1 --2- 172.21.15.26:0/1650138964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970844d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a96a00640 1 -- 172.21.15.26:0/1650138964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597080c4210 con 0x559708452760 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a9cc00640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7a840a2cb0 0x7f7a840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a96a00640 1 --2- 172.21.15.26:0/1650138964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x559708454bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009cf0 tx=0x7f7a880117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e720143c3be41 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a9cc00640 1 -- 172.21.15.26:0/2377039166 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7a900cf2e0 con 0x7f7aa0074400 2024-06-13T07:05:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a9fe00640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7a840a2cb0 0x7f7a840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a94c00640 1 -- 172.21.15.26:0/1650138964 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88012450 con 0x559708452760 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a94c00640 1 -- 172.21.15.26:0/1650138964 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a88012ad0 con 0x559708452760 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.683+0000 7f7a94c00640 1 -- 172.21.15.26:0/1650138964 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8800fb80 con 0x559708452760 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7aa95ce740 1 -- 172.21.15.26:0/1650138964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 msgr2=0x559708454bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/1650138964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x559708454bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009cf0 tx=0x7f7a880117b0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7a9fe00640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7a840a2cb0 0x7f7a840a5170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7a980305b0 tx=0x7f7a980058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7aa95ce740 1 -- 172.21.15.26:0/1650138964 shutdown_connections 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/1650138964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x559708454bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/1650138964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x5597084520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/1650138964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970844d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7aa95ce740 1 -- 172.21.15.26:0/1650138964 >> 172.21.15.26:0/1650138964 conn(0x559708414fb0 msgr2=0x559708448680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.684+0000 7f7aa95ce740 1 -- 172.21.15.26:0/1650138964 shutdown_connections 2024-06-13T07:05:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.685+0000 7f7aa95ce740 1 -- 172.21.15.26:0/1650138964 wait complete. 2024-06-13T07:05:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.685+0000 7f7aa95ce740 1 -- start start 2024-06-13T07:05:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.685+0000 7f7aa95ce740 1 Processor -- start 2024-06-13T07:05:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55970844cc00 0x559708456ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x559708457420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559708452760 0x55970845e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559708455710 con 0x559708452760 2024-06-13T07:05:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55970836b0b0 con 0x55970844cc00 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559708455890 con 0x55970844d5f0 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7aa95ce740 10 client.0 _ll_register_callbacks cb 0x559708356f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x559708457420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x559708457420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44728/0 (socket says 172.21.15.26:44728) 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7a96a00640 1 -- 172.21.15.26:0/56191923 learned_addr learned my addr 172.21.15.26:0/56191923 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7a96a00640 1 -- 172.21.15.26:0/56191923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55970844cc00 msgr2=0x559708456ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7a96a00640 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55970844cc00 0x559708456ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.686+0000 7f7a96a00640 1 -- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559708452760 msgr2=0x55970845e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7a9fe00640 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559708452760 0x55970845e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7a96a00640 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559708452760 0x55970845e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7a96a00640 1 -- 172.21.15.26:0/56191923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559708190620 con 0x55970844d5f0 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7a9fe00640 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559708452760 0x55970845e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7a96a00640 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x559708457420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7a88011cb0 tx=0x7f7a880097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7089c821893eb9dd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7a8d600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7a8d600640 1 -- 172.21.15.26:0/56191923 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88029020 con 0x55970844d5f0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7a8d600640 1 -- 172.21.15.26:0/56191923 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7a88012660 con 0x55970844d5f0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.687+0000 7f7aa95ce740 1 -- 172.21.15.26:0/56191923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a88012ad0 con 0x55970844d5f0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.688+0000 7f7a8d600640 1 -- 172.21.15.26:0/56191923 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7a88034050 con 0x55970844d5f0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.688+0000 7f7aa95ce740 10 client.25933 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.688+0000 7f7aa95ce740 1 -- 172.21.15.26:0/56191923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559708461820 con 0x55970844d5f0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.688+0000 7f7a8d600640 1 -- 172.21.15.26:0/56191923 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a88012ad0 con 0x55970844d5f0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.688+0000 7f7a8d600640 1 -- 172.21.15.26:0/56191923 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f7a88005470 con 0x55970844d5f0 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.689+0000 7f7aa95ce740 10 client.25933 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.689+0000 7f7aa95ce740 10 client.25933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.689+0000 7f7aa95ce740 20 client.25933 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.689+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559708462cd0 0x5597084650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.689+0000 7f7aa95ce740 4 client.25933 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.689+0000 7f7aa95ce740 1 -- 172.21.15.26:0/56191923 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597083bdb50 con 0x559708462cd0 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.689+0000 7f7a9f400640 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559708462cd0 0x5597084650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.690+0000 7f7a9f400640 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559708462cd0 0x5597084650c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe7df4e304bcc9cc server_cookie=b42f86a4e7a620b6 in_seq=0 out_seq=0 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.690+0000 7f7a8d600640 10 client.25933 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.690+0000 7f7a8d600640 1 -- 172.21.15.26:0/56191923 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597083bdb50 con 0x559708462cd0 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.690+0000 7f7a8d600640 10 client.25933 handle_command_reply: tid=0 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.690+0000 7f7aa95ce740 1 client.25933 shutdown 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 -- 172.21.15.26:0/56191923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 msgr2=0x559708457420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x559708457420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7a88011cb0 tx=0x7f7a880097f0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 -- 172.21.15.26:0/56191923 shutdown_connections 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559708462cd0 0x5597084650c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559708452760 0x55970845e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x559708457420 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/56191923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55970844cc00 0x559708456ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 -- 172.21.15.26:0/56191923 >> 172.21.15.26:0/56191923 conn(0x559708414fb0 msgr2=0x559708447840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7a8d600640 0 client.25933 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 -- 172.21.15.26:0/56191923 shutdown_connections 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 1 -- 172.21.15.26:0/56191923 wait complete. 2024-06-13T07:05:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.691+0000 7f7aa95ce740 20 client.25933 trim_cache size 0 max 16384 2024-06-13T07:05:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.714+0000 7f7aa95ce740 1 Processor -- start 2024-06-13T07:05:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.714+0000 7f7aa95ce740 1 -- start start 2024-06-13T07:05:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970844e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970844ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01af190 con 0x55970844cc00 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01aca30 con 0x559708452760 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01aa880 con 0x55970844d5f0 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7a9fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970844ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7a9f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970844e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7a9fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37760/0 (socket says 172.21.15.26:37760) 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.715+0000 7f7a9fe00640 1 -- 172.21.15.26:0/3375764537 learned_addr learned my addr 172.21.15.26:0/3375764537 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a96a00640 1 -- 172.21.15.26:0/3375764537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 msgr2=0x55970845e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a96a00640 1 --2- 172.21.15.26:0/3375764537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a96a00640 1 -- 172.21.15.26:0/3375764537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 msgr2=0x55970844e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a96a00640 1 --2- 172.21.15.26:0/3375764537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970844e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a96a00640 1 -- 172.21.15.26:0/3375764537 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559708449330 con 0x55970844d5f0 2024-06-13T07:05:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a96a00640 1 --2- 172.21.15.26:0/3375764537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970844ee10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800a010 tx=0x7f7a880092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed57c97287799830 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a8fe00640 1 -- 172.21.15.26:0/3375764537 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88029070 con 0x55970844d5f0 2024-06-13T07:05:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a8fe00640 1 -- 172.21.15.26:0/3375764537 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a88004b80 con 0x55970844d5f0 2024-06-13T07:05:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.716+0000 7f7a8fe00640 1 -- 172.21.15.26:0/3375764537 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8800fc20 con 0x55970844d5f0 2024-06-13T07:05:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 -- 172.21.15.26:0/3375764537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 msgr2=0x55970844ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/3375764537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970844ee10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800a010 tx=0x7f7a880092a0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 -- 172.21.15.26:0/3375764537 shutdown_connections 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/3375764537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/3375764537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970844ee10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/3375764537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970844e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 -- 172.21.15.26:0/3375764537 >> 172.21.15.26:0/3375764537 conn(0x559708414fb0 msgr2=0x559708456ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 -- 172.21.15.26:0/3375764537 shutdown_connections 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.717+0000 7f7aa95ce740 1 -- 172.21.15.26:0/3375764537 wait complete. 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.718+0000 7f7aa95ce740 1 -- start start 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.718+0000 7f7aa95ce740 1 Processor -- start 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.718+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970845b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.718+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970845bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.718+0000 7f7aa95ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.718+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01aaef0 con 0x55970844cc00 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.718+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01ac360 con 0x559708452760 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.718+0000 7f7aa95ce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01ab5c0 con 0x55970844d5f0 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7aa95ce740 10 client.0 _ll_register_callbacks cb 0x7f7a90009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a9f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970845b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970845bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970845bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44742/0 (socket says 172.21.15.26:44742) 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a9f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970845b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43896/0 (socket says 172.21.15.26:43896) 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a96a00640 1 -- 172.21.15.26:0/645044678 learned_addr learned my addr 172.21.15.26:0/645044678 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a9fe00640 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a96a00640 1 -- 172.21.15.26:0/645044678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 msgr2=0x55970845caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a96a00640 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a96a00640 1 -- 172.21.15.26:0/645044678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 msgr2=0x55970845b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a96a00640 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970845b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.719+0000 7f7a96a00640 1 -- 172.21.15.26:0/645044678 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55970845d0e0 con 0x55970844d5f0 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.720+0000 7f7a96a00640 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970845bae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7a880095f0 tx=0x7f7a88005940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f94536d37913e576 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.720+0000 7f7a8d600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.720+0000 7f7a8d600640 1 -- 172.21.15.26:0/645044678 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8803c070 con 0x55970844d5f0 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.720+0000 7f7aa95ce740 1 -- 172.21.15.26:0/645044678 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a88004b80 con 0x55970844d5f0 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.720+0000 7f7a8d600640 1 -- 172.21.15.26:0/645044678 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7a8800fbc0 con 0x55970844d5f0 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.720+0000 7f7a8d600640 1 -- 172.21.15.26:0/645044678 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7a88029040 con 0x55970844d5f0 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.720+0000 7f7aa95ce740 10 client.25939 fetch_fsmap learned FSMap version 34 2024-06-13T07:05:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.720+0000 7f7aa95ce740 1 -- 172.21.15.26:0/645044678 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55970845e3f0 con 0x55970844d5f0 2024-06-13T07:05:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7a8d600640 1 -- 172.21.15.26:0/645044678 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a88004b80 con 0x55970844d5f0 2024-06-13T07:05:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7a8d600640 1 -- 172.21.15.26:0/645044678 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f7a880052c0 con 0x55970844d5f0 2024-06-13T07:05:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7aa95ce740 10 client.25939 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:05:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7aa95ce740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:05:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7aa95ce740 20 client.25939 populate_metadata read hostname 'smithi026' 2024-06-13T07:05:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5597084cdb90 0x5597084cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7aa95ce740 4 client.25939 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7aa95ce740 1 -- 172.21.15.26:0/645044678 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597083b7ea0 con 0x5597084cdb90 2024-06-13T07:05:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.721+0000 7f7a9f400640 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5597084cdb90 0x5597084cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.722+0000 7f7a9f400640 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5597084cdb90 0x5597084cff80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7f69fbe611572c server_cookie=95b7e57fe19b3c1e in_seq=0 out_seq=0 2024-06-13T07:05:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.722+0000 7f7a8d600640 10 client.25939 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7a8d600640 1 -- 172.21.15.26:0/645044678 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5597083b7ea0 con 0x5597084cdb90 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7a8d600640 10 client.25939 handle_command_reply: tid=0 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 client.25939 shutdown 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 -- 172.21.15.26:0/645044678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 msgr2=0x55970845bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970845bae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7a880095f0 tx=0x7f7a88005940 comp rx=0 tx=0).stop 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 -- 172.21.15.26:0/645044678 shutdown_connections 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5597084cdb90 0x5597084cff80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559708452760 0x55970845caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55970844d5f0 0x55970845bae0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 --2- 172.21.15.26:0/645044678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55970844cc00 0x55970845b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7a8d600640 0 client.25939 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.815+0000 7f7aa95ce740 1 -- 172.21.15.26:0/645044678 >> 172.21.15.26:0/645044678 conn(0x559708414fb0 msgr2=0x559708456df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.816+0000 7f7aa95ce740 1 -- 172.21.15.26:0/645044678 shutdown_connections 2024-06-13T07:05:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.816+0000 7f7aa95ce740 1 -- 172.21.15.26:0/645044678 wait complete. 2024-06-13T07:05:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.816+0000 7f7aa95ce740 20 client.25939 trim_cache size 0 max 16384 2024-06-13T07:05:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:05:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:05:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 -- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7a840a2cb0 msgr2=0x7f7a840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7a840a2cb0 0x7f7a840a5170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7a980305b0 tx=0x7f7a980058b0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 -- 172.21.15.26:0/2377039166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0074400 msgr2=0x7f7aa0199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0074400 0x7f7aa0199dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c420 tx=0x7f7a90009f90 comp rx=0 tx=0).stop 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 -- 172.21.15.26:0/2377039166 shutdown_connections 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7a840a2cb0 0x7f7a840a5170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa01098c0 0x7f7aa01a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00759e0 0x7f7aa019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 --2- 172.21.15.26:0/2377039166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0074400 0x7f7aa0199dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 -- 172.21.15.26:0/2377039166 >> 172.21.15.26:0/2377039166 conn(0x7f7aa00fcf90 msgr2=0x7f7aa01057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 -- 172.21.15.26:0/2377039166 shutdown_connections 2024-06-13T07:05:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:05:58.817+0000 7f7aa6600640 1 -- 172.21.15.26:0/2377039166 wait complete. 2024-06-13T07:05:59.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 211479 4.50 MB/sec execute 133 sec latency 53.062 ms 2024-06-13T07:06:00.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:05:59 smithi026 ceph-mon[32041]: pgmap v674: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-13T07:06:00.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:05:59 smithi135 ceph-mon[36276]: pgmap v674: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-13T07:06:00.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:05:59 smithi136 ceph-mon[37229]: pgmap v674: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-13T07:06:00.530 INFO:tasks.workunit.client.0.smithi026.stdout: 1 212484 4.52 MB/sec execute 134 sec latency 23.420 ms 2024-06-13T07:06:01.531 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213195 4.53 MB/sec execute 135 sec latency 22.202 ms 2024-06-13T07:06:02.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:01 smithi026 ceph-mon[32041]: pgmap v675: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:06:02.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:01 smithi135 ceph-mon[36276]: pgmap v675: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:06:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:01 smithi136 ceph-mon[37229]: pgmap v675: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:06:02.531 INFO:tasks.workunit.client.0.smithi026.stdout: 1 213702 4.51 MB/sec execute 136 sec latency 35.150 ms 2024-06-13T07:06:03.531 INFO:tasks.workunit.client.0.smithi026.stdout: 1 215015 4.51 MB/sec execute 137 sec latency 48.079 ms 2024-06-13T07:06:04.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:03 smithi026 ceph-mon[32041]: pgmap v676: 129 pgs: 129 active+clean; 64 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-06-13T07:06:04.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:03 smithi135 ceph-mon[36276]: pgmap v676: 129 pgs: 129 active+clean; 64 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-06-13T07:06:04.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:03 smithi136 ceph-mon[37229]: pgmap v676: 129 pgs: 129 active+clean; 64 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-06-13T07:06:04.531 INFO:tasks.workunit.client.0.smithi026.stdout: 1 216174 4.54 MB/sec execute 138 sec latency 27.359 ms 2024-06-13T07:06:05.532 INFO:tasks.workunit.client.0.smithi026.stdout: 1 216811 4.54 MB/sec execute 139 sec latency 23.923 ms 2024-06-13T07:06:06.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:05 smithi026 ceph-mon[32041]: pgmap v677: 129 pgs: 129 active+clean; 64 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:06:06.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:05 smithi135 ceph-mon[36276]: pgmap v677: 129 pgs: 129 active+clean; 64 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:06:06.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:05 smithi136 ceph-mon[37229]: pgmap v677: 129 pgs: 129 active+clean; 64 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:06:06.532 INFO:tasks.workunit.client.0.smithi026.stdout: 1 217336 4.51 MB/sec execute 140 sec latency 39.805 ms 2024-06-13T07:06:07.532 INFO:tasks.workunit.client.0.smithi026.stdout: 1 218296 4.52 MB/sec execute 141 sec latency 28.125 ms 2024-06-13T07:06:08.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:07 smithi026 ceph-mon[32041]: pgmap v678: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-06-13T07:06:08.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:07 smithi135 ceph-mon[36276]: pgmap v678: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-06-13T07:06:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:07 smithi136 ceph-mon[37229]: pgmap v678: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-06-13T07:06:08.532 INFO:tasks.workunit.client.0.smithi026.stdout: 1 219632 4.54 MB/sec execute 142 sec latency 59.428 ms 2024-06-13T07:06:09.532 INFO:tasks.workunit.client.0.smithi026.stdout: 1 220356 4.55 MB/sec execute 143 sec latency 25.804 ms 2024-06-13T07:06:10.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:09 smithi026 ceph-mon[32041]: pgmap v679: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-13T07:06:10.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:09 smithi135 ceph-mon[36276]: pgmap v679: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-13T07:06:10.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:09 smithi136 ceph-mon[37229]: pgmap v679: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-13T07:06:10.532 INFO:tasks.workunit.client.0.smithi026.stdout: 1 220893 4.52 MB/sec execute 144 sec latency 33.844 ms 2024-06-13T07:06:11.532 INFO:tasks.workunit.client.0.smithi026.stdout: 1 222166 4.53 MB/sec execute 145 sec latency 33.112 ms 2024-06-13T07:06:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.986+0000 7f94d9e00640 1 -- 172.21.15.26:0/801104903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d4104c30 msgr2=0x7f94d4107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.986+0000 7f94d9e00640 1 --2- 172.21.15.26:0/801104903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d4104c30 0x7f94d4107020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009900 tx=0x7f94c002a410 comp rx=0 tx=0).stop 2024-06-13T07:06:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.986+0000 7f94d9e00640 1 -- 172.21.15.26:0/801104903 shutdown_connections 2024-06-13T07:06:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.986+0000 7f94d9e00640 1 --2- 172.21.15.26:0/801104903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4107560 0x7f94d4109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.986+0000 7f94d9e00640 1 --2- 172.21.15.26:0/801104903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d4104c30 0x7f94d4107020 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.986+0000 7f94d9e00640 1 --2- 172.21.15.26:0/801104903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d4102300 0x7f94d41046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.986+0000 7f94d9e00640 1 -- 172.21.15.26:0/801104903 >> 172.21.15.26:0/801104903 conn(0x7f94d40fbc90 msgr2=0x7f94d40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.987+0000 7f94d9e00640 1 -- 172.21.15.26:0/801104903 shutdown_connections 2024-06-13T07:06:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.987+0000 7f94d9e00640 1 -- 172.21.15.26:0/801104903 wait complete. 2024-06-13T07:06:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.988+0000 7f94d9e00640 1 Processor -- start 2024-06-13T07:06:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.988+0000 7f94d9e00640 1 -- start start 2024-06-13T07:06:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.988+0000 7f94d9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d4102300 0x7f94d4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.988+0000 7f94d9e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4104c30 0x7f94d419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.988+0000 7f94d9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d4107560 0x7f94d41a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.988+0000 7f94d9e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94d410d4f0 con 0x7f94d4102300 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.988+0000 7f94d9e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94d410d370 con 0x7f94d4104c30 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.988+0000 7f94d9e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94d410d670 con 0x7f94d4107560 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4104c30 0x7f94d419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4104c30 0x7f94d419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39832/0 (socket says 172.21.15.26:39832) 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 -- 172.21.15.26:0/116175829 learned_addr learned my addr 172.21.15.26:0/116175829 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d9400640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d4107560 0x7f94d41a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d8a00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d4102300 0x7f94d4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 -- 172.21.15.26:0/116175829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d4107560 msgr2=0x7f94d41a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d4107560 0x7f94d41a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 -- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d4102300 msgr2=0x7f94d4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d4102300 0x7f94d4199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 -- 172.21.15.26:0/116175829 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d41a19d0 con 0x7f94d4104c30 2024-06-13T07:06:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.989+0000 7f94d3e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4104c30 0x7f94d419a300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94c0031440 tx=0x7f94c0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.990+0000 7f94d1600640 1 -- 172.21.15.26:0/116175829 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c003f030 con 0x7f94d4104c30 2024-06-13T07:06:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.990+0000 7f94d9e00640 1 -- 172.21.15.26:0/116175829 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d41a1c60 con 0x7f94d4104c30 2024-06-13T07:06:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.990+0000 7f94d9e00640 1 -- 172.21.15.26:0/116175829 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41a21a0 con 0x7f94d4104c30 2024-06-13T07:06:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.990+0000 7f94d1600640 1 -- 172.21.15.26:0/116175829 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c0003ce0 con 0x7f94d4104c30 2024-06-13T07:06:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.990+0000 7f94d1600640 1 -- 172.21.15.26:0/116175829 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0035070 con 0x7f94d4104c30 2024-06-13T07:06:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.993+0000 7f94d1600640 1 -- 172.21.15.26:0/116175829 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94c0042030 con 0x7f94d4104c30 2024-06-13T07:06:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.994+0000 7f94de194740 1 Processor -- start 2024-06-13T07:06:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.994+0000 7f94de194740 1 -- start start 2024-06-13T07:06:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.994+0000 7f94de194740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d391d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.994+0000 7f94de194740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d39222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.994+0000 7f94de194740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d3924db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.994+0000 7f94de194740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5613d3880420 con 0x5613d3922920 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.994+0000 7f94de194740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613d3827c50 con 0x5613d391d7b0 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.994+0000 7f94de194740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5613d387c3e0 con 0x5613d391cdc0 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d391d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d391d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37736/0 (socket says 172.21.15.26:37736) 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 -- 172.21.15.26:0/3936802526 learned_addr learned my addr 172.21.15.26:0/3936802526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d8a00640 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d39222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d3e00640 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d3924db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 -- 172.21.15.26:0/3936802526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 msgr2=0x5613d39222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d39222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 -- 172.21.15.26:0/3936802526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 msgr2=0x5613d3924db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d3924db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 -- 172.21.15.26:0/3936802526 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d3594210 con 0x5613d391cdc0 2024-06-13T07:06:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d3e00640 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d3924db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.995+0000 7f94d9400640 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d391d1e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009fd0 tx=0x7f94cc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e8d109a4460862 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.996+0000 7f94c5c00640 1 -- 172.21.15.26:0/3936802526 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004d20 con 0x5613d391cdc0 2024-06-13T07:06:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.996+0000 7f94c5c00640 1 -- 172.21.15.26:0/3936802526 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc004050 con 0x5613d391cdc0 2024-06-13T07:06:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.996+0000 7f94c5c00640 1 -- 172.21.15.26:0/3936802526 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc005260 con 0x5613d391cdc0 2024-06-13T07:06:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.996+0000 7f94d1600640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94bc0a2c00 0x7f94bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.996+0000 7f94d1600640 1 -- 172.21.15.26:0/116175829 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94c00efd80 con 0x7f94d4104c30 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94d8a00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94bc0a2c00 0x7f94bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 -- 172.21.15.26:0/3936802526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 msgr2=0x5613d391d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d391d1e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009fd0 tx=0x7f94cc0118d0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 -- 172.21.15.26:0/3936802526 shutdown_connections 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d39222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d3924db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 --2- 172.21.15.26:0/3936802526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d391d1e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 -- 172.21.15.26:0/3936802526 >> 172.21.15.26:0/3936802526 conn(0x5613d38e4fb0 msgr2=0x5613d3918840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 -- 172.21.15.26:0/3936802526 shutdown_connections 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.997+0000 7f94de194740 1 -- 172.21.15.26:0/3936802526 wait complete. 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.998+0000 7f94d8a00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94bc0a2c00 0x7f94bc0a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5613d39196f0 tx=0x7f94c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.998+0000 7f94de194740 1 -- start start 2024-06-13T07:06:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.998+0000 7f94de194740 1 Processor -- start 2024-06-13T07:06:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94de194740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d3927130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94de194740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d3927670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94de194740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d392e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94de194740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5613d39258d0 con 0x5613d3922920 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94de194740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613d383b0b0 con 0x5613d391d7b0 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94de194740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5613d3925a50 con 0x5613d391cdc0 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94de194740 10 client.0 _ll_register_callbacks cb 0x5613d3826f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94d8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d3927130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94d8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d3927130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37750/0 (socket says 172.21.15.26:37750) 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94d8a00640 1 -- 172.21.15.26:0/2196362385 learned_addr learned my addr 172.21.15.26:0/2196362385 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:11.999+0000 7f94d9400640 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d392e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94d3e00640 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d3927670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94d8a00640 1 -- 172.21.15.26:0/2196362385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 msgr2=0x5613d3927670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94d8a00640 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d3927670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94d8a00640 1 -- 172.21.15.26:0/2196362385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 msgr2=0x5613d392e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94d8a00640 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d392e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94d8a00640 1 -- 172.21.15.26:0/2196362385 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d3660620 con 0x5613d391cdc0 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94d3e00640 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d3927670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94d8a00640 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d3927130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94c8013010 tx=0x7f94c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e4f478494b6c1d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.000+0000 7f94ba000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.001+0000 7f94ba000640 1 -- 172.21.15.26:0/2196362385 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8009960 con 0x5613d391cdc0 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.001+0000 7f94ba000640 1 -- 172.21.15.26:0/2196362385 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94c8004050 con 0x5613d391cdc0 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.001+0000 7f94de194740 1 -- 172.21.15.26:0/2196362385 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94cc004050 con 0x5613d391cdc0 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.001+0000 7f94ba000640 1 -- 172.21.15.26:0/2196362385 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94c8004250 con 0x5613d391cdc0 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.001+0000 7f94de194740 10 client.25951 fetch_fsmap learned FSMap version 34 2024-06-13T07:06:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.001+0000 7f94de194740 1 -- 172.21.15.26:0/2196362385 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5613d3931af0 con 0x5613d391cdc0 2024-06-13T07:06:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.002+0000 7f94ba000640 1 -- 172.21.15.26:0/2196362385 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c8033070 con 0x5613d391cdc0 2024-06-13T07:06:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.002+0000 7f94ba000640 1 -- 172.21.15.26:0/2196362385 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f94c8027070 con 0x5613d391cdc0 2024-06-13T07:06:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.002+0000 7f94de194740 10 client.25951 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:06:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.002+0000 7f94de194740 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:06:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.002+0000 7f94de194740 20 client.25951 populate_metadata read hostname 'smithi026' 2024-06-13T07:06:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.002+0000 7f94de194740 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5613d3933230 0x5613d39356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.002+0000 7f94de194740 4 client.25951 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.002+0000 7f94de194740 1 -- 172.21.15.26:0/2196362385 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d388db50 con 0x5613d3933230 2024-06-13T07:06:12.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:11 smithi026 ceph-mon[32041]: pgmap v680: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-06-13T07:06:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.003+0000 7f94d3e00640 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5613d3933230 0x5613d39356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.003+0000 7f94d3e00640 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5613d3933230 0x5613d39356c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6052949219e75340 server_cookie=93729b164482cea in_seq=0 out_seq=0 2024-06-13T07:06:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.003+0000 7f94ba000640 10 client.25951 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.004+0000 7f94ba000640 1 -- 172.21.15.26:0/2196362385 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5613d388db50 con 0x5613d3933230 2024-06-13T07:06:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.004+0000 7f94ba000640 10 client.25951 handle_command_reply: tid=0 2024-06-13T07:06:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.004+0000 7f94de194740 1 client.25951 shutdown 2024-06-13T07:06:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.004+0000 7f94de194740 1 -- 172.21.15.26:0/2196362385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 msgr2=0x5613d3927130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.004+0000 7f94de194740 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d3927130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94c8013010 tx=0x7f94c8004730 comp rx=0 tx=0).stop 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 1 -- 172.21.15.26:0/2196362385 shutdown_connections 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5613d3933230 0x5613d39356c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d3927670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d3922920 0x5613d392e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 1 --2- 172.21.15.26:0/2196362385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d391cdc0 0x5613d3927130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94ba000640 0 client.25951 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 1 -- 172.21.15.26:0/2196362385 >> 172.21.15.26:0/2196362385 conn(0x5613d38e4fb0 msgr2=0x5613d39179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 1 -- 172.21.15.26:0/2196362385 shutdown_connections 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 1 -- 172.21.15.26:0/2196362385 wait complete. 2024-06-13T07:06:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.005+0000 7f94de194740 20 client.25951 trim_cache size 0 max 16384 2024-06-13T07:06:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.026+0000 7f94de194740 1 Processor -- start 2024-06-13T07:06:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.026+0000 7f94de194740 1 -- start start 2024-06-13T07:06:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94de194740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391cdc0 0x5613d3927450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94de194740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d392e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94de194740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 0x5613d391b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94de194740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41af0d0 con 0x5613d391cdc0 2024-06-13T07:06:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94de194740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41ac970 con 0x5613d391d7b0 2024-06-13T07:06:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94de194740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41aa7c0 con 0x5613d3922920 2024-06-13T07:06:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94d8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391cdc0 0x5613d3927450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d392e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94d8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391cdc0 0x5613d3927450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49610/0 (socket says 172.21.15.26:49610) 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94d9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 0x5613d391b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94d8a00640 1 -- 172.21.15.26:0/2139971981 learned_addr learned my addr 172.21.15.26:0/2139971981 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.027+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d392e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39862/0 (socket says 172.21.15.26:39862) 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94d8a00640 1 -- 172.21.15.26:0/2139971981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 msgr2=0x5613d391b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94d8a00640 1 --2- 172.21.15.26:0/2139971981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 0x5613d391b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94d8a00640 1 -- 172.21.15.26:0/2139971981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 msgr2=0x5613d392e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94d8a00640 1 --2- 172.21.15.26:0/2139971981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d392e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94d8a00640 1 -- 172.21.15.26:0/2139971981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d39194f0 con 0x5613d391cdc0 2024-06-13T07:06:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94d8a00640 1 --2- 172.21.15.26:0/2139971981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391cdc0 0x5613d3927450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94c8012d20 tx=0x7f94c8020900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e406b039579e32 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94c5200640 1 -- 172.21.15.26:0/2139971981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c803c070 con 0x5613d391cdc0 2024-06-13T07:06:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94c5200640 1 -- 172.21.15.26:0/2139971981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c8033040 con 0x5613d391cdc0 2024-06-13T07:06:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.028+0000 7f94c5200640 1 -- 172.21.15.26:0/2139971981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8029680 con 0x5613d391cdc0 2024-06-13T07:06:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.029+0000 7f94de194740 1 -- 172.21.15.26:0/2139971981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391cdc0 msgr2=0x5613d3927450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.029+0000 7f94de194740 1 --2- 172.21.15.26:0/2139971981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391cdc0 0x5613d3927450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94c8012d20 tx=0x7f94c8020900 comp rx=0 tx=0).stop 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.029+0000 7f94de194740 1 -- 172.21.15.26:0/2139971981 shutdown_connections 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.029+0000 7f94de194740 1 --2- 172.21.15.26:0/2139971981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391d7b0 0x5613d392e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.029+0000 7f94de194740 1 --2- 172.21.15.26:0/2139971981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 0x5613d391b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.029+0000 7f94de194740 1 --2- 172.21.15.26:0/2139971981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391cdc0 0x5613d3927450 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.029+0000 7f94de194740 1 -- 172.21.15.26:0/2139971981 >> 172.21.15.26:0/2139971981 conn(0x5613d38e4fb0 msgr2=0x5613d3926d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.030+0000 7f94de194740 1 -- 172.21.15.26:0/2139971981 shutdown_connections 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.030+0000 7f94de194740 1 -- 172.21.15.26:0/2139971981 wait complete. 2024-06-13T07:06:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.030+0000 7f94de194740 1 -- start start 2024-06-13T07:06:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.030+0000 7f94de194740 1 Processor -- start 2024-06-13T07:06:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94de194740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391cdc0 0x5613d392be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94de194740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391d7b0 0x5613d392c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94de194740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 0x5613d392d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94de194740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41aae30 con 0x5613d391d7b0 2024-06-13T07:06:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94de194740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41ac2a0 con 0x5613d391cdc0 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94de194740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41ab500 con 0x5613d3922920 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94de194740 10 client.0 _ll_register_callbacks cb 0x7f94c8009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94d8a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391cdc0 0x5613d392be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391d7b0 0x5613d392c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94d8a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391cdc0 0x5613d392be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39864/0 (socket says 172.21.15.26:39864) 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94d8a00640 1 -- 172.21.15.26:0/277336147 learned_addr learned my addr 172.21.15.26:0/277336147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.031+0000 7f94d9400640 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 0x5613d392d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94d8a00640 1 -- 172.21.15.26:0/277336147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 msgr2=0x5613d392d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94d8a00640 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 0x5613d392d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94d8a00640 1 -- 172.21.15.26:0/277336147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391d7b0 msgr2=0x5613d392c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94d8a00640 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391d7b0 0x5613d392c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94d8a00640 1 -- 172.21.15.26:0/277336147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d392d960 con 0x5613d391cdc0 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94d8a00640 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391cdc0 0x5613d392be20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94c8019040 tx=0x7f94c8004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc886a88ee2eb39 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94ba000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:06:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94ba000640 1 -- 172.21.15.26:0/277336147 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c803c070 con 0x5613d391cdc0 2024-06-13T07:06:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94ba000640 1 -- 172.21.15.26:0/277336147 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94c8005040 con 0x5613d391cdc0 2024-06-13T07:06:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.032+0000 7f94de194740 1 -- 172.21.15.26:0/277336147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94c8033040 con 0x5613d391cdc0 2024-06-13T07:06:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.033+0000 7f94ba000640 1 -- 172.21.15.26:0/277336147 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94c8048050 con 0x5613d391cdc0 2024-06-13T07:06:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.033+0000 7f94de194740 10 client.25913 fetch_fsmap learned FSMap version 34 2024-06-13T07:06:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.033+0000 7f94de194740 1 -- 172.21.15.26:0/277336147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5613d399dfa0 con 0x5613d391cdc0 2024-06-13T07:06:12.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:11 smithi135 ceph-mon[36276]: pgmap v680: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-06-13T07:06:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.033+0000 7f94ba000640 1 -- 172.21.15.26:0/277336147 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c8033040 con 0x5613d391cdc0 2024-06-13T07:06:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.034+0000 7f94ba000640 1 -- 172.21.15.26:0/277336147 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f94c8027040 con 0x5613d391cdc0 2024-06-13T07:06:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.034+0000 7f94de194740 10 client.25913 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:06:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.034+0000 7f94de194740 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:06:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.034+0000 7f94de194740 20 client.25913 populate_metadata read hostname 'smithi026' 2024-06-13T07:06:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.034+0000 7f94de194740 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5613d399e830 0x5613d39a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.034+0000 7f94de194740 4 client.25913 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:06:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.034+0000 7f94de194740 1 -- 172.21.15.26:0/277336147 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613d3887ea0 con 0x5613d399e830 2024-06-13T07:06:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.034+0000 7f94d3e00640 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5613d399e830 0x5613d39a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.035+0000 7f94d3e00640 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5613d399e830 0x5613d39a0c50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c482287f685e190 server_cookie=f4fa886f84404c27 in_seq=0 out_seq=0 2024-06-13T07:06:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.035+0000 7f94ba000640 10 client.25913 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:12.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:11 smithi136 ceph-mon[37229]: pgmap v680: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-06-13T07:06:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94ba000640 1 -- 172.21.15.26:0/277336147 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5613d3887ea0 con 0x5613d399e830 2024-06-13T07:06:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94ba000640 10 client.25913 handle_command_reply: tid=0 2024-06-13T07:06:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 client.25913 shutdown 2024-06-13T07:06:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 -- 172.21.15.26:0/277336147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391cdc0 msgr2=0x5613d392be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391cdc0 0x5613d392be20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94c8019040 tx=0x7f94c8004180 comp rx=0 tx=0).stop 2024-06-13T07:06:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 -- 172.21.15.26:0/277336147 shutdown_connections 2024-06-13T07:06:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5613d399e830 0x5613d39a0c50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5613d3922920 0x5613d392d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5613d391d7b0 0x5613d392c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 --2- 172.21.15.26:0/277336147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d391cdc0 0x5613d392be20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94ba000640 0 client.25913 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 -- 172.21.15.26:0/277336147 >> 172.21.15.26:0/277336147 conn(0x5613d38e4fb0 msgr2=0x5613d3926bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 -- 172.21.15.26:0/277336147 shutdown_connections 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.132+0000 7f94de194740 1 -- 172.21.15.26:0/277336147 wait complete. 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94de194740 20 client.25913 trim_cache size 0 max 16384 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:06:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:06:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 -- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94bc0a2c00 msgr2=0x7f94bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94bc0a2c00 0x7f94bc0a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5613d39196f0 tx=0x7f94c8005f90 comp rx=0 tx=0).stop 2024-06-13T07:06:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 -- 172.21.15.26:0/116175829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4104c30 msgr2=0x7f94d419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4104c30 0x7f94d419a300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94c0031440 tx=0x7f94c0032ce0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 -- 172.21.15.26:0/116175829 shutdown_connections 2024-06-13T07:06:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94bc0a2c00 0x7f94bc0a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4104c30 0x7f94d419a300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d4107560 0x7f94d41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 --2- 172.21.15.26:0/116175829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d4102300 0x7f94d4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.133+0000 7f94d9e00640 1 -- 172.21.15.26:0/116175829 >> 172.21.15.26:0/116175829 conn(0x7f94d40fbc90 msgr2=0x7f94d4101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.134+0000 7f94d9e00640 1 -- 172.21.15.26:0/116175829 shutdown_connections 2024-06-13T07:06:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:12.134+0000 7f94d9e00640 1 -- 172.21.15.26:0/116175829 wait complete. 2024-06-13T07:06:12.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 223313 4.55 MB/sec execute 146 sec latency 39.796 ms 2024-06-13T07:06:13.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 223945 4.55 MB/sec execute 147 sec latency 42.669 ms 2024-06-13T07:06:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:13 smithi026 ceph-mon[32041]: pgmap v681: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:06:14.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:13 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:06:14.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:13 smithi135 ceph-mon[36276]: pgmap v681: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:06:14.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:13 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:06:14.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:13 smithi136 ceph-mon[37229]: pgmap v681: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:06:14.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:13 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:06:14.532 INFO:tasks.workunit.client.0.smithi026.stdout: 1 224923 4.56 MB/sec execute 148 sec latency 23.299 ms 2024-06-13T07:06:15.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 226392 4.57 MB/sec execute 149 sec latency 22.307 ms 2024-06-13T07:06:16.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:15 smithi026 ceph-mon[32041]: pgmap v682: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:06:16.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:06:16.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:06:16.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:06:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:15 smithi135 ceph-mon[36276]: pgmap v682: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:06:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:06:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:06:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:06:16.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:15 smithi136 ceph-mon[37229]: pgmap v682: 129 pgs: 129 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:06:16.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:06:16.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:06:16.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:06:16.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 227279 4.59 MB/sec execute 150 sec latency 21.946 ms 2024-06-13T07:06:17.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 227658 4.56 MB/sec execute 151 sec latency 39.186 ms 2024-06-13T07:06:18.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:17 smithi026 ceph-mon[32041]: pgmap v683: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 187 op/s 2024-06-13T07:06:18.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:17 smithi135 ceph-mon[36276]: pgmap v683: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 187 op/s 2024-06-13T07:06:18.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:17 smithi136 ceph-mon[37229]: pgmap v683: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 187 op/s 2024-06-13T07:06:18.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 228705 4.57 MB/sec execute 152 sec latency 52.941 ms 2024-06-13T07:06:19.028 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:18 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:06:19.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:18 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:06:19.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:18 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:06:19.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 230035 4.58 MB/sec execute 153 sec latency 32.948 ms 2024-06-13T07:06:20.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:19 smithi135 ceph-mon[36276]: pgmap v684: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:06:20.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:19 smithi136 ceph-mon[37229]: pgmap v684: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:06:20.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:19 smithi026 ceph-mon[32041]: pgmap v684: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-06-13T07:06:20.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 230864 4.59 MB/sec execute 154 sec latency 23.913 ms 2024-06-13T07:06:21.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 231326 4.57 MB/sec execute 155 sec latency 35.288 ms 2024-06-13T07:06:22.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:21 smithi135 ceph-mon[36276]: pgmap v685: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-06-13T07:06:22.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:21 smithi136 ceph-mon[37229]: pgmap v685: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-06-13T07:06:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:21 smithi026 ceph-mon[32041]: pgmap v685: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-06-13T07:06:22.533 INFO:tasks.workunit.client.0.smithi026.stdout: 1 232263 4.57 MB/sec execute 156 sec latency 25.284 ms 2024-06-13T07:06:23.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 233537 4.59 MB/sec execute 157 sec latency 83.230 ms 2024-06-13T07:06:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:23 smithi136 ceph-mon[37229]: pgmap v686: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:06:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:23 smithi026 ceph-mon[32041]: pgmap v686: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:06:24.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:23 smithi135 ceph-mon[36276]: pgmap v686: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:06:24.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 234475 4.60 MB/sec execute 158 sec latency 25.814 ms 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 -- 172.21.15.26:0/871640122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c100720 msgr2=0x7fba3c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 --2- 172.21.15.26:0/871640122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c100720 0x7fba3c100b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009860 tx=0x7fba2c02a340 comp rx=0 tx=0).stop 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 -- 172.21.15.26:0/871640122 shutdown_connections 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 --2- 172.21.15.26:0/871640122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba3c1022d0 0x7fba3c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 --2- 172.21.15.26:0/871640122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba3c101910 0x7fba3c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 --2- 172.21.15.26:0/871640122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c100720 0x7fba3c100b40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 -- 172.21.15.26:0/871640122 >> 172.21.15.26:0/871640122 conn(0x7fba3c0fbc90 msgr2=0x7fba3c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 -- 172.21.15.26:0/871640122 shutdown_connections 2024-06-13T07:06:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.294+0000 7fba41400640 1 -- 172.21.15.26:0/871640122 wait complete. 2024-06-13T07:06:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.295+0000 7fba41400640 1 Processor -- start 2024-06-13T07:06:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.295+0000 7fba41400640 1 -- start start 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba41400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba3c100720 0x7fba3c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba41400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c101910 0x7fba3c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba41400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba3c1022d0 0x7fba3c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba41400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c10d480 con 0x7fba3c100720 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba41400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c10d300 con 0x7fba3c101910 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba41400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c10d600 con 0x7fba3c1022d0 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba33e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c101910 0x7fba3c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba40a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba3c1022d0 0x7fba3c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba33e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c101910 0x7fba3c19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51196/0 (socket says 172.21.15.26:51196) 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba33e00640 1 -- 172.21.15.26:0/3577439005 learned_addr learned my addr 172.21.15.26:0/3577439005 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba40a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba3c1022d0 0x7fba3c1a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60140/0 (socket says 172.21.15.26:60140) 2024-06-13T07:06:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba3be00640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba3c100720 0x7fba3c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba40a00640 1 -- 172.21.15.26:0/3577439005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c101910 msgr2=0x7fba3c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba40a00640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c101910 0x7fba3c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba40a00640 1 -- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba3c100720 msgr2=0x7fba3c199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba40a00640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba3c100720 0x7fba3c199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba40a00640 1 -- 172.21.15.26:0/3577439005 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c1a1990 con 0x7fba3c1022d0 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba33e00640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c101910 0x7fba3c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.296+0000 7fba3be00640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba3c100720 0x7fba3c199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.297+0000 7fba40a00640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba3c1022d0 0x7fba3c1a1320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba34009fd0 tx=0x7fba3400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.297+0000 7fba39600640 1 -- 172.21.15.26:0/3577439005 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3400ce20 con 0x7fba3c1022d0 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.297+0000 7fba39600640 1 -- 172.21.15.26:0/3577439005 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba34005020 con 0x7fba3c1022d0 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.297+0000 7fba41400640 1 -- 172.21.15.26:0/3577439005 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c1a1c80 con 0x7fba3c1022d0 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.297+0000 7fba39600640 1 -- 172.21.15.26:0/3577439005 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34010720 con 0x7fba3c1022d0 2024-06-13T07:06:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.297+0000 7fba41400640 1 -- 172.21.15.26:0/3577439005 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c1a21d0 con 0x7fba3c1022d0 2024-06-13T07:06:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.300+0000 7fba45774740 1 Processor -- start 2024-06-13T07:06:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.301+0000 7fba39600640 1 -- 172.21.15.26:0/3577439005 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba34005540 con 0x7fba3c1022d0 2024-06-13T07:06:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.301+0000 7fba45774740 1 -- start start 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba45774740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba45774740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x56288798e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba45774740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887990db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba45774740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5628878ec420 con 0x56288798e920 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba45774740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562887893c50 con 0x562887988dc0 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba45774740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5628878e83e0 con 0x5628879897b0 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba33e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x56288798e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba33e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879891e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51212/0 (socket says 172.21.15.26:51212) 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba33e00640 1 -- 172.21.15.26:0/384771305 learned_addr learned my addr 172.21.15.26:0/384771305 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.302+0000 7fba33e00640 1 --2- 172.21.15.26:0/384771305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887990db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.303+0000 7fba3be00640 1 -- 172.21.15.26:0/384771305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 msgr2=0x5628879891e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.303+0000 7fba3be00640 1 --2- 172.21.15.26:0/384771305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879891e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.303+0000 7fba3be00640 1 -- 172.21.15.26:0/384771305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 msgr2=0x562887990db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.303+0000 7fba3be00640 1 --2- 172.21.15.26:0/384771305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887990db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.303+0000 7fba3be00640 1 -- 172.21.15.26:0/384771305 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562887600210 con 0x5628879897b0 2024-06-13T07:06:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.303+0000 7fba3be00640 1 --2- 172.21.15.26:0/384771305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x56288798e2b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba2c035040 tx=0x7fba2c005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff24d6c99a934c3e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.303+0000 7fba39600640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba200a2c00 0x7fba200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.304+0000 7fba31200640 1 -- 172.21.15.26:0/384771305 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c040070 con 0x5628879897b0 2024-06-13T07:06:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.304+0000 7fba31200640 1 -- 172.21.15.26:0/384771305 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba2c004020 con 0x5628879897b0 2024-06-13T07:06:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.304+0000 7fba39600640 1 -- 172.21.15.26:0/3577439005 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fba340cfbc0 con 0x7fba3c1022d0 2024-06-13T07:06:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.304+0000 7fba31200640 1 -- 172.21.15.26:0/384771305 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c037040 con 0x5628879897b0 2024-06-13T07:06:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.304+0000 7fba33e00640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba200a2c00 0x7fba200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 -- 172.21.15.26:0/384771305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 msgr2=0x56288798e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 --2- 172.21.15.26:0/384771305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x56288798e2b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba2c035040 tx=0x7fba2c005d10 comp rx=0 tx=0).stop 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba33e00640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba200a2c00 0x7fba200a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x56288798a010 tx=0x7fba24013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 -- 172.21.15.26:0/384771305 shutdown_connections 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 --2- 172.21.15.26:0/384771305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887990db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 --2- 172.21.15.26:0/384771305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x56288798e2b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 --2- 172.21.15.26:0/384771305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879891e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 -- 172.21.15.26:0/384771305 >> 172.21.15.26:0/384771305 conn(0x562887950fb0 msgr2=0x562887984840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 -- 172.21.15.26:0/384771305 shutdown_connections 2024-06-13T07:06:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.305+0000 7fba45774740 1 -- 172.21.15.26:0/384771305 wait complete. 2024-06-13T07:06:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 1 -- start start 2024-06-13T07:06:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 1 Processor -- start 2024-06-13T07:06:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x562887993630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56288798e920 0x56288799a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562887991870 con 0x5628879897b0 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5628878a70b0 con 0x562887988dc0 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5628879919f0 con 0x56288798e920 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.306+0000 7fba45774740 10 client.0 _ll_register_callbacks cb 0x562887892f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x562887993630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x562887993630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45206/0 (socket says 172.21.15.26:45206) 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba33e00640 1 -- 172.21.15.26:0/804700265 learned_addr learned my addr 172.21.15.26:0/804700265 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba3be00640 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba40a00640 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56288798e920 0x56288799a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba3be00640 1 -- 172.21.15.26:0/804700265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56288798e920 msgr2=0x56288799a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba3be00640 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56288798e920 0x56288799a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba3be00640 1 -- 172.21.15.26:0/804700265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 msgr2=0x562887993630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba3be00640 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x562887993630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba3be00640 1 -- 172.21.15.26:0/804700265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628876cc620 con 0x562887988dc0 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.307+0000 7fba33e00640 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x562887993630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.308+0000 7fba3be00640 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879930f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5628879856f0 tx=0x7fba2c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bed39853703b874 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.308+0000 7fba29600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.308+0000 7fba29600640 1 -- 172.21.15.26:0/804700265 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c048030 con 0x562887988dc0 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.308+0000 7fba29600640 1 -- 172.21.15.26:0/804700265 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fba2c0043e0 con 0x562887988dc0 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.308+0000 7fba45774740 1 -- 172.21.15.26:0/804700265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba2c004020 con 0x562887988dc0 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.308+0000 7fba29600640 1 -- 172.21.15.26:0/804700265 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba2c055050 con 0x562887988dc0 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.308+0000 7fba45774740 10 client.25925 fetch_fsmap learned FSMap version 35 2024-06-13T07:06:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.308+0000 7fba45774740 1 -- 172.21.15.26:0/804700265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56288799d9e0 con 0x562887988dc0 2024-06-13T07:06:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba29600640 1 -- 172.21.15.26:0/804700265 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba2c004020 con 0x562887988dc0 2024-06-13T07:06:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba29600640 1 -- 172.21.15.26:0/804700265 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fba2c0046c0 con 0x562887988dc0 2024-06-13T07:06:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba45774740 10 client.25925 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:06:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba45774740 10 client.25925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba45774740 20 client.25925 populate_metadata read hostname 'smithi026' 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba45774740 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56288799ed30 0x5628879a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba45774740 4 client.25925 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba45774740 1 -- 172.21.15.26:0/804700265 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628878f9b50 con 0x56288799ed30 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.309+0000 7fba33e00640 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56288799ed30 0x5628879a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.310+0000 7fba33e00640 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56288799ed30 0x5628879a1120 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935eb4e68ff4a0d6 server_cookie=a0cab68035fef31f in_seq=0 out_seq=0 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.310+0000 7fba29600640 10 client.25925 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba29600640 1 -- 172.21.15.26:0/804700265 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5628878f9b50 con 0x56288799ed30 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba29600640 10 client.25925 handle_command_reply: tid=0 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 client.25925 shutdown 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 -- 172.21.15.26:0/804700265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 msgr2=0x5628879930f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879930f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5628879856f0 tx=0x7fba2c005f90 comp rx=0 tx=0).stop 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 -- 172.21.15.26:0/804700265 shutdown_connections 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56288799ed30 0x5628879a1120 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56288798e920 0x56288799a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x562887993630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 --2- 172.21.15.26:0/804700265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x5628879930f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 -- 172.21.15.26:0/804700265 >> 172.21.15.26:0/804700265 conn(0x562887950fb0 msgr2=0x5628879839a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba29600640 0 client.25925 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 -- 172.21.15.26:0/804700265 shutdown_connections 2024-06-13T07:06:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 1 -- 172.21.15.26:0/804700265 wait complete. 2024-06-13T07:06:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.311+0000 7fba45774740 20 client.25925 trim_cache size 0 max 16384 2024-06-13T07:06:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.329+0000 7fba45774740 1 Processor -- start 2024-06-13T07:06:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.329+0000 7fba45774740 1 -- start start 2024-06-13T07:06:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba45774740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562887988dc0 0x5628879880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba45774740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x56288799a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba45774740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56288798e920 0x562887983820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba45774740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1aefb0 con 0x5628879897b0 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba45774740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1ac880 con 0x56288798e920 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba45774740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1aa6d0 con 0x562887988dc0 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562887988dc0 0x5628879880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba40a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56288798e920 0x562887983820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x56288799a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba40a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56288798e920 0x562887983820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51226/0 (socket says 172.21.15.26:51226) 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562887988dc0 0x5628879880a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60164/0 (socket says 172.21.15.26:60164) 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.330+0000 7fba40a00640 1 -- 172.21.15.26:0/917574905 learned_addr learned my addr 172.21.15.26:0/917574905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.331+0000 7fba40a00640 1 -- 172.21.15.26:0/917574905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562887988dc0 msgr2=0x5628879880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.331+0000 7fba40a00640 1 --2- 172.21.15.26:0/917574905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562887988dc0 0x5628879880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.331+0000 7fba40a00640 1 -- 172.21.15.26:0/917574905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 msgr2=0x56288799a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.331+0000 7fba40a00640 1 --2- 172.21.15.26:0/917574905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x56288799a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.331+0000 7fba40a00640 1 -- 172.21.15.26:0/917574905 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288798dda0 con 0x56288798e920 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.331+0000 7fba40a00640 1 --2- 172.21.15.26:0/917574905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56288798e920 0x562887983820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fba340989a0 tx=0x7fba340cd140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ab6ae6db432ec8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.331+0000 7fba2be00640 1 -- 172.21.15.26:0/917574905 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340b4070 con 0x56288798e920 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.331+0000 7fba2be00640 1 -- 172.21.15.26:0/917574905 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba340a0380 con 0x56288798e920 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.332+0000 7fba2be00640 1 -- 172.21.15.26:0/917574905 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340a47b0 con 0x56288798e920 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.332+0000 7fba45774740 1 -- 172.21.15.26:0/917574905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56288798e920 msgr2=0x562887983820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.332+0000 7fba45774740 1 --2- 172.21.15.26:0/917574905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56288798e920 0x562887983820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fba340989a0 tx=0x7fba340cd140 comp rx=0 tx=0).stop 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.333+0000 7fba45774740 1 -- 172.21.15.26:0/917574905 shutdown_connections 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.333+0000 7fba45774740 1 --2- 172.21.15.26:0/917574905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56288798e920 0x562887983820 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.333+0000 7fba45774740 1 --2- 172.21.15.26:0/917574905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628879897b0 0x56288799a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.333+0000 7fba45774740 1 --2- 172.21.15.26:0/917574905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562887988dc0 0x5628879880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.333+0000 7fba45774740 1 -- 172.21.15.26:0/917574905 >> 172.21.15.26:0/917574905 conn(0x562887950fb0 msgr2=0x562887983110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.333+0000 7fba45774740 1 -- 172.21.15.26:0/917574905 shutdown_connections 2024-06-13T07:06:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.333+0000 7fba45774740 1 -- 172.21.15.26:0/917574905 wait complete. 2024-06-13T07:06:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.333+0000 7fba45774740 1 -- start start 2024-06-13T07:06:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba45774740 1 Processor -- start 2024-06-13T07:06:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba45774740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x562887997770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba45774740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x562887997cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba45774740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887998cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba45774740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1aad40 con 0x56288798e920 2024-06-13T07:06:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba45774740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1ac1b0 con 0x562887988dc0 2024-06-13T07:06:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba45774740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1ab410 con 0x5628879897b0 2024-06-13T07:06:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba45774740 10 client.0 _ll_register_callbacks cb 0x7fba24009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba33e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x562887997cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba33e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x562887997cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60170/0 (socket says 172.21.15.26:60170) 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba33e00640 1 -- 172.21.15.26:0/2570473661 learned_addr learned my addr 172.21.15.26:0/2570473661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba3be00640 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x562887997770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.334+0000 7fba40a00640 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887998cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba33e00640 1 -- 172.21.15.26:0/2570473661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 msgr2=0x562887997770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba33e00640 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x562887997770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba33e00640 1 -- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 msgr2=0x562887998cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba33e00640 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887998cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba33e00640 1 -- 172.21.15.26:0/2570473661 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628879992b0 con 0x5628879897b0 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba40a00640 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887998cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba33e00640 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x562887997cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fba24003bd0 tx=0x7fba24004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af39ecfbcb0635c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba29600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:06:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba29600640 1 -- 172.21.15.26:0/2570473661 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24003e90 con 0x5628879897b0 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba45774740 1 -- 172.21.15.26:0/2570473661 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba340a0380 con 0x5628879897b0 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.335+0000 7fba29600640 1 -- 172.21.15.26:0/2570473661 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fba2402c3f0 con 0x5628879897b0 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba29600640 1 -- 172.21.15.26:0/2570473661 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba2401fe10 con 0x5628879897b0 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba45774740 10 client.25975 fetch_fsmap learned FSMap version 35 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba45774740 1 -- 172.21.15.26:0/2570473661 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56288799a690 con 0x5628879897b0 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba29600640 1 -- 172.21.15.26:0/2570473661 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba340a0380 con 0x5628879897b0 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba29600640 1 -- 172.21.15.26:0/2570473661 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fba24035070 con 0x5628879897b0 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba45774740 10 client.25975 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba45774740 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba45774740 20 client.25975 populate_metadata read hostname 'smithi026' 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba45774740 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562887a09ef0 0x562887a0c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba45774740 4 client.25975 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:06:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.336+0000 7fba45774740 1 -- 172.21.15.26:0/2570473661 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628878f3ea0 con 0x562887a09ef0 2024-06-13T07:06:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.337+0000 7fba3be00640 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562887a09ef0 0x562887a0c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.337+0000 7fba3be00640 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562887a09ef0 0x562887a0c310 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cecdb5c10b81d90 server_cookie=a45730189ed32658 in_seq=0 out_seq=0 2024-06-13T07:06:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.337+0000 7fba29600640 10 client.25975 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba29600640 1 -- 172.21.15.26:0/2570473661 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fba2c049030 con 0x562887a09ef0 2024-06-13T07:06:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba29600640 10 client.25975 handle_command_reply: tid=0 2024-06-13T07:06:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 client.25975 shutdown 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 -- 172.21.15.26:0/2570473661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 msgr2=0x562887997cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x562887997cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fba24003bd0 tx=0x7fba24004730 comp rx=0 tx=0).stop 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 -- 172.21.15.26:0/2570473661 shutdown_connections 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562887a09ef0 0x562887a0c310 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56288798e920 0x562887998cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628879897b0 0x562887997cb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 --2- 172.21.15.26:0/2570473661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562887988dc0 0x562887997770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba45774740 1 -- 172.21.15.26:0/2570473661 >> 172.21.15.26:0/2570473661 conn(0x562887950fb0 msgr2=0x5628879839a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.424+0000 7fba29600640 0 client.25975 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.425+0000 7fba45774740 1 -- 172.21.15.26:0/2570473661 shutdown_connections 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.425+0000 7fba45774740 1 -- 172.21.15.26:0/2570473661 wait complete. 2024-06-13T07:06:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.425+0000 7fba45774740 20 client.25975 trim_cache size 0 max 16384 2024-06-13T07:06:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:06:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:06:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:06:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 -- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba200a2c00 msgr2=0x7fba200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba200a2c00 0x7fba200a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x56288798a010 tx=0x7fba24013040 comp rx=0 tx=0).stop 2024-06-13T07:06:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 -- 172.21.15.26:0/3577439005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba3c1022d0 msgr2=0x7fba3c1a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba3c1022d0 0x7fba3c1a1320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba34009fd0 tx=0x7fba3400ef10 comp rx=0 tx=0).stop 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 -- 172.21.15.26:0/3577439005 shutdown_connections 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fba200a2c00 0x7fba200a50c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c101910 0x7fba3c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba3c1022d0 0x7fba3c1a1320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 --2- 172.21.15.26:0/3577439005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba3c100720 0x7fba3c199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 -- 172.21.15.26:0/3577439005 >> 172.21.15.26:0/3577439005 conn(0x7fba3c0fbc90 msgr2=0x7fba3c109520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 -- 172.21.15.26:0/3577439005 shutdown_connections 2024-06-13T07:06:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:25.426+0000 7fba41400640 1 -- 172.21.15.26:0/3577439005 wait complete. 2024-06-13T07:06:25.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 234957 4.57 MB/sec execute 159 sec latency 45.534 ms 2024-06-13T07:06:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:25 smithi136 ceph-mon[37229]: pgmap v687: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-13T07:06:26.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:25 smithi026 ceph-mon[32041]: pgmap v687: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-13T07:06:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:25 smithi135 ceph-mon[36276]: pgmap v687: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-13T07:06:26.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 236351 4.58 MB/sec execute 160 sec latency 20.952 ms 2024-06-13T07:06:27.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 237321 4.60 MB/sec execute 161 sec latency 26.359 ms 2024-06-13T07:06:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:27 smithi136 ceph-mon[37229]: pgmap v688: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:06:28.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:27 smithi026 ceph-mon[32041]: pgmap v688: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:06:28.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:27 smithi135 ceph-mon[36276]: pgmap v688: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:06:28.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 238034 4.60 MB/sec execute 162 sec latency 32.525 ms 2024-06-13T07:06:29.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 238477 4.58 MB/sec execute 163 sec latency 46.034 ms 2024-06-13T07:06:30.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:29 smithi136 ceph-mon[37229]: pgmap v689: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:06:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:29 smithi026 ceph-mon[32041]: pgmap v689: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:06:30.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:29 smithi135 ceph-mon[36276]: pgmap v689: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:06:30.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 239388 4.58 MB/sec execute 164 sec latency 30.493 ms 2024-06-13T07:06:31.534 INFO:tasks.workunit.client.0.smithi026.stdout: 1 240686 4.60 MB/sec execute 165 sec latency 24.117 ms 2024-06-13T07:06:32.058 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:31 smithi026 ceph-mon[32041]: pgmap v690: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-06-13T07:06:32.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:31 smithi136 ceph-mon[37229]: pgmap v690: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-06-13T07:06:32.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:31 smithi135 ceph-mon[36276]: pgmap v690: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-06-13T07:06:32.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 241541 4.61 MB/sec execute 166 sec latency 18.616 ms 2024-06-13T07:06:33.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 242028 4.58 MB/sec execute 167 sec latency 54.730 ms 2024-06-13T07:06:34.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:33 smithi136 ceph-mon[37229]: pgmap v691: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-06-13T07:06:34.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:33 smithi026 ceph-mon[32041]: pgmap v691: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-06-13T07:06:34.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:33 smithi135 ceph-mon[36276]: pgmap v691: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-06-13T07:06:34.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 243287 4.59 MB/sec execute 168 sec latency 27.770 ms 2024-06-13T07:06:35.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 244445 4.61 MB/sec execute 169 sec latency 20.172 ms 2024-06-13T07:06:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:35 smithi136 ceph-mon[37229]: pgmap v692: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:06:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:35 smithi026 ceph-mon[32041]: pgmap v692: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:06:36.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:35 smithi135 ceph-mon[36276]: pgmap v692: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:06:36.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 245150 4.61 MB/sec execute 170 sec latency 26.417 ms 2024-06-13T07:06:37.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 245730 4.59 MB/sec execute 171 sec latency 28.275 ms 2024-06-13T07:06:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:37 smithi136 ceph-mon[37229]: pgmap v693: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-13T07:06:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:37 smithi026 ceph-mon[32041]: pgmap v693: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-13T07:06:38.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:37 smithi135 ceph-mon[36276]: pgmap v693: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-13T07:06:38.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 246970 4.59 MB/sec execute 172 sec latency 59.623 ms 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 -- 172.21.15.26:0/3354906115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdec8102a20 msgr2=0x7fdec8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 --2- 172.21.15.26:0/3354906115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdec8102a20 0x7fdec8102ea0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009860 tx=0x7fdeb402a340 comp rx=0 tx=0).stop 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 -- 172.21.15.26:0/3354906115 shutdown_connections 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 --2- 172.21.15.26:0/3354906115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec81033e0 0x7fdec8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 --2- 172.21.15.26:0/3354906115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdec8102a20 0x7fdec8102ea0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 --2- 172.21.15.26:0/3354906115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8101830 0x7fdec8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 -- 172.21.15.26:0/3354906115 >> 172.21.15.26:0/3354906115 conn(0x7fdec80fcda0 msgr2=0x7fdec80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 -- 172.21.15.26:0/3354906115 shutdown_connections 2024-06-13T07:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.571+0000 7fdecca00640 1 -- 172.21.15.26:0/3354906115 wait complete. 2024-06-13T07:06:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.572+0000 7fdecca00640 1 Processor -- start 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.572+0000 7fdecca00640 1 -- start start 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdecca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8101830 0x7fdec8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdecca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdec8102a20 0x7fdec819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdecca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec81033e0 0x7fdec81a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdecca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdec810d4b0 con 0x7fdec8102a20 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdecca00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdec810d330 con 0x7fdec81033e0 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdecca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdec810d630 con 0x7fdec8101830 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdec7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec81033e0 0x7fdec81a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdec8102a20 0x7fdec819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdec7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec81033e0 0x7fdec81a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39460/0 (socket says 172.21.15.26:39460) 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdec7e00640 1 -- 172.21.15.26:0/1447908072 learned_addr learned my addr 172.21.15.26:0/1447908072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.573+0000 7fdec7400640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8101830 0x7fdec8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdec7e00640 1 -- 172.21.15.26:0/1447908072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8101830 msgr2=0x7fdec8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdec7e00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8101830 0x7fdec8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdec7e00640 1 -- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdec8102a20 msgr2=0x7fdec819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdec7e00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdec8102a20 0x7fdec819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdec7e00640 1 -- 172.21.15.26:0/1447908072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec81a1960 con 0x7fdec81033e0 2024-06-13T07:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdec7e00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec81033e0 0x7fdec81a12f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdec000eb10 tx=0x7fdec000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdebfe00640 1 -- 172.21.15.26:0/1447908072 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec000ce20 con 0x7fdec81033e0 2024-06-13T07:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdecca00640 1 -- 172.21.15.26:0/1447908072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec81a1c50 con 0x7fdec81033e0 2024-06-13T07:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdebfe00640 1 -- 172.21.15.26:0/1447908072 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec0005020 con 0x7fdec81033e0 2024-06-13T07:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdebfe00640 1 -- 172.21.15.26:0/1447908072 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0010690 con 0x7fdec81033e0 2024-06-13T07:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.574+0000 7fdecca00640 1 -- 172.21.15.26:0/1447908072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec81a2170 con 0x7fdec81033e0 2024-06-13T07:06:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.578+0000 7fdebfe00640 1 -- 172.21.15.26:0/1447908072 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdec0005540 con 0x7fdec81033e0 2024-06-13T07:06:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.578+0000 7fded0df4740 1 Processor -- start 2024-06-13T07:06:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.578+0000 7fded0df4740 1 -- start start 2024-06-13T07:06:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fded0df4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fded0df4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fded0df4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fded0df4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557f7e0b7420 con 0x557f7e159920 2024-06-13T07:06:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fded0df4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557f7e05ec50 con 0x557f7e1547b0 2024-06-13T07:06:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fded0df4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557f7e0b33e0 con 0x557f7e153dc0 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fdec7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fdec7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39462/0 (socket says 172.21.15.26:39462) 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fdec7400640 1 -- 172.21.15.26:0/4048490980 learned_addr learned my addr 172.21.15.26:0/4048490980 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.579+0000 7fdec6a00640 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdebfe00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdeac0a2b80 0x7fdeac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdebfe00640 1 -- 172.21.15.26:0/1447908072 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdec00cfa50 con 0x7fdec81033e0 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdec7400640 1 -- 172.21.15.26:0/4048490980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 msgr2=0x557f7e1541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdec7400640 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e1541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdec7400640 1 -- 172.21.15.26:0/4048490980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 msgr2=0x557f7e15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdec7400640 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e15bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdec7400640 1 -- 172.21.15.26:0/4048490980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f7ddcb210 con 0x557f7e1547b0 2024-06-13T07:06:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdec6a00640 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdec7e00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdeac0a2b80 0x7fdeac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.580+0000 7fdec7400640 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1592b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009ca0 tx=0x7fdeb8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2956b0cf4fb0b12 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.581+0000 7fdeb3e00640 1 -- 172.21.15.26:0/4048490980 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8004220 con 0x557f7e1547b0 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.581+0000 7fdeb3e00640 1 -- 172.21.15.26:0/4048490980 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb80043c0 con 0x557f7e1547b0 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.581+0000 7fdeb3e00640 1 -- 172.21.15.26:0/4048490980 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8019810 con 0x557f7e1547b0 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 -- 172.21.15.26:0/4048490980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 msgr2=0x557f7e1592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fdec7e00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdeac0a2b80 0x7fdeac0a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdec00049d0 tx=0x7fdec0005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1592b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009ca0 tx=0x7fdeb8011810 comp rx=0 tx=0).stop 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 -- 172.21.15.26:0/4048490980 shutdown_connections 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1592b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 --2- 172.21.15.26:0/4048490980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 -- 172.21.15.26:0/4048490980 >> 172.21.15.26:0/4048490980 conn(0x557f7e11bfb0 msgr2=0x557f7e14f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 -- 172.21.15.26:0/4048490980 shutdown_connections 2024-06-13T07:06:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.582+0000 7fded0df4740 1 -- 172.21.15.26:0/4048490980 wait complete. 2024-06-13T07:06:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 1 -- start start 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 1 Processor -- start 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 0x557f7e15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e15e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e165670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557f7e15c8d0 con 0x557f7e153dc0 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557f7e0720b0 con 0x557f7e1547b0 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557f7e15ca50 con 0x557f7e159920 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.583+0000 7fded0df4740 10 client.0 _ll_register_callbacks cb 0x557f7e05df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e15e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39468/0 (socket says 172.21.15.26:39468) 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec7400640 1 -- 172.21.15.26:0/75005762 learned_addr learned my addr 172.21.15.26:0/75005762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec6a00640 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 0x557f7e15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec6a00640 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e165670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec7400640 1 -- 172.21.15.26:0/75005762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 msgr2=0x557f7e165670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec7400640 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e165670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec7400640 1 -- 172.21.15.26:0/75005762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 msgr2=0x557f7e15e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec7400640 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 0x557f7e15e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.584+0000 7fdec7400640 1 -- 172.21.15.26:0/75005762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7de97620 con 0x557f7e1547b0 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.585+0000 7fdec7400640 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e15e630 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8006010 tx=0x7fdeb8003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e41631cb4bc1a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.585+0000 7fdeb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.585+0000 7fdeb0c00640 1 -- 172.21.15.26:0/75005762 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8028070 con 0x557f7e1547b0 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.585+0000 7fded0df4740 1 -- 172.21.15.26:0/75005762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb80043c0 con 0x557f7e1547b0 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.585+0000 7fdeb0c00640 1 -- 172.21.15.26:0/75005762 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdeb8005040 con 0x557f7e1547b0 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.585+0000 7fdeb0c00640 1 -- 172.21.15.26:0/75005762 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdeb8034050 con 0x557f7e1547b0 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.585+0000 7fded0df4740 10 client.25943 fetch_fsmap learned FSMap version 35 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.585+0000 7fded0df4740 1 -- 172.21.15.26:0/75005762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557f7e168a30 con 0x557f7e1547b0 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fdeb0c00640 1 -- 172.21.15.26:0/75005762 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb80043c0 con 0x557f7e1547b0 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fdeb0c00640 1 -- 172.21.15.26:0/75005762 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fdeb8004ad0 con 0x557f7e1547b0 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fded0df4740 10 client.25943 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:06:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fded0df4740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fded0df4740 20 client.25943 populate_metadata read hostname 'smithi026' 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fded0df4740 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557f7e169eb0 0x557f7e16c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fded0df4740 4 client.25943 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fded0df4740 1 -- 172.21.15.26:0/75005762 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f7e0c4b50 con 0x557f7e169eb0 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.586+0000 7fdec6a00640 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557f7e169eb0 0x557f7e16c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.587+0000 7fdec6a00640 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557f7e169eb0 0x557f7e16c330 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6302bbf7052ffa3 server_cookie=32c7a4cfd70ccb83 in_seq=0 out_seq=0 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.587+0000 7fdeb0c00640 10 client.25943 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fdeb0c00640 1 -- 172.21.15.26:0/75005762 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f7e0c4b50 con 0x557f7e169eb0 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fdeb0c00640 10 client.25943 handle_command_reply: tid=0 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 client.25943 shutdown 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 -- 172.21.15.26:0/75005762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 msgr2=0x557f7e15e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e15e630 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8006010 tx=0x7fdeb8003820 comp rx=0 tx=0).stop 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 -- 172.21.15.26:0/75005762 shutdown_connections 2024-06-13T07:06:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557f7e169eb0 0x557f7e16c330 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e15e630 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e165670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 --2- 172.21.15.26:0/75005762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 0x557f7e15e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fdeb0c00640 0 client.25943 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 -- 172.21.15.26:0/75005762 >> 172.21.15.26:0/75005762 conn(0x557f7e11bfb0 msgr2=0x557f7e14e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 -- 172.21.15.26:0/75005762 shutdown_connections 2024-06-13T07:06:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 1 -- 172.21.15.26:0/75005762 wait complete. 2024-06-13T07:06:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.588+0000 7fded0df4740 20 client.25943 trim_cache size 0 max 16384 2024-06-13T07:06:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.611+0000 7fded0df4740 1 Processor -- start 2024-06-13T07:06:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fded0df4740 1 -- start start 2024-06-13T07:06:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fded0df4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 0x557f7e14f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fded0df4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e165630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fded0df4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e14f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fded0df4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81af080 con 0x557f7e153dc0 2024-06-13T07:06:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fded0df4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81ac950 con 0x557f7e1547b0 2024-06-13T07:06:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fded0df4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81aa7a0 con 0x557f7e159920 2024-06-13T07:06:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fdec7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fdec7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e165630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39480/0 (socket says 172.21.15.26:39480) 2024-06-13T07:06:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 0x557f7e14f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.612+0000 7fdec7400640 1 -- 172.21.15.26:0/920974239 learned_addr learned my addr 172.21.15.26:0/920974239 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdec6a00640 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e14f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdec7400640 1 -- 172.21.15.26:0/920974239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 msgr2=0x557f7e14f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdec7400640 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e14f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdec7400640 1 -- 172.21.15.26:0/920974239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 msgr2=0x557f7e14f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdec7400640 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 0x557f7e14f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdec7400640 1 -- 172.21.15.26:0/920974239 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f7e14f0d0 con 0x557f7e1547b0 2024-06-13T07:06:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdec6a00640 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdec7400640 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e165630 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80092a0 tx=0x7fdeb8005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b225430c10174964 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdeb3400640 1 -- 172.21.15.26:0/920974239 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8028040 con 0x557f7e1547b0 2024-06-13T07:06:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdeb3400640 1 -- 172.21.15.26:0/920974239 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb80097c0 con 0x557f7e1547b0 2024-06-13T07:06:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.613+0000 7fdeb3400640 1 -- 172.21.15.26:0/920974239 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80052f0 con 0x557f7e1547b0 2024-06-13T07:06:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.614+0000 7fded0df4740 1 -- 172.21.15.26:0/920974239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 msgr2=0x557f7e165630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.614+0000 7fded0df4740 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e165630 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80092a0 tx=0x7fdeb8005970 comp rx=0 tx=0).stop 2024-06-13T07:06:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.614+0000 7fded0df4740 1 -- 172.21.15.26:0/920974239 shutdown_connections 2024-06-13T07:06:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.614+0000 7fded0df4740 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e165630 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.614+0000 7fded0df4740 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e159920 0x557f7e14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.614+0000 7fded0df4740 1 --2- 172.21.15.26:0/920974239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e153dc0 0x557f7e14f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.614+0000 7fded0df4740 1 -- 172.21.15.26:0/920974239 >> 172.21.15.26:0/920974239 conn(0x557f7e11bfb0 msgr2=0x557f7e155a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.615+0000 7fded0df4740 1 -- 172.21.15.26:0/920974239 shutdown_connections 2024-06-13T07:06:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.615+0000 7fded0df4740 1 -- 172.21.15.26:0/920974239 wait complete. 2024-06-13T07:06:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.615+0000 7fded0df4740 1 -- start start 2024-06-13T07:06:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.616+0000 7fded0df4740 1 Processor -- start 2024-06-13T07:06:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.616+0000 7fded0df4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e162190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.616+0000 7fded0df4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fded0df4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e1d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fded0df4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81aae10 con 0x557f7e159920 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fded0df4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81ac280 con 0x557f7e1547b0 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fded0df4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81ab4e0 con 0x557f7e153dc0 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fded0df4740 10 client.0 _ll_register_callbacks cb 0x7fdeb8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39490/0 (socket says 172.21.15.26:39490) 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec7400640 1 -- 172.21.15.26:0/652154893 learned_addr learned my addr 172.21.15.26:0/652154893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec6a00640 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e1d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec7400640 1 -- 172.21.15.26:0/652154893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 msgr2=0x557f7e162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec7400640 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e162190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec7400640 1 -- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 msgr2=0x557f7e1d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec7400640 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e1d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec7400640 1 -- 172.21.15.26:0/652154893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7e165770 con 0x557f7e1547b0 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.617+0000 7fdec6a00640 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e1d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fdec7400640 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1626d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80099b0 tx=0x7fdeb8019940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6199b565de430c32 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fdeb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fdeb0c00640 1 -- 172.21.15.26:0/652154893 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8028040 con 0x557f7e1547b0 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fdeb0c00640 1 -- 172.21.15.26:0/652154893 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdeb8019a80 con 0x557f7e1547b0 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fded0df4740 1 -- 172.21.15.26:0/652154893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb80097c0 con 0x557f7e1547b0 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fdeb0c00640 1 -- 172.21.15.26:0/652154893 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdeb804b050 con 0x557f7e1547b0 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fded0df4740 10 client.25949 fetch_fsmap learned FSMap version 35 2024-06-13T07:06:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fded0df4740 1 -- 172.21.15.26:0/652154893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557f7e1d3f90 con 0x557f7e1547b0 2024-06-13T07:06:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.618+0000 7fdeb0c00640 1 -- 172.21.15.26:0/652154893 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb80097c0 con 0x557f7e1547b0 2024-06-13T07:06:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.619+0000 7fdeb0c00640 1 -- 172.21.15.26:0/652154893 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fdeb8004410 con 0x557f7e1547b0 2024-06-13T07:06:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.619+0000 7fded0df4740 10 client.25949 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:06:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.619+0000 7fded0df4740 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:06:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.619+0000 7fded0df4740 20 client.25949 populate_metadata read hostname 'smithi026' 2024-06-13T07:06:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.619+0000 7fded0df4740 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557f7e1d4850 0x557f7e1d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.619+0000 7fded0df4740 4 client.25949 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:06:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.619+0000 7fded0df4740 1 -- 172.21.15.26:0/652154893 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f7e15d530 con 0x557f7e1d4850 2024-06-13T07:06:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.619+0000 7fdec6a00640 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557f7e1d4850 0x557f7e1d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.620+0000 7fdec6a00640 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557f7e1d4850 0x557f7e1d6c70 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7e406841769536 server_cookie=c5bcbabfbc21552c in_seq=0 out_seq=0 2024-06-13T07:06:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.620+0000 7fdeb0c00640 10 client.25949 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fdeb0c00640 1 -- 172.21.15.26:0/652154893 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557f7e15d530 con 0x557f7e1d4850 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fdeb0c00640 10 client.25949 handle_command_reply: tid=0 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 client.25949 shutdown 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 -- 172.21.15.26:0/652154893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 msgr2=0x557f7e1626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1626d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80099b0 tx=0x7fdeb8019940 comp rx=0 tx=0).stop 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 -- 172.21.15.26:0/652154893 shutdown_connections 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557f7e1d4850 0x557f7e1d6c70 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f7e1547b0 0x557f7e1626d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557f7e159920 0x557f7e1d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 --2- 172.21.15.26:0/652154893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f7e153dc0 0x557f7e162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fded0df4740 1 -- 172.21.15.26:0/652154893 >> 172.21.15.26:0/652154893 conn(0x557f7e11bfb0 msgr2=0x557f7e14f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.701+0000 7fdeb0c00640 0 client.25949 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.702+0000 7fded0df4740 1 -- 172.21.15.26:0/652154893 shutdown_connections 2024-06-13T07:06:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.702+0000 7fded0df4740 1 -- 172.21.15.26:0/652154893 wait complete. 2024-06-13T07:06:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.702+0000 7fded0df4740 20 client.25949 trim_cache size 0 max 16384 2024-06-13T07:06:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:06:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:06:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:06:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 -- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdeac0a2b80 msgr2=0x7fdeac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdeac0a2b80 0x7fdeac0a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdec00049d0 tx=0x7fdec0005940 comp rx=0 tx=0).stop 2024-06-13T07:06:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 -- 172.21.15.26:0/1447908072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec81033e0 msgr2=0x7fdec81a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec81033e0 0x7fdec81a12f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdec000eb10 tx=0x7fdec000ef10 comp rx=0 tx=0).stop 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 -- 172.21.15.26:0/1447908072 shutdown_connections 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdeac0a2b80 0x7fdeac0a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec81033e0 0x7fdec81a12f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdec8102a20 0x7fdec819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 --2- 172.21.15.26:0/1447908072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8101830 0x7fdec8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 -- 172.21.15.26:0/1447908072 >> 172.21.15.26:0/1447908072 conn(0x7fdec80fcda0 msgr2=0x7fdec8073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 -- 172.21.15.26:0/1447908072 shutdown_connections 2024-06-13T07:06:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:38.703+0000 7fdecca00640 1 -- 172.21.15.26:0/1447908072 wait complete. 2024-06-13T07:06:39.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 248331 4.63 MB/sec execute 173 sec latency 38.199 ms 2024-06-13T07:06:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:39 smithi136 ceph-mon[37229]: pgmap v694: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:06:40.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:39 smithi026 ceph-mon[32041]: pgmap v694: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:06:40.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:39 smithi135 ceph-mon[36276]: pgmap v694: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:06:40.535 INFO:tasks.workunit.client.0.smithi026.stdout: 1 248772 4.62 MB/sec execute 174 sec latency 31.861 ms 2024-06-13T07:06:41.536 INFO:tasks.workunit.client.0.smithi026.stdout: 1 249422 4.60 MB/sec execute 175 sec latency 37.073 ms 2024-06-13T07:06:42.084 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:41 smithi135 ceph-mon[36276]: pgmap v695: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:06:42.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:41 smithi136 ceph-mon[37229]: pgmap v695: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:06:42.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:41 smithi026 ceph-mon[32041]: pgmap v695: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-06-13T07:06:42.536 INFO:tasks.workunit.client.0.smithi026.stdout: 1 250533 4.60 MB/sec execute 176 sec latency 28.930 ms 2024-06-13T07:06:43.536 INFO:tasks.workunit.client.0.smithi026.stdout: 1 251655 4.62 MB/sec execute 177 sec latency 28.899 ms 2024-06-13T07:06:44.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:43 smithi136 ceph-mon[37229]: pgmap v696: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-06-13T07:06:44.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:43 smithi026 ceph-mon[32041]: pgmap v696: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-06-13T07:06:44.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:43 smithi135 ceph-mon[36276]: pgmap v696: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-06-13T07:06:44.536 INFO:tasks.workunit.client.0.smithi026.stdout: 1 252342 4.62 MB/sec execute 178 sec latency 37.883 ms 2024-06-13T07:06:45.536 INFO:tasks.workunit.client.0.smithi026.stdout: 1 253117 4.61 MB/sec execute 179 sec latency 28.565 ms 2024-06-13T07:06:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:45 smithi136 ceph-mon[37229]: pgmap v697: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:06:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:45 smithi026 ceph-mon[32041]: pgmap v697: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:06:46.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:45 smithi135 ceph-mon[36276]: pgmap v697: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:06:46.536 INFO:tasks.workunit.client.0.smithi026.stdout: 1 254365 4.61 MB/sec execute 180 sec latency 24.206 ms 2024-06-13T07:06:47.536 INFO:tasks.workunit.client.0.smithi026.stdout: 1 255516 4.64 MB/sec execute 181 sec latency 21.648 ms 2024-06-13T07:06:48.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:47 smithi136 ceph-mon[37229]: pgmap v698: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-06-13T07:06:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:47 smithi026 ceph-mon[32041]: pgmap v698: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-06-13T07:06:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:47 smithi135 ceph-mon[36276]: pgmap v698: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-06-13T07:06:48.536 INFO:tasks.workunit.client.0.smithi026.stdout: 1 255973 4.62 MB/sec execute 182 sec latency 28.391 ms 2024-06-13T07:06:49.537 INFO:tasks.workunit.client.0.smithi026.stdout: 1 256927 4.63 MB/sec execute 183 sec latency 33.378 ms 2024-06-13T07:06:50.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:49 smithi136 ceph-mon[37229]: pgmap v699: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-13T07:06:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:49 smithi026 ceph-mon[32041]: pgmap v699: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-13T07:06:50.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:49 smithi135 ceph-mon[36276]: pgmap v699: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-13T07:06:50.537 INFO:tasks.workunit.client.0.smithi026.stdout: 1 258188 4.63 MB/sec execute 184 sec latency 21.638 ms 2024-06-13T07:06:51.537 INFO:tasks.workunit.client.0.smithi026.stdout: 1 259207 4.65 MB/sec execute 185 sec latency 19.325 ms 2024-06-13T07:06:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.855+0000 7fa8a4000640 1 -- 172.21.15.26:0/3420148554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 msgr2=0x7fa89c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.855+0000 7fa8a4000640 1 --2- 172.21.15.26:0/3420148554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 0x7fa89c104870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa890009ca0 tx=0x7fa89002a8c0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.856+0000 7fa8a4000640 1 -- 172.21.15.26:0/3420148554 shutdown_connections 2024-06-13T07:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.856+0000 7fa8a4000640 1 --2- 172.21.15.26:0/3420148554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa89c1076e0 0x7fa89c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.856+0000 7fa8a4000640 1 --2- 172.21.15.26:0/3420148554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c104db0 0x7fa89c1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.856+0000 7fa8a4000640 1 --2- 172.21.15.26:0/3420148554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 0x7fa89c104870 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.856+0000 7fa8a4000640 1 -- 172.21.15.26:0/3420148554 >> 172.21.15.26:0/3420148554 conn(0x7fa89c0fbe70 msgr2=0x7fa89c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.856+0000 7fa8a4000640 1 -- 172.21.15.26:0/3420148554 shutdown_connections 2024-06-13T07:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.856+0000 7fa8a4000640 1 -- 172.21.15.26:0/3420148554 wait complete. 2024-06-13T07:06:51.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.857+0000 7fa8a4000640 1 Processor -- start 2024-06-13T07:06:51.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.857+0000 7fa8a4000640 1 -- start start 2024-06-13T07:06:51.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a4000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 0x7fa89c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a4000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa89c104db0 0x7fa89c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a4000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1076e0 0x7fa89c19f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a4000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c10d6f0 con 0x7fa89c1076e0 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a4000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c10d570 con 0x7fa89c102480 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a4000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c10d870 con 0x7fa89c104db0 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 0x7fa89c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 0x7fa89c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:41844/0 (socket says 172.21.15.26:41844) 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a1600640 1 -- 172.21.15.26:0/1693051318 learned_addr learned my addr 172.21.15.26:0/1693051318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa89c104db0 0x7fa89c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.858+0000 7fa8a2000640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1076e0 0x7fa89c19f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa8a1600640 1 -- 172.21.15.26:0/1693051318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa89c104db0 msgr2=0x7fa89c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa8a1600640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa89c104db0 0x7fa89c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa8a1600640 1 -- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1076e0 msgr2=0x7fa89c19f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa8a1600640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1076e0 0x7fa89c19f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa8a1600640 1 -- 172.21.15.26:0/1693051318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c19f8e0 con 0x7fa89c102480 2024-06-13T07:06:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa8a1600640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 0x7fa89c197ca0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa890000f80 tx=0x7fa8900319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa89a000640 1 -- 172.21.15.26:0/1693051318 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890003b70 con 0x7fa89c102480 2024-06-13T07:06:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa89a000640 1 -- 172.21.15.26:0/1693051318 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa890003d10 con 0x7fa89c102480 2024-06-13T07:06:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa8a4000640 1 -- 172.21.15.26:0/1693051318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c19fb70 con 0x7fa89c102480 2024-06-13T07:06:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.859+0000 7fa89a000640 1 -- 172.21.15.26:0/1693051318 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89002fc00 con 0x7fa89c102480 2024-06-13T07:06:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.860+0000 7fa8a4000640 1 -- 172.21.15.26:0/1693051318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c1a0110 con 0x7fa89c102480 2024-06-13T07:06:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.863+0000 7fa89a000640 1 -- 172.21.15.26:0/1693051318 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa890004040 con 0x7fa89c102480 2024-06-13T07:06:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.864+0000 7fa8a6d8c740 1 Processor -- start 2024-06-13T07:06:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.864+0000 7fa8a6d8c740 1 -- start start 2024-06-13T07:06:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.864+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb234020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.864+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.864+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb23bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.864+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559cdb197420 con 0x559cdb2345f0 2024-06-13T07:06:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.864+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559cdb13ec50 con 0x559cdb239760 2024-06-13T07:06:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.864+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559cdb1933e0 con 0x559cdb233c00 2024-06-13T07:06:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb234020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb234020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41046/0 (socket says 172.21.15.26:41046) 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb23bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a2000640 1 -- 172.21.15.26:0/24282681 learned_addr learned my addr 172.21.15.26:0/24282681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a1600640 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a2000640 1 -- 172.21.15.26:0/24282681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 msgr2=0x559cdb23bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a2000640 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb23bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a2000640 1 -- 172.21.15.26:0/24282681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 msgr2=0x559cdb2390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a2000640 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2390f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa89a000640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa8840a2cb0 0x7fa8840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a2000640 1 -- 172.21.15.26:0/24282681 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cdaeab210 con 0x559cdb233c00 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb23bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.865+0000 7fa89a000640 1 -- 172.21.15.26:0/1693051318 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa8900f0810 con 0x7fa89c102480 2024-06-13T07:06:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.866+0000 7fa8a2000640 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb234020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa894009ca0 tx=0x7fa894011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4062b93122dd66 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.866+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa8840a2cb0 0x7fa8840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.866+0000 7fa88e600640 1 -- 172.21.15.26:0/24282681 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89401e070 con 0x559cdb233c00 2024-06-13T07:06:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.866+0000 7fa88e600640 1 -- 172.21.15.26:0/24282681 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa894009e50 con 0x559cdb233c00 2024-06-13T07:06:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.866+0000 7fa88e600640 1 -- 172.21.15.26:0/24282681 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89400ea80 con 0x559cdb233c00 2024-06-13T07:06:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/24282681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 msgr2=0x559cdb234020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb234020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa894009ca0 tx=0x7fa894011810 comp rx=0 tx=0).stop 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa8840a2cb0 0x7fa8840a5170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa89c1991a0 tx=0x7fa888005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/24282681 shutdown_connections 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb23bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/24282681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb234020 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/24282681 >> 172.21.15.26:0/24282681 conn(0x559cdb1fbfb0 msgr2=0x559cdb22f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/24282681 shutdown_connections 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.867+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/24282681 wait complete. 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.868+0000 7fa8a6d8c740 1 -- start start 2024-06-13T07:06:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.868+0000 7fa8a6d8c740 1 Processor -- start 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb233c00 0x559cdb23df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb23e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb245510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559cdb23c710 con 0x559cdb2345f0 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559cdb1520b0 con 0x559cdb233c00 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559cdb23c890 con 0x559cdb239760 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a6d8c740 10 client.0 _ll_register_callbacks cb 0x559cdb13df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb233c00 0x559cdb23df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb245510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb233c00 0x559cdb23df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:41860/0 (socket says 172.21.15.26:41860) 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb245510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41060/0 (socket says 172.21.15.26:41060) 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a1600640 1 -- 172.21.15.26:0/674770643 learned_addr learned my addr 172.21.15.26:0/674770643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb23e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a1600640 1 -- 172.21.15.26:0/674770643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 msgr2=0x559cdb245510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a1600640 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb245510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a1600640 1 -- 172.21.15.26:0/674770643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 msgr2=0x559cdb23e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a1600640 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb23e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a1600640 1 -- 172.21.15.26:0/674770643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cdaf77620 con 0x559cdb233c00 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.869+0000 7fa8a2000640 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb245510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.870+0000 7fa8a1600640 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb233c00 0x559cdb23df90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa890006010 tx=0x7fa8900099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ab0e2ae3b4dcdd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.870+0000 7fa882a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.870+0000 7fa882a00640 1 -- 172.21.15.26:0/674770643 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900bc880 con 0x559cdb233c00 2024-06-13T07:06:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.870+0000 7fa882a00640 1 -- 172.21.15.26:0/674770643 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa8900b93b0 con 0x559cdb233c00 2024-06-13T07:06:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.870+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/674770643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa894009e50 con 0x559cdb233c00 2024-06-13T07:06:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.870+0000 7fa882a00640 1 -- 172.21.15.26:0/674770643 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8900bc6a0 con 0x559cdb233c00 2024-06-13T07:06:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.870+0000 7fa8a6d8c740 10 client.25961 fetch_fsmap learned FSMap version 35 2024-06-13T07:06:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.870+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/674770643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559cdb23c710 con 0x559cdb233c00 2024-06-13T07:06:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa882a00640 1 -- 172.21.15.26:0/674770643 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa894009e50 con 0x559cdb233c00 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa882a00640 1 -- 172.21.15.26:0/674770643 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa8900bcfa0 con 0x559cdb233c00 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa8a6d8c740 10 client.25961 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa8a6d8c740 10 client.25961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa8a6d8c740 20 client.25961 populate_metadata read hostname 'smithi026' 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559cdb249fa0 0x559cdb24c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa8a6d8c740 4 client.25961 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/674770643 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cdb1a4b50 con 0x559cdb249fa0 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.871+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559cdb249fa0 0x559cdb24c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.872+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559cdb249fa0 0x559cdb24c3d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41a0d096436672d server_cookie=f2572175a352972e in_seq=0 out_seq=0 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.872+0000 7fa882a00640 10 client.25961 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa882a00640 1 -- 172.21.15.26:0/674770643 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559cdb1a4b50 con 0x559cdb249fa0 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa882a00640 10 client.25961 handle_command_reply: tid=0 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 client.25961 shutdown 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/674770643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb233c00 msgr2=0x559cdb23df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb233c00 0x559cdb23df90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa890006010 tx=0x7fa8900099c0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/674770643 shutdown_connections 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559cdb249fa0 0x559cdb24c3d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb245510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb23e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/674770643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb233c00 0x559cdb23df90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/674770643 >> 172.21.15.26:0/674770643 conn(0x559cdb1fbfb0 msgr2=0x559cdb22e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa882a00640 0 client.25961 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.873+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/674770643 shutdown_connections 2024-06-13T07:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.874+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/674770643 wait complete. 2024-06-13T07:06:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.874+0000 7fa8a6d8c740 20 client.25961 trim_cache size 0 max 16384 2024-06-13T07:06:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.893+0000 7fa8a6d8c740 1 Processor -- start 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.893+0000 7fa8a6d8c740 1 -- start start 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.893+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb233c00 0x559cdb232610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.893+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb2345f0 0x559cdb2454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.893+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb232b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.893+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1aad30 con 0x559cdb233c00 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.893+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1a8600 con 0x559cdb2345f0 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.893+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1a6450 con 0x559cdb239760 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb2345f0 0x559cdb2454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb232b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb232b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41068/0 (socket says 172.21.15.26:41068) 2024-06-13T07:06:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb2345f0 0x559cdb2454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:41876/0 (socket says 172.21.15.26:41876) 2024-06-13T07:06:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 -- 172.21.15.26:0/2212489501 learned_addr learned my addr 172.21.15.26:0/2212489501 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb233c00 0x559cdb232610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 -- 172.21.15.26:0/2212489501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 msgr2=0x559cdb232b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/2212489501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb232b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 -- 172.21.15.26:0/2212489501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb233c00 msgr2=0x559cdb232610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/2212489501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb233c00 0x559cdb232610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 -- 172.21.15.26:0/2212489501 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cdb233230 con 0x559cdb2345f0 2024-06-13T07:06:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.894+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/2212489501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb2345f0 0x559cdb2454a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa888013010 tx=0x7fa888017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13380974bb09f030 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.895+0000 7fa88dc00640 1 -- 172.21.15.26:0/2212489501 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888022c60 con 0x559cdb2345f0 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.895+0000 7fa88dc00640 1 -- 172.21.15.26:0/2212489501 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa888022e00 con 0x559cdb2345f0 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.895+0000 7fa88dc00640 1 -- 172.21.15.26:0/2212489501 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888020a10 con 0x559cdb2345f0 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/2212489501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb2345f0 msgr2=0x559cdb2454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/2212489501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb2345f0 0x559cdb2454a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa888013010 tx=0x7fa888017000 comp rx=0 tx=0).stop 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/2212489501 shutdown_connections 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/2212489501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb2345f0 0x559cdb2454a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/2212489501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb239760 0x559cdb232b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/2212489501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb233c00 0x559cdb232610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/2212489501 >> 172.21.15.26:0/2212489501 conn(0x559cdb1fbfb0 msgr2=0x559cdb22f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/2212489501 shutdown_connections 2024-06-13T07:06:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/2212489501 wait complete. 2024-06-13T07:06:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.896+0000 7fa8a6d8c740 1 -- start start 2024-06-13T07:06:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a6d8c740 1 Processor -- start 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb242280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a6d8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb2b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1a6ac0 con 0x559cdb2345f0 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1a7f30 con 0x559cdb239760 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a6d8c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1a7190 con 0x559cdb233c00 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a6d8c740 10 client.0 _ll_register_callbacks cb 0x7fa8900f6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a2000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb2b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb242280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a2000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb2b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:41888/0 (socket says 172.21.15.26:41888) 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.897+0000 7fa8a2000640 1 -- 172.21.15.26:0/1551774606 learned_addr learned my addr 172.21.15.26:0/1551774606 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:06:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa8a2000640 1 -- 172.21.15.26:0/1551774606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 msgr2=0x559cdb242280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa8a2000640 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb242280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa8a2000640 1 -- 172.21.15.26:0/1551774606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 msgr2=0x559cdb2427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa8a2000640 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa8a2000640 1 -- 172.21.15.26:0/1551774606 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cdb2b31b0 con 0x559cdb239760 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa8a1600640 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb242280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa8a0c00640 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa8a2000640 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb2b2c70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa894005940 tx=0x7fa8940096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a9bb5395ee93a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.898+0000 7fa882a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.899+0000 7fa882a00640 1 -- 172.21.15.26:0/1551774606 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89401e040 con 0x559cdb239760 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.899+0000 7fa882a00640 1 -- 172.21.15.26:0/1551774606 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa894003a00 con 0x559cdb239760 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.899+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/1551774606 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa888022e00 con 0x559cdb239760 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.899+0000 7fa882a00640 1 -- 172.21.15.26:0/1551774606 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa894034050 con 0x559cdb239760 2024-06-13T07:06:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.899+0000 7fa8a6d8c740 10 client.25967 fetch_fsmap learned FSMap version 35 2024-06-13T07:06:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.899+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/1551774606 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559cdb2b4360 con 0x559cdb239760 2024-06-13T07:06:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.899+0000 7fa882a00640 1 -- 172.21.15.26:0/1551774606 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa888022e00 con 0x559cdb239760 2024-06-13T07:06:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.899+0000 7fa882a00640 1 -- 172.21.15.26:0/1551774606 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa89402e4d0 con 0x559cdb239760 2024-06-13T07:06:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa8a6d8c740 10 client.25967 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:06:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa8a6d8c740 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:06:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa8a6d8c740 20 client.25967 populate_metadata read hostname 'smithi026' 2024-06-13T07:06:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559cdb2b4b20 0x559cdb2b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa8a6d8c740 4 client.25967 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:06:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/1551774606 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cdb19eea0 con 0x559cdb2b4b20 2024-06-13T07:06:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa8a1600640 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559cdb2b4b20 0x559cdb2b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa8a1600640 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559cdb2b4b20 0x559cdb2b6f40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9292c84b7b20aa1 server_cookie=fc641e2ab0648e75 in_seq=0 out_seq=0 2024-06-13T07:06:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.900+0000 7fa882a00640 10 client.25967 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.987+0000 7fa882a00640 1 -- 172.21.15.26:0/1551774606 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559cdb19eea0 con 0x559cdb2b4b20 2024-06-13T07:06:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.987+0000 7fa882a00640 10 client.25967 handle_command_reply: tid=0 2024-06-13T07:06:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.987+0000 7fa8a6d8c740 1 client.25967 shutdown 2024-06-13T07:06:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/1551774606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 msgr2=0x559cdb2b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb2b2c70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa894005940 tx=0x7fa8940096b0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/1551774606 shutdown_connections 2024-06-13T07:06:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559cdb2b4b20 0x559cdb2b6f40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cdb239760 0x559cdb2b2c70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cdb2345f0 0x559cdb2427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 --2- 172.21.15.26:0/1551774606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cdb233c00 0x559cdb242280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa882a00640 0 client.25967 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:06:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/1551774606 >> 172.21.15.26:0/1551774606 conn(0x559cdb1fbfb0 msgr2=0x559cdb232610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/1551774606 shutdown_connections 2024-06-13T07:06:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.989+0000 7fa8a6d8c740 1 -- 172.21.15.26:0/1551774606 wait complete. 2024-06-13T07:06:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.990+0000 7fa8a6d8c740 20 client.25967 trim_cache size 0 max 16384 2024-06-13T07:06:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:06:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:06:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:06:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.990+0000 7fa8a4000640 1 -- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa8840a2cb0 msgr2=0x7fa8840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.990+0000 7fa8a4000640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa8840a2cb0 0x7fa8840a5170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa89c1991a0 tx=0x7fa888005f90 comp rx=0 tx=0).stop 2024-06-13T07:06:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.990+0000 7fa8a4000640 1 -- 172.21.15.26:0/1693051318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 msgr2=0x7fa89c197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.990+0000 7fa8a4000640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 0x7fa89c197ca0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa890000f80 tx=0x7fa8900319f0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.991+0000 7fa8a4000640 1 -- 172.21.15.26:0/1693051318 shutdown_connections 2024-06-13T07:06:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.991+0000 7fa8a4000640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa8840a2cb0 0x7fa8840a5170 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.991+0000 7fa8a4000640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa89c1076e0 0x7fa89c19f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.991+0000 7fa8a4000640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa89c104db0 0x7fa89c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.991+0000 7fa8a4000640 1 --2- 172.21.15.26:0/1693051318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa89c102480 0x7fa89c197ca0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.991+0000 7fa8a4000640 1 -- 172.21.15.26:0/1693051318 >> 172.21.15.26:0/1693051318 conn(0x7fa89c0fbe70 msgr2=0x7fa89c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.991+0000 7fa8a4000640 1 -- 172.21.15.26:0/1693051318 shutdown_connections 2024-06-13T07:06:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:06:51.991+0000 7fa8a4000640 1 -- 172.21.15.26:0/1693051318 wait complete. 2024-06-13T07:06:52.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:51 smithi136 ceph-mon[37229]: pgmap v700: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:06:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:51 smithi026 ceph-mon[32041]: pgmap v700: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:06:52.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:51 smithi135 ceph-mon[36276]: pgmap v700: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:06:52.537 INFO:tasks.workunit.client.0.smithi026.stdout: 1 259691 4.63 MB/sec execute 186 sec latency 46.042 ms 2024-06-13T07:06:53.537 INFO:tasks.workunit.client.0.smithi026.stdout: 1 260733 4.63 MB/sec execute 187 sec latency 21.160 ms 2024-06-13T07:06:54.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:53 smithi136 ceph-mon[37229]: pgmap v701: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:06:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:53 smithi026 ceph-mon[32041]: pgmap v701: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:06:54.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:53 smithi135 ceph-mon[36276]: pgmap v701: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:06:54.537 INFO:tasks.workunit.client.0.smithi026.stdout: 1 261979 4.65 MB/sec execute 188 sec latency 31.377 ms 2024-06-13T07:06:55.537 INFO:tasks.workunit.client.0.smithi026.stdout: 1 262852 4.66 MB/sec execute 189 sec latency 48.551 ms 2024-06-13T07:06:56.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:55 smithi136 ceph-mon[37229]: pgmap v702: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-06-13T07:06:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:55 smithi026 ceph-mon[32041]: pgmap v702: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-06-13T07:06:56.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:55 smithi135 ceph-mon[36276]: pgmap v702: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-06-13T07:06:56.537 INFO:tasks.workunit.client.0.smithi026.stdout: 1 263339 4.63 MB/sec execute 190 sec latency 29.910 ms 2024-06-13T07:06:57.538 INFO:tasks.workunit.client.0.smithi026.stdout: 1 264323 4.63 MB/sec execute 191 sec latency 24.967 ms 2024-06-13T07:06:58.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:57 smithi136 ceph-mon[37229]: pgmap v703: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-06-13T07:06:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:57 smithi026 ceph-mon[32041]: pgmap v703: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-06-13T07:06:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:57 smithi135 ceph-mon[36276]: pgmap v703: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-06-13T07:06:58.538 INFO:tasks.workunit.client.0.smithi026.stdout: 1 265443 4.65 MB/sec execute 192 sec latency 20.296 ms 2024-06-13T07:06:59.538 INFO:tasks.workunit.client.0.smithi026.stdout: 1 266384 4.66 MB/sec execute 193 sec latency 54.016 ms 2024-06-13T07:07:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:06:59 smithi026 ceph-mon[32041]: pgmap v704: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-13T07:07:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:06:59 smithi135 ceph-mon[36276]: pgmap v704: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-13T07:07:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:06:59 smithi136 ceph-mon[37229]: pgmap v704: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-13T07:07:00.538 INFO:tasks.workunit.client.0.smithi026.stdout: 1 266800 4.64 MB/sec execute 194 sec latency 86.365 ms 2024-06-13T07:07:01.538 INFO:tasks.workunit.client.0.smithi026.stdout: 1 267764 4.64 MB/sec execute 195 sec latency 31.823 ms 2024-06-13T07:07:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:01 smithi026 ceph-mon[32041]: pgmap v705: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:07:02.273 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:01 smithi135 ceph-mon[36276]: pgmap v705: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:07:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:01 smithi136 ceph-mon[37229]: pgmap v705: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:07:02.538 INFO:tasks.workunit.client.0.smithi026.stdout: 1 269042 4.65 MB/sec execute 196 sec latency 19.117 ms 2024-06-13T07:07:03.538 INFO:tasks.workunit.client.0.smithi026.stdout: 1 269877 4.66 MB/sec execute 197 sec latency 21.942 ms 2024-06-13T07:07:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:03 smithi026 ceph-mon[32041]: pgmap v706: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:07:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:03 smithi135 ceph-mon[36276]: pgmap v706: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:07:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:03 smithi136 ceph-mon[37229]: pgmap v706: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:07:04.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 270394 4.64 MB/sec execute 198 sec latency 66.817 ms 2024-06-13T07:07:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 -- 172.21.15.26:0/4040220795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec103c30 msgr2=0x7f11ec104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 --2- 172.21.15.26:0/4040220795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec103c30 0x7f11ec104050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009900 tx=0x7f11e002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 -- 172.21.15.26:0/4040220795 shutdown_connections 2024-06-13T07:07:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 --2- 172.21.15.26:0/4040220795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec1057e0 0x7f11ec10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 --2- 172.21.15.26:0/4040220795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec104e20 0x7f11ec1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 --2- 172.21.15.26:0/4040220795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec103c30 0x7f11ec104050 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 -- 172.21.15.26:0/4040220795 >> 172.21.15.26:0/4040220795 conn(0x7f11ec0ff1c0 msgr2=0x7f11ec1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 -- 172.21.15.26:0/4040220795 shutdown_connections 2024-06-13T07:07:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.140+0000 7f11f3e00640 1 -- 172.21.15.26:0/4040220795 wait complete. 2024-06-13T07:07:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.142+0000 7f11f3e00640 1 Processor -- start 2024-06-13T07:07:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.142+0000 7f11f3e00640 1 -- start start 2024-06-13T07:07:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.142+0000 7f11f3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec103c30 0x7f11ec199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.142+0000 7f11f3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec104e20 0x7f11ec19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.142+0000 7f11f3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec1057e0 0x7f11ec1a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.142+0000 7f11f3e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10f900 con 0x7f11ec1057e0 2024-06-13T07:07:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.142+0000 7f11f3e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10f780 con 0x7f11ec103c30 2024-06-13T07:07:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.142+0000 7f11f3e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10fa80 con 0x7f11ec104e20 2024-06-13T07:07:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec103c30 0x7f11ec199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec103c30 0x7f11ec199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47896/0 (socket says 172.21.15.26:47896) 2024-06-13T07:07:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f0a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec104e20 0x7f11ec19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1400640 1 -- 172.21.15.26:0/2912587245 learned_addr learned my addr 172.21.15.26:0/2912587245 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec1057e0 0x7f11ec1a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1400640 1 -- 172.21.15.26:0/2912587245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec104e20 msgr2=0x7f11ec19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1400640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec104e20 0x7f11ec19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1400640 1 -- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec1057e0 msgr2=0x7f11ec1a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1400640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec1057e0 0x7f11ec1a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f1400640 1 -- 172.21.15.26:0/2912587245 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec1a1aa0 con 0x7f11ec103c30 2024-06-13T07:07:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.143+0000 7f11f0a00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec104e20 0x7f11ec19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.144+0000 7f11f1400640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec103c30 0x7f11ec199e80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11e0031440 tx=0x7f11e0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.144+0000 7f11ea000640 1 -- 172.21.15.26:0/2912587245 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e003f030 con 0x7f11ec103c30 2024-06-13T07:07:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.144+0000 7f11ea000640 1 -- 172.21.15.26:0/2912587245 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11e0003ce0 con 0x7f11ec103c30 2024-06-13T07:07:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.144+0000 7f11ea000640 1 -- 172.21.15.26:0/2912587245 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0035070 con 0x7f11ec103c30 2024-06-13T07:07:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.144+0000 7f11f3e00640 1 -- 172.21.15.26:0/2912587245 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec1a1d30 con 0x7f11ec103c30 2024-06-13T07:07:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.144+0000 7f11f3e00640 1 -- 172.21.15.26:0/2912587245 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec1a21c0 con 0x7f11ec103c30 2024-06-13T07:07:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.148+0000 7f11ea000640 1 -- 172.21.15.26:0/2912587245 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f11e0042030 con 0x7f11ec103c30 2024-06-13T07:07:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.148+0000 7f11f6bdf740 1 Processor -- start 2024-06-13T07:07:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f6bdf740 1 -- start start 2024-06-13T07:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a1c00 0x556aa89a2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a7760 0x556aa89a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556aa8905420 con 0x556aa89a1c00 2024-06-13T07:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556aa88acc50 con 0x556aa89a7760 2024-06-13T07:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556aa89013e0 con 0x556aa89a25f0 2024-06-13T07:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a1c00 0x556aa89a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f0a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a7760 0x556aa89a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a1c00 0x556aa89a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43208/0 (socket says 172.21.15.26:43208) 2024-06-13T07:07:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f0a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a7760 0x556aa89a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47912/0 (socket says 172.21.15.26:47912) 2024-06-13T07:07:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.149+0000 7f11f1e00640 1 -- 172.21.15.26:0/172850380 learned_addr learned my addr 172.21.15.26:0/172850380 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11f0a00640 1 -- 172.21.15.26:0/172850380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 msgr2=0x556aa89a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11f0a00640 1 --2- 172.21.15.26:0/172850380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11ea000640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11d40a2cb0 0x7f11d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11f0a00640 1 -- 172.21.15.26:0/172850380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a1c00 msgr2=0x556aa89a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11f0a00640 1 --2- 172.21.15.26:0/172850380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a1c00 0x556aa89a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11f0a00640 1 -- 172.21.15.26:0/172850380 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa8619210 con 0x556aa89a7760 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11ea000640 1 -- 172.21.15.26:0/2912587245 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f11e00efc20 con 0x7f11ec103c30 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11f0a00640 1 --2- 172.21.15.26:0/172850380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a7760 0x556aa89a9bf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009fa0 tx=0x7f11d80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0b7bf1d1bebd7d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.150+0000 7f11f1e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11d40a2cb0 0x7f11d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.151+0000 7f11de600640 1 -- 172.21.15.26:0/172850380 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8004030 con 0x556aa89a7760 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.151+0000 7f11de600640 1 -- 172.21.15.26:0/172850380 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d80041d0 con 0x556aa89a7760 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.151+0000 7f11de600640 1 -- 172.21.15.26:0/172850380 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800f910 con 0x556aa89a7760 2024-06-13T07:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 -- 172.21.15.26:0/172850380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a7760 msgr2=0x556aa89a9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/172850380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a7760 0x556aa89a9bf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009fa0 tx=0x7f11d80118d0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f1e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11d40a2cb0 0x7f11d40a5170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009fd0 tx=0x7f11e4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 -- 172.21.15.26:0/172850380 shutdown_connections 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/172850380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a7760 0x556aa89a9bf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/172850380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/172850380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a1c00 0x556aa89a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 -- 172.21.15.26:0/172850380 >> 172.21.15.26:0/172850380 conn(0x556aa8969fb0 msgr2=0x556aa899d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 -- 172.21.15.26:0/172850380 shutdown_connections 2024-06-13T07:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.152+0000 7f11f6bdf740 1 -- 172.21.15.26:0/172850380 wait complete. 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.153+0000 7f11f6bdf740 1 -- start start 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.153+0000 7f11f6bdf740 1 Processor -- start 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556aa89aa710 con 0x556aa89a7760 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556aa88c00b0 con 0x556aa89a1c00 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556aa89aa890 con 0x556aa89a25f0 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f6bdf740 10 client.0 _ll_register_callbacks cb 0x556aa88abf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f0a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47926/0 (socket says 172.21.15.26:47926) 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1400640 1 -- 172.21.15.26:0/533788718 learned_addr learned my addr 172.21.15.26:0/533788718 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1e00640 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1400640 1 -- 172.21.15.26:0/533788718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 msgr2=0x556aa89ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1400640 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1400640 1 -- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 msgr2=0x556aa89b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1400640 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1400640 1 -- 172.21.15.26:0/533788718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa86e5620 con 0x556aa89a1c00 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.154+0000 7f11f1e00640 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11f1400640 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89abfa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11e00b8ab0 tx=0x7f11e0005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec206f3eda6a8cc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11d2a00640 1 -- 172.21.15.26:0/533788718 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0049070 con 0x556aa89a1c00 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11d2a00640 1 -- 172.21.15.26:0/533788718 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f11e003b460 con 0x556aa89a1c00 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11f6bdf740 1 -- 172.21.15.26:0/533788718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d80041d0 con 0x556aa89a1c00 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11d2a00640 1 -- 172.21.15.26:0/533788718 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f11e00bbeb0 con 0x556aa89a1c00 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11f6bdf740 10 client.25979 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11f6bdf740 1 -- 172.21.15.26:0/533788718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556aa89b6820 con 0x556aa89a1c00 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.155+0000 7f11d2a00640 1 -- 172.21.15.26:0/533788718 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d80041d0 con 0x556aa89a1c00 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.156+0000 7f11d2a00640 1 -- 172.21.15.26:0/533788718 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f11e003bcc0 con 0x556aa89a1c00 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.156+0000 7f11f6bdf740 10 client.25979 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.156+0000 7f11f6bdf740 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.156+0000 7f11f6bdf740 20 client.25979 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.156+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556aa89b7f80 0x556aa89ba440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.156+0000 7f11f6bdf740 4 client.25979 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.156+0000 7f11f6bdf740 1 -- 172.21.15.26:0/533788718 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aa8912b50 con 0x556aa89b7f80 2024-06-13T07:07:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.156+0000 7f11f0a00640 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556aa89b7f80 0x556aa89ba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.157+0000 7f11f0a00640 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556aa89b7f80 0x556aa89ba440 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c7143f0ae96562 server_cookie=f886efb063ac6b95 in_seq=0 out_seq=0 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.157+0000 7f11d2a00640 10 client.25979 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11d2a00640 1 -- 172.21.15.26:0/533788718 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556aa8912b50 con 0x556aa89b7f80 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11d2a00640 10 client.25979 handle_command_reply: tid=0 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 client.25979 shutdown 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 -- 172.21.15.26:0/533788718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 msgr2=0x556aa89abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89abfa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11e00b8ab0 tx=0x7f11e0005e50 comp rx=0 tx=0).stop 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 -- 172.21.15.26:0/533788718 shutdown_connections 2024-06-13T07:07:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556aa89b7f80 0x556aa89ba440 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/533788718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89abfa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11d2a00640 0 client.25979 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.158+0000 7f11f6bdf740 1 -- 172.21.15.26:0/533788718 >> 172.21.15.26:0/533788718 conn(0x556aa8969fb0 msgr2=0x556aa899c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.159+0000 7f11f6bdf740 1 -- 172.21.15.26:0/533788718 shutdown_connections 2024-06-13T07:07:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.159+0000 7f11f6bdf740 1 -- 172.21.15.26:0/533788718 wait complete. 2024-06-13T07:07:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.159+0000 7f11f6bdf740 20 client.25979 trim_cache size 0 max 16384 2024-06-13T07:07:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.179+0000 7f11f6bdf740 1 Processor -- start 2024-06-13T07:07:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.179+0000 7f11f6bdf740 1 -- start start 2024-06-13T07:07:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.179+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.179+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.179+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.179+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1af1c0 con 0x556aa89a7760 2024-06-13T07:07:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.179+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1aca90 con 0x556aa89a1c00 2024-06-13T07:07:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.179+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1aa8e0 con 0x556aa89a25f0 2024-06-13T07:07:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47934/0 (socket says 172.21.15.26:47934) 2024-06-13T07:07:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 -- 172.21.15.26:0/1535089614 learned_addr learned my addr 172.21.15.26:0/1535089614 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f0a00640 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1e00640 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 -- 172.21.15.26:0/1535089614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 msgr2=0x556aa89b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 -- 172.21.15.26:0/1535089614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 msgr2=0x556aa89abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 -- 172.21.15.26:0/1535089614 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa89a29d0 con 0x556aa89a1c00 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1e00640 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.180+0000 7f11f1400640 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89a0e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11e00f3400 tx=0x7f11e0005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd0b94d512851552 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.181+0000 7f11ddc00640 1 -- 172.21.15.26:0/1535089614 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0049070 con 0x556aa89a1c00 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.181+0000 7f11ddc00640 1 -- 172.21.15.26:0/1535089614 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11e00bc0e0 con 0x556aa89a1c00 2024-06-13T07:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.181+0000 7f11ddc00640 1 -- 172.21.15.26:0/1535089614 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e00bba40 con 0x556aa89a1c00 2024-06-13T07:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 -- 172.21.15.26:0/1535089614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 msgr2=0x556aa89a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89a0e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11e00f3400 tx=0x7f11e0005a40 comp rx=0 tx=0).stop 2024-06-13T07:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 -- 172.21.15.26:0/1535089614 shutdown_connections 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a7760 0x556aa89abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a25f0 0x556aa89b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/1535089614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89a0e00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 -- 172.21.15.26:0/1535089614 >> 172.21.15.26:0/1535089614 conn(0x556aa8969fb0 msgr2=0x556aa899d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 -- 172.21.15.26:0/1535089614 shutdown_connections 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 -- 172.21.15.26:0/1535089614 wait complete. 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 -- start start 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.182+0000 7f11f6bdf740 1 Processor -- start 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a25f0 0x556aa89b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f6bdf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a7760 0x556aa89b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1aaf50 con 0x556aa89a25f0 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1ac3c0 con 0x556aa89a1c00 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f6bdf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1ab620 con 0x556aa89a7760 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f6bdf740 10 client.0 _ll_register_callbacks cb 0x7f11e00f3350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47942/0 (socket says 172.21.15.26:47942) 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1400640 1 -- 172.21.15.26:0/108747200 learned_addr learned my addr 172.21.15.26:0/108747200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1e00640 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a7760 0x556aa89b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f0a00640 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a25f0 0x556aa89b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1400640 1 -- 172.21.15.26:0/108747200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a7760 msgr2=0x556aa89b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1400640 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a7760 0x556aa89b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1400640 1 -- 172.21.15.26:0/108747200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a25f0 msgr2=0x556aa89b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1400640 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a25f0 0x556aa89b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.183+0000 7f11f1400640 1 -- 172.21.15.26:0/108747200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa89b2730 con 0x556aa89a1c00 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.184+0000 7f11f1400640 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89b0bf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f11e00096c0 tx=0x7f11e00bc570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a46db493ab310e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.184+0000 7f11d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.184+0000 7f11d2a00640 1 -- 172.21.15.26:0/108747200 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0049050 con 0x556aa89a1c00 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.184+0000 7f11d2a00640 1 -- 172.21.15.26:0/108747200 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f11e00c4810 con 0x556aa89a1c00 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.184+0000 7f11f6bdf740 1 -- 172.21.15.26:0/108747200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e00bc0e0 con 0x556aa89a1c00 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.184+0000 7f11d2a00640 1 -- 172.21.15.26:0/108747200 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f11e00a2050 con 0x556aa89a1c00 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.184+0000 7f11f6bdf740 10 client.25985 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.184+0000 7f11f6bdf740 1 -- 172.21.15.26:0/108747200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556aa8a22da0 con 0x556aa89a1c00 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11d2a00640 1 -- 172.21.15.26:0/108747200 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11e00bc0e0 con 0x556aa89a1c00 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11d2a00640 1 -- 172.21.15.26:0/108747200 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f11e00bb530 con 0x556aa89a1c00 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11f6bdf740 10 client.25985 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11f6bdf740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11f6bdf740 20 client.25985 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556aa8a23560 0x556aa8a25980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11f6bdf740 4 client.25985 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11f6bdf740 1 -- 172.21.15.26:0/108747200 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556aa890cea0 con 0x556aa8a23560 2024-06-13T07:07:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.185+0000 7f11f0a00640 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556aa8a23560 0x556aa8a25980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.186+0000 7f11f0a00640 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556aa8a23560 0x556aa8a25980 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eeb8f4e5a6ef656 server_cookie=ea534d8ddcc7d78b in_seq=0 out_seq=0 2024-06-13T07:07:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.186+0000 7f11d2a00640 10 client.25985 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.264+0000 7f11d2a00640 1 -- 172.21.15.26:0/108747200 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556aa890cea0 con 0x556aa8a23560 2024-06-13T07:07:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.264+0000 7f11d2a00640 10 client.25985 handle_command_reply: tid=0 2024-06-13T07:07:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.264+0000 7f11f6bdf740 1 client.25985 shutdown 2024-06-13T07:07:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.264+0000 7f11f6bdf740 1 -- 172.21.15.26:0/108747200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 msgr2=0x556aa89b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.264+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89b0bf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f11e00096c0 tx=0x7f11e00bc570 comp rx=0 tx=0).stop 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.264+0000 7f11f6bdf740 1 -- 172.21.15.26:0/108747200 shutdown_connections 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556aa8a23560 0x556aa8a25980 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556aa89a7760 0x556aa89b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556aa89a25f0 0x556aa89b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11d2a00640 0 client.25985 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11f6bdf740 1 --2- 172.21.15.26:0/108747200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556aa89a1c00 0x556aa89b0bf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11f6bdf740 1 -- 172.21.15.26:0/108747200 >> 172.21.15.26:0/108747200 conn(0x556aa8969fb0 msgr2=0x556aa89abb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11f6bdf740 1 -- 172.21.15.26:0/108747200 shutdown_connections 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11f6bdf740 1 -- 172.21.15.26:0/108747200 wait complete. 2024-06-13T07:07:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.265+0000 7f11f6bdf740 20 client.25985 trim_cache size 0 max 16384 2024-06-13T07:07:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:07:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:07:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 -- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11d40a2cb0 msgr2=0x7f11d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11d40a2cb0 0x7f11d40a5170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009fd0 tx=0x7f11e4004a40 comp rx=0 tx=0).stop 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 -- 172.21.15.26:0/2912587245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec103c30 msgr2=0x7f11ec199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec103c30 0x7f11ec199e80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11e0031440 tx=0x7f11e0032ce0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 -- 172.21.15.26:0/2912587245 shutdown_connections 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f11d40a2cb0 0x7f11d40a5170 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec1057e0 0x7f11ec1a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec104e20 0x7f11ec19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 --2- 172.21.15.26:0/2912587245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec103c30 0x7f11ec199e80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 -- 172.21.15.26:0/2912587245 >> 172.21.15.26:0/2912587245 conn(0x7f11ec0ff1c0 msgr2=0x7f11ec108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 -- 172.21.15.26:0/2912587245 shutdown_connections 2024-06-13T07:07:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:05.266+0000 7f11f3e00640 1 -- 172.21.15.26:0/2912587245 wait complete. 2024-06-13T07:07:05.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 271780 4.64 MB/sec execute 199 sec latency 46.337 ms 2024-06-13T07:07:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:05 smithi026 ceph-mon[32041]: pgmap v707: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:07:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:05 smithi135 ceph-mon[36276]: pgmap v707: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:07:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:05 smithi136 ceph-mon[37229]: pgmap v707: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:07:06.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 272789 4.66 MB/sec execute 200 sec latency 25.405 ms 2024-06-13T07:07:07.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 273469 4.66 MB/sec execute 201 sec latency 22.602 ms 2024-06-13T07:07:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:07 smithi026 ceph-mon[32041]: pgmap v708: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:07:08.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:07 smithi135 ceph-mon[36276]: pgmap v708: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:07:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:07 smithi136 ceph-mon[37229]: pgmap v708: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:07:08.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 273873 4.64 MB/sec execute 202 sec latency 44.032 ms 2024-06-13T07:07:09.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 274915 4.64 MB/sec execute 203 sec latency 38.223 ms 2024-06-13T07:07:10.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:09 smithi026 ceph-mon[32041]: pgmap v709: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:07:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:09 smithi135 ceph-mon[36276]: pgmap v709: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:07:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:09 smithi136 ceph-mon[37229]: pgmap v709: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:07:10.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 276137 4.66 MB/sec execute 204 sec latency 22.848 ms 2024-06-13T07:07:11.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 276957 4.66 MB/sec execute 205 sec latency 28.507 ms 2024-06-13T07:07:12.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:11 smithi026 ceph-mon[32041]: pgmap v710: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-06-13T07:07:12.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:11 smithi135 ceph-mon[36276]: pgmap v710: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-06-13T07:07:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:11 smithi136 ceph-mon[37229]: pgmap v710: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-06-13T07:07:12.539 INFO:tasks.workunit.client.0.smithi026.stdout: 1 277313 4.64 MB/sec execute 206 sec latency 36.859 ms 2024-06-13T07:07:13.540 INFO:tasks.workunit.client.0.smithi026.stdout: 1 278146 4.64 MB/sec execute 207 sec latency 34.035 ms 2024-06-13T07:07:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:13 smithi026 ceph-mon[32041]: pgmap v711: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-06-13T07:07:14.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:13 smithi135 ceph-mon[36276]: pgmap v711: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-06-13T07:07:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:13 smithi136 ceph-mon[37229]: pgmap v711: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-06-13T07:07:14.540 INFO:tasks.workunit.client.0.smithi026.stdout: 1 279289 4.64 MB/sec execute 208 sec latency 20.271 ms 2024-06-13T07:07:15.540 INFO:tasks.workunit.client.0.smithi026.stdout: 1 280477 4.67 MB/sec execute 209 sec latency 25.767 ms 2024-06-13T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:15 smithi136 ceph-mon[37229]: pgmap v712: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-06-13T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:07:16.226 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:15 smithi135 ceph-mon[36276]: pgmap v712: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-06-13T07:07:16.226 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:07:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:15 smithi026 ceph-mon[32041]: pgmap v712: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-06-13T07:07:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:07:16.540 INFO:tasks.workunit.client.0.smithi026.stdout: 1 280928 4.65 MB/sec execute 210 sec latency 24.892 ms 2024-06-13T07:07:17.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:17 smithi026 ceph-mon[32041]: pgmap v713: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:07:17.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:17 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:17 smithi135 ceph-mon[36276]: pgmap v713: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:07:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:17 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.540 INFO:tasks.workunit.client.0.smithi026.stdout: 1 282446 4.65 MB/sec execute 211 sec latency 24.051 ms 2024-06-13T07:07:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:17 smithi136 ceph-mon[37229]: pgmap v713: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:07:17.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:17.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:17 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 -- 172.21.15.26:0/1974716770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68101230 msgr2=0x7efc681016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 --2- 172.21.15.26:0/1974716770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68101230 0x7efc681016b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efc5000aa10 tx=0x7efc5002b720 comp rx=0 tx=0).stop 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 -- 172.21.15.26:0/1974716770 shutdown_connections 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 --2- 172.21.15.26:0/1974716770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68101230 0x7efc681016b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 --2- 172.21.15.26:0/1974716770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68101bf0 0x7efc68108480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 --2- 172.21.15.26:0/1974716770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc68100390 0x7efc681007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 -- 172.21.15.26:0/1974716770 >> 172.21.15.26:0/1974716770 conn(0x7efc680fb1c0 msgr2=0x7efc680fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 -- 172.21.15.26:0/1974716770 shutdown_connections 2024-06-13T07:07:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.421+0000 7efc6d000640 1 -- 172.21.15.26:0/1974716770 wait complete. 2024-06-13T07:07:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.422+0000 7efc6d000640 1 Processor -- start 2024-06-13T07:07:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc6d000640 1 -- start start 2024-06-13T07:07:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc6d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc68100390 0x7efc68103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc6d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68101230 0x7efc68102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc6d000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68101bf0 0x7efc68102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc6d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810b4f0 con 0x7efc68101230 2024-06-13T07:07:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc6d000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810b370 con 0x7efc68101bf0 2024-06-13T07:07:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc6d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810b670 con 0x7efc68100390 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc65e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc68100390 0x7efc68103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc66800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68101bf0 0x7efc68102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc65e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc68100390 0x7efc68103fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60148/0 (socket says 172.21.15.26:60148) 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.423+0000 7efc65e00640 1 -- 172.21.15.26:0/1516452870 learned_addr learned my addr 172.21.15.26:0/1516452870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.424+0000 7efc5d400640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68101230 0x7efc68102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.424+0000 7efc65e00640 1 -- 172.21.15.26:0/1516452870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68101bf0 msgr2=0x7efc68102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.424+0000 7efc65e00640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68101bf0 0x7efc68102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.424+0000 7efc65e00640 1 -- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68101230 msgr2=0x7efc68102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.424+0000 7efc65e00640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68101230 0x7efc68102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.424+0000 7efc65e00640 1 -- 172.21.15.26:0/1516452870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc681033e0 con 0x7efc68100390 2024-06-13T07:07:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.424+0000 7efc65e00640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc68100390 0x7efc68103fd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efc5800eb70 tx=0x7efc5800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.425+0000 7efc5f400640 1 -- 172.21.15.26:0/1516452870 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58003b70 con 0x7efc68100390 2024-06-13T07:07:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.425+0000 7efc5f400640 1 -- 172.21.15.26:0/1516452870 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5800ee70 con 0x7efc68100390 2024-06-13T07:07:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.425+0000 7efc6d000640 1 -- 172.21.15.26:0/1516452870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc681a1dd0 con 0x7efc68100390 2024-06-13T07:07:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.425+0000 7efc5f400640 1 -- 172.21.15.26:0/1516452870 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58010830 con 0x7efc68100390 2024-06-13T07:07:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.425+0000 7efc6d000640 1 -- 172.21.15.26:0/1516452870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc681a21d0 con 0x7efc68100390 2024-06-13T07:07:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.429+0000 7efc5f400640 1 -- 172.21.15.26:0/1516452870 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc58004020 con 0x7efc68100390 2024-06-13T07:07:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc6fe51740 1 Processor -- start 2024-06-13T07:07:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc6fe51740 1 -- start start 2024-06-13T07:07:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556163526c00 0x556163527020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x55616352c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55616352c760 0x55616352ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc6fe51740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55616348a420 con 0x556163526c00 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc6fe51740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556163431c50 con 0x5561635275f0 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc6fe51740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561634863e0 con 0x55616352c760 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.430+0000 7efc5d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55616352c760 0x55616352ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc65e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x55616352c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc5d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55616352c760 0x55616352ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60162/0 (socket says 172.21.15.26:60162) 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc5d400640 1 -- 172.21.15.26:0/3872063021 learned_addr learned my addr 172.21.15.26:0/3872063021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc66800640 1 --2- 172.21.15.26:0/3872063021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556163526c00 0x556163527020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc65e00640 1 -- 172.21.15.26:0/3872063021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55616352c760 msgr2=0x55616352ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc65e00640 1 --2- 172.21.15.26:0/3872063021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55616352c760 0x55616352ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc65e00640 1 -- 172.21.15.26:0/3872063021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556163526c00 msgr2=0x556163527020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc65e00640 1 --2- 172.21.15.26:0/3872063021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556163526c00 0x556163527020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc65e00640 1 -- 172.21.15.26:0/3872063021 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616319e210 con 0x5561635275f0 2024-06-13T07:07:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.431+0000 7efc65e00640 1 --2- 172.21.15.26:0/3872063021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x55616352c0f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc580cd130 tx=0x7efc580d3ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdab651d884d295d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.432+0000 7efc57400640 1 -- 172.21.15.26:0/3872063021 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580e3070 con 0x5561635275f0 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.432+0000 7efc57400640 1 -- 172.21.15.26:0/3872063021 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc580d8d30 con 0x5561635275f0 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.432+0000 7efc57400640 1 -- 172.21.15.26:0/3872063021 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580dd3c0 con 0x5561635275f0 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.432+0000 7efc5f400640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc4c0a2cb0 0x7efc4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.432+0000 7efc6fe51740 1 -- 172.21.15.26:0/3872063021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 msgr2=0x55616352c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.432+0000 7efc6fe51740 1 --2- 172.21.15.26:0/3872063021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x55616352c0f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc580cd130 tx=0x7efc580d3ab0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.433+0000 7efc5f400640 1 -- 172.21.15.26:0/1516452870 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7efc580cfa50 con 0x7efc68100390 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.433+0000 7efc66800640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc4c0a2cb0 0x7efc4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.433+0000 7efc6fe51740 1 -- 172.21.15.26:0/3872063021 shutdown_connections 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.433+0000 7efc6fe51740 1 --2- 172.21.15.26:0/3872063021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x55616352c0f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.433+0000 7efc6fe51740 1 --2- 172.21.15.26:0/3872063021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55616352c760 0x55616352ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.433+0000 7efc6fe51740 1 --2- 172.21.15.26:0/3872063021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556163526c00 0x556163527020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.433+0000 7efc6fe51740 1 -- 172.21.15.26:0/3872063021 >> 172.21.15.26:0/3872063021 conn(0x5561634eefb0 msgr2=0x556163522680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.433+0000 7efc6fe51740 1 -- 172.21.15.26:0/3872063021 shutdown_connections 2024-06-13T07:07:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.434+0000 7efc6fe51740 1 -- 172.21.15.26:0/3872063021 wait complete. 2024-06-13T07:07:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.434+0000 7efc66800640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc4c0a2cb0 0x7efc4c0a5170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x556163527e50 tx=0x7efc6000b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.434+0000 7efc6fe51740 1 -- start start 2024-06-13T07:07:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.434+0000 7efc6fe51740 1 Processor -- start 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.435+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x556163539770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.435+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x556163539cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.435+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55616352c760 0x556163540cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.435+0000 7efc6fe51740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55616352f710 con 0x55616352c760 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.435+0000 7efc6fe51740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5561634450b0 con 0x5561635275f0 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.435+0000 7efc6fe51740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55616352f890 con 0x556163526c00 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.435+0000 7efc6fe51740 10 client.0 _ll_register_callbacks cb 0x556163430f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc5d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x556163539cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc5d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x556163539cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:60484/0 (socket says 172.21.15.26:60484) 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc5d400640 1 -- 172.21.15.26:0/2069539008 learned_addr learned my addr 172.21.15.26:0/2069539008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc65e00640 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x556163539770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc66800640 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55616352c760 0x556163540cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc65e00640 1 -- 172.21.15.26:0/2069539008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 msgr2=0x556163539cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc65e00640 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x556163539cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc65e00640 1 -- 172.21.15.26:0/2069539008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55616352c760 msgr2=0x556163540cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc65e00640 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55616352c760 0x556163540cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.436+0000 7efc65e00640 1 -- 172.21.15.26:0/2069539008 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616326a620 con 0x556163526c00 2024-06-13T07:07:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.437+0000 7efc65e00640 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x556163539770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc5809bbc0 tx=0x7efc580d3fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd20a4181f6b6be5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.437+0000 7efc4be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.437+0000 7efc4be00640 1 -- 172.21.15.26:0/2069539008 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58098160 con 0x556163526c00 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.437+0000 7efc4be00640 1 -- 172.21.15.26:0/2069539008 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7efc580cd270 con 0x556163526c00 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.437+0000 7efc6fe51740 1 -- 172.21.15.26:0/2069539008 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc580d8d30 con 0x556163526c00 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.437+0000 7efc4be00640 1 -- 172.21.15.26:0/2069539008 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efc5809b820 con 0x556163526c00 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.437+0000 7efc6fe51740 10 client.26038 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.437+0000 7efc6fe51740 1 -- 172.21.15.26:0/2069539008 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55616352f710 con 0x556163526c00 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc4be00640 1 -- 172.21.15.26:0/2069539008 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc580d8d30 con 0x556163526c00 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc4be00640 1 -- 172.21.15.26:0/2069539008 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7efc58004310 con 0x556163526c00 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc6fe51740 10 client.26038 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc6fe51740 10 client.26038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc6fe51740 20 client.26038 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556163534be0 0x556163536ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc6fe51740 4 client.26038 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc6fe51740 1 -- 172.21.15.26:0/2069539008 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556163497b50 con 0x556163534be0 2024-06-13T07:07:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.438+0000 7efc5d400640 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556163534be0 0x556163536ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.439+0000 7efc5d400640 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556163534be0 0x556163536ff0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee7dbd81485b770 server_cookie=ce3b15b6d9344fee in_seq=0 out_seq=0 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.439+0000 7efc4be00640 10 client.26038 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc4be00640 1 -- 172.21.15.26:0/2069539008 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556163497b50 con 0x556163534be0 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc4be00640 10 client.26038 handle_command_reply: tid=0 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc6fe51740 1 client.26038 shutdown 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc6fe51740 1 -- 172.21.15.26:0/2069539008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 msgr2=0x556163539770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x556163539770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc5809bbc0 tx=0x7efc580d3fb0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc6fe51740 1 -- 172.21.15.26:0/2069539008 shutdown_connections 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556163534be0 0x556163536ff0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561635275f0 0x556163539cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55616352c760 0x556163540cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.440+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2069539008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x556163539770 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.441+0000 7efc6fe51740 1 -- 172.21.15.26:0/2069539008 >> 172.21.15.26:0/2069539008 conn(0x5561634eefb0 msgr2=0x556163521840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.441+0000 7efc4be00640 0 client.26038 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.441+0000 7efc6fe51740 1 -- 172.21.15.26:0/2069539008 shutdown_connections 2024-06-13T07:07:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.441+0000 7efc6fe51740 1 -- 172.21.15.26:0/2069539008 wait complete. 2024-06-13T07:07:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.441+0000 7efc6fe51740 20 client.26038 trim_cache size 0 max 16384 2024-06-13T07:07:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.461+0000 7efc6fe51740 1 Processor -- start 2024-06-13T07:07:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.461+0000 7efc6fe51740 1 -- start start 2024-06-13T07:07:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x5561635255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x556163540c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 0x556163525af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc6fe51740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc681aadd0 con 0x5561635275f0 2024-06-13T07:07:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc6fe51740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a86a0 con 0x55616352c760 2024-06-13T07:07:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc6fe51740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a64f0 con 0x556163526c00 2024-06-13T07:07:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc65e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x5561635255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc65e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x5561635255b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60184/0 (socket says 172.21.15.26:60184) 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc65e00640 1 -- 172.21.15.26:0/4173305276 learned_addr learned my addr 172.21.15.26:0/4173305276 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc66800640 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 0x556163525af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc5d400640 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x556163540c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc65e00640 1 -- 172.21.15.26:0/4173305276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 msgr2=0x556163525af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.462+0000 7efc65e00640 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 0x556163525af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.463+0000 7efc65e00640 1 -- 172.21.15.26:0/4173305276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 msgr2=0x556163540c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.463+0000 7efc65e00640 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x556163540c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.463+0000 7efc65e00640 1 -- 172.21.15.26:0/4173305276 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556163522770 con 0x556163526c00 2024-06-13T07:07:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.463+0000 7efc5d400640 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x556163540c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.463+0000 7efc65e00640 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x5561635255b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efc58098320 tx=0x7efc5809a3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99a8faade321e975 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.463+0000 7efc56a00640 1 -- 172.21.15.26:0/4173305276 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5809ba00 con 0x556163526c00 2024-06-13T07:07:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.463+0000 7efc56a00640 1 -- 172.21.15.26:0/4173305276 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5809c5c0 con 0x556163526c00 2024-06-13T07:07:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.463+0000 7efc56a00640 1 -- 172.21.15.26:0/4173305276 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5809bc90 con 0x556163526c00 2024-06-13T07:07:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.464+0000 7efc6fe51740 1 -- 172.21.15.26:0/4173305276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 msgr2=0x5561635255b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.464+0000 7efc6fe51740 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x5561635255b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efc58098320 tx=0x7efc5809a3d0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.464+0000 7efc6fe51740 1 -- 172.21.15.26:0/4173305276 shutdown_connections 2024-06-13T07:07:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.464+0000 7efc6fe51740 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 0x556163525af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.464+0000 7efc6fe51740 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x556163540c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.464+0000 7efc6fe51740 1 --2- 172.21.15.26:0/4173305276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x5561635255b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.464+0000 7efc6fe51740 1 -- 172.21.15.26:0/4173305276 >> 172.21.15.26:0/4173305276 conn(0x5561634eefb0 msgr2=0x556163528970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.465+0000 7efc6fe51740 1 -- 172.21.15.26:0/4173305276 shutdown_connections 2024-06-13T07:07:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.465+0000 7efc6fe51740 1 -- 172.21.15.26:0/4173305276 wait complete. 2024-06-13T07:07:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.465+0000 7efc6fe51740 1 -- start start 2024-06-13T07:07:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.465+0000 7efc6fe51740 1 Processor -- start 2024-06-13T07:07:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x55616353e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x55616353e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc6fe51740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 0x55616353f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc6fe51740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a6b60 con 0x5561635275f0 2024-06-13T07:07:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc6fe51740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a7fd0 con 0x55616352c760 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc6fe51740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a7230 con 0x556163526c00 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc6fe51740 10 client.0 _ll_register_callbacks cb 0x7efc580e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc65e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x55616353e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc65e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x55616353e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60198/0 (socket says 172.21.15.26:60198) 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc65e00640 1 -- 172.21.15.26:0/2912271809 learned_addr learned my addr 172.21.15.26:0/2912271809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc5d400640 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x55616353e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.466+0000 7efc66800640 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 0x55616353f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc65e00640 1 -- 172.21.15.26:0/2912271809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 msgr2=0x55616353f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc65e00640 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 0x55616353f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc65e00640 1 -- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 msgr2=0x55616353e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc65e00640 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x55616353e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc65e00640 1 -- 172.21.15.26:0/2912271809 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616353fed0 con 0x556163526c00 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc5d400640 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x55616353e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc65e00640 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x55616353e390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc580d2040 tx=0x7efc580a48d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4d3311f97358e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc4be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:07:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc4be00640 1 -- 172.21.15.26:0/2912271809 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5809bc90 con 0x556163526c00 2024-06-13T07:07:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.467+0000 7efc4be00640 1 -- 172.21.15.26:0/2912271809 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7efc5809c330 con 0x556163526c00 2024-06-13T07:07:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.468+0000 7efc6fe51740 1 -- 172.21.15.26:0/2912271809 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc5809c5c0 con 0x556163526c00 2024-06-13T07:07:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.470+0000 7efc4be00640 1 -- 172.21.15.26:0/2912271809 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efc5809ba00 con 0x556163526c00 2024-06-13T07:07:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.470+0000 7efc6fe51740 10 client.26044 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.470+0000 7efc6fe51740 1 -- 172.21.15.26:0/2912271809 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55616353fed0 con 0x556163526c00 2024-06-13T07:07:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.470+0000 7efc4be00640 1 -- 172.21.15.26:0/2912271809 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5809c5c0 con 0x556163526c00 2024-06-13T07:07:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.470+0000 7efc4be00640 1 -- 172.21.15.26:0/2912271809 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7efc5809b310 con 0x556163526c00 2024-06-13T07:07:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.471+0000 7efc6fe51740 10 client.26044 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.471+0000 7efc6fe51740 10 client.26044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.471+0000 7efc6fe51740 20 client.26044 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.471+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5561635b86b0 0x5561635baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.471+0000 7efc6fe51740 4 client.26044 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.471+0000 7efc6fe51740 1 -- 172.21.15.26:0/2912271809 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556163491ea0 con 0x5561635b86b0 2024-06-13T07:07:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.472+0000 7efc5d400640 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5561635b86b0 0x5561635baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.473+0000 7efc5d400640 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5561635b86b0 0x5561635baaa0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e5f10a9cd21270 server_cookie=67e7fb9d3ed1c535 in_seq=0 out_seq=0 2024-06-13T07:07:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.473+0000 7efc4be00640 10 client.26044 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:18.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:18 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:18.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:18 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:18.540 INFO:tasks.workunit.client.0.smithi026.stdout: 1 283597 4.67 MB/sec execute 212 sec latency 43.272 ms 2024-06-13T07:07:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.584+0000 7efc4be00640 1 -- 172.21.15.26:0/2912271809 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556163491ea0 con 0x5561635b86b0 2024-06-13T07:07:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.584+0000 7efc4be00640 10 client.26044 handle_command_reply: tid=0 2024-06-13T07:07:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.584+0000 7efc6fe51740 1 client.26044 shutdown 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.584+0000 7efc6fe51740 1 -- 172.21.15.26:0/2912271809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 msgr2=0x55616353e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.584+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x55616353e390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc580d2040 tx=0x7efc580a48d0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc6fe51740 1 -- 172.21.15.26:0/2912271809 shutdown_connections 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5561635b86b0 0x5561635baaa0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55616352c760 0x55616353f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561635275f0 0x55616353e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc6fe51740 1 --2- 172.21.15.26:0/2912271809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556163526c00 0x55616353e390 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc6fe51740 1 -- 172.21.15.26:0/2912271809 >> 172.21.15.26:0/2912271809 conn(0x5561634eefb0 msgr2=0x5561635255b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc4be00640 0 client.26044 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc6fe51740 1 -- 172.21.15.26:0/2912271809 shutdown_connections 2024-06-13T07:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.585+0000 7efc6fe51740 1 -- 172.21.15.26:0/2912271809 wait complete. 2024-06-13T07:07:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.586+0000 7efc6fe51740 20 client.26044 trim_cache size 0 max 16384 2024-06-13T07:07:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:07:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:07:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:07:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 -- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc4c0a2cb0 msgr2=0x7efc4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc4c0a2cb0 0x7efc4c0a5170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x556163527e50 tx=0x7efc6000b2e0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 -- 172.21.15.26:0/1516452870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc68100390 msgr2=0x7efc68103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc68100390 0x7efc68103fd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efc5800eb70 tx=0x7efc5800c6a0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 -- 172.21.15.26:0/1516452870 shutdown_connections 2024-06-13T07:07:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efc4c0a2cb0 0x7efc4c0a5170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68101bf0 0x7efc68102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc68101230 0x7efc68102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 --2- 172.21.15.26:0/1516452870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc68100390 0x7efc68103fd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.587+0000 7efc6d000640 1 -- 172.21.15.26:0/1516452870 >> 172.21.15.26:0/1516452870 conn(0x7efc680fb1c0 msgr2=0x7efc68106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.588+0000 7efc6d000640 1 -- 172.21.15.26:0/1516452870 shutdown_connections 2024-06-13T07:07:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:18.588+0000 7efc6d000640 1 -- 172.21.15.26:0/1516452870 wait complete. 2024-06-13T07:07:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:18.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:18 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:19 smithi026 ceph-mon[32041]: pgmap v714: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:07:19.528 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:19 smithi136 ceph-mon[37229]: pgmap v714: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:07:19.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:19 smithi135 ceph-mon[36276]: pgmap v714: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:07:19.540 INFO:tasks.workunit.client.0.smithi026.stdout: 1 284337 4.67 MB/sec execute 213 sec latency 61.347 ms 2024-06-13T07:07:20.540 INFO:tasks.workunit.client.0.smithi026.stdout: 1 285479 4.67 MB/sec execute 214 sec latency 34.826 ms 2024-06-13T07:07:20.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:20.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:20 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:20 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:20 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:21.541 INFO:tasks.workunit.client.0.smithi026.stdout: 1 286685 4.68 MB/sec execute 215 sec latency 26.413 ms 2024-06-13T07:07:21.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:21 smithi136 ceph-mon[37229]: pgmap v715: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-13T07:07:21.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:07:21.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:07:21.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:21 smithi026 ceph-mon[32041]: pgmap v715: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-13T07:07:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:07:21.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:07:21.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:21.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:21 smithi135 ceph-mon[36276]: pgmap v715: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-13T07:07:21.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:07:21.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:07:21.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:07:22.541 INFO:tasks.workunit.client.0.smithi026.stdout: 1 287578 4.69 MB/sec execute 216 sec latency 25.830 ms 2024-06-13T07:07:23.541 INFO:tasks.workunit.client.0.smithi026.stdout: 1 287925 4.67 MB/sec execute 217 sec latency 34.925 ms 2024-06-13T07:07:24.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:23 smithi026 ceph-mon[32041]: pgmap v716: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-13T07:07:24.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:23 smithi135 ceph-mon[36276]: pgmap v716: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-13T07:07:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:23 smithi136 ceph-mon[37229]: pgmap v716: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-13T07:07:24.541 INFO:tasks.workunit.client.0.smithi026.stdout: 1 288717 4.67 MB/sec execute 218 sec latency 85.617 ms 2024-06-13T07:07:25.541 INFO:tasks.workunit.client.0.smithi026.stdout: 1 290092 4.68 MB/sec execute 219 sec latency 39.864 ms 2024-06-13T07:07:26.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:25 smithi026 ceph-mon[32041]: pgmap v717: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-06-13T07:07:26.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:25 smithi135 ceph-mon[36276]: pgmap v717: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-06-13T07:07:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:25 smithi136 ceph-mon[37229]: pgmap v717: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-06-13T07:07:26.541 INFO:tasks.workunit.client.0.smithi026.stdout: 1 291036 4.69 MB/sec execute 220 sec latency 21.889 ms 2024-06-13T07:07:27.541 INFO:tasks.workunit.client.0.smithi026.stdout: 1 291362 4.67 MB/sec execute 221 sec latency 33.590 ms 2024-06-13T07:07:28.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:27 smithi026 ceph-mon[32041]: pgmap v718: 129 pgs: 129 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:07:28.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:27 smithi135 ceph-mon[36276]: pgmap v718: 129 pgs: 129 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:07:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:27 smithi136 ceph-mon[37229]: pgmap v718: 129 pgs: 129 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:07:28.541 INFO:tasks.workunit.client.0.smithi026.stdout: 1 292093 4.66 MB/sec execute 222 sec latency 38.527 ms 2024-06-13T07:07:29.542 INFO:tasks.workunit.client.0.smithi026.stdout: 1 293292 4.66 MB/sec execute 223 sec latency 41.675 ms 2024-06-13T07:07:30.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:29 smithi026 ceph-mon[32041]: pgmap v719: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-06-13T07:07:30.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:29 smithi135 ceph-mon[36276]: pgmap v719: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-06-13T07:07:30.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:29 smithi136 ceph-mon[37229]: pgmap v719: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-06-13T07:07:30.542 INFO:tasks.workunit.client.0.smithi026.stdout: 1 294498 4.69 MB/sec execute 224 sec latency 48.652 ms 2024-06-13T07:07:31.542 INFO:tasks.workunit.client.0.smithi026.stdout: 1 294926 4.68 MB/sec execute 225 sec latency 23.444 ms 2024-06-13T07:07:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.740+0000 7f1acee00640 1 -- 172.21.15.26:0/360434955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ac81057d0 msgr2=0x7f1ac810c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.740+0000 7f1acee00640 1 --2- 172.21.15.26:0/360434955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ac81057d0 0x7f1ac810c060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009860 tx=0x7f1ac002a340 comp rx=0 tx=0).stop 2024-06-13T07:07:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.740+0000 7f1acee00640 1 -- 172.21.15.26:0/360434955 shutdown_connections 2024-06-13T07:07:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.740+0000 7f1acee00640 1 --2- 172.21.15.26:0/360434955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ac81057d0 0x7f1ac810c060 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.740+0000 7f1acee00640 1 --2- 172.21.15.26:0/360434955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8104e10 0x7f1ac8105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.740+0000 7f1acee00640 1 --2- 172.21.15.26:0/360434955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac8103c20 0x7f1ac8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.740+0000 7f1acee00640 1 -- 172.21.15.26:0/360434955 >> 172.21.15.26:0/360434955 conn(0x7f1ac80ff190 msgr2=0x7f1ac81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.741+0000 7f1acee00640 1 -- 172.21.15.26:0/360434955 shutdown_connections 2024-06-13T07:07:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.741+0000 7f1acee00640 1 -- 172.21.15.26:0/360434955 wait complete. 2024-06-13T07:07:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.742+0000 7f1acee00640 1 Processor -- start 2024-06-13T07:07:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.742+0000 7f1acee00640 1 -- start start 2024-06-13T07:07:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.742+0000 7f1acee00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8103c20 0x7f1ac8199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.742+0000 7f1acee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac8104e10 0x7f1ac819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.742+0000 7f1acee00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ac81057d0 0x7f1ac81a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.742+0000 7f1acee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810f880 con 0x7f1ac8104e10 2024-06-13T07:07:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.742+0000 7f1acee00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810f700 con 0x7f1ac81057d0 2024-06-13T07:07:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.742+0000 7f1acee00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810fa00 con 0x7f1ac8103c20 2024-06-13T07:07:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1abf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac8104e10 0x7f1ac819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8103c20 0x7f1ac8199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1abf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac8104e10 0x7f1ac819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59310/0 (socket says 172.21.15.26:59310) 2024-06-13T07:07:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1abf400640 1 -- 172.21.15.26:0/77978002 learned_addr learned my addr 172.21.15.26:0/77978002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8103c20 0x7f1ac8199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57040/0 (socket says 172.21.15.26:57040) 2024-06-13T07:07:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1acce00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ac81057d0 0x7f1ac81a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1ac7e00640 1 -- 172.21.15.26:0/77978002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ac81057d0 msgr2=0x7f1ac81a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1ac7e00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ac81057d0 0x7f1ac81a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1ac7e00640 1 -- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac8104e10 msgr2=0x7f1ac819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1ac7e00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac8104e10 0x7f1ac819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1ac7e00640 1 -- 172.21.15.26:0/77978002 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac81a1a20 con 0x7f1ac8103c20 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.743+0000 7f1ac7e00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8103c20 0x7f1ac8199e00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005fd0 tx=0x7f1ab800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.744+0000 7f1ac5600640 1 -- 172.21.15.26:0/77978002 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8003970 con 0x7f1ac8103c20 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.744+0000 7f1ac5600640 1 -- 172.21.15.26:0/77978002 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab8004030 con 0x7f1ac8103c20 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.744+0000 7f1acee00640 1 -- 172.21.15.26:0/77978002 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac81a1d10 con 0x7f1ac8103c20 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.744+0000 7f1ac5600640 1 -- 172.21.15.26:0/77978002 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80107f0 con 0x7f1ac8103c20 2024-06-13T07:07:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.744+0000 7f1acee00640 1 -- 172.21.15.26:0/77978002 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a21f0 con 0x7f1ac8103c20 2024-06-13T07:07:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.748+0000 7f1ac5600640 1 -- 172.21.15.26:0/77978002 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ab8003b10 con 0x7f1ac8103c20 2024-06-13T07:07:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.749+0000 7f1ad1c38740 1 Processor -- start 2024-06-13T07:07:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.749+0000 7f1ad1c38740 1 -- start start 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.749+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4ef020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.749+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.749+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 0x55877f4f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.749+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55877f452420 con 0x55877f4eec00 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.749+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55877f3f9c50 con 0x55877f4f4760 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.749+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55877f44e3e0 con 0x55877f4ef5f0 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1acce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1acce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4ef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59320/0 (socket says 172.21.15.26:59320) 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1acce00640 1 -- 172.21.15.26:0/950014691 learned_addr learned my addr 172.21.15.26:0/950014691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1abf400640 1 --2- 172.21.15.26:0/950014691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 0x55877f4f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1acce00640 1 -- 172.21.15.26:0/950014691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 msgr2=0x55877f4f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1acce00640 1 --2- 172.21.15.26:0/950014691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1acce00640 1 -- 172.21.15.26:0/950014691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 msgr2=0x55877f4f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1acce00640 1 --2- 172.21.15.26:0/950014691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 0x55877f4f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.750+0000 7f1acce00640 1 -- 172.21.15.26:0/950014691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877f166210 con 0x55877f4eec00 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.751+0000 7f1ac5600640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1aac0a2cb0 0x7f1aac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.751+0000 7f1acce00640 1 --2- 172.21.15.26:0/950014691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4ef020 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009830 tx=0x7f1ac0005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a6759f52505c74 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.751+0000 7f1ac5600640 1 -- 172.21.15.26:0/77978002 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1ab80cfbf0 con 0x7f1ac8103c20 2024-06-13T07:07:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.751+0000 7f1ac7e00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1aac0a2cb0 0x7f1aac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.751+0000 7f1abd600640 1 -- 172.21.15.26:0/950014691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0041020 con 0x55877f4eec00 2024-06-13T07:07:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.751+0000 7f1abd600640 1 -- 172.21.15.26:0/950014691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac0004040 con 0x55877f4eec00 2024-06-13T07:07:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.751+0000 7f1abd600640 1 -- 172.21.15.26:0/950014691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0037040 con 0x55877f4eec00 2024-06-13T07:07:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.752+0000 7f1ad1c38740 1 -- 172.21.15.26:0/950014691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 msgr2=0x55877f4ef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.752+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/950014691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4ef020 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009830 tx=0x7f1ac0005d60 comp rx=0 tx=0).stop 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.752+0000 7f1ac7e00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1aac0a2cb0 0x7f1aac0a5170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80a4740 tx=0x7f1ab80d3630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 -- 172.21.15.26:0/950014691 shutdown_connections 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/950014691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 0x55877f4f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/950014691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/950014691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4ef020 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 -- 172.21.15.26:0/950014691 >> 172.21.15.26:0/950014691 conn(0x55877f4b6fb0 msgr2=0x55877f4ea680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 -- 172.21.15.26:0/950014691 shutdown_connections 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 -- 172.21.15.26:0/950014691 wait complete. 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 -- start start 2024-06-13T07:07:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.753+0000 7f1ad1c38740 1 Processor -- start 2024-06-13T07:07:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f500480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55877f4f76b0 con 0x55877f4ef5f0 2024-06-13T07:07:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55877f40d0b0 con 0x55877f4eec00 2024-06-13T07:07:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55877f4f7830 con 0x55877f4f4760 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1ad1c38740 10 client.0 _ll_register_callbacks cb 0x55877f3f8f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1abf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1acce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f500480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1abf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59330/0 (socket says 172.21.15.26:59330) 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1abf400640 1 -- 172.21.15.26:0/272766321 learned_addr learned my addr 172.21.15.26:0/272766321 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1acce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f500480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57060/0 (socket says 172.21.15.26:57060) 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.754+0000 7f1acce00640 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1abf400640 1 -- 172.21.15.26:0/272766321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 msgr2=0x55877f500480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1abf400640 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f500480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1abf400640 1 -- 172.21.15.26:0/272766321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4eec00 msgr2=0x55877f4f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1abf400640 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1abf400640 1 -- 172.21.15.26:0/272766321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877f232620 con 0x55877f4ef5f0 2024-06-13T07:07:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1acce00640 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1abf400640 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00127d0 tx=0x7f1ab0012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e33b35fd01982 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1ab6000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1ab6000640 1 -- 172.21.15.26:0/272766321 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000f8c0 con 0x55877f4ef5f0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1ab6000640 1 -- 172.21.15.26:0/272766321 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1ab000fa60 con 0x55877f4ef5f0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.755+0000 7f1ad1c38740 1 -- 172.21.15.26:0/272766321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac0004040 con 0x55877f4ef5f0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ab6000640 1 -- 172.21.15.26:0/272766321 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1ab000fc60 con 0x55877f4ef5f0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ad1c38740 10 client.15915 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ad1c38740 1 -- 172.21.15.26:0/272766321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55877f5038c0 con 0x55877f4ef5f0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ab6000640 1 -- 172.21.15.26:0/272766321 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac0004040 con 0x55877f4ef5f0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ab6000640 1 -- 172.21.15.26:0/272766321 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1ab001c420 con 0x55877f4ef5f0 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ad1c38740 10 client.15915 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ad1c38740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ad1c38740 20 client.15915 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.756+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877f504fb0 0x55877f507430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.757+0000 7f1ad1c38740 4 client.15915 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.757+0000 7f1ad1c38740 1 -- 172.21.15.26:0/272766321 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55877f45fb50 con 0x55877f504fb0 2024-06-13T07:07:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.757+0000 7f1acce00640 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877f504fb0 0x55877f507430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.757+0000 7f1acce00640 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877f504fb0 0x55877f507430 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0a6680ab78c781 server_cookie=978fe4ecf1bfda84 in_seq=0 out_seq=0 2024-06-13T07:07:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.757+0000 7f1ab6000640 10 client.15915 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.758+0000 7f1ab6000640 1 -- 172.21.15.26:0/272766321 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55877f45fb50 con 0x55877f504fb0 2024-06-13T07:07:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.758+0000 7f1ab6000640 10 client.15915 handle_command_reply: tid=0 2024-06-13T07:07:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.758+0000 7f1ad1c38740 1 client.15915 shutdown 2024-06-13T07:07:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.758+0000 7f1ad1c38740 1 -- 172.21.15.26:0/272766321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4ef5f0 msgr2=0x55877f4f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.758+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00127d0 tx=0x7f1ab0012c90 comp rx=0 tx=0).stop 2024-06-13T07:07:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 1 -- 172.21.15.26:0/272766321 shutdown_connections 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877f504fb0 0x55877f507430 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f500480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/272766321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 1 -- 172.21.15.26:0/272766321 >> 172.21.15.26:0/272766321 conn(0x55877f4b6fb0 msgr2=0x55877f4e9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ab6000640 0 client.15915 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 1 -- 172.21.15.26:0/272766321 shutdown_connections 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 1 -- 172.21.15.26:0/272766321 wait complete. 2024-06-13T07:07:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.759+0000 7f1ad1c38740 20 client.15915 trim_cache size 0 max 16384 2024-06-13T07:07:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.779+0000 7f1ad1c38740 1 Processor -- start 2024-06-13T07:07:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.779+0000 7f1ad1c38740 1 -- start start 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 0x55877f500440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81af130 con 0x55877f4eec00 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81aca00 con 0x55877f4f4760 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81aa850 con 0x55877f4ef5f0 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1abf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1abf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57066/0 (socket says 172.21.15.26:57066) 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1abf400640 1 -- 172.21.15.26:0/1184337906 learned_addr learned my addr 172.21.15.26:0/1184337906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1acce00640 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1acce00640 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 0x55877f500440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1abf400640 1 -- 172.21.15.26:0/1184337906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 msgr2=0x55877f500440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1abf400640 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 0x55877f500440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1abf400640 1 -- 172.21.15.26:0/1184337906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 msgr2=0x55877f4f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1abf400640 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.780+0000 7f1abf400640 1 -- 172.21.15.26:0/1184337906 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877f4eb330 con 0x55877f4ef5f0 2024-06-13T07:07:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.781+0000 7f1abf400640 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0000c00 tx=0x7f1ab000ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3743acdb9b0505ef server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.781+0000 7f1abcc00640 1 -- 172.21.15.26:0/1184337906 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab001cbd0 con 0x55877f4ef5f0 2024-06-13T07:07:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.781+0000 7f1abcc00640 1 -- 172.21.15.26:0/1184337906 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab001cd70 con 0x55877f4ef5f0 2024-06-13T07:07:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.781+0000 7f1abcc00640 1 -- 172.21.15.26:0/1184337906 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000faa0 con 0x55877f4ef5f0 2024-06-13T07:07:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 -- 172.21.15.26:0/1184337906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 msgr2=0x55877f4f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0000c00 tx=0x7f1ab000ff90 comp rx=0 tx=0).stop 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 -- 172.21.15.26:0/1184337906 shutdown_connections 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4f4760 0x55877f500440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4ef5f0 0x55877f4f9440 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/1184337906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 -- 172.21.15.26:0/1184337906 >> 172.21.15.26:0/1184337906 conn(0x55877f4b6fb0 msgr2=0x55877f4ea100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 -- 172.21.15.26:0/1184337906 shutdown_connections 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 -- 172.21.15.26:0/1184337906 wait complete. 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.782+0000 7f1ad1c38740 1 -- start start 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1ad1c38740 1 Processor -- start 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4ef5f0 0x55877f4fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1ad1c38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f4fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81aaec0 con 0x55877f4eec00 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81ac330 con 0x55877f4ef5f0 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1ad1c38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81ab590 con 0x55877f4f4760 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1ad1c38740 10 client.0 _ll_register_callbacks cb 0x7f1ab0009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1abf400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4ef5f0 0x55877f4fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1abf400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4ef5f0 0x55877f4fdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39170/0 (socket says 172.21.15.26:39170) 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1abf400640 1 -- 172.21.15.26:0/634185551 learned_addr learned my addr 172.21.15.26:0/634185551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.783+0000 7f1acce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1acce00640 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f4fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1abf400640 1 -- 172.21.15.26:0/634185551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 msgr2=0x55877f4fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1abf400640 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f4fecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1abf400640 1 -- 172.21.15.26:0/634185551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 msgr2=0x55877f4fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1abf400640 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1abf400640 1 -- 172.21.15.26:0/634185551 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877f4ff2b0 con 0x55877f4ef5f0 2024-06-13T07:07:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1acce00640 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1abf400640 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4ef5f0 0x55877f4fdd00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0012cd0 tx=0x7f1ab0009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de782b91cd0bc2e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1ab6000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:07:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.784+0000 7f1ab6000640 1 -- 172.21.15.26:0/634185551 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0038060 con 0x55877f4ef5f0 2024-06-13T07:07:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 1 -- 172.21.15.26:0/634185551 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab001cd70 con 0x55877f4ef5f0 2024-06-13T07:07:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ab6000640 1 -- 172.21.15.26:0/634185551 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1ab0005240 con 0x55877f4ef5f0 2024-06-13T07:07:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ab6000640 1 -- 172.21.15.26:0/634185551 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1ab001cbd0 con 0x55877f4ef5f0 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 10 client.26021 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 1 -- 172.21.15.26:0/634185551 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55877f5005c0 con 0x55877f4ef5f0 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ab6000640 1 -- 172.21.15.26:0/634185551 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab001cd70 con 0x55877f4ef5f0 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ab6000640 1 -- 172.21.15.26:0/634185551 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1ab002a430 con 0x55877f4ef5f0 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 10 client.26021 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 20 client.26021 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877f5702c0 0x55877f5726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 4 client.26021 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.785+0000 7f1ad1c38740 1 -- 172.21.15.26:0/634185551 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55877f459ea0 con 0x55877f5702c0 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.786+0000 7f1acce00640 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877f5702c0 0x55877f5726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.786+0000 7f1acce00640 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877f5702c0 0x55877f5726e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee82e7f9eeb0d7c server_cookie=313b8df434db9a54 in_seq=0 out_seq=0 2024-06-13T07:07:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.786+0000 7f1ab6000640 10 client.26021 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.883+0000 7f1ab6000640 1 -- 172.21.15.26:0/634185551 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55877f459ea0 con 0x55877f5702c0 2024-06-13T07:07:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.883+0000 7f1ab6000640 10 client.26021 handle_command_reply: tid=0 2024-06-13T07:07:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.883+0000 7f1ad1c38740 1 client.26021 shutdown 2024-06-13T07:07:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 -- 172.21.15.26:0/634185551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4ef5f0 msgr2=0x55877f4fdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4ef5f0 0x55877f4fdd00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0012cd0 tx=0x7f1ab0009f70 comp rx=0 tx=0).stop 2024-06-13T07:07:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 -- 172.21.15.26:0/634185551 shutdown_connections 2024-06-13T07:07:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877f5702c0 0x55877f5726e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877f4ef5f0 0x55877f4fdd00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877f4f4760 0x55877f4fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 --2- 172.21.15.26:0/634185551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877f4eec00 0x55877f4fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 -- 172.21.15.26:0/634185551 >> 172.21.15.26:0/634185551 conn(0x55877f4b6fb0 msgr2=0x55877f4f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ab6000640 0 client.26021 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.884+0000 7f1ad1c38740 1 -- 172.21.15.26:0/634185551 shutdown_connections 2024-06-13T07:07:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.885+0000 7f1ad1c38740 1 -- 172.21.15.26:0/634185551 wait complete. 2024-06-13T07:07:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.885+0000 7f1ad1c38740 20 client.26021 trim_cache size 0 max 16384 2024-06-13T07:07:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:07:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:07:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:07:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.886+0000 7f1acee00640 1 -- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1aac0a2cb0 msgr2=0x7f1aac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.886+0000 7f1acee00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1aac0a2cb0 0x7f1aac0a5170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80a4740 tx=0x7f1ab80d3630 comp rx=0 tx=0).stop 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.886+0000 7f1acee00640 1 -- 172.21.15.26:0/77978002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8103c20 msgr2=0x7f1ac8199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.886+0000 7f1acee00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8103c20 0x7f1ac8199e00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005fd0 tx=0x7f1ab800ef90 comp rx=0 tx=0).stop 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.887+0000 7f1acee00640 1 -- 172.21.15.26:0/77978002 shutdown_connections 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.887+0000 7f1acee00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1aac0a2cb0 0x7f1aac0a5170 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.887+0000 7f1acee00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ac81057d0 0x7f1ac81a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.887+0000 7f1acee00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ac8104e10 0x7f1ac819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.887+0000 7f1acee00640 1 --2- 172.21.15.26:0/77978002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8103c20 0x7f1ac8199e00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.887+0000 7f1acee00640 1 -- 172.21.15.26:0/77978002 >> 172.21.15.26:0/77978002 conn(0x7f1ac80ff190 msgr2=0x7f1ac8108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.887+0000 7f1acee00640 1 -- 172.21.15.26:0/77978002 shutdown_connections 2024-06-13T07:07:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:31.887+0000 7f1acee00640 1 -- 172.21.15.26:0/77978002 wait complete. 2024-06-13T07:07:32.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:31 smithi026 ceph-mon[32041]: pgmap v720: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:07:32.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:31 smithi135 ceph-mon[36276]: pgmap v720: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:07:32.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:31 smithi136 ceph-mon[37229]: pgmap v720: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:07:32.542 INFO:tasks.workunit.client.0.smithi026.stdout: 1 295850 4.68 MB/sec execute 226 sec latency 34.693 ms 2024-06-13T07:07:33.542 INFO:tasks.workunit.client.0.smithi026.stdout: 1 297028 4.67 MB/sec execute 227 sec latency 19.123 ms 2024-06-13T07:07:34.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:33 smithi026 ceph-mon[32041]: pgmap v721: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-13T07:07:34.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:33 smithi135 ceph-mon[36276]: pgmap v721: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-13T07:07:34.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:33 smithi136 ceph-mon[37229]: pgmap v721: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-13T07:07:34.542 INFO:tasks.workunit.client.0.smithi026.stdout: 1 298177 4.70 MB/sec execute 228 sec latency 21.675 ms 2024-06-13T07:07:35.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 298578 4.68 MB/sec execute 229 sec latency 46.207 ms 2024-06-13T07:07:36.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:35 smithi026 ceph-mon[32041]: pgmap v722: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:07:36.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:35 smithi135 ceph-mon[36276]: pgmap v722: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:07:36.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:35 smithi136 ceph-mon[37229]: pgmap v722: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:07:36.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 299768 4.68 MB/sec execute 230 sec latency 19.840 ms 2024-06-13T07:07:37.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 300956 4.69 MB/sec execute 231 sec latency 23.033 ms 2024-06-13T07:07:38.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:37 smithi026 ceph-mon[32041]: pgmap v723: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:07:38.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:37 smithi135 ceph-mon[36276]: pgmap v723: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:07:38.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:37 smithi136 ceph-mon[37229]: pgmap v723: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:07:38.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 301847 4.70 MB/sec execute 232 sec latency 23.400 ms 2024-06-13T07:07:39.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 302303 4.68 MB/sec execute 233 sec latency 43.417 ms 2024-06-13T07:07:40.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:39 smithi026 ceph-mon[32041]: pgmap v724: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-06-13T07:07:40.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:39 smithi135 ceph-mon[36276]: pgmap v724: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-06-13T07:07:40.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:39 smithi136 ceph-mon[37229]: pgmap v724: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-06-13T07:07:40.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 303501 4.68 MB/sec execute 234 sec latency 31.549 ms 2024-06-13T07:07:41.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 304729 4.70 MB/sec execute 235 sec latency 23.271 ms 2024-06-13T07:07:42.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:41 smithi135 ceph-mon[36276]: pgmap v725: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-06-13T07:07:42.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:41 smithi136 ceph-mon[37229]: pgmap v725: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-06-13T07:07:42.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:41 smithi026 ceph-mon[32041]: pgmap v725: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-06-13T07:07:42.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 305440 4.70 MB/sec execute 236 sec latency 19.870 ms 2024-06-13T07:07:43.543 INFO:tasks.workunit.client.0.smithi026.stdout: 1 305939 4.68 MB/sec execute 237 sec latency 31.637 ms 2024-06-13T07:07:44.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:43 smithi135 ceph-mon[36276]: pgmap v726: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:07:44.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:43 smithi136 ceph-mon[37229]: pgmap v726: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:07:44.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:43 smithi026 ceph-mon[32041]: pgmap v726: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:07:44.544 INFO:tasks.workunit.client.0.smithi026.stdout: 1 307266 4.69 MB/sec execute 238 sec latency 23.457 ms 2024-06-13T07:07:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.090+0000 7f9493200640 1 -- 172.21.15.26:0/1216006227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948c103c30 msgr2=0x7f948c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.090+0000 7f9493200640 1 --2- 172.21.15.26:0/1216006227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948c103c30 0x7f948c104050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f947c009900 tx=0x7f947c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.090+0000 7f9493200640 1 -- 172.21.15.26:0/1216006227 shutdown_connections 2024-06-13T07:07:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.090+0000 7f9493200640 1 --2- 172.21.15.26:0/1216006227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c104e20 0x7f948c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.090+0000 7f9493200640 1 --2- 172.21.15.26:0/1216006227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c1057e0 0x7f948c10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.090+0000 7f9493200640 1 --2- 172.21.15.26:0/1216006227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948c103c30 0x7f948c104050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.090+0000 7f9493200640 1 -- 172.21.15.26:0/1216006227 >> 172.21.15.26:0/1216006227 conn(0x7f948c0ff1c0 msgr2=0x7f948c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.091+0000 7f9493200640 1 -- 172.21.15.26:0/1216006227 shutdown_connections 2024-06-13T07:07:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.091+0000 7f9493200640 1 -- 172.21.15.26:0/1216006227 wait complete. 2024-06-13T07:07:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.092+0000 7f9493200640 1 Processor -- start 2024-06-13T07:07:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.092+0000 7f9493200640 1 -- start start 2024-06-13T07:07:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.092+0000 7f9493200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948c103c30 0x7f948c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.092+0000 7f9493200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c104e20 0x7f948c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.092+0000 7f9493200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c1057e0 0x7f948c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.092+0000 7f9493200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f948c10f8a0 con 0x7f948c104e20 2024-06-13T07:07:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.092+0000 7f9493200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f948c10f720 con 0x7f948c1057e0 2024-06-13T07:07:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.092+0000 7f9493200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f948c10fa20 con 0x7f948c103c30 2024-06-13T07:07:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f948be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948c103c30 0x7f948c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9491200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c1057e0 0x7f948c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9483400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c104e20 0x7f948c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9491200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c1057e0 0x7f948c1a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38168/0 (socket says 172.21.15.26:38168) 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9491200640 1 -- 172.21.15.26:0/1453171575 learned_addr learned my addr 172.21.15.26:0/1453171575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9491200640 1 -- 172.21.15.26:0/1453171575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948c103c30 msgr2=0x7f948c199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9491200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948c103c30 0x7f948c199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9491200640 1 -- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c104e20 msgr2=0x7f948c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9491200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c104e20 0x7f948c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.093+0000 7f9491200640 1 -- 172.21.15.26:0/1453171575 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c1a1ad0 con 0x7f948c1057e0 2024-06-13T07:07:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.094+0000 7f9491200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c1057e0 0x7f948c1a1460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f948400eb10 tx=0x7f948400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.094+0000 7f9489600640 1 -- 172.21.15.26:0/1453171575 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484004b20 con 0x7f948c1057e0 2024-06-13T07:07:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.094+0000 7f9493200640 1 -- 172.21.15.26:0/1453171575 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c1a1dc0 con 0x7f948c1057e0 2024-06-13T07:07:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.094+0000 7f9489600640 1 -- 172.21.15.26:0/1453171575 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9484004cc0 con 0x7f948c1057e0 2024-06-13T07:07:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.094+0000 7f9489600640 1 -- 172.21.15.26:0/1453171575 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484010760 con 0x7f948c1057e0 2024-06-13T07:07:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.094+0000 7f9493200640 1 -- 172.21.15.26:0/1453171575 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c1a2230 con 0x7f948c1057e0 2024-06-13T07:07:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9489600640 1 -- 172.21.15.26:0/1453171575 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9484005020 con 0x7f948c1057e0 2024-06-13T07:07:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9496015740 1 Processor -- start 2024-06-13T07:07:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9496015740 1 -- start start 2024-06-13T07:07:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9496015740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6560020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9496015740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae65650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9496015740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae6567bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9496015740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563ae64c3420 con 0x563ae6565760 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9496015740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563ae646ac50 con 0x563ae655fc00 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.098+0000 7f9496015740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ae64bf3e0 con 0x563ae65605f0 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f948be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae65650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f9483400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6560020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f948be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae65650f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44650/0 (socket says 172.21.15.26:44650) 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f948be00640 1 -- 172.21.15.26:0/1504652943 learned_addr learned my addr 172.21.15.26:0/1504652943 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f9483400640 1 --2- 172.21.15.26:0/1504652943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae6567bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f948be00640 1 -- 172.21.15.26:0/1504652943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 msgr2=0x563ae6560020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f948be00640 1 --2- 172.21.15.26:0/1504652943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6560020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f948be00640 1 -- 172.21.15.26:0/1504652943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 msgr2=0x563ae6567bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f948be00640 1 --2- 172.21.15.26:0/1504652943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae6567bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.099+0000 7f948be00640 1 -- 172.21.15.26:0/1504652943 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae61d7210 con 0x563ae65605f0 2024-06-13T07:07:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.100+0000 7f9489600640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94700a2c30 0x7f94700a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.100+0000 7f948be00640 1 --2- 172.21.15.26:0/1504652943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae65650f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f947c031610 tx=0x7f947c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de19e91175c4c7de server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.100+0000 7f9489600640 1 -- 172.21.15.26:0/1453171575 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f94840cf9e0 con 0x7f948c1057e0 2024-06-13T07:07:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.100+0000 7f9481600640 1 -- 172.21.15.26:0/1504652943 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c040070 con 0x563ae65605f0 2024-06-13T07:07:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.100+0000 7f9483400640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94700a2c30 0x7f94700a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.100+0000 7f9481600640 1 -- 172.21.15.26:0/1504652943 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f947c004030 con 0x563ae65605f0 2024-06-13T07:07:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.100+0000 7f9481600640 1 -- 172.21.15.26:0/1504652943 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c035040 con 0x563ae65605f0 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 -- 172.21.15.26:0/1504652943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 msgr2=0x563ae65650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9483400640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94700a2c30 0x7f94700a50f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x563ae6560e50 tx=0x7f9474013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 --2- 172.21.15.26:0/1504652943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae65650f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f947c031610 tx=0x7f947c005de0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 -- 172.21.15.26:0/1504652943 shutdown_connections 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 --2- 172.21.15.26:0/1504652943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae6567bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 --2- 172.21.15.26:0/1504652943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae65650f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 --2- 172.21.15.26:0/1504652943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6560020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 -- 172.21.15.26:0/1504652943 >> 172.21.15.26:0/1504652943 conn(0x563ae6527fb0 msgr2=0x563ae655b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 -- 172.21.15.26:0/1504652943 shutdown_connections 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.101+0000 7f9496015740 1 -- 172.21.15.26:0/1504652943 wait complete. 2024-06-13T07:07:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.102+0000 7f9496015740 1 -- start start 2024-06-13T07:07:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.102+0000 7f9496015740 1 Processor -- start 2024-06-13T07:07:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9496015740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9496015740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae656a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9496015740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae65714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9496015740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563ae65686b0 con 0x563ae6565760 2024-06-13T07:07:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9496015740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563ae647e0b0 con 0x563ae655fc00 2024-06-13T07:07:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9496015740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ae6568830 con 0x563ae65605f0 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9496015740 10 client.0 _ll_register_callbacks cb 0x563ae6469f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9483400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae656a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f948be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f948be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38192/0 (socket says 172.21.15.26:38192) 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f948be00640 1 -- 172.21.15.26:0/3241834223 learned_addr learned my addr 172.21.15.26:0/3241834223 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9483400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae656a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44658/0 (socket says 172.21.15.26:44658) 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.103+0000 7f9491200640 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae65714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f948be00640 1 -- 172.21.15.26:0/3241834223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 msgr2=0x563ae656a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f948be00640 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae656a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f948be00640 1 -- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 msgr2=0x563ae65714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f948be00640 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae65714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f948be00640 1 -- 172.21.15.26:0/3241834223 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae62a3620 con 0x563ae655fc00 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f9491200640 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae65714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f948be00640 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f947c02a910 tx=0x7f947c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e3d2620085b9fd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f947a000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f947a000640 1 -- 172.21.15.26:0/3241834223 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c035070 con 0x563ae655fc00 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f947a000640 1 -- 172.21.15.26:0/3241834223 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f947c0043e0 con 0x563ae655fc00 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.104+0000 7f9496015740 1 -- 172.21.15.26:0/3241834223 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f947c004030 con 0x563ae655fc00 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f947a000640 1 -- 172.21.15.26:0/3241834223 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f947c058050 con 0x563ae655fc00 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f9496015740 10 client.26033 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f9496015740 1 -- 172.21.15.26:0/3241834223 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563ae6574820 con 0x563ae655fc00 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f947a000640 1 -- 172.21.15.26:0/3241834223 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f947c004030 con 0x563ae655fc00 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f947a000640 1 -- 172.21.15.26:0/3241834223 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f947c0035f0 con 0x563ae655fc00 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f9496015740 10 client.26033 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f9496015740 10 client.26033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f9496015740 20 client.26033 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f9496015740 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ae6575b70 0x563ae6577f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f9496015740 4 client.26033 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.105+0000 7f9496015740 1 -- 172.21.15.26:0/3241834223 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae64d0b50 con 0x563ae6575b70 2024-06-13T07:07:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.106+0000 7f9483400640 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ae6575b70 0x563ae6577f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.106+0000 7f9483400640 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ae6575b70 0x563ae6577f60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb176d7991cea550 server_cookie=7e3240c1b1abc7c5 in_seq=0 out_seq=0 2024-06-13T07:07:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.106+0000 7f947a000640 10 client.26033 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.107+0000 7f947a000640 1 -- 172.21.15.26:0/3241834223 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ae64d0b50 con 0x563ae6575b70 2024-06-13T07:07:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.107+0000 7f947a000640 10 client.26033 handle_command_reply: tid=0 2024-06-13T07:07:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.107+0000 7f9496015740 1 client.26033 shutdown 2024-06-13T07:07:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 -- 172.21.15.26:0/3241834223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 msgr2=0x563ae6569f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f947c02a910 tx=0x7f947c009680 comp rx=0 tx=0).stop 2024-06-13T07:07:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 -- 172.21.15.26:0/3241834223 shutdown_connections 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ae6575b70 0x563ae6577f60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae6565760 0x563ae65714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae65605f0 0x563ae656a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 --2- 172.21.15.26:0/3241834223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569f30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 -- 172.21.15.26:0/3241834223 >> 172.21.15.26:0/3241834223 conn(0x563ae6527fb0 msgr2=0x563ae655a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f947a000640 0 client.26033 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 -- 172.21.15.26:0/3241834223 shutdown_connections 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 1 -- 172.21.15.26:0/3241834223 wait complete. 2024-06-13T07:07:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.108+0000 7f9496015740 20 client.26033 trim_cache size 0 max 16384 2024-06-13T07:07:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9496015740 1 Processor -- start 2024-06-13T07:07:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9496015740 1 -- start start 2024-06-13T07:07:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9496015740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9496015740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 0x563ae6571470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9496015740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae6565760 0x563ae6569a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9496015740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1af190 con 0x563ae65605f0 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9496015740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1aca60 con 0x563ae655fc00 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9496015740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1aa8b0 con 0x563ae6565760 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f948be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9491200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae6565760 0x563ae6569a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f948be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38194/0 (socket says 172.21.15.26:38194) 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f948be00640 1 -- 172.21.15.26:0/2279746205 learned_addr learned my addr 172.21.15.26:0/2279746205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.130+0000 7f9483400640 1 --2- 172.21.15.26:0/2279746205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 0x563ae6571470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9491200640 1 -- 172.21.15.26:0/2279746205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 msgr2=0x563ae6569550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9491200640 1 --2- 172.21.15.26:0/2279746205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9491200640 1 -- 172.21.15.26:0/2279746205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 msgr2=0x563ae6571470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9491200640 1 --2- 172.21.15.26:0/2279746205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 0x563ae6571470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9491200640 1 -- 172.21.15.26:0/2279746205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae6564c40 con 0x563ae6565760 2024-06-13T07:07:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9491200640 1 --2- 172.21.15.26:0/2279746205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae6565760 0x563ae6569a90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94840cd140 tx=0x7f948409bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fba910ea428fd89 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9480c00640 1 -- 172.21.15.26:0/2279746205 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948409be50 con 0x563ae6565760 2024-06-13T07:07:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9480c00640 1 -- 172.21.15.26:0/2279746205 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948409e030 con 0x563ae6565760 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.131+0000 7f9480c00640 1 -- 172.21.15.26:0/2279746205 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94840a4630 con 0x563ae6565760 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 -- 172.21.15.26:0/2279746205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae6565760 msgr2=0x563ae6569a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 --2- 172.21.15.26:0/2279746205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae6565760 0x563ae6569a90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94840cd140 tx=0x7f948409bce0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 -- 172.21.15.26:0/2279746205 shutdown_connections 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 --2- 172.21.15.26:0/2279746205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae6565760 0x563ae6569a90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 --2- 172.21.15.26:0/2279746205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 0x563ae6571470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 --2- 172.21.15.26:0/2279746205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae655fc00 0x563ae6569550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 -- 172.21.15.26:0/2279746205 >> 172.21.15.26:0/2279746205 conn(0x563ae6527fb0 msgr2=0x563ae655edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 -- 172.21.15.26:0/2279746205 shutdown_connections 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.132+0000 7f9496015740 1 -- 172.21.15.26:0/2279746205 wait complete. 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.133+0000 7f9496015740 1 -- start start 2024-06-13T07:07:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.133+0000 7f9496015740 1 Processor -- start 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.133+0000 7f9496015740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae655fc00 0x563ae656e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.133+0000 7f9496015740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 0x563ae656edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f9496015740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae6565760 0x563ae656fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f9496015740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1aaf20 con 0x563ae65605f0 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f9496015740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1ac390 con 0x563ae6565760 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f9496015740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1ab5f0 con 0x563ae655fc00 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f9496015740 10 client.0 _ll_register_callbacks cb 0x7f94740099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f948be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae655fc00 0x563ae656e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f9483400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 0x563ae656edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f948be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae655fc00 0x563ae656e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44676/0 (socket says 172.21.15.26:44676) 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f9491200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae6565760 0x563ae656fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f948be00640 1 -- 172.21.15.26:0/516939320 learned_addr learned my addr 172.21.15.26:0/516939320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f948be00640 1 -- 172.21.15.26:0/516939320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae6565760 msgr2=0x563ae656fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f948be00640 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae6565760 0x563ae656fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f948be00640 1 -- 172.21.15.26:0/516939320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 msgr2=0x563ae656edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f948be00640 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 0x563ae656edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.134+0000 7f948be00640 1 -- 172.21.15.26:0/516939320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae65703f0 con 0x563ae655fc00 2024-06-13T07:07:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f948be00640 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae655fc00 0x563ae656e8b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f947c02a910 tx=0x7f947c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c32e1a4f02cd6088 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f947a000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:07:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f947a000640 1 -- 172.21.15.26:0/516939320 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c061060 con 0x563ae655fc00 2024-06-13T07:07:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f947a000640 1 -- 172.21.15.26:0/516939320 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f947c031ac0 con 0x563ae655fc00 2024-06-13T07:07:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f9496015740 1 -- 172.21.15.26:0/516939320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f948409e030 con 0x563ae655fc00 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f947a000640 1 -- 172.21.15.26:0/516939320 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f947c035070 con 0x563ae655fc00 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f9496015740 10 client.26077 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f9496015740 1 -- 172.21.15.26:0/516939320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563ae6571700 con 0x563ae655fc00 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.135+0000 7f947a000640 1 -- 172.21.15.26:0/516939320 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948409e030 con 0x563ae655fc00 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.136+0000 7f947a000640 1 -- 172.21.15.26:0/516939320 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f947c03c440 con 0x563ae655fc00 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.136+0000 7f9496015740 10 client.26077 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.136+0000 7f9496015740 10 client.26077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.136+0000 7f9496015740 20 client.26077 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.136+0000 7f9496015740 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ae65e0d10 0x563ae65e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.136+0000 7f9496015740 4 client.26077 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.136+0000 7f9496015740 1 -- 172.21.15.26:0/516939320 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ae64caea0 con 0x563ae65e0d10 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.136+0000 7f9483400640 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ae65e0d10 0x563ae65e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.137+0000 7f9483400640 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ae65e0d10 0x563ae65e3100 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44bc985460a0529 server_cookie=d7ef335fd27267bc in_seq=0 out_seq=0 2024-06-13T07:07:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.137+0000 7f947a000640 10 client.26077 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.218+0000 7f947a000640 1 -- 172.21.15.26:0/516939320 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563ae64caea0 con 0x563ae65e0d10 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.218+0000 7f947a000640 10 client.26077 handle_command_reply: tid=0 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.218+0000 7f9496015740 1 client.26077 shutdown 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 -- 172.21.15.26:0/516939320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae655fc00 msgr2=0x563ae656e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae655fc00 0x563ae656e8b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f947c02a910 tx=0x7f947c0038a0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 -- 172.21.15.26:0/516939320 shutdown_connections 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ae65e0d10 0x563ae65e3100 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ae6565760 0x563ae656fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ae65605f0 0x563ae656edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 --2- 172.21.15.26:0/516939320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ae655fc00 0x563ae656e8b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 -- 172.21.15.26:0/516939320 >> 172.21.15.26:0/516939320 conn(0x563ae6527fb0 msgr2=0x563ae6559fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f947a000640 0 client.26077 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 -- 172.21.15.26:0/516939320 shutdown_connections 2024-06-13T07:07:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 1 -- 172.21.15.26:0/516939320 wait complete. 2024-06-13T07:07:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.219+0000 7f9496015740 20 client.26077 trim_cache size 0 max 16384 2024-06-13T07:07:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:07:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:07:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:07:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 -- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94700a2c30 msgr2=0x7f94700a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94700a2c30 0x7f94700a50f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x563ae6560e50 tx=0x7f9474013040 comp rx=0 tx=0).stop 2024-06-13T07:07:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 -- 172.21.15.26:0/1453171575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c1057e0 msgr2=0x7f948c1a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c1057e0 0x7f948c1a1460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f948400eb10 tx=0x7f948400c2d0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 -- 172.21.15.26:0/1453171575 shutdown_connections 2024-06-13T07:07:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f94700a2c30 0x7f94700a50f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c1057e0 0x7f948c1a1460 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c104e20 0x7f948c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 --2- 172.21.15.26:0/1453171575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948c103c30 0x7f948c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.220+0000 7f9493200640 1 -- 172.21.15.26:0/1453171575 >> 172.21.15.26:0/1453171575 conn(0x7f948c0ff1c0 msgr2=0x7f948c1080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.221+0000 7f9493200640 1 -- 172.21.15.26:0/1453171575 shutdown_connections 2024-06-13T07:07:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:45.221+0000 7f9493200640 1 -- 172.21.15.26:0/1453171575 wait complete. 2024-06-13T07:07:45.544 INFO:tasks.workunit.client.0.smithi026.stdout: 1 308423 4.70 MB/sec execute 239 sec latency 49.038 ms 2024-06-13T07:07:46.024 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:45 smithi026 ceph-mon[32041]: pgmap v727: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:07:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:45 smithi135 ceph-mon[36276]: pgmap v727: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:07:46.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:45 smithi136 ceph-mon[37229]: pgmap v727: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:07:46.544 INFO:tasks.workunit.client.0.smithi026.stdout: 1 309053 4.70 MB/sec execute 240 sec latency 27.663 ms 2024-06-13T07:07:47.544 INFO:tasks.workunit.client.0.smithi026.stdout: 1 309755 4.69 MB/sec execute 241 sec latency 28.722 ms 2024-06-13T07:07:48.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:47 smithi135 ceph-mon[36276]: pgmap v728: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-06-13T07:07:48.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:47 smithi136 ceph-mon[37229]: pgmap v728: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-06-13T07:07:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:47 smithi026 ceph-mon[32041]: pgmap v728: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-06-13T07:07:48.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 311031 4.69 MB/sec execute 242 sec latency 19.830 ms 2024-06-13T07:07:49.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 312121 4.72 MB/sec execute 243 sec latency 24.031 ms 2024-06-13T07:07:50.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:49 smithi135 ceph-mon[36276]: pgmap v729: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:07:50.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:49 smithi136 ceph-mon[37229]: pgmap v729: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:07:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:49 smithi026 ceph-mon[32041]: pgmap v729: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:07:50.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 312582 4.70 MB/sec execute 244 sec latency 44.139 ms 2024-06-13T07:07:51.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 313297 4.69 MB/sec execute 245 sec latency 48.797 ms 2024-06-13T07:07:52.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:51 smithi135 ceph-mon[36276]: pgmap v730: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-13T07:07:52.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:51 smithi136 ceph-mon[37229]: pgmap v730: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-13T07:07:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:51 smithi026 ceph-mon[32041]: pgmap v730: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-13T07:07:52.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 314563 4.69 MB/sec execute 246 sec latency 33.776 ms 2024-06-13T07:07:53.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 315530 4.70 MB/sec execute 247 sec latency 20.460 ms 2024-06-13T07:07:54.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:53 smithi136 ceph-mon[37229]: pgmap v731: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:07:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:53 smithi026 ceph-mon[32041]: pgmap v731: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:07:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:53 smithi135 ceph-mon[36276]: pgmap v731: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:07:54.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 316107 4.70 MB/sec execute 248 sec latency 35.567 ms 2024-06-13T07:07:55.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 316683 4.69 MB/sec execute 249 sec latency 49.670 ms 2024-06-13T07:07:56.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:55 smithi136 ceph-mon[37229]: pgmap v732: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-13T07:07:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:55 smithi026 ceph-mon[32041]: pgmap v732: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-13T07:07:56.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:55 smithi135 ceph-mon[36276]: pgmap v732: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-13T07:07:56.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 318140 4.69 MB/sec execute 250 sec latency 35.232 ms 2024-06-13T07:07:57.546 INFO:tasks.workunit.client.0.smithi026.stdout: 1 319193 4.72 MB/sec execute 251 sec latency 21.850 ms 2024-06-13T07:07:58.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:57 smithi136 ceph-mon[37229]: pgmap v733: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:07:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:57 smithi026 ceph-mon[32041]: pgmap v733: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:07:58.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:57 smithi135 ceph-mon[36276]: pgmap v733: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-13T07:07:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.376+0000 7f9e67800640 1 -- 172.21.15.26:0/4177771034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e601035b0 msgr2=0x7f9e60109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.376+0000 7f9e67800640 1 --2- 172.21.15.26:0/4177771034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e601035b0 0x7f9e60109e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009ca0 tx=0x7f9e5802a8c0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.377+0000 7f9e67800640 1 -- 172.21.15.26:0/4177771034 shutdown_connections 2024-06-13T07:07:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.377+0000 7f9e67800640 1 --2- 172.21.15.26:0/4177771034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e601035b0 0x7f9e60109e40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.377+0000 7f9e67800640 1 --2- 172.21.15.26:0/4177771034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60102bf0 0x7f9e60103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.377+0000 7f9e67800640 1 --2- 172.21.15.26:0/4177771034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e60101a00 0x7f9e60101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.377+0000 7f9e67800640 1 -- 172.21.15.26:0/4177771034 >> 172.21.15.26:0/4177771034 conn(0x7f9e600fcf90 msgr2=0x7f9e600ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.377+0000 7f9e67800640 1 -- 172.21.15.26:0/4177771034 shutdown_connections 2024-06-13T07:07:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.377+0000 7f9e67800640 1 -- 172.21.15.26:0/4177771034 wait complete. 2024-06-13T07:07:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.378+0000 7f9e67800640 1 Processor -- start 2024-06-13T07:07:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.378+0000 7f9e67800640 1 -- start start 2024-06-13T07:07:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.378+0000 7f9e67800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60101a00 0x7f9e60075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.378+0000 7f9e67800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e60102bf0 0x7f9e60073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.378+0000 7f9e67800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e601035b0 0x7f9e600742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.378+0000 7f9e67800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010d6b0 con 0x7f9e60101a00 2024-06-13T07:07:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.378+0000 7f9e67800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010d530 con 0x7f9e60102bf0 2024-06-13T07:07:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.378+0000 7f9e67800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010d830 con 0x7f9e601035b0 2024-06-13T07:07:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e65800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e601035b0 0x7f9e600742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e60102bf0 0x7f9e60073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e64e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60101a00 0x7f9e60075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e65800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e601035b0 0x7f9e600742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40402/0 (socket says 172.21.15.26:40402) 2024-06-13T07:07:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e60102bf0 0x7f9e60073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:32850/0 (socket says 172.21.15.26:32850) 2024-06-13T07:07:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e65800640 1 -- 172.21.15.26:0/1961532744 learned_addr learned my addr 172.21.15.26:0/1961532744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e5fe00640 1 -- 172.21.15.26:0/1961532744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e601035b0 msgr2=0x7f9e600742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e601035b0 0x7f9e600742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e5fe00640 1 -- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60101a00 msgr2=0x7f9e60075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60101a00 0x7f9e60075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.379+0000 7f9e5fe00640 1 -- 172.21.15.26:0/1961532744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e60074b60 con 0x7f9e60102bf0 2024-06-13T07:07:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.380+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e60102bf0 0x7f9e60073d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00e9f0 tx=0x7f9e4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.380+0000 7f9e5d600640 1 -- 172.21.15.26:0/1961532744 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00ccc0 con 0x7f9e60102bf0 2024-06-13T07:07:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.380+0000 7f9e67800640 1 -- 172.21.15.26:0/1961532744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e601a1cf0 con 0x7f9e60102bf0 2024-06-13T07:07:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.380+0000 7f9e67800640 1 -- 172.21.15.26:0/1961532744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e601a21d0 con 0x7f9e60102bf0 2024-06-13T07:07:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.380+0000 7f9e5d600640 1 -- 172.21.15.26:0/1961532744 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c010910 con 0x7f9e60102bf0 2024-06-13T07:07:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.380+0000 7f9e5d600640 1 -- 172.21.15.26:0/1961532744 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c018960 con 0x7f9e60102bf0 2024-06-13T07:07:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e6a7c5740 1 Processor -- start 2024-06-13T07:07:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e6a7c5740 1 -- start start 2024-06-13T07:07:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e5d600640 1 -- 172.21.15.26:0/1961532744 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e4c010430 con 0x7f9e60102bf0 2024-06-13T07:07:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359759020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x56235975e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x562359760bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5623596bc420 con 0x56235975e760 2024-06-13T07:07:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562359663c50 con 0x562359758c00 2024-06-13T07:07:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.384+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5623596b83e0 con 0x5623597595f0 2024-06-13T07:07:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359759020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e64e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x56235975e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359759020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:32852/0 (socket says 172.21.15.26:32852) 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 -- 172.21.15.26:0/2172545664 learned_addr learned my addr 172.21.15.26:0/2172545664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x562359760bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 -- 172.21.15.26:0/2172545664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 msgr2=0x56235975e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x56235975e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 -- 172.21.15.26:0/2172545664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 msgr2=0x562359760bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x562359760bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 -- 172.21.15.26:0/2172545664 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623593d0210 con 0x562359758c00 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e64e00640 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x56235975e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.385+0000 7f9e65800640 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359759020 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009430 tx=0x7f9e58005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3ff0949097cd684 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.386+0000 7f9e52000640 1 -- 172.21.15.26:0/2172545664 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58031650 con 0x562359758c00 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.386+0000 7f9e52000640 1 -- 172.21.15.26:0/2172545664 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e58036030 con 0x562359758c00 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.386+0000 7f9e52000640 1 -- 172.21.15.26:0/2172545664 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004800 con 0x562359758c00 2024-06-13T07:07:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.386+0000 7f9e5d600640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e480a2cb0 0x7f9e480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.386+0000 7f9e5d600640 1 -- 172.21.15.26:0/1961532744 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9e4c0cf770 con 0x7f9e60102bf0 2024-06-13T07:07:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/2172545664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 msgr2=0x562359759020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359759020 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009430 tx=0x7f9e58005ef0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e64e00640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e480a2cb0 0x7f9e480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/2172545664 shutdown_connections 2024-06-13T07:07:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x562359760bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x56235975e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/2172545664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359759020 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/2172545664 >> 172.21.15.26:0/2172545664 conn(0x562359720fb0 msgr2=0x562359754680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/2172545664 shutdown_connections 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.387+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/2172545664 wait complete. 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.388+0000 7f9e6a7c5740 1 -- start start 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.388+0000 7f9e64e00640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e480a2cb0 0x7f9e480a5170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x562359755530 tx=0x7f9e54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.388+0000 7f9e6a7c5740 1 Processor -- start 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.388+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359762d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.388+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x5623597632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.388+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x56235976a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.388+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562359761530 con 0x56235975e760 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5623596770b0 con 0x562359758c00 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5623597616b0 con 0x5623597595f0 2024-06-13T07:07:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e6a7c5740 10 client.0 _ll_register_callbacks cb 0x562359662f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e64e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359762d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x5623597632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e64e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359762d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:32860/0 (socket says 172.21.15.26:32860) 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e64e00640 1 -- 172.21.15.26:0/4120874581 learned_addr learned my addr 172.21.15.26:0/4120874581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e65800640 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x56235976a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x5623597632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40426/0 (socket says 172.21.15.26:40426) 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e64e00640 1 -- 172.21.15.26:0/4120874581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 msgr2=0x5623597632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e64e00640 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x5623597632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e64e00640 1 -- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 msgr2=0x56235976a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e64e00640 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x56235976a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e64e00640 1 -- 172.21.15.26:0/4120874581 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235949c620 con 0x562359758c00 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.389+0000 7f9e65800640 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x56235976a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.390+0000 7f9e64e00640 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359762d90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e5401ea00 tx=0x7f9e54004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3794094ed18eda25 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.390+0000 7f9e46a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.390+0000 7f9e46a00640 1 -- 172.21.15.26:0/4120874581 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54005020 con 0x562359758c00 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.390+0000 7f9e46a00640 1 -- 172.21.15.26:0/4120874581 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9e54018030 con 0x562359758c00 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.390+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/4120874581 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e58036030 con 0x562359758c00 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.390+0000 7f9e46a00640 1 -- 172.21.15.26:0/4120874581 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9e54034050 con 0x562359758c00 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.390+0000 7f9e6a7c5740 10 client.26051 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.390+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/4120874581 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56235976d6f0 con 0x562359758c00 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e46a00640 1 -- 172.21.15.26:0/4120874581 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e58036030 con 0x562359758c00 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e46a00640 1 -- 172.21.15.26:0/4120874581 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9e5400f480 con 0x562359758c00 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e6a7c5740 10 client.26051 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e6a7c5740 10 client.26051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e6a7c5740 20 client.26051 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56235976ee40 0x5623597712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e6a7c5740 4 client.26051 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/4120874581 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623596c9b50 con 0x56235976ee40 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.391+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56235976ee40 0x5623597712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.392+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56235976ee40 0x5623597712e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98515481a60c72 server_cookie=80a6aeb8fb993a56 in_seq=0 out_seq=0 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.392+0000 7f9e46a00640 10 client.26051 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e46a00640 1 -- 172.21.15.26:0/4120874581 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5623596c9b50 con 0x56235976ee40 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e46a00640 10 client.26051 handle_command_reply: tid=0 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 client.26051 shutdown 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/4120874581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 msgr2=0x562359762d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359762d90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e5401ea00 tx=0x7f9e54004730 comp rx=0 tx=0).stop 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/4120874581 shutdown_connections 2024-06-13T07:07:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56235976ee40 0x5623597712e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x56235976a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623597595f0 0x5623597632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e46a00640 0 client.26051 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/4120874581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562359758c00 0x562359762d90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.393+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/4120874581 >> 172.21.15.26:0/4120874581 conn(0x562359720fb0 msgr2=0x562359753840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.394+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/4120874581 shutdown_connections 2024-06-13T07:07:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.394+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/4120874581 wait complete. 2024-06-13T07:07:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.394+0000 7f9e6a7c5740 20 client.26051 trim_cache size 0 max 16384 2024-06-13T07:07:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.412+0000 7f9e6a7c5740 1 Processor -- start 2024-06-13T07:07:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.412+0000 7f9e6a7c5740 1 -- start start 2024-06-13T07:07:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.412+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 0x562359754810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623597595f0 0x56235976a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x5623597629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40012610 con 0x56235975e760 2024-06-13T07:07:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4000fee0 con 0x5623597595f0 2024-06-13T07:07:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4000dd30 con 0x562359758c00 2024-06-13T07:07:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623597595f0 0x56235976a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623597595f0 0x56235976a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:32876/0 (socket says 172.21.15.26:32876) 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e5fe00640 1 -- 172.21.15.26:0/3032552716 learned_addr learned my addr 172.21.15.26:0/3032552716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e65800640 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x5623597629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e64e00640 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 0x562359754810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e5fe00640 1 -- 172.21.15.26:0/3032552716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 msgr2=0x562359754810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 0x562359754810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e5fe00640 1 -- 172.21.15.26:0/3032552716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 msgr2=0x5623597629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x5623597629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e5fe00640 1 -- 172.21.15.26:0/3032552716 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235975db90 con 0x5623597595f0 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.413+0000 7f9e65800640 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x5623597629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.414+0000 7f9e5fe00640 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623597595f0 0x56235976a2a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c09bbc0 tx=0x7f9e4c0d4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f291cf6abb4d80e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.414+0000 7f9e51600640 1 -- 172.21.15.26:0/3032552716 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c010720 con 0x5623597595f0 2024-06-13T07:07:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.414+0000 7f9e51600640 1 -- 172.21.15.26:0/3032552716 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c099f30 con 0x5623597595f0 2024-06-13T07:07:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.414+0000 7f9e51600640 1 -- 172.21.15.26:0/3032552716 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0a4360 con 0x5623597595f0 2024-06-13T07:07:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3032552716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623597595f0 msgr2=0x56235976a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623597595f0 0x56235976a2a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c09bbc0 tx=0x7f9e4c0d4040 comp rx=0 tx=0).stop 2024-06-13T07:07:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3032552716 shutdown_connections 2024-06-13T07:07:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623597595f0 0x56235976a2a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56235975e760 0x5623597629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3032552716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 0x562359754810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3032552716 >> 172.21.15.26:0/3032552716 conn(0x562359720fb0 msgr2=0x562359754100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3032552716 shutdown_connections 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3032552716 wait complete. 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.415+0000 7f9e6a7c5740 1 -- start start 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e6a7c5740 1 Processor -- start 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 0x5623597672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5623597595f0 0x562359767830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e6a7c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56235975e760 0x562359768840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4000e3a0 con 0x5623597595f0 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4000f810 con 0x56235975e760 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e6a7c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4000ea70 con 0x562359758c00 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e6a7c5740 10 client.0 _ll_register_callbacks cb 0x7f9e54009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e65800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56235975e760 0x562359768840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5623597595f0 0x562359767830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e65800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56235975e760 0x562359768840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:32880/0 (socket says 172.21.15.26:32880) 2024-06-13T07:07:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e65800640 1 -- 172.21.15.26:0/3677615715 learned_addr learned my addr 172.21.15.26:0/3677615715 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:07:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.416+0000 7f9e64e00640 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 0x5623597672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.417+0000 7f9e65800640 1 -- 172.21.15.26:0/3677615715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 msgr2=0x5623597672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.417+0000 7f9e65800640 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 0x5623597672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.417+0000 7f9e65800640 1 -- 172.21.15.26:0/3677615715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5623597595f0 msgr2=0x562359767830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.417+0000 7f9e65800640 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5623597595f0 0x562359767830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.417+0000 7f9e65800640 1 -- 172.21.15.26:0/3677615715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562359768e30 con 0x56235975e760 2024-06-13T07:07:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.417+0000 7f9e65800640 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56235975e760 0x562359768840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9e58006010 tx=0x7f9e58003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=906ff9590cf8fe6d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.417+0000 7f9e46a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.418+0000 7f9e46a00640 1 -- 172.21.15.26:0/3677615715 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58058070 con 0x56235975e760 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.418+0000 7f9e46a00640 1 -- 172.21.15.26:0/3677615715 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9e58050ad0 con 0x56235975e760 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.418+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3677615715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4c099f30 con 0x56235975e760 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.418+0000 7f9e46a00640 1 -- 172.21.15.26:0/3677615715 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9e5802af40 con 0x56235975e760 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.418+0000 7f9e6a7c5740 10 client.26057 fetch_fsmap learned FSMap version 35 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.418+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3677615715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56235976a140 con 0x56235975e760 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.418+0000 7f9e46a00640 1 -- 172.21.15.26:0/3677615715 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c099f30 con 0x56235975e760 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.418+0000 7f9e46a00640 1 -- 172.21.15.26:0/3677615715 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9e5802f610 con 0x56235975e760 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.419+0000 7f9e6a7c5740 10 client.26057 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.419+0000 7f9e6a7c5740 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.419+0000 7f9e6a7c5740 20 client.26057 populate_metadata read hostname 'smithi026' 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.419+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5623597d9cd0 0x5623597dc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.419+0000 7f9e6a7c5740 4 client.26057 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.419+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3677615715 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623596c3ea0 con 0x5623597d9cd0 2024-06-13T07:07:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.419+0000 7f9e64e00640 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5623597d9cd0 0x5623597dc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.420+0000 7f9e64e00640 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5623597d9cd0 0x5623597dc0c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979d82bf6d7230eb server_cookie=144a0c70c1c5c221 in_seq=0 out_seq=0 2024-06-13T07:07:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.420+0000 7f9e46a00640 10 client.26057 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e46a00640 1 -- 172.21.15.26:0/3677615715 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5623596c3ea0 con 0x5623597d9cd0 2024-06-13T07:07:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e46a00640 10 client.26057 handle_command_reply: tid=0 2024-06-13T07:07:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 client.26057 shutdown 2024-06-13T07:07:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3677615715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56235975e760 msgr2=0x562359768840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56235975e760 0x562359768840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9e58006010 tx=0x7f9e58003ba0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3677615715 shutdown_connections 2024-06-13T07:07:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5623597d9cd0 0x5623597dc0c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56235975e760 0x562359768840 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5623597595f0 0x562359767830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 --2- 172.21.15.26:0/3677615715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562359758c00 0x5623597672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e46a00640 0 client.26057 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:07:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.510+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3677615715 >> 172.21.15.26:0/3677615715 conn(0x562359720fb0 msgr2=0x562359762930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.511+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3677615715 shutdown_connections 2024-06-13T07:07:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.511+0000 7f9e6a7c5740 1 -- 172.21.15.26:0/3677615715 wait complete. 2024-06-13T07:07:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.511+0000 7f9e6a7c5740 20 client.26057 trim_cache size 0 max 16384 2024-06-13T07:07:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:07:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:07:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:07:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 -- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e480a2cb0 msgr2=0x7f9e480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e480a2cb0 0x7f9e480a5170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x562359755530 tx=0x7f9e54005f90 comp rx=0 tx=0).stop 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 -- 172.21.15.26:0/1961532744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e60102bf0 msgr2=0x7f9e60073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e60102bf0 0x7f9e60073d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00e9f0 tx=0x7f9e4c00eeb0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 -- 172.21.15.26:0/1961532744 shutdown_connections 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f9e480a2cb0 0x7f9e480a5170 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e60102bf0 0x7f9e60073d90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e601035b0 0x7f9e600742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 --2- 172.21.15.26:0/1961532744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60101a00 0x7f9e60075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 -- 172.21.15.26:0/1961532744 >> 172.21.15.26:0/1961532744 conn(0x7f9e600fcf90 msgr2=0x7f9e60105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 -- 172.21.15.26:0/1961532744 shutdown_connections 2024-06-13T07:07:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:07:58.512+0000 7f9e67800640 1 -- 172.21.15.26:0/1961532744 wait complete. 2024-06-13T07:07:58.545 INFO:tasks.workunit.client.0.smithi026.stdout: 1 319656 4.71 MB/sec execute 252 sec latency 83.260 ms 2024-06-13T07:07:59.546 INFO:tasks.workunit.client.0.smithi026.stdout: 1 320225 4.69 MB/sec execute 253 sec latency 35.751 ms 2024-06-13T07:08:00.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:07:59 smithi136 ceph-mon[37229]: pgmap v734: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:08:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:07:59 smithi026 ceph-mon[32041]: pgmap v734: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:08:00.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:07:59 smithi135 ceph-mon[36276]: pgmap v734: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:08:00.546 INFO:tasks.workunit.client.0.smithi026.stdout: 1 321446 4.69 MB/sec execute 254 sec latency 59.069 ms 2024-06-13T07:08:01.546 INFO:tasks.workunit.client.0.smithi026.stdout: 1 322605 4.71 MB/sec execute 255 sec latency 41.944 ms 2024-06-13T07:08:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:01 smithi136 ceph-mon[37229]: pgmap v735: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:08:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:01 smithi026 ceph-mon[32041]: pgmap v735: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:08:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:01 smithi135 ceph-mon[36276]: pgmap v735: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:08:02.546 INFO:tasks.workunit.client.0.smithi026.stdout: 1 323179 4.71 MB/sec execute 256 sec latency 31.559 ms 2024-06-13T07:08:03.546 INFO:tasks.workunit.client.0.smithi026.stdout: 1 323668 4.69 MB/sec execute 257 sec latency 32.990 ms 2024-06-13T07:08:04.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:03 smithi136 ceph-mon[37229]: pgmap v736: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:08:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:03 smithi026 ceph-mon[32041]: pgmap v736: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:08:04.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:03 smithi135 ceph-mon[36276]: pgmap v736: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:08:04.547 INFO:tasks.workunit.client.0.smithi026.stdout: 1 324571 4.69 MB/sec execute 258 sec latency 38.714 ms 2024-06-13T07:08:05.547 INFO:tasks.workunit.client.0.smithi026.stdout: 1 325688 4.70 MB/sec execute 259 sec latency 76.417 ms 2024-06-13T07:08:06.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:05 smithi136 ceph-mon[37229]: pgmap v737: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:08:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:05 smithi026 ceph-mon[32041]: pgmap v737: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:08:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:05 smithi135 ceph-mon[36276]: pgmap v737: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-13T07:08:06.547 INFO:tasks.workunit.client.0.smithi026.stdout: 1 326600 4.71 MB/sec execute 260 sec latency 48.208 ms 2024-06-13T07:08:07.547 INFO:tasks.workunit.client.0.smithi026.stdout: 1 326946 4.69 MB/sec execute 261 sec latency 33.795 ms 2024-06-13T07:08:08.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:07 smithi136 ceph-mon[37229]: pgmap v738: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:08:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:07 smithi026 ceph-mon[32041]: pgmap v738: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:08:08.285 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:07 smithi135 ceph-mon[36276]: pgmap v738: 129 pgs: 129 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:08:08.547 INFO:tasks.workunit.client.0.smithi026.stdout: 1 327856 4.69 MB/sec execute 262 sec latency 30.317 ms 2024-06-13T07:08:09.547 INFO:tasks.workunit.client.0.smithi026.stdout: 1 328942 4.68 MB/sec execute 263 sec latency 23.552 ms 2024-06-13T07:08:10.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:09 smithi136 ceph-mon[37229]: pgmap v739: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:08:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:09 smithi026 ceph-mon[32041]: pgmap v739: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:08:10.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:09 smithi135 ceph-mon[36276]: pgmap v739: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:08:10.548 INFO:tasks.workunit.client.0.smithi026.stdout: 1 329989 4.70 MB/sec execute 264 sec latency 21.736 ms 2024-06-13T07:08:11.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 330422 4.69 MB/sec execute 265 sec latency 38.683 ms 2024-06-13T07:08:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 -- 172.21.15.26:0/2777797137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 msgr2=0x7f6aec104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 --2- 172.21.15.26:0/2777797137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 0x7f6aec104050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009900 tx=0x7f6adc02a410 comp rx=0 tx=0).stop 2024-06-13T07:08:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 -- 172.21.15.26:0/2777797137 shutdown_connections 2024-06-13T07:08:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 --2- 172.21.15.26:0/2777797137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6aec1057e0 0x7f6aec10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 --2- 172.21.15.26:0/2777797137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec104e20 0x7f6aec1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 --2- 172.21.15.26:0/2777797137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 0x7f6aec104050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 -- 172.21.15.26:0/2777797137 >> 172.21.15.26:0/2777797137 conn(0x7f6aec0ff1c0 msgr2=0x7f6aec1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 -- 172.21.15.26:0/2777797137 shutdown_connections 2024-06-13T07:08:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.682+0000 7f6af3200640 1 -- 172.21.15.26:0/2777797137 wait complete. 2024-06-13T07:08:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.683+0000 7f6af3200640 1 Processor -- start 2024-06-13T07:08:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.683+0000 7f6af3200640 1 -- start start 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af3200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 0x7f6aec199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec104e20 0x7f6aec19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af3200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6aec1057e0 0x7f6aec1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10f8b0 con 0x7f6aec104e20 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af3200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10f730 con 0x7f6aec103c30 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af3200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10fa30 con 0x7f6aec1057e0 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec104e20 0x7f6aec19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6aec1057e0 0x7f6aec1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 0x7f6aec199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6aec1057e0 0x7f6aec1a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56784/0 (socket says 172.21.15.26:56784) 2024-06-13T07:08:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.684+0000 7f6af1200640 1 -- 172.21.15.26:0/823919280 learned_addr learned my addr 172.21.15.26:0/823919280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6aebe00640 1 -- 172.21.15.26:0/823919280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6aec1057e0 msgr2=0x7f6aec1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6aebe00640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6aec1057e0 0x7f6aec1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6aebe00640 1 -- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec104e20 msgr2=0x7f6aec19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6aebe00640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec104e20 0x7f6aec19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6aebe00640 1 -- 172.21.15.26:0/823919280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec1a1a60 con 0x7f6aec103c30 2024-06-13T07:08:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec104e20 0x7f6aec19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6aebe00640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 0x7f6aec199e40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009230 tx=0x7f6adc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6ae9600640 1 -- 172.21.15.26:0/823919280 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc03f070 con 0x7f6aec103c30 2024-06-13T07:08:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6af3200640 1 -- 172.21.15.26:0/823919280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aec1a1cf0 con 0x7f6aec103c30 2024-06-13T07:08:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6ae9600640 1 -- 172.21.15.26:0/823919280 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc003cd0 con 0x7f6aec103c30 2024-06-13T07:08:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.685+0000 7f6ae9600640 1 -- 172.21.15.26:0/823919280 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc035070 con 0x7f6aec103c30 2024-06-13T07:08:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.686+0000 7f6af3200640 1 -- 172.21.15.26:0/823919280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aec1a21b0 con 0x7f6aec103c30 2024-06-13T07:08:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.690+0000 7f6af5fef740 1 Processor -- start 2024-06-13T07:08:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.690+0000 7f6ae9600640 1 -- 172.21.15.26:0/823919280 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6adc03b460 con 0x7f6aec103c30 2024-06-13T07:08:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.690+0000 7f6af5fef740 1 -- start start 2024-06-13T07:08:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.690+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b3c00 0x55c8398b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.690+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.690+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af5fef740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c839817420 con 0x55c8398b45f0 2024-06-13T07:08:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af5fef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c8397bec50 con 0x55c8398b9760 2024-06-13T07:08:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af5fef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c8398133e0 con 0x55c8398b3c00 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b3c00 0x55c8398b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b3c00 0x55c8398b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56796/0 (socket says 172.21.15.26:56796) 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af1200640 1 -- 172.21.15.26:0/2756920118 learned_addr learned my addr 172.21.15.26:0/2756920118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6aebe00640 1 --2- 172.21.15.26:0/2756920118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af1200640 1 -- 172.21.15.26:0/2756920118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 msgr2=0x55c8398bbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af1200640 1 --2- 172.21.15.26:0/2756920118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398bbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af1200640 1 -- 172.21.15.26:0/2756920118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 msgr2=0x55c8398b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af1200640 1 --2- 172.21.15.26:0/2756920118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398b90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.691+0000 7f6af1200640 1 -- 172.21.15.26:0/2756920118 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83952b210 con 0x55c8398b3c00 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.692+0000 7f6af1200640 1 --2- 172.21.15.26:0/2756920118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b3c00 0x55c8398b4020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4011570 tx=0x7f6ae4011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37bab85e28526348 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.692+0000 7f6ae1600640 1 -- 172.21.15.26:0/2756920118 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004b20 con 0x55c8398b3c00 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.692+0000 7f6ae9600640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ad00a2cb0 0x7f6ad00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.692+0000 7f6ae1600640 1 -- 172.21.15.26:0/2756920118 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae4004cc0 con 0x55c8398b3c00 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.692+0000 7f6ae1600640 1 -- 172.21.15.26:0/2756920118 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4005500 con 0x55c8398b3c00 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.692+0000 7f6ae9600640 1 -- 172.21.15.26:0/823919280 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6adc0ef7b0 con 0x7f6aec103c30 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.692+0000 7f6aebe00640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ad00a2cb0 0x7f6ad00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 -- 172.21.15.26:0/2756920118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b3c00 msgr2=0x55c8398b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 --2- 172.21.15.26:0/2756920118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b3c00 0x55c8398b4020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4011570 tx=0x7f6ae4011a30 comp rx=0 tx=0).stop 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 -- 172.21.15.26:0/2756920118 shutdown_connections 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 --2- 172.21.15.26:0/2756920118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398bbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 --2- 172.21.15.26:0/2756920118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 --2- 172.21.15.26:0/2756920118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b3c00 0x55c8398b4020 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 -- 172.21.15.26:0/2756920118 >> 172.21.15.26:0/2756920118 conn(0x55c83987bfb0 msgr2=0x55c8398af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6aebe00640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ad00a2cb0 0x7f6ad00a5170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009900 tx=0x7f6adc02a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 -- 172.21.15.26:0/2756920118 shutdown_connections 2024-06-13T07:08:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.693+0000 7f6af5fef740 1 -- 172.21.15.26:0/2756920118 wait complete. 2024-06-13T07:08:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 1 -- start start 2024-06-13T07:08:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 1 Processor -- start 2024-06-13T07:08:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b3c00 0x55c8398bdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398c5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c8398bc710 con 0x55c8398b45f0 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c8397d20b0 con 0x55c8398b3c00 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c8398bc890 con 0x55c8398b9760 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.694+0000 7f6af5fef740 10 client.0 _ll_register_callbacks cb 0x55c8397bdf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6af1200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b3c00 0x55c8398bdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6af1200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b3c00 0x55c8398bdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56412/0 (socket says 172.21.15.26:56412) 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398be3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39268/0 (socket says 172.21.15.26:39268) 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6af1200640 1 -- 172.21.15.26:0/3684438738 learned_addr learned my addr 172.21.15.26:0/3684438738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6af1200640 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6af1200640 1 -- 172.21.15.26:0/3684438738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 msgr2=0x55c8398c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6af1200640 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6af1200640 1 -- 172.21.15.26:0/3684438738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 msgr2=0x55c8398be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.695+0000 7f6af1200640 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.696+0000 7f6af1200640 1 -- 172.21.15.26:0/3684438738 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8395f7620 con 0x55c8398b3c00 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.696+0000 7f6af1200640 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b3c00 0x55c8398bdea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005c40 tx=0x7f6ae4005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b399e6bdab96ccc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.696+0000 7f6ada000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.696+0000 7f6ada000640 1 -- 172.21.15.26:0/3684438738 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae402c020 con 0x55c8398b3c00 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.696+0000 7f6af5fef740 1 -- 172.21.15.26:0/3684438738 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae4004cc0 con 0x55c8398b3c00 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6ada000640 1 -- 172.21.15.26:0/3684438738 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6ae4016040 con 0x55c8398b3c00 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6ada000640 1 -- 172.21.15.26:0/3684438738 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6ae4037050 con 0x55c8398b3c00 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6af5fef740 10 client.26069 fetch_fsmap learned FSMap version 35 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6af5fef740 1 -- 172.21.15.26:0/3684438738 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c8398c87e0 con 0x55c8398b3c00 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6ada000640 1 -- 172.21.15.26:0/3684438738 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae4004cc0 con 0x55c8398b3c00 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6ada000640 1 -- 172.21.15.26:0/3684438738 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6ae4028450 con 0x55c8398b3c00 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6af5fef740 10 client.26069 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:08:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6af5fef740 10 client.26069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:08:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6af5fef740 20 client.26069 populate_metadata read hostname 'smithi026' 2024-06-13T07:08:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c8398c9d00 0x55c8398cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.697+0000 7f6af5fef740 4 client.26069 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.698+0000 7f6af5fef740 1 -- 172.21.15.26:0/3684438738 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c839824b50 con 0x55c8398c9d00 2024-06-13T07:08:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.698+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c8398c9d00 0x55c8398cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.698+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c8398c9d00 0x55c8398cc0f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90aaea5a47566d99 server_cookie=b797f4c8d432669d in_seq=0 out_seq=0 2024-06-13T07:08:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.698+0000 7f6ada000640 10 client.26069 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.699+0000 7f6ada000640 1 -- 172.21.15.26:0/3684438738 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c839824b50 con 0x55c8398c9d00 2024-06-13T07:08:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.699+0000 7f6ada000640 10 client.26069 handle_command_reply: tid=0 2024-06-13T07:08:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.699+0000 7f6af5fef740 1 client.26069 shutdown 2024-06-13T07:08:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6af5fef740 1 -- 172.21.15.26:0/3684438738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b3c00 msgr2=0x55c8398bdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b3c00 0x55c8398bdea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005c40 tx=0x7f6ae4005bd0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6af5fef740 1 -- 172.21.15.26:0/3684438738 shutdown_connections 2024-06-13T07:08:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c8398c9d00 0x55c8398cc0f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b45f0 0x55c8398be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3684438738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b3c00 0x55c8398bdea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6af5fef740 1 -- 172.21.15.26:0/3684438738 >> 172.21.15.26:0/3684438738 conn(0x55c83987bfb0 msgr2=0x55c8398ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.700+0000 7f6ada000640 0 client.26069 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.701+0000 7f6af5fef740 1 -- 172.21.15.26:0/3684438738 shutdown_connections 2024-06-13T07:08:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.701+0000 7f6af5fef740 1 -- 172.21.15.26:0/3684438738 wait complete. 2024-06-13T07:08:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.701+0000 7f6af5fef740 20 client.26069 trim_cache size 0 max 16384 2024-06-13T07:08:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af5fef740 1 Processor -- start 2024-06-13T07:08:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af5fef740 1 -- start start 2024-06-13T07:08:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398af100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b45f0 0x55c8398c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af5fef740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1af0a0 con 0x55c8398b3c00 2024-06-13T07:08:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af5fef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ac970 con 0x55c8398b45f0 2024-06-13T07:08:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af5fef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1aa7c0 con 0x55c8398b9760 2024-06-13T07:08:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6af1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39282/0 (socket says 172.21.15.26:39282) 2024-06-13T07:08:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6af1200640 1 -- 172.21.15.26:0/4009632411 learned_addr learned my addr 172.21.15.26:0/4009632411 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.724+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b45f0 0x55c8398c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6af1200640 1 --2- 172.21.15.26:0/4009632411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398bdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6af1200640 1 -- 172.21.15.26:0/4009632411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b45f0 msgr2=0x55c8398c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6af1200640 1 --2- 172.21.15.26:0/4009632411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b45f0 0x55c8398c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6af1200640 1 -- 172.21.15.26:0/4009632411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 msgr2=0x55c8398af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6af1200640 1 --2- 172.21.15.26:0/4009632411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6af1200640 1 -- 172.21.15.26:0/4009632411 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8398cc750 con 0x55c8398b9760 2024-06-13T07:08:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6af1200640 1 --2- 172.21.15.26:0/4009632411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398bdab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005d70 tx=0x7f6ae400ff10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8a5968ebc0ef0e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.725+0000 7f6ae0c00640 1 -- 172.21.15.26:0/4009632411 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae402c070 con 0x55c8398b9760 2024-06-13T07:08:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.726+0000 7f6ae0c00640 1 -- 172.21.15.26:0/4009632411 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae4028650 con 0x55c8398b9760 2024-06-13T07:08:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.726+0000 7f6ae0c00640 1 -- 172.21.15.26:0/4009632411 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400f820 con 0x55c8398b9760 2024-06-13T07:08:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 -- 172.21.15.26:0/4009632411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 msgr2=0x55c8398bdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 --2- 172.21.15.26:0/4009632411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398bdab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005d70 tx=0x7f6ae400ff10 comp rx=0 tx=0).stop 2024-06-13T07:08:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 -- 172.21.15.26:0/4009632411 shutdown_connections 2024-06-13T07:08:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 --2- 172.21.15.26:0/4009632411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b45f0 0x55c8398c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 --2- 172.21.15.26:0/4009632411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b9760 0x55c8398bdab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 --2- 172.21.15.26:0/4009632411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 -- 172.21.15.26:0/4009632411 >> 172.21.15.26:0/4009632411 conn(0x55c83987bfb0 msgr2=0x55c8398b8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 -- 172.21.15.26:0/4009632411 shutdown_connections 2024-06-13T07:08:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 -- 172.21.15.26:0/4009632411 wait complete. 2024-06-13T07:08:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.727+0000 7f6af5fef740 1 -- start start 2024-06-13T07:08:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6af5fef740 1 Processor -- start 2024-06-13T07:08:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b45f0 0x55c8398c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6af5fef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6af5fef740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1aae30 con 0x55c8398b3c00 2024-06-13T07:08:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6af5fef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ac2a0 con 0x55c8398b9760 2024-06-13T07:08:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6af5fef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ab500 con 0x55c8398b45f0 2024-06-13T07:08:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6af5fef740 10 client.0 _ll_register_callbacks cb 0x7f6ae4005320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b45f0 0x55c8398c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.728+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b45f0 0x55c8398c2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56816/0 (socket says 172.21.15.26:56816) 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.729+0000 7f6ae3e00640 1 -- 172.21.15.26:0/3238546294 learned_addr learned my addr 172.21.15.26:0/3238546294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.729+0000 7f6af1200640 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.729+0000 7f6af1200640 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.729+0000 7f6af1200640 1 -- 172.21.15.26:0/3238546294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b45f0 msgr2=0x55c8398c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.729+0000 7f6af1200640 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b45f0 0x55c8398c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.729+0000 7f6af1200640 1 -- 172.21.15.26:0/3238546294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 msgr2=0x55c8398c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.729+0000 7f6af1200640 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.729+0000 7f6af1200640 1 -- 172.21.15.26:0/3238546294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8398c4080 con 0x55c8398b9760 2024-06-13T07:08:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.730+0000 7f6af1200640 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398c3a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000f80 tx=0x7f6ae40096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f649f44f0c92f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.730+0000 7f6ada000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:08:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.730+0000 7f6ada000640 1 -- 172.21.15.26:0/3238546294 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae402c030 con 0x55c8398b9760 2024-06-13T07:08:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.730+0000 7f6af5fef740 1 -- 172.21.15.26:0/3238546294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae4028650 con 0x55c8398b9760 2024-06-13T07:08:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.730+0000 7f6ada000640 1 -- 172.21.15.26:0/3238546294 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6ae4004ec0 con 0x55c8398b9760 2024-06-13T07:08:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.730+0000 7f6ada000640 1 -- 172.21.15.26:0/3238546294 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6ae40050e0 con 0x55c8398b9760 2024-06-13T07:08:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6af5fef740 10 client.26075 fetch_fsmap learned FSMap version 35 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6af5fef740 1 -- 172.21.15.26:0/3238546294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c8398c5390 con 0x55c8398b9760 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6ada000640 1 -- 172.21.15.26:0/3238546294 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae400faa0 con 0x55c8398b9760 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6ada000640 1 -- 172.21.15.26:0/3238546294 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6ae4052650 con 0x55c8398b9760 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6af5fef740 10 client.26075 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6af5fef740 10 client.26075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6af5fef740 20 client.26075 populate_metadata read hostname 'smithi026' 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c83993d460 0x55c83993f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6af5fef740 4 client.26075 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.731+0000 7f6af5fef740 1 -- 172.21.15.26:0/3238546294 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6ae4028650 con 0x55c83993d460 2024-06-13T07:08:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.732+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c83993d460 0x55c83993f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.732+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c83993d460 0x55c83993f850 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4247edebfb5840 server_cookie=618ca8882ce7fb9e in_seq=0 out_seq=0 2024-06-13T07:08:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.732+0000 7f6ada000640 10 client.26075 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.807+0000 7f6ada000640 1 -- 172.21.15.26:0/3238546294 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f6ae4028650 con 0x55c83993d460 2024-06-13T07:08:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.807+0000 7f6ada000640 10 client.26075 handle_command_reply: tid=0 2024-06-13T07:08:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 client.26075 shutdown 2024-06-13T07:08:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 -- 172.21.15.26:0/3238546294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 msgr2=0x55c8398c3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398c3a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000f80 tx=0x7f6ae40096c0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 -- 172.21.15.26:0/3238546294 shutdown_connections 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c83993d460 0x55c83993f850 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8398b9760 0x55c8398c3a90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c8398b45f0 0x55c8398c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6ada000640 0 client.26075 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 --2- 172.21.15.26:0/3238546294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8398b3c00 0x55c8398c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 -- 172.21.15.26:0/3238546294 >> 172.21.15.26:0/3238546294 conn(0x55c83987bfb0 msgr2=0x55c8398bdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 -- 172.21.15.26:0/3238546294 shutdown_connections 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 1 -- 172.21.15.26:0/3238546294 wait complete. 2024-06-13T07:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.808+0000 7f6af5fef740 20 client.26075 trim_cache size 0 max 16384 2024-06-13T07:08:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:08:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:08:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 -- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ad00a2cb0 msgr2=0x7f6ad00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ad00a2cb0 0x7f6ad00a5170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009900 tx=0x7f6adc02a410 comp rx=0 tx=0).stop 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 -- 172.21.15.26:0/823919280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 msgr2=0x7f6aec199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 0x7f6aec199e40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009230 tx=0x7f6adc004270 comp rx=0 tx=0).stop 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 -- 172.21.15.26:0/823919280 shutdown_connections 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ad00a2cb0 0x7f6ad00a5170 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6aec1057e0 0x7f6aec1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec104e20 0x7f6aec19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 --2- 172.21.15.26:0/823919280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aec103c30 0x7f6aec199e40 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009230 tx=0x7f6adc004270 comp rx=0 tx=0).stop 2024-06-13T07:08:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.809+0000 7f6af3200640 1 -- 172.21.15.26:0/823919280 >> 172.21.15.26:0/823919280 conn(0x7f6aec0ff1c0 msgr2=0x7f6aec108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.810+0000 7f6af3200640 1 -- 172.21.15.26:0/823919280 shutdown_connections 2024-06-13T07:08:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:11.810+0000 7f6af3200640 1 -- 172.21.15.26:0/823919280 wait complete. 2024-06-13T07:08:12.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:11 smithi136 ceph-mon[37229]: pgmap v740: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-13T07:08:12.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:11 smithi026 ceph-mon[32041]: pgmap v740: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-13T07:08:12.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:11 smithi135 ceph-mon[36276]: pgmap v740: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-13T07:08:12.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 331320 4.69 MB/sec execute 266 sec latency 42.542 ms 2024-06-13T07:08:13.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 332444 4.69 MB/sec execute 267 sec latency 20.044 ms 2024-06-13T07:08:14.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:13 smithi136 ceph-mon[37229]: pgmap v741: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-13T07:08:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:13 smithi026 ceph-mon[32041]: pgmap v741: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-13T07:08:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:13 smithi135 ceph-mon[36276]: pgmap v741: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-13T07:08:14.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 333513 4.70 MB/sec execute 268 sec latency 21.618 ms 2024-06-13T07:08:15.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 333878 4.69 MB/sec execute 269 sec latency 29.096 ms 2024-06-13T07:08:16.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:15 smithi136 ceph-mon[37229]: pgmap v742: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:08:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:15 smithi026 ceph-mon[32041]: pgmap v742: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:08:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:15 smithi135 ceph-mon[36276]: pgmap v742: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:08:16.549 INFO:tasks.workunit.client.0.smithi026.stdout: 1 334650 4.68 MB/sec execute 270 sec latency 80.685 ms 2024-06-13T07:08:17.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 335895 4.69 MB/sec execute 271 sec latency 34.005 ms 2024-06-13T07:08:18.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:17 smithi136 ceph-mon[37229]: pgmap v743: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:08:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:17 smithi026 ceph-mon[32041]: pgmap v743: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:08:18.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:17 smithi135 ceph-mon[36276]: pgmap v743: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:08:18.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 336947 4.70 MB/sec execute 272 sec latency 23.930 ms 2024-06-13T07:08:19.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 337376 4.69 MB/sec execute 273 sec latency 24.502 ms 2024-06-13T07:08:20.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:19 smithi136 ceph-mon[37229]: pgmap v744: 129 pgs: 129 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:08:20.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:19 smithi026 ceph-mon[32041]: pgmap v744: 129 pgs: 129 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:08:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:19 smithi135 ceph-mon[36276]: pgmap v744: 129 pgs: 129 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-13T07:08:20.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 337889 4.68 MB/sec execute 274 sec latency 31.371 ms 2024-06-13T07:08:21.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 339395 4.69 MB/sec execute 275 sec latency 28.029 ms 2024-06-13T07:08:22.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:21 smithi136 ceph-mon[37229]: pgmap v745: 129 pgs: 129 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-13T07:08:22.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:08:22.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:21 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:21 smithi026 ceph-mon[32041]: pgmap v745: 129 pgs: 129 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-13T07:08:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:08:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:21 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:21 smithi135 ceph-mon[36276]: pgmap v745: 129 pgs: 129 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-13T07:08:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:08:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:21 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:22.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 340722 4.71 MB/sec execute 276 sec latency 25.782 ms 2024-06-13T07:08:23.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:08:23.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:08:23.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:23.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:08:23.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:08:23.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:23.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:08:23.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:08:23.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:08:23.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 341119 4.69 MB/sec execute 277 sec latency 32.810 ms 2024-06-13T07:08:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:23 smithi136 ceph-mon[37229]: pgmap v746: 129 pgs: 129 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-13T07:08:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:23 smithi026 ceph-mon[32041]: pgmap v746: 129 pgs: 129 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-13T07:08:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:23 smithi135 ceph-mon[36276]: pgmap v746: 129 pgs: 129 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-13T07:08:24.550 INFO:tasks.workunit.client.0.smithi026.stdout: 1 342027 4.70 MB/sec execute 278 sec latency 25.822 ms 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 -- 172.21.15.26:0/3498345754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd01075c0 msgr2=0x7f4fd01099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 --2- 172.21.15.26:0/3498345754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd01075c0 0x7f4fd01099b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009860 tx=0x7f4fc802a340 comp rx=0 tx=0).stop 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 -- 172.21.15.26:0/3498345754 shutdown_connections 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 --2- 172.21.15.26:0/3498345754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fd0104c90 0x7f4fd0107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 --2- 172.21.15.26:0/3498345754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd01075c0 0x7f4fd01099b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 --2- 172.21.15.26:0/3498345754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd0102360 0x7f4fd0104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 -- 172.21.15.26:0/3498345754 >> 172.21.15.26:0/3498345754 conn(0x7f4fd00fbc90 msgr2=0x7f4fd00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 -- 172.21.15.26:0/3498345754 shutdown_connections 2024-06-13T07:08:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.979+0000 7f4fd5800640 1 -- 172.21.15.26:0/3498345754 wait complete. 2024-06-13T07:08:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.980+0000 7f4fd5800640 1 Processor -- start 2024-06-13T07:08:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fd5800640 1 -- start start 2024-06-13T07:08:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fd5800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fd0102360 0x7f4fd0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fd5800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0104c90 0x7f4fd019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fd5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd01075c0 0x7f4fd01a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fd5800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd010d4b0 con 0x7f4fd01075c0 2024-06-13T07:08:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fd5800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd010d330 con 0x7f4fd0102360 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fd5800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd010d630 con 0x7f4fd0104c90 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fd0102360 0x7f4fd0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fd0102360 0x7f4fd0199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:45218/0 (socket says 172.21.15.26:45218) 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fcfe00640 1 -- 172.21.15.26:0/2816551934 learned_addr learned my addr 172.21.15.26:0/2816551934 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fcf400640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0104c90 0x7f4fd019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.981+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd01075c0 0x7f4fd01a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fcfe00640 1 -- 172.21.15.26:0/2816551934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0104c90 msgr2=0x7f4fd019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0104c90 0x7f4fd019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fcfe00640 1 -- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd01075c0 msgr2=0x7f4fd01a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd01075c0 0x7f4fd01a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fcfe00640 1 -- 172.21.15.26:0/2816551934 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd01a1930 con 0x7f4fd0102360 2024-06-13T07:08:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd01075c0 0x7f4fd01a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fd0102360 0x7f4fd0199d20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000c420 tx=0x7f4fc0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fccc00640 1 -- 172.21.15.26:0/2816551934 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0003970 con 0x7f4fd0102360 2024-06-13T07:08:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fccc00640 1 -- 172.21.15.26:0/2816551934 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc0004020 con 0x7f4fd0102360 2024-06-13T07:08:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fd5800640 1 -- 172.21.15.26:0/2816551934 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd01a1c20 con 0x7f4fd0102360 2024-06-13T07:08:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fccc00640 1 -- 172.21.15.26:0/2816551934 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000bad0 con 0x7f4fd0102360 2024-06-13T07:08:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.982+0000 7f4fd5800640 1 -- 172.21.15.26:0/2816551934 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd01a21c0 con 0x7f4fd0102360 2024-06-13T07:08:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.986+0000 7f4fccc00640 1 -- 172.21.15.26:0/2816551934 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4fc001d020 con 0x7f4fd0102360 2024-06-13T07:08:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.987+0000 7f4fd9baa740 1 Processor -- start 2024-06-13T07:08:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.987+0000 7f4fd9baa740 1 -- start start 2024-06-13T07:08:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.987+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7b9ddc0 0x5583a7b9e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.987+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.987+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7ba3920 0x5583a7ba5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.987+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5583a7b01420 con 0x5583a7b9e7b0 2024-06-13T07:08:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.987+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583a7aa8c50 con 0x5583a7b9ddc0 2024-06-13T07:08:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.987+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5583a7afd3e0 con 0x5583a7ba3920 2024-06-13T07:08:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7ba3920 0x5583a7ba5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7ba3920 0x5583a7ba5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33946/0 (socket says 172.21.15.26:33946) 2024-06-13T07:08:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcf400640 1 -- 172.21.15.26:0/551162372 learned_addr learned my addr 172.21.15.26:0/551162372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/551162372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7b9ddc0 0x5583a7b9e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcf400640 1 -- 172.21.15.26:0/551162372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7b9ddc0 msgr2=0x5583a7b9e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcf400640 1 --2- 172.21.15.26:0/551162372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7b9ddc0 0x5583a7b9e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcf400640 1 -- 172.21.15.26:0/551162372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 msgr2=0x5583a7ba32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcf400640 1 --2- 172.21.15.26:0/551162372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fcf400640 1 -- 172.21.15.26:0/551162372 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a7815210 con 0x5583a7ba3920 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.988+0000 7f4fccc00640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4fb40a2c00 0x7f4fb40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.989+0000 7f4fcf400640 1 --2- 172.21.15.26:0/551162372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7ba3920 0x5583a7ba5db0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009c90 tx=0x7f4fb8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671fd9eb1bb9f0f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.989+0000 7f4fccc00640 1 -- 172.21.15.26:0/2816551934 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4fc00cf3d0 con 0x7f4fd0102360 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.989+0000 7f4fc5600640 1 -- 172.21.15.26:0/551162372 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb801e070 con 0x5583a7ba3920 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.989+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4fb40a2c00 0x7f4fb40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.989+0000 7f4fc5600640 1 -- 172.21.15.26:0/551162372 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb8019b10 con 0x5583a7ba3920 2024-06-13T07:08:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.989+0000 7f4fc5600640 1 -- 172.21.15.26:0/551162372 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800eac0 con 0x5583a7ba3920 2024-06-13T07:08:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 -- 172.21.15.26:0/551162372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7ba3920 msgr2=0x5583a7ba5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4fb40a2c00 0x7f4fb40a50c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009830 tx=0x7f4fc8030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/551162372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7ba3920 0x5583a7ba5db0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009c90 tx=0x7f4fb8011810 comp rx=0 tx=0).stop 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 -- 172.21.15.26:0/551162372 shutdown_connections 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/551162372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7ba3920 0x5583a7ba5db0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/551162372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/551162372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7b9ddc0 0x5583a7b9e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 -- 172.21.15.26:0/551162372 >> 172.21.15.26:0/551162372 conn(0x5583a7b65fb0 msgr2=0x5583a7b99840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 -- 172.21.15.26:0/551162372 shutdown_connections 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.990+0000 7f4fd9baa740 1 -- 172.21.15.26:0/551162372 wait complete. 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 1 -- start start 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 1 Processor -- start 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7baf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5583a7ba68d0 con 0x5583a7b9e7b0 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583a7abc0b0 con 0x5583a7ba3920 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5583a7ba6a50 con 0x5583a7b9ddc0 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.991+0000 7f4fd9baa740 10 client.0 _ll_register_callbacks cb 0x5583a7aa7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fd4e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7baf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fd4e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7baf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:45230/0 (socket says 172.21.15.26:45230) 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fd4e00640 1 -- 172.21.15.26:0/1882035055 learned_addr learned my addr 172.21.15.26:0/1882035055 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcf400640 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcfe00640 1 -- 172.21.15.26:0/1882035055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 msgr2=0x5583a7baf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7baf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcfe00640 1 -- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 msgr2=0x5583a7ba86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcfe00640 1 -- 172.21.15.26:0/1882035055 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a78e1620 con 0x5583a7b9ddc0 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcf400640 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7baf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba8160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5583a7b9f220 tx=0x7f4fc0097ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=352fbf5ecebba86 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.992+0000 7f4fbe000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.993+0000 7f4fbe000640 1 -- 172.21.15.26:0/1882035055 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0099920 con 0x5583a7b9ddc0 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.993+0000 7f4fbe000640 1 -- 172.21.15.26:0/1882035055 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4fc00041d0 con 0x5583a7b9ddc0 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.993+0000 7f4fd9baa740 1 -- 172.21.15.26:0/1882035055 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb8019b10 con 0x5583a7b9ddc0 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.993+0000 7f4fbe000640 1 -- 172.21.15.26:0/1882035055 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4fc0004540 con 0x5583a7b9ddc0 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.993+0000 7f4fd9baa740 10 client.26125 fetch_fsmap learned FSMap version 35 2024-06-13T07:08:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.993+0000 7f4fd9baa740 1 -- 172.21.15.26:0/1882035055 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5583a7bb2ab0 con 0x5583a7b9ddc0 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.993+0000 7f4fbe000640 1 -- 172.21.15.26:0/1882035055 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb8019b10 con 0x5583a7b9ddc0 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.994+0000 7f4fbe000640 1 -- 172.21.15.26:0/1882035055 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4fc0084ce0 con 0x5583a7b9ddc0 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.994+0000 7f4fd9baa740 10 client.26125 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.994+0000 7f4fd9baa740 10 client.26125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.994+0000 7f4fd9baa740 20 client.26125 populate_metadata read hostname 'smithi026' 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.994+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5583a7bb41d0 0x5583a7bb6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.994+0000 7f4fd9baa740 4 client.26125 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.994+0000 7f4fd9baa740 1 -- 172.21.15.26:0/1882035055 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583a7b0eb50 con 0x5583a7bb41d0 2024-06-13T07:08:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.994+0000 7f4fcf400640 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5583a7bb41d0 0x5583a7bb6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.995+0000 7f4fcf400640 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5583a7bb41d0 0x5583a7bb6620 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b71daa8a1cdab2e9 server_cookie=262ee4a1088d2401 in_seq=0 out_seq=0 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.995+0000 7f4fbe000640 10 client.26125 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fbe000640 1 -- 172.21.15.26:0/1882035055 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583a7b0eb50 con 0x5583a7bb41d0 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fbe000640 10 client.26125 handle_command_reply: tid=0 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 client.26125 shutdown 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 -- 172.21.15.26:0/1882035055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 msgr2=0x5583a7ba8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba8160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5583a7b9f220 tx=0x7f4fc0097ab0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 -- 172.21.15.26:0/1882035055 shutdown_connections 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5583a7bb41d0 0x5583a7bb6620 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7baf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7ba86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/1882035055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba8160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fbe000640 0 client.26125 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 -- 172.21.15.26:0/1882035055 >> 172.21.15.26:0/1882035055 conn(0x5583a7b65fb0 msgr2=0x5583a7b989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 -- 172.21.15.26:0/1882035055 shutdown_connections 2024-06-13T07:08:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 1 -- 172.21.15.26:0/1882035055 wait complete. 2024-06-13T07:08:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:24.996+0000 7f4fd9baa740 20 client.26125 trim_cache size 0 max 16384 2024-06-13T07:08:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.016+0000 7f4fd9baa740 1 Processor -- start 2024-06-13T07:08:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.016+0000 7f4fd9baa740 1 -- start start 2024-06-13T07:08:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.016+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.016+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7baf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.016+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7b9c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.016+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd01af0d0 con 0x5583a7b9e7b0 2024-06-13T07:08:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.016+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd01ac9a0 con 0x5583a7ba3920 2024-06-13T07:08:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.016+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd01aa7f0 con 0x5583a7b9ddc0 2024-06-13T07:08:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7baf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7baf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54114/0 (socket says 172.21.15.26:54114) 2024-06-13T07:08:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcf400640 1 -- 172.21.15.26:0/3300999275 learned_addr learned my addr 172.21.15.26:0/3300999275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/3300999275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7b9c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcfe00640 1 -- 172.21.15.26:0/3300999275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 msgr2=0x5583a7b9c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/3300999275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7b9c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcfe00640 1 -- 172.21.15.26:0/3300999275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 msgr2=0x5583a7baf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/3300999275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7baf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.017+0000 7f4fcfe00640 1 -- 172.21.15.26:0/3300999275 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a7b9a4f0 con 0x5583a7b9ddc0 2024-06-13T07:08:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.018+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/3300999275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba3160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0098120 tx=0x7f4fc009b200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e2a8dd3bef852c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.018+0000 7f4fc4c00640 1 -- 172.21.15.26:0/3300999275 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc009b320 con 0x5583a7b9ddc0 2024-06-13T07:08:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.018+0000 7f4fc4c00640 1 -- 172.21.15.26:0/3300999275 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc00a4220 con 0x5583a7b9ddc0 2024-06-13T07:08:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.018+0000 7f4fc4c00640 1 -- 172.21.15.26:0/3300999275 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc009b7e0 con 0x5583a7b9ddc0 2024-06-13T07:08:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3300999275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 msgr2=0x5583a7ba3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3300999275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba3160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0098120 tx=0x7f4fc009b200 comp rx=0 tx=0).stop 2024-06-13T07:08:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3300999275 shutdown_connections 2024-06-13T07:08:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3300999275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7b9c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3300999275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9e7b0 0x5583a7baf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3300999275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9ddc0 0x5583a7ba3160 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3300999275 >> 172.21.15.26:0/3300999275 conn(0x5583a7b65fb0 msgr2=0x5583a7bb00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3300999275 shutdown_connections 2024-06-13T07:08:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.019+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3300999275 wait complete. 2024-06-13T07:08:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.020+0000 7f4fd9baa740 1 -- start start 2024-06-13T07:08:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.020+0000 7f4fd9baa740 1 Processor -- start 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9ddc0 0x5583a7bac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9e7b0 0x5583a7bac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd9baa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7c1cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd01aae60 con 0x5583a7b9ddc0 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd01ac2d0 con 0x5583a7ba3920 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd9baa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd01ab530 con 0x5583a7b9e7b0 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd9baa740 10 client.0 _ll_register_callbacks cb 0x7f4fc0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9ddc0 0x5583a7bac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd4e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7c1cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd4e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7c1cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:45240/0 (socket says 172.21.15.26:45240) 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fd4e00640 1 -- 172.21.15.26:0/3724604777 learned_addr learned my addr 172.21.15.26:0/3724604777 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.021+0000 7f4fcf400640 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9e7b0 0x5583a7bac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fd4e00640 1 -- 172.21.15.26:0/3724604777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9e7b0 msgr2=0x5583a7bac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9e7b0 0x5583a7bac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fd4e00640 1 -- 172.21.15.26:0/3724604777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9ddc0 msgr2=0x5583a7bac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9ddc0 0x5583a7bac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fd4e00640 1 -- 172.21.15.26:0/3724604777 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a7baf850 con 0x5583a7ba3920 2024-06-13T07:08:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fd4e00640 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7c1cef0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009860 tx=0x7f4fc802a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4350387f821adf00 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fbe000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fbe000640 1 -- 172.21.15.26:0/3724604777 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc804d070 con 0x5583a7ba3920 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fbe000640 1 -- 172.21.15.26:0/3724604777 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4fc80048c0 con 0x5583a7ba3920 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.022+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3724604777 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc00a4220 con 0x5583a7ba3920 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fbe000640 1 -- 172.21.15.26:0/3724604777 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4fc805b050 con 0x5583a7ba3920 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fd9baa740 10 client.26093 fetch_fsmap learned FSMap version 35 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3724604777 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5583a7c1e400 con 0x5583a7ba3920 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fbe000640 1 -- 172.21.15.26:0/3724604777 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc00a4220 con 0x5583a7ba3920 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fbe000640 1 -- 172.21.15.26:0/3724604777 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4fc8046040 con 0x5583a7ba3920 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fd9baa740 10 client.26093 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fd9baa740 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fd9baa740 20 client.26093 populate_metadata read hostname 'smithi026' 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5583a7c1ee00 0x5583a7c21220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fd9baa740 4 client.26093 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.023+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3724604777 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583a7b08ea0 con 0x5583a7c1ee00 2024-06-13T07:08:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.024+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5583a7c1ee00 0x5583a7c21220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.024+0000 7f4fcfe00640 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5583a7c1ee00 0x5583a7c21220 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b34cc3941d9463 server_cookie=6c049b6314da4aa9 in_seq=0 out_seq=0 2024-06-13T07:08:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.024+0000 7f4fbe000640 10 client.26093 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.124+0000 7f4fbe000640 1 -- 172.21.15.26:0/3724604777 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5583a7b08ea0 con 0x5583a7c1ee00 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.124+0000 7f4fbe000640 10 client.26093 handle_command_reply: tid=0 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 client.26093 shutdown 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3724604777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 msgr2=0x5583a7c1cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7c1cef0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009860 tx=0x7f4fc802a340 comp rx=0 tx=0).stop 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3724604777 shutdown_connections 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5583a7c1ee00 0x5583a7c21220 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583a7ba3920 0x5583a7c1cef0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583a7b9e7b0 0x5583a7bac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 --2- 172.21.15.26:0/3724604777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583a7b9ddc0 0x5583a7bac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fbe000640 0 client.26093 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3724604777 >> 172.21.15.26:0/3724604777 conn(0x5583a7b65fb0 msgr2=0x5583a7bb00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3724604777 shutdown_connections 2024-06-13T07:08:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.125+0000 7f4fd9baa740 1 -- 172.21.15.26:0/3724604777 wait complete. 2024-06-13T07:08:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.126+0000 7f4fd9baa740 20 client.26093 trim_cache size 0 max 16384 2024-06-13T07:08:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:08:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:08:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:08:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.126+0000 7f4fd5800640 1 -- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4fb40a2c00 msgr2=0x7f4fb40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.126+0000 7f4fd5800640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4fb40a2c00 0x7f4fb40a50c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009830 tx=0x7f4fc8030570 comp rx=0 tx=0).stop 2024-06-13T07:08:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 -- 172.21.15.26:0/2816551934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fd0102360 msgr2=0x7f4fd0199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fd0102360 0x7f4fd0199d20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000c420 tx=0x7f4fc0009f90 comp rx=0 tx=0).stop 2024-06-13T07:08:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 -- 172.21.15.26:0/2816551934 shutdown_connections 2024-06-13T07:08:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4fb40a2c00 0x7f4fb40a50c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd01075c0 0x7f4fd01a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0104c90 0x7f4fd019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 --2- 172.21.15.26:0/2816551934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fd0102360 0x7f4fd0199d20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 -- 172.21.15.26:0/2816551934 >> 172.21.15.26:0/2816551934 conn(0x7f4fd00fbc90 msgr2=0x7f4fd0101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 -- 172.21.15.26:0/2816551934 shutdown_connections 2024-06-13T07:08:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:25.127+0000 7f4fd5800640 1 -- 172.21.15.26:0/2816551934 wait complete. 2024-06-13T07:08:25.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 343420 4.70 MB/sec execute 279 sec latency 38.585 ms 2024-06-13T07:08:26.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:25 smithi136 ceph-mon[37229]: pgmap v747: 129 pgs: 129 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:08:26.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:25 smithi026 ceph-mon[32041]: pgmap v747: 129 pgs: 129 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:08:26.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:25 smithi135 ceph-mon[36276]: pgmap v747: 129 pgs: 129 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:08:26.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 344372 4.71 MB/sec execute 280 sec latency 39.517 ms 2024-06-13T07:08:27.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 344870 4.70 MB/sec execute 281 sec latency 32.097 ms 2024-06-13T07:08:28.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:27 smithi136 ceph-mon[37229]: pgmap v748: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-06-13T07:08:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:27 smithi026 ceph-mon[32041]: pgmap v748: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-06-13T07:08:28.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:27 smithi135 ceph-mon[36276]: pgmap v748: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-06-13T07:08:28.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 346158 4.70 MB/sec execute 282 sec latency 24.737 ms 2024-06-13T07:08:29.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 347083 4.71 MB/sec execute 283 sec latency 23.064 ms 2024-06-13T07:08:30.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:29 smithi136 ceph-mon[37229]: pgmap v749: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-13T07:08:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:29 smithi026 ceph-mon[32041]: pgmap v749: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-13T07:08:30.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:29 smithi135 ceph-mon[36276]: pgmap v749: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-13T07:08:30.551 INFO:tasks.workunit.client.0.smithi026.stdout: 1 347938 4.71 MB/sec execute 284 sec latency 19.632 ms 2024-06-13T07:08:31.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 348523 4.70 MB/sec execute 285 sec latency 38.524 ms 2024-06-13T07:08:32.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:31 smithi026 ceph-mon[32041]: pgmap v750: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-06-13T07:08:32.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:31 smithi135 ceph-mon[36276]: pgmap v750: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-06-13T07:08:32.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:31 smithi136 ceph-mon[37229]: pgmap v750: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-06-13T07:08:32.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 349980 4.70 MB/sec execute 286 sec latency 50.953 ms 2024-06-13T07:08:33.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 350948 4.71 MB/sec execute 287 sec latency 24.010 ms 2024-06-13T07:08:34.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:33 smithi026 ceph-mon[32041]: pgmap v751: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-06-13T07:08:34.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:33 smithi135 ceph-mon[36276]: pgmap v751: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-06-13T07:08:34.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:33 smithi136 ceph-mon[37229]: pgmap v751: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-06-13T07:08:34.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 351542 4.71 MB/sec execute 288 sec latency 19.595 ms 2024-06-13T07:08:35.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 352041 4.70 MB/sec execute 289 sec latency 38.498 ms 2024-06-13T07:08:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:35 smithi026 ceph-mon[32041]: pgmap v752: 129 pgs: 129 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-06-13T07:08:36.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:35 smithi135 ceph-mon[36276]: pgmap v752: 129 pgs: 129 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-06-13T07:08:36.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:35 smithi136 ceph-mon[37229]: pgmap v752: 129 pgs: 129 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-06-13T07:08:36.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 353198 4.70 MB/sec execute 290 sec latency 57.125 ms 2024-06-13T07:08:37.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 354484 4.72 MB/sec execute 291 sec latency 23.778 ms 2024-06-13T07:08:38.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:37 smithi026 ceph-mon[32041]: pgmap v753: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-06-13T07:08:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 -- 172.21.15.26:0/528920766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe37c102480 msgr2=0x7fe37c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 --2- 172.21.15.26:0/528920766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe37c102480 0x7fe37c104870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009900 tx=0x7fe36c02a410 comp rx=0 tx=0).stop 2024-06-13T07:08:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 -- 172.21.15.26:0/528920766 shutdown_connections 2024-06-13T07:08:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 --2- 172.21.15.26:0/528920766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c104db0 0x7fe37c1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 --2- 172.21.15.26:0/528920766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c1076e0 0x7fe37c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 --2- 172.21.15.26:0/528920766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe37c102480 0x7fe37c104870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 -- 172.21.15.26:0/528920766 >> 172.21.15.26:0/528920766 conn(0x7fe37c0fbe70 msgr2=0x7fe37c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 -- 172.21.15.26:0/528920766 shutdown_connections 2024-06-13T07:08:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.281+0000 7fe382400640 1 -- 172.21.15.26:0/528920766 wait complete. 2024-06-13T07:08:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.282+0000 7fe382400640 1 Processor -- start 2024-06-13T07:08:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.282+0000 7fe382400640 1 -- start start 2024-06-13T07:08:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.282+0000 7fe382400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c102480 0x7fe37c195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.282+0000 7fe382400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe37c104db0 0x7fe37c195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.282+0000 7fe382400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c1076e0 0x7fe37c19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.282+0000 7fe382400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10b5a0 con 0x7fe37c104db0 2024-06-13T07:08:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.282+0000 7fe382400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10b420 con 0x7fe37c1076e0 2024-06-13T07:08:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.282+0000 7fe382400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10b720 con 0x7fe37c102480 2024-06-13T07:08:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c102480 0x7fe37c195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe37c104db0 0x7fe37c195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c102480 0x7fe37c195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34430/0 (socket says 172.21.15.26:34430) 2024-06-13T07:08:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37b400640 1 -- 172.21.15.26:0/2738990507 learned_addr learned my addr 172.21.15.26:0/2738990507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37be00640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c1076e0 0x7fe37c19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.284 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:37 smithi135 ceph-mon[36276]: pgmap v753: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-06-13T07:08:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37b400640 1 -- 172.21.15.26:0/2738990507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c1076e0 msgr2=0x7fe37c19d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37b400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c1076e0 0x7fe37c19d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37b400640 1 -- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe37c104db0 msgr2=0x7fe37c195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37b400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe37c104db0 0x7fe37c195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.283+0000 7fe37b400640 1 -- 172.21.15.26:0/2738990507 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c19d6b0 con 0x7fe37c102480 2024-06-13T07:08:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.284+0000 7fe37b400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c102480 0x7fe37c195a90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009230 tx=0x7fe36c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.284+0000 7fe373e00640 1 -- 172.21.15.26:0/2738990507 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c03f030 con 0x7fe37c102480 2024-06-13T07:08:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.284+0000 7fe373e00640 1 -- 172.21.15.26:0/2738990507 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36c003ce0 con 0x7fe37c102480 2024-06-13T07:08:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.284+0000 7fe373e00640 1 -- 172.21.15.26:0/2738990507 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c035070 con 0x7fe37c102480 2024-06-13T07:08:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.284+0000 7fe382400640 1 -- 172.21.15.26:0/2738990507 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c19d940 con 0x7fe37c102480 2024-06-13T07:08:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.284+0000 7fe382400640 1 -- 172.21.15.26:0/2738990507 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c19de80 con 0x7fe37c102480 2024-06-13T07:08:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe384f22740 1 Processor -- start 2024-06-13T07:08:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe384f22740 1 -- start start 2024-06-13T07:08:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe373e00640 1 -- 172.21.15.26:0/2738990507 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe36c042030 con 0x7fe37c102480 2024-06-13T07:08:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe384f22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe384f22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc6f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe384f22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc71bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe384f22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cafdbcd420 con 0x55cafdc69c00 2024-06-13T07:08:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe384f22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cafdb74c50 con 0x55cafdc6a5f0 2024-06-13T07:08:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.289+0000 7fe384f22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cafdbc93e0 con 0x55cafdc6f760 2024-06-13T07:08:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc6f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43502/0 (socket says 172.21.15.26:43502) 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37be00640 1 -- 172.21.15.26:0/850940942 learned_addr learned my addr 172.21.15.26:0/850940942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37aa00640 1 --2- 172.21.15.26:0/850940942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc71bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37aa00640 1 -- 172.21.15.26:0/850940942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 msgr2=0x55cafdc6f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37aa00640 1 --2- 172.21.15.26:0/850940942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc6f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37aa00640 1 -- 172.21.15.26:0/850940942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 msgr2=0x55cafdc6a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37aa00640 1 --2- 172.21.15.26:0/850940942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.290+0000 7fe37aa00640 1 -- 172.21.15.26:0/850940942 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cafd8e1210 con 0x55cafdc6f760 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.291+0000 7fe37aa00640 1 --2- 172.21.15.26:0/850940942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc71bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe37c069bb0 tx=0x7fe368011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6826b7bfeb64e3e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.291+0000 7fe370c00640 1 -- 172.21.15.26:0/850940942 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36801e070 con 0x55cafdc6f760 2024-06-13T07:08:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.291+0000 7fe370c00640 1 -- 172.21.15.26:0/850940942 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe368009d70 con 0x55cafdc6f760 2024-06-13T07:08:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.291+0000 7fe370c00640 1 -- 172.21.15.26:0/850940942 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680187a0 con 0x55cafdc6f760 2024-06-13T07:08:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.291+0000 7fe373e00640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3600a70c0 0x7fe3600a9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe373e00640 1 -- 172.21.15.26:0/2738990507 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe36c0ef910 con 0x7fe37c102480 2024-06-13T07:08:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 -- 172.21.15.26:0/850940942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 msgr2=0x55cafdc71bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 --2- 172.21.15.26:0/850940942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc71bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe37c069bb0 tx=0x7fe368011810 comp rx=0 tx=0).stop 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe37be00640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3600a70c0 0x7fe3600a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 -- 172.21.15.26:0/850940942 shutdown_connections 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 --2- 172.21.15.26:0/850940942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc6f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 --2- 172.21.15.26:0/850940942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc71bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 --2- 172.21.15.26:0/850940942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 -- 172.21.15.26:0/850940942 >> 172.21.15.26:0/850940942 conn(0x55cafdc31fb0 msgr2=0x55cafdc65680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 -- 172.21.15.26:0/850940942 shutdown_connections 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.292+0000 7fe384f22740 1 -- 172.21.15.26:0/850940942 wait complete. 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.293+0000 7fe37be00640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3600a70c0 0x7fe3600a9580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55cafdc6ae50 tx=0x7fe374009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.293+0000 7fe384f22740 1 -- start start 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.293+0000 7fe384f22740 1 Processor -- start 2024-06-13T07:08:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.293+0000 7fe384f22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc73fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.293+0000 7fe384f22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 0x55cafdc744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.293+0000 7fe384f22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 0x55cafdc7b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.293+0000 7fe384f22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cafdc72710 con 0x55cafdc6a5f0 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe384f22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cafdb880b0 con 0x55cafdc6f760 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe384f22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cafdc72890 con 0x55cafdc69c00 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe384f22740 10 client.0 _ll_register_callbacks cb 0x55cafdb73f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc73fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc73fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34438/0 (socket says 172.21.15.26:34438) 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 -- 172.21.15.26:0/2967906402 learned_addr learned my addr 172.21.15.26:0/2967906402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37be00640 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 0x55cafdc7b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37aa00640 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 0x55cafdc744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 -- 172.21.15.26:0/2967906402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 msgr2=0x55cafdc7b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 0x55cafdc7b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 -- 172.21.15.26:0/2967906402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 msgr2=0x55cafdc744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 0x55cafdc744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 -- 172.21.15.26:0/2967906402 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cafd9ad620 con 0x55cafdc69c00 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.294+0000 7fe37b400640 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc73fa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0b86f0 tx=0x7fe36c0b7960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2654b76fee5c00fe server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.295+0000 7fe365600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.295+0000 7fe365600640 1 -- 172.21.15.26:0/2967906402 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c049070 con 0x55cafdc69c00 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.295+0000 7fe365600640 1 -- 172.21.15.26:0/2967906402 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe36c003800 con 0x55cafdc69c00 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.295+0000 7fe384f22740 1 -- 172.21.15.26:0/2967906402 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe368009d70 con 0x55cafdc69c00 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.295+0000 7fe365600640 1 -- 172.21.15.26:0/2967906402 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe36c0bb680 con 0x55cafdc69c00 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.295+0000 7fe384f22740 10 client.26143 fetch_fsmap learned FSMap version 35 2024-06-13T07:08:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.295+0000 7fe384f22740 1 -- 172.21.15.26:0/2967906402 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cafdc72710 con 0x55cafdc69c00 2024-06-13T07:08:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.295+0000 7fe365600640 1 -- 172.21.15.26:0/2967906402 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe368009d70 con 0x55cafdc69c00 2024-06-13T07:08:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.296+0000 7fe365600640 1 -- 172.21.15.26:0/2967906402 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe36c0d3070 con 0x55cafdc69c00 2024-06-13T07:08:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.296+0000 7fe384f22740 10 client.26143 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:08:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.296+0000 7fe384f22740 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:08:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.296+0000 7fe384f22740 20 client.26143 populate_metadata read hostname 'smithi026' 2024-06-13T07:08:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.296+0000 7fe384f22740 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55cafdc7ffe0 0x55cafdc82430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.296+0000 7fe384f22740 4 client.26143 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.296+0000 7fe384f22740 1 -- 172.21.15.26:0/2967906402 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cafdbdab50 con 0x55cafdc7ffe0 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.296+0000 7fe37aa00640 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55cafdc7ffe0 0x55cafdc82430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.297+0000 7fe37aa00640 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55cafdc7ffe0 0x55cafdc82430 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772ed507ea8e6ecf server_cookie=1f7ce00900bdbf8d in_seq=0 out_seq=0 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.297+0000 7fe365600640 10 client.26143 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe365600640 1 -- 172.21.15.26:0/2967906402 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cafdbdab50 con 0x55cafdc7ffe0 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe365600640 10 client.26143 handle_command_reply: tid=0 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 client.26143 shutdown 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 -- 172.21.15.26:0/2967906402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 msgr2=0x55cafdc73fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc73fa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0b86f0 tx=0x7fe36c0b7960 comp rx=0 tx=0).stop 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 -- 172.21.15.26:0/2967906402 shutdown_connections 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55cafdc7ffe0 0x55cafdc82430 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 0x55cafdc7b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 0x55cafdc744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 --2- 172.21.15.26:0/2967906402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc73fa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe365600640 0 client.26143 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.298+0000 7fe384f22740 1 -- 172.21.15.26:0/2967906402 >> 172.21.15.26:0/2967906402 conn(0x55cafdc31fb0 msgr2=0x55cafdc64840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.299+0000 7fe384f22740 1 -- 172.21.15.26:0/2967906402 shutdown_connections 2024-06-13T07:08:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.299+0000 7fe384f22740 1 -- 172.21.15.26:0/2967906402 wait complete. 2024-06-13T07:08:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.299+0000 7fe384f22740 20 client.26143 trim_cache size 0 max 16384 2024-06-13T07:08:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.316+0000 7fe384f22740 1 Processor -- start 2024-06-13T07:08:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe384f22740 1 -- start start 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe384f22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe384f22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc7b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe384f22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc73bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe384f22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1aad00 con 0x55cafdc69c00 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe384f22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1a85d0 con 0x55cafdc6a5f0 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe384f22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1a6420 con 0x55cafdc6f760 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe37b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc7b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe37be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc73bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc7b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:40190/0 (socket says 172.21.15.26:40190) 2024-06-13T07:08:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe37aa00640 1 -- 172.21.15.26:0/1324355677 learned_addr learned my addr 172.21.15.26:0/1324355677 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.317+0000 7fe37b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43512/0 (socket says 172.21.15.26:43512) 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.318+0000 7fe37aa00640 1 -- 172.21.15.26:0/1324355677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 msgr2=0x55cafdc73bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.318+0000 7fe37aa00640 1 --2- 172.21.15.26:0/1324355677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc73bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.318+0000 7fe37aa00640 1 -- 172.21.15.26:0/1324355677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 msgr2=0x55cafdc6af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.318+0000 7fe37aa00640 1 --2- 172.21.15.26:0/1324355677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.318+0000 7fe37aa00640 1 -- 172.21.15.26:0/1324355677 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cafdc69190 con 0x55cafdc6a5f0 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.318+0000 7fe37aa00640 1 --2- 172.21.15.26:0/1324355677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc7b440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe3680099b0 tx=0x7fe368005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5901b2fd9ade89 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.318+0000 7fe367e00640 1 -- 172.21.15.26:0/1324355677 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36801e040 con 0x55cafdc6a5f0 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe367e00640 1 -- 172.21.15.26:0/1324355677 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36801f040 con 0x55cafdc6a5f0 2024-06-13T07:08:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe367e00640 1 -- 172.21.15.26:0/1324355677 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368019410 con 0x55cafdc6a5f0 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe384f22740 1 -- 172.21.15.26:0/1324355677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 msgr2=0x55cafdc7b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe384f22740 1 --2- 172.21.15.26:0/1324355677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc7b440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe3680099b0 tx=0x7fe368005dc0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe384f22740 1 -- 172.21.15.26:0/1324355677 shutdown_connections 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe384f22740 1 --2- 172.21.15.26:0/1324355677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6a5f0 0x55cafdc7b440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe384f22740 1 --2- 172.21.15.26:0/1324355677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc6f760 0x55cafdc73bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe384f22740 1 --2- 172.21.15.26:0/1324355677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc69c00 0x55cafdc6af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.319+0000 7fe384f22740 1 -- 172.21.15.26:0/1324355677 >> 172.21.15.26:0/1324355677 conn(0x55cafdc31fb0 msgr2=0x55cafdc65160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.320+0000 7fe384f22740 1 -- 172.21.15.26:0/1324355677 shutdown_connections 2024-06-13T07:08:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.320+0000 7fe384f22740 1 -- 172.21.15.26:0/1324355677 wait complete. 2024-06-13T07:08:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.320+0000 7fe384f22740 1 -- start start 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.320+0000 7fe384f22740 1 Processor -- start 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe384f22740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc78880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe384f22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 0x55cafdc78dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe384f22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 0x55cafdc79dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe384f22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1a6a90 con 0x55cafdc6a5f0 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe384f22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1a7f00 con 0x55cafdc6f760 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe384f22740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1a7160 con 0x55cafdc69c00 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe384f22740 10 client.0 _ll_register_callbacks cb 0x7fe36c0f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc78880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc78880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34442/0 (socket says 172.21.15.26:34442) 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37b400640 1 -- 172.21.15.26:0/949041416 learned_addr learned my addr 172.21.15.26:0/949041416 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37aa00640 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 0x55cafdc78dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37be00640 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 0x55cafdc79dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37b400640 1 -- 172.21.15.26:0/949041416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 msgr2=0x55cafdc79dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37b400640 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 0x55cafdc79dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37b400640 1 -- 172.21.15.26:0/949041416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 msgr2=0x55cafdc78dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37b400640 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 0x55cafdc78dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.321+0000 7fe37b400640 1 -- 172.21.15.26:0/949041416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cafdc7a3c0 con 0x55cafdc69c00 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe37b400640 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc78880 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0314e0 tx=0x7fe36c0b8610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a13592c103b832 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe365600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe365600640 1 -- 172.21.15.26:0/949041416 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c049040 con 0x55cafdc69c00 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe365600640 1 -- 172.21.15.26:0/949041416 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe36c0bbf80 con 0x55cafdc69c00 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe384f22740 1 -- 172.21.15.26:0/949041416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36801f040 con 0x55cafdc69c00 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe365600640 1 -- 172.21.15.26:0/949041416 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe36c0044e0 con 0x55cafdc69c00 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe384f22740 10 client.26149 fetch_fsmap learned FSMap version 35 2024-06-13T07:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe384f22740 1 -- 172.21.15.26:0/949041416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cafdc7b740 con 0x55cafdc69c00 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.322+0000 7fe365600640 1 -- 172.21.15.26:0/949041416 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36801f040 con 0x55cafdc69c00 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.323+0000 7fe365600640 1 -- 172.21.15.26:0/949041416 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe36c0d3040 con 0x55cafdc69c00 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.323+0000 7fe384f22740 10 client.26149 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.323+0000 7fe384f22740 10 client.26149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.323+0000 7fe384f22740 20 client.26149 populate_metadata read hostname 'smithi026' 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.323+0000 7fe384f22740 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55cafdceb1e0 0x55cafdced600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.323+0000 7fe384f22740 4 client.26149 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.323+0000 7fe384f22740 1 -- 172.21.15.26:0/949041416 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cafdbd4ea0 con 0x55cafdceb1e0 2024-06-13T07:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.323+0000 7fe37aa00640 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55cafdceb1e0 0x55cafdced600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.324+0000 7fe37aa00640 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55cafdceb1e0 0x55cafdced600 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707905de70fe496e server_cookie=65f5a11f2a7520ca in_seq=0 out_seq=0 2024-06-13T07:08:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.324+0000 7fe365600640 10 client.26149 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:38.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:37 smithi136 ceph-mon[37229]: pgmap v753: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.407+0000 7fe365600640 1 -- 172.21.15.26:0/949041416 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cafdbd4ea0 con 0x55cafdceb1e0 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.407+0000 7fe365600640 10 client.26149 handle_command_reply: tid=0 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.407+0000 7fe384f22740 1 client.26149 shutdown 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 -- 172.21.15.26:0/949041416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 msgr2=0x55cafdc78880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc78880 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0314e0 tx=0x7fe36c0b8610 comp rx=0 tx=0).stop 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 -- 172.21.15.26:0/949041416 shutdown_connections 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55cafdceb1e0 0x55cafdced600 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cafdc6f760 0x55cafdc79dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cafdc6a5f0 0x55cafdc78dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 --2- 172.21.15.26:0/949041416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cafdc69c00 0x55cafdc78880 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe365600640 0 client.26149 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 -- 172.21.15.26:0/949041416 >> 172.21.15.26:0/949041416 conn(0x55cafdc31fb0 msgr2=0x55cafdc73bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 -- 172.21.15.26:0/949041416 shutdown_connections 2024-06-13T07:08:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 1 -- 172.21.15.26:0/949041416 wait complete. 2024-06-13T07:08:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.408+0000 7fe384f22740 20 client.26149 trim_cache size 0 max 16384 2024-06-13T07:08:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:08:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:08:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:08:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 -- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3600a70c0 msgr2=0x7fe3600a9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3600a70c0 0x7fe3600a9580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55cafdc6ae50 tx=0x7fe374009500 comp rx=0 tx=0).stop 2024-06-13T07:08:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 -- 172.21.15.26:0/2738990507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c102480 msgr2=0x7fe37c195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c102480 0x7fe37c195a90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009230 tx=0x7fe36c0042f0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 -- 172.21.15.26:0/2738990507 shutdown_connections 2024-06-13T07:08:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fe3600a70c0 0x7fe3600a9580 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c1076e0 0x7fe37c19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe37c104db0 0x7fe37c195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 --2- 172.21.15.26:0/2738990507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c102480 0x7fe37c195a90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 -- 172.21.15.26:0/2738990507 >> 172.21.15.26:0/2738990507 conn(0x7fe37c0fbe70 msgr2=0x7fe37c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 -- 172.21.15.26:0/2738990507 shutdown_connections 2024-06-13T07:08:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:38.409+0000 7fe382400640 1 -- 172.21.15.26:0/2738990507 wait complete. 2024-06-13T07:08:38.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 355106 4.72 MB/sec execute 292 sec latency 75.644 ms 2024-06-13T07:08:39.552 INFO:tasks.workunit.client.0.smithi026.stdout: 1 355669 4.70 MB/sec execute 293 sec latency 33.891 ms 2024-06-13T07:08:40.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:39 smithi026 ceph-mon[32041]: pgmap v754: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:08:40.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:39 smithi135 ceph-mon[36276]: pgmap v754: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:08:40.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:39 smithi136 ceph-mon[37229]: pgmap v754: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:08:40.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 356841 4.70 MB/sec execute 294 sec latency 27.627 ms 2024-06-13T07:08:41.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 357867 4.71 MB/sec execute 295 sec latency 49.203 ms 2024-06-13T07:08:42.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:41 smithi026 ceph-mon[32041]: pgmap v755: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:08:42.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:41 smithi135 ceph-mon[36276]: pgmap v755: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:08:42.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:41 smithi136 ceph-mon[37229]: pgmap v755: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:08:42.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 358649 4.72 MB/sec execute 296 sec latency 28.644 ms 2024-06-13T07:08:43.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 359308 4.71 MB/sec execute 297 sec latency 43.737 ms 2024-06-13T07:08:44.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:43 smithi026 ceph-mon[32041]: pgmap v756: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:08:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:43 smithi135 ceph-mon[36276]: pgmap v756: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:08:44.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:43 smithi136 ceph-mon[37229]: pgmap v756: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:08:44.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 360633 4.71 MB/sec execute 298 sec latency 34.628 ms 2024-06-13T07:08:45.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 361584 4.72 MB/sec execute 299 sec latency 24.572 ms 2024-06-13T07:08:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:45 smithi026 ceph-mon[32041]: pgmap v757: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:08:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:45 smithi135 ceph-mon[36276]: pgmap v757: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:08:46.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:45 smithi136 ceph-mon[37229]: pgmap v757: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:08:46.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 362179 4.72 MB/sec execute 300 sec latency 35.622 ms 2024-06-13T07:08:47.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 362723 4.70 MB/sec execute 301 sec latency 34.593 ms 2024-06-13T07:08:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:47 smithi026 ceph-mon[32041]: pgmap v758: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:08:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:47 smithi135 ceph-mon[36276]: pgmap v758: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:08:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:47 smithi136 ceph-mon[37229]: pgmap v758: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:08:48.553 INFO:tasks.workunit.client.0.smithi026.stdout: 1 364051 4.71 MB/sec execute 302 sec latency 24.530 ms 2024-06-13T07:08:49.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 364978 4.71 MB/sec execute 303 sec latency 20.635 ms 2024-06-13T07:08:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:49 smithi026 ceph-mon[32041]: pgmap v759: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:08:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:49 smithi135 ceph-mon[36276]: pgmap v759: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:08:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:49 smithi136 ceph-mon[37229]: pgmap v759: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:08:50.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 365700 4.72 MB/sec execute 304 sec latency 20.378 ms 2024-06-13T07:08:51.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 366120 4.70 MB/sec execute 305 sec latency 45.618 ms 2024-06-13T07:08:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 -- 172.21.15.26:0/423751009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4103c20 msgr2=0x7f8ce4104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 --2- 172.21.15.26:0/423751009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4103c20 0x7f8ce4104040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009900 tx=0x7f8cd402a3f0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 -- 172.21.15.26:0/423751009 shutdown_connections 2024-06-13T07:08:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 --2- 172.21.15.26:0/423751009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ce4104e10 0x7f8ce4105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 --2- 172.21.15.26:0/423751009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ce41057d0 0x7f8ce410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 --2- 172.21.15.26:0/423751009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4103c20 0x7f8ce4104040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 -- 172.21.15.26:0/423751009 >> 172.21.15.26:0/423751009 conn(0x7f8ce40ff190 msgr2=0x7f8ce41015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 -- 172.21.15.26:0/423751009 shutdown_connections 2024-06-13T07:08:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.592+0000 7f8cea000640 1 -- 172.21.15.26:0/423751009 wait complete. 2024-06-13T07:08:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.593+0000 7f8cea000640 1 Processor -- start 2024-06-13T07:08:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8cea000640 1 -- start start 2024-06-13T07:08:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8cea000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ce4103c20 0x7f8ce4199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8cea000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4104e10 0x7f8ce419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8cea000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ce41057d0 0x7f8ce41a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8cea000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410f8e0 con 0x7f8ce4103c20 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8cea000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410f760 con 0x7f8ce41057d0 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8cea000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410fa60 con 0x7f8ce4104e10 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ce41057d0 0x7f8ce41a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8ce2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4104e10 0x7f8ce419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ce4103c20 0x7f8ce4199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ce41057d0 0x7f8ce41a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59950/0 (socket says 172.21.15.26:59950) 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8ce2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4104e10 0x7f8ce419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44732/0 (socket says 172.21.15.26:44732) 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.594+0000 7f8ce3e00640 1 -- 172.21.15.26:0/1853377693 learned_addr learned my addr 172.21.15.26:0/1853377693 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.595+0000 7f8ce3e00640 1 -- 172.21.15.26:0/1853377693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4104e10 msgr2=0x7f8ce419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.595+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4104e10 0x7f8ce419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.595+0000 7f8ce3e00640 1 -- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ce4103c20 msgr2=0x7f8ce4199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.595+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ce4103c20 0x7f8ce4199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.595+0000 7f8ce3e00640 1 -- 172.21.15.26:0/1853377693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce41a1ab0 con 0x7f8ce41057d0 2024-06-13T07:08:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.595+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ce41057d0 0x7f8ce41a1440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00eb10 tx=0x7f8cdc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.596+0000 7f8cdbe00640 1 -- 172.21.15.26:0/1853377693 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc004b20 con 0x7f8ce41057d0 2024-06-13T07:08:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.596+0000 7f8cdbe00640 1 -- 172.21.15.26:0/1853377693 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cdc004cc0 con 0x7f8ce41057d0 2024-06-13T07:08:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.596+0000 7f8cea000640 1 -- 172.21.15.26:0/1853377693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce41a1da0 con 0x7f8ce41057d0 2024-06-13T07:08:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.596+0000 7f8cdbe00640 1 -- 172.21.15.26:0/1853377693 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc010790 con 0x7f8ce41057d0 2024-06-13T07:08:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.596+0000 7f8cea000640 1 -- 172.21.15.26:0/1853377693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce41a2260 con 0x7f8ce41057d0 2024-06-13T07:08:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.599+0000 7f8cecfc9740 1 Processor -- start 2024-06-13T07:08:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.600+0000 7f8cdbe00640 1 -- 172.21.15.26:0/1853377693 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8cdc005020 con 0x7f8ce41057d0 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.600+0000 7f8cecfc9740 1 -- start start 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.600+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389deda020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.600+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dedf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.600+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 0x56389dee1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.600+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56389de3d420 con 0x56389dedf760 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.600+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56389dde4c50 con 0x56389deda5f0 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.600+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56389de393e0 con 0x56389ded9c00 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dedf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dedf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59954/0 (socket says 172.21.15.26:59954) 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce3400640 1 -- 172.21.15.26:0/2642200231 learned_addr learned my addr 172.21.15.26:0/2642200231 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389deda020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 0x56389dee1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce3400640 1 -- 172.21.15.26:0/2642200231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 msgr2=0x56389deda020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce3400640 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389deda020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce3400640 1 -- 172.21.15.26:0/2642200231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 msgr2=0x56389dee1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce3400640 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 0x56389dee1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.601+0000 7f8ce3400640 1 -- 172.21.15.26:0/2642200231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56389db51210 con 0x56389deda5f0 2024-06-13T07:08:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.602+0000 7f8ce3400640 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dedf0f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4006010 tx=0x7f8cd4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bfe52dfbbecf31e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.602+0000 7f8cdbe00640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8cc80a2cb0 0x7f8cc80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.602+0000 7f8ccfe00640 1 -- 172.21.15.26:0/2642200231 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4035040 con 0x56389deda5f0 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.602+0000 7f8ccfe00640 1 -- 172.21.15.26:0/2642200231 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd4004430 con 0x56389deda5f0 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.602+0000 7f8cdbe00640 1 -- 172.21.15.26:0/1853377693 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8cdc0cf9c0 con 0x7f8ce41057d0 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.602+0000 7f8ccfe00640 1 -- 172.21.15.26:0/2642200231 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40316b0 con 0x56389deda5f0 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.602+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8cc80a2cb0 0x7f8cc80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 -- 172.21.15.26:0/2642200231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 msgr2=0x56389dedf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dedf0f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4006010 tx=0x7f8cd4004040 comp rx=0 tx=0).stop 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8cc80a2cb0 0x7f8cc80a5170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56389dedae50 tx=0x7f8cd0013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 -- 172.21.15.26:0/2642200231 shutdown_connections 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dedf0f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 0x56389dee1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/2642200231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389deda020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 -- 172.21.15.26:0/2642200231 >> 172.21.15.26:0/2642200231 conn(0x56389dea1fb0 msgr2=0x56389ded5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 -- 172.21.15.26:0/2642200231 shutdown_connections 2024-06-13T07:08:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.603+0000 7f8cecfc9740 1 -- 172.21.15.26:0/2642200231 wait complete. 2024-06-13T07:08:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.604+0000 7f8cecfc9740 1 -- start start 2024-06-13T07:08:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.604+0000 7f8cecfc9740 1 Processor -- start 2024-06-13T07:08:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389dee3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dee4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 0x56389deeb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56389dee26b0 con 0x56389dedf760 2024-06-13T07:08:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56389ddf80b0 con 0x56389deda5f0 2024-06-13T07:08:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56389dee2830 con 0x56389ded9c00 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389dee3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389dee3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44754/0 (socket says 172.21.15.26:44754) 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8cecfc9740 10 client.0 _ll_register_callbacks cb 0x56389dde3f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce3400640 1 -- 172.21.15.26:0/724422646 learned_addr learned my addr 172.21.15.26:0/724422646 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dee4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce3400640 1 -- 172.21.15.26:0/724422646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 msgr2=0x56389dee4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce3400640 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dee4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce3400640 1 -- 172.21.15.26:0/724422646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 msgr2=0x56389deeb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce3400640 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 0x56389deeb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.605+0000 7f8ce3400640 1 -- 172.21.15.26:0/724422646 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389dc1d620 con 0x56389ded9c00 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.606+0000 7f8ce3400640 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389dee3f10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8cd402a9e0 tx=0x7f8cd40092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a80e464be3f5d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.606+0000 7f8cccc00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.606+0000 7f8cccc00640 1 -- 172.21.15.26:0/724422646 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4035040 con 0x56389ded9c00 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.606+0000 7f8cccc00640 1 -- 172.21.15.26:0/724422646 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8cd4005000 con 0x56389ded9c00 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.606+0000 7f8cecfc9740 1 -- 172.21.15.26:0/724422646 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd4004430 con 0x56389ded9c00 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cccc00640 1 -- 172.21.15.26:0/724422646 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8cd4005200 con 0x56389ded9c00 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cecfc9740 10 client.26161 fetch_fsmap learned FSMap version 35 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cecfc9740 1 -- 172.21.15.26:0/724422646 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56389dee26b0 con 0x56389ded9c00 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cccc00640 1 -- 172.21.15.26:0/724422646 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd4004430 con 0x56389ded9c00 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cccc00640 1 -- 172.21.15.26:0/724422646 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8cd4055030 con 0x56389ded9c00 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cecfc9740 10 client.26161 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:08:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cecfc9740 10 client.26161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:08:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cecfc9740 20 client.26161 populate_metadata read hostname 'smithi026' 2024-06-13T07:08:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56389deefab0 0x56389def1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cecfc9740 4 client.26161 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.607+0000 7f8cecfc9740 1 -- 172.21.15.26:0/724422646 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56389de4ab50 con 0x56389deefab0 2024-06-13T07:08:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.608+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56389deefab0 0x56389def1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.609+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56389deefab0 0x56389def1ea0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72f7c8ce0762f22 server_cookie=d0a65d4af1d7bf31 in_seq=0 out_seq=0 2024-06-13T07:08:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.609+0000 7f8cccc00640 10 client.26161 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cccc00640 1 -- 172.21.15.26:0/724422646 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56389de4ab50 con 0x56389deefab0 2024-06-13T07:08:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cccc00640 10 client.26161 handle_command_reply: tid=0 2024-06-13T07:08:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 client.26161 shutdown 2024-06-13T07:08:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 -- 172.21.15.26:0/724422646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 msgr2=0x56389dee3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389dee3f10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8cd402a9e0 tx=0x7f8cd40092e0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 -- 172.21.15.26:0/724422646 shutdown_connections 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56389deefab0 0x56389def1ea0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389deda5f0 0x56389dee4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389dedf760 0x56389deeb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/724422646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389ded9c00 0x56389dee3f10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 -- 172.21.15.26:0/724422646 >> 172.21.15.26:0/724422646 conn(0x56389dea1fb0 msgr2=0x56389ded4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cccc00640 0 client.26161 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 -- 172.21.15.26:0/724422646 shutdown_connections 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 1 -- 172.21.15.26:0/724422646 wait complete. 2024-06-13T07:08:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.610+0000 7f8cecfc9740 20 client.26161 trim_cache size 0 max 16384 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.631+0000 7f8cecfc9740 1 Processor -- start 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8cecfc9740 1 -- start start 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389ded8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389deeb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389dee3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41af270 con 0x56389deda5f0 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41acb40 con 0x56389ded9c00 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41aa990 con 0x56389dedf760 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389ded8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389dee3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8ce2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389deeb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389ded8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59970/0 (socket says 172.21.15.26:59970) 2024-06-13T07:08:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389dee3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44770/0 (socket says 172.21.15.26:44770) 2024-06-13T07:08:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8ce3400640 1 -- 172.21.15.26:0/553479525 learned_addr learned my addr 172.21.15.26:0/553479525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.632+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389deeb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48396/0 (socket says 172.21.15.26:48396) 2024-06-13T07:08:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ce3e00640 1 -- 172.21.15.26:0/553479525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 msgr2=0x56389ded8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389ded8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ce3e00640 1 -- 172.21.15.26:0/553479525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 msgr2=0x56389deeb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389deeb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ce3e00640 1 -- 172.21.15.26:0/553479525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56389ded91e0 con 0x56389dedf760 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389deeb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ce3400640 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389ded8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389dee3530 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0cd070 tx=0x7f8cdc09b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf19907852543d8b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ccf400640 1 -- 172.21.15.26:0/553479525 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc09bdd0 con 0x56389dedf760 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ccf400640 1 -- 172.21.15.26:0/553479525 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cdc0a0180 con 0x56389dedf760 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.633+0000 7f8ccf400640 1 -- 172.21.15.26:0/553479525 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc09b220 con 0x56389dedf760 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 -- 172.21.15.26:0/553479525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 msgr2=0x56389dee3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389dee3530 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0cd070 tx=0x7f8cdc09b910 comp rx=0 tx=0).stop 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 -- 172.21.15.26:0/553479525 shutdown_connections 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389dee3530 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389deeb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/553479525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389ded8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 -- 172.21.15.26:0/553479525 >> 172.21.15.26:0/553479525 conn(0x56389dea1fb0 msgr2=0x56389ded8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 -- 172.21.15.26:0/553479525 shutdown_connections 2024-06-13T07:08:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.634+0000 7f8cecfc9740 1 -- 172.21.15.26:0/553479525 wait complete. 2024-06-13T07:08:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.635+0000 7f8cecfc9740 1 -- start start 2024-06-13T07:08:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.635+0000 7f8cecfc9740 1 Processor -- start 2024-06-13T07:08:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389dee83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389dee88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8cecfc9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389df58720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41ab000 con 0x56389deda5f0 2024-06-13T07:08:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41ac470 con 0x56389ded9c00 2024-06-13T07:08:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8cecfc9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41ab6d0 con 0x56389dedf760 2024-06-13T07:08:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8cecfc9740 10 client.0 _ll_register_callbacks cb 0x7f8cd00097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389df58720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389dee83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389df58720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44778/0 (socket says 172.21.15.26:44778) 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 -- 172.21.15.26:0/129651383 learned_addr learned my addr 172.21.15.26:0/129651383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce2a00640 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389dee88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 -- 172.21.15.26:0/129651383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 msgr2=0x56389dee83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389dee83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 -- 172.21.15.26:0/129651383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 msgr2=0x56389dee88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389dee88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 -- 172.21.15.26:0/129651383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389df58c60 con 0x56389dedf760 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8ce3e00640 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389df58720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc09bd60 tx=0x7f8cdc00cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc5b9acf52f0766 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.636+0000 7f8cccc00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.637+0000 7f8cccc00640 1 -- 172.21.15.26:0/129651383 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc09bdd0 con 0x56389dedf760 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.637+0000 7f8cecfc9740 1 -- 172.21.15.26:0/129651383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cdc0a0180 con 0x56389dedf760 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.637+0000 7f8cccc00640 1 -- 172.21.15.26:0/129651383 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8cdc0a4720 con 0x56389dedf760 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.637+0000 7f8cccc00640 1 -- 172.21.15.26:0/129651383 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8cdc0a4a40 con 0x56389dedf760 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.637+0000 7f8cecfc9740 10 client.26167 fetch_fsmap learned FSMap version 35 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.637+0000 7f8cecfc9740 1 -- 172.21.15.26:0/129651383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56389df59ea0 con 0x56389dedf760 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.637+0000 7f8cccc00640 1 -- 172.21.15.26:0/129651383 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cdc0a0180 con 0x56389dedf760 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.637+0000 7f8cccc00640 1 -- 172.21.15.26:0/129651383 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8cdc082030 con 0x56389dedf760 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.638+0000 7f8cecfc9740 10 client.26167 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.638+0000 7f8cecfc9740 10 client.26167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:08:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.638+0000 7f8cecfc9740 20 client.26167 populate_metadata read hostname 'smithi026' 2024-06-13T07:08:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.638+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56389df5a830 0x56389df5cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.638+0000 7f8cecfc9740 4 client.26167 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.638+0000 7f8cecfc9740 1 -- 172.21.15.26:0/129651383 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56389de44ea0 con 0x56389df5a830 2024-06-13T07:08:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.638+0000 7f8ce3400640 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56389df5a830 0x56389df5cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.639+0000 7f8ce3400640 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56389df5a830 0x56389df5cc50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e807c8ef0e17edd server_cookie=2ca8ff036359329f in_seq=0 out_seq=0 2024-06-13T07:08:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.639+0000 7f8cccc00640 10 client.26167 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.723+0000 7f8cccc00640 1 -- 172.21.15.26:0/129651383 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56389de44ea0 con 0x56389df5a830 2024-06-13T07:08:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.723+0000 7f8cccc00640 10 client.26167 handle_command_reply: tid=0 2024-06-13T07:08:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.724+0000 7f8cecfc9740 1 client.26167 shutdown 2024-06-13T07:08:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.724+0000 7f8cecfc9740 1 -- 172.21.15.26:0/129651383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 msgr2=0x56389df58720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.724+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389df58720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc09bd60 tx=0x7f8cdc00cee0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.725+0000 7f8cecfc9740 1 -- 172.21.15.26:0/129651383 shutdown_connections 2024-06-13T07:08:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.725+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56389df5a830 0x56389df5cc50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.725+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56389dedf760 0x56389df58720 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.725+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56389deda5f0 0x56389dee88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.725+0000 7f8cecfc9740 1 --2- 172.21.15.26:0/129651383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56389ded9c00 0x56389dee83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.725+0000 7f8cecfc9740 1 -- 172.21.15.26:0/129651383 >> 172.21.15.26:0/129651383 conn(0x56389dea1fb0 msgr2=0x56389ded3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.725+0000 7f8cccc00640 0 client.26167 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:08:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.726+0000 7f8cecfc9740 1 -- 172.21.15.26:0/129651383 shutdown_connections 2024-06-13T07:08:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.726+0000 7f8cecfc9740 1 -- 172.21.15.26:0/129651383 wait complete. 2024-06-13T07:08:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.727+0000 7f8cecfc9740 20 client.26167 trim_cache size 0 max 16384 2024-06-13T07:08:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:08:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:08:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:08:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 -- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8cc80a2cb0 msgr2=0x7f8cc80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8cc80a2cb0 0x7f8cc80a5170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56389dedae50 tx=0x7f8cd0013040 comp rx=0 tx=0).stop 2024-06-13T07:08:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 -- 172.21.15.26:0/1853377693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ce41057d0 msgr2=0x7f8ce41a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ce41057d0 0x7f8ce41a1440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00eb10 tx=0x7f8cdc00c2d0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 -- 172.21.15.26:0/1853377693 shutdown_connections 2024-06-13T07:08:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8cc80a2cb0 0x7f8cc80a5170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ce41057d0 0x7f8ce41a1440 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ce4104e10 0x7f8ce419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 --2- 172.21.15.26:0/1853377693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ce4103c20 0x7f8ce4199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.728+0000 7f8cea000640 1 -- 172.21.15.26:0/1853377693 >> 172.21.15.26:0/1853377693 conn(0x7f8ce40ff190 msgr2=0x7f8ce41080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.729+0000 7f8cea000640 1 -- 172.21.15.26:0/1853377693 shutdown_connections 2024-06-13T07:08:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:08:51.729+0000 7f8cea000640 1 -- 172.21.15.26:0/1853377693 wait complete. 2024-06-13T07:08:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:51 smithi026 ceph-mon[32041]: pgmap v760: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:08:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:51 smithi135 ceph-mon[36276]: pgmap v760: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:08:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:51 smithi136 ceph-mon[37229]: pgmap v760: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:08:52.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 367257 4.70 MB/sec execute 306 sec latency 64.808 ms 2024-06-13T07:08:53.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 368508 4.71 MB/sec execute 307 sec latency 27.394 ms 2024-06-13T07:08:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:53 smithi026 ceph-mon[32041]: pgmap v761: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:08:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:53 smithi135 ceph-mon[36276]: pgmap v761: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:08:54.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:53 smithi136 ceph-mon[37229]: pgmap v761: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-13T07:08:54.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 369195 4.72 MB/sec execute 308 sec latency 24.439 ms 2024-06-13T07:08:55.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 369673 4.70 MB/sec execute 309 sec latency 35.497 ms 2024-06-13T07:08:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:55 smithi026 ceph-mon[32041]: pgmap v762: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:08:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:55 smithi135 ceph-mon[36276]: pgmap v762: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:08:56.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:55 smithi136 ceph-mon[37229]: pgmap v762: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:08:56.554 INFO:tasks.workunit.client.0.smithi026.stdout: 1 370551 4.71 MB/sec execute 310 sec latency 33.668 ms 2024-06-13T07:08:57.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 371856 4.71 MB/sec execute 311 sec latency 62.439 ms 2024-06-13T07:08:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:57 smithi026 ceph-mon[32041]: pgmap v763: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-13T07:08:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:57 smithi135 ceph-mon[36276]: pgmap v763: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-13T07:08:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:57 smithi136 ceph-mon[37229]: pgmap v763: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-13T07:08:58.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 372708 4.72 MB/sec execute 312 sec latency 26.662 ms 2024-06-13T07:08:59.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 373098 4.70 MB/sec execute 313 sec latency 36.044 ms 2024-06-13T07:09:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:08:59 smithi026 ceph-mon[32041]: pgmap v764: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:09:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:08:59 smithi135 ceph-mon[36276]: pgmap v764: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:09:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:08:59 smithi136 ceph-mon[37229]: pgmap v764: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-13T07:09:00.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 374080 4.71 MB/sec execute 314 sec latency 26.991 ms 2024-06-13T07:09:01.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 375182 4.71 MB/sec execute 315 sec latency 20.936 ms 2024-06-13T07:09:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:01 smithi026 ceph-mon[32041]: pgmap v765: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-06-13T07:09:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:01 smithi135 ceph-mon[36276]: pgmap v765: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-06-13T07:09:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:01 smithi136 ceph-mon[37229]: pgmap v765: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-06-13T07:09:02.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 376191 4.72 MB/sec execute 316 sec latency 37.409 ms 2024-06-13T07:09:03.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 376527 4.71 MB/sec execute 317 sec latency 35.933 ms 2024-06-13T07:09:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:03 smithi026 ceph-mon[32041]: pgmap v766: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-13T07:09:04.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:03 smithi135 ceph-mon[36276]: pgmap v766: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-13T07:09:04.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:03 smithi136 ceph-mon[37229]: pgmap v766: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-13T07:09:04.555 INFO:tasks.workunit.client.0.smithi026.stdout: 1 377225 4.70 MB/sec execute 318 sec latency 36.554 ms 2024-06-13T07:09:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 -- 172.21.15.26:0/3416676464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce70100a10 msgr2=0x7fce70100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 --2- 172.21.15.26:0/3416676464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce70100a10 0x7fce70100e70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009900 tx=0x7fce5c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 -- 172.21.15.26:0/3416676464 shutdown_connections 2024-06-13T07:09:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 --2- 172.21.15.26:0/3416676464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce70100a10 0x7fce70100e70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 --2- 172.21.15.26:0/3416676464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce701013b0 0x7fce70101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 --2- 172.21.15.26:0/3416676464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce701000b0 0x7fce701004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 -- 172.21.15.26:0/3416676464 >> 172.21.15.26:0/3416676464 conn(0x7fce700fbc90 msgr2=0x7fce700fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 -- 172.21.15.26:0/3416676464 shutdown_connections 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.901+0000 7fce6fe00640 1 -- 172.21.15.26:0/3416676464 wait complete. 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.902+0000 7fce6fe00640 1 Processor -- start 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.902+0000 7fce6fe00640 1 -- start start 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce701000b0 0x7fce70199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce70100a10 0x7fce7019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce701013b0 0x7fce701a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce7010f6e0 con 0x7fce701013b0 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6fe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fce7010f560 con 0x7fce701000b0 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce7010f860 con 0x7fce70100a10 2024-06-13T07:09:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce70100a10 0x7fce7019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce701000b0 0x7fce70199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce701013b0 0x7fce701a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce701000b0 0x7fce70199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:58704/0 (socket says 172.21.15.26:58704) 2024-06-13T07:09:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.903+0000 7fce6ea00640 1 -- 172.21.15.26:0/2325353307 learned_addr learned my addr 172.21.15.26:0/2325353307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce6ea00640 1 -- 172.21.15.26:0/2325353307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce70100a10 msgr2=0x7fce7019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce6ea00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce70100a10 0x7fce7019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce6ea00640 1 -- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce701013b0 msgr2=0x7fce701a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce6ea00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce701013b0 0x7fce701a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce6ea00640 1 -- 172.21.15.26:0/2325353307 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce701a1970 con 0x7fce701000b0 2024-06-13T07:09:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce6ea00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce701000b0 0x7fce70199d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9f0 tx=0x7fce6000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce67400640 1 -- 172.21.15.26:0/2325353307 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6000ccc0 con 0x7fce701000b0 2024-06-13T07:09:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce67400640 1 -- 172.21.15.26:0/2325353307 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce6000ce60 con 0x7fce701000b0 2024-06-13T07:09:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce6fe00640 1 -- 172.21.15.26:0/2325353307 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce701a1c60 con 0x7fce701000b0 2024-06-13T07:09:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce67400640 1 -- 172.21.15.26:0/2325353307 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60010690 con 0x7fce701000b0 2024-06-13T07:09:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.904+0000 7fce6fe00640 1 -- 172.21.15.26:0/2325353307 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701a2130 con 0x7fce701000b0 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.908+0000 7fce67400640 1 -- 172.21.15.26:0/2325353307 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fce600046c0 con 0x7fce701000b0 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce785ae740 1 Processor -- start 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce785ae740 1 -- start start 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce785ae740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce785ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce785ae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7e9f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce785ae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e955420 con 0x55aa7e9f7920 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce785ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e8fcc50 con 0x55aa7e9f27b0 2024-06-13T07:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce785ae740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e9513e0 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce6ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce6f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce6f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40156/0 (socket says 172.21.15.26:40156) 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.909+0000 7fce6f400640 1 -- 172.21.15.26:0/1802649810 learned_addr learned my addr 172.21.15.26:0/1802649810 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6e000640 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7e9f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6f400640 1 -- 172.21.15.26:0/1802649810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 msgr2=0x55aa7e9f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6f400640 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6f400640 1 -- 172.21.15.26:0/1802649810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 msgr2=0x55aa7e9f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6f400640 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7e9f9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6f400640 1 -- 172.21.15.26:0/1802649810 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7e669210 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6ea00640 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6e000640 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7e9f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce67400640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fce540a2b80 0x7fce540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.910+0000 7fce6f400640 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9f21e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fce68011440 tx=0x7fce68011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c20ca7be91960f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.911+0000 7fce67400640 1 -- 172.21.15.26:0/2325353307 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fce600cf8b0 con 0x7fce701000b0 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.911+0000 7fce5be00640 1 -- 172.21.15.26:0/1802649810 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68003c30 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.911+0000 7fce6ea00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fce540a2b80 0x7fce540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.911+0000 7fce5be00640 1 -- 172.21.15.26:0/1802649810 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce68003dd0 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.911+0000 7fce5be00640 1 -- 172.21.15.26:0/1802649810 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6800fac0 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 -- 172.21.15.26:0/1802649810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f1dc0 msgr2=0x55aa7e9f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9f21e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fce68011440 tx=0x7fce68011900 comp rx=0 tx=0).stop 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce6ea00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fce540a2b80 0x7fce540a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9c0 tx=0x7fce600d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 -- 172.21.15.26:0/1802649810 shutdown_connections 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7e9f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 --2- 172.21.15.26:0/1802649810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9f21e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 -- 172.21.15.26:0/1802649810 >> 172.21.15.26:0/1802649810 conn(0x55aa7e9b9fb0 msgr2=0x55aa7e9ed840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 -- 172.21.15.26:0/1802649810 shutdown_connections 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.912+0000 7fce785ae740 1 -- 172.21.15.26:0/1802649810 wait complete. 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.913+0000 7fce785ae740 1 -- start start 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.913+0000 7fce785ae740 1 Processor -- start 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.913+0000 7fce785ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.913+0000 7fce785ae740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.913+0000 7fce785ae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7ea036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce785ae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e9fa8d0 con 0x55aa7e9f7920 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce785ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e9100b0 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce785ae740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e9faa50 con 0x55aa7e9f27b0 2024-06-13T07:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce785ae740 10 client.0 _ll_register_callbacks cb 0x55aa7e8fbf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40160/0 (socket says 172.21.15.26:40160) 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6e000640 1 -- 172.21.15.26:0/2359372250 learned_addr learned my addr 172.21.15.26:0/2359372250 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6f400640 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6f400640 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7ea036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6e000640 1 -- 172.21.15.26:0/2359372250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f1dc0 msgr2=0x55aa7e9fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6e000640 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9fc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6e000640 1 -- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 msgr2=0x55aa7ea036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6e000640 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7ea036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6e000640 1 -- 172.21.15.26:0/2359372250 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa7e735620 con 0x55aa7e9f27b0 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.914+0000 7fce6f400640 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7ea036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.915+0000 7fce6e000640 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9fc660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce5c006010 tx=0x7fce5c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1e0aadb42797d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.915+0000 7fce58c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.915+0000 7fce58c00640 1 -- 172.21.15.26:0/2359372250 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c035040 con 0x55aa7e9f27b0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.915+0000 7fce58c00640 1 -- 172.21.15.26:0/2359372250 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fce5c005450 con 0x55aa7e9f27b0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.915+0000 7fce785ae740 1 -- 172.21.15.26:0/2359372250 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce68003dd0 con 0x55aa7e9f27b0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce58c00640 1 -- 172.21.15.26:0/2359372250 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fce5c04e050 con 0x55aa7e9f27b0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce785ae740 10 client.26179 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce785ae740 1 -- 172.21.15.26:0/2359372250 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aa7ea06a80 con 0x55aa7e9f27b0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce58c00640 1 -- 172.21.15.26:0/2359372250 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce68003dd0 con 0x55aa7e9f27b0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce58c00640 1 -- 172.21.15.26:0/2359372250 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fce5c0319e0 con 0x55aa7e9f27b0 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce785ae740 10 client.26179 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce785ae740 10 client.26179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce785ae740 20 client.26179 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce785ae740 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55aa7ea081c0 0x55aa7ea0a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce785ae740 4 client.26179 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce785ae740 1 -- 172.21.15.26:0/2359372250 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa7e962b50 con 0x55aa7ea081c0 2024-06-13T07:09:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.916+0000 7fce6f400640 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55aa7ea081c0 0x55aa7ea0a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.917+0000 7fce6f400640 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55aa7ea081c0 0x55aa7ea0a5b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=812d99b8acda28e1 server_cookie=448d85fd3c7a7f0c in_seq=0 out_seq=0 2024-06-13T07:09:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.917+0000 7fce58c00640 10 client.26179 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce58c00640 1 -- 172.21.15.26:0/2359372250 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aa7e962b50 con 0x55aa7ea081c0 2024-06-13T07:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce58c00640 10 client.26179 handle_command_reply: tid=0 2024-06-13T07:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 client.26179 shutdown 2024-06-13T07:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 -- 172.21.15.26:0/2359372250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f27b0 msgr2=0x55aa7e9fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9fc660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce5c006010 tx=0x7fce5c0042f0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 -- 172.21.15.26:0/2359372250 shutdown_connections 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55aa7ea081c0 0x55aa7ea0a5b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f7920 0x55aa7ea036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9fc660 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 --2- 172.21.15.26:0/2359372250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f1dc0 0x55aa7e9fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 -- 172.21.15.26:0/2359372250 >> 172.21.15.26:0/2359372250 conn(0x55aa7e9b9fb0 msgr2=0x55aa7e9ec9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce58c00640 0 client.26179 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.918+0000 7fce785ae740 1 -- 172.21.15.26:0/2359372250 shutdown_connections 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.919+0000 7fce785ae740 1 -- 172.21.15.26:0/2359372250 wait complete. 2024-06-13T07:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.919+0000 7fce785ae740 20 client.26179 trim_cache size 0 max 16384 2024-06-13T07:09:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.938+0000 7fce785ae740 1 Processor -- start 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.938+0000 7fce785ae740 1 -- start start 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce785ae740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7e9eda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce785ae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce785ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9e8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce785ae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce701af0e0 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce785ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fce701ac9b0 con 0x55aa7e9f27b0 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce785ae740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce701aa800 con 0x55aa7e9b9fb0 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce6f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7e9eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce6e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce6f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7e9eda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40172/0 (socket says 172.21.15.26:40172) 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce6f400640 1 -- 172.21.15.26:0/1729133758 learned_addr learned my addr 172.21.15.26:0/1729133758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.939+0000 7fce6f400640 1 --2- 172.21.15.26:0/1729133758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce6e000640 1 -- 172.21.15.26:0/1729133758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 msgr2=0x55aa7e9eda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce6e000640 1 --2- 172.21.15.26:0/1729133758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7e9eda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce6e000640 1 -- 172.21.15.26:0/1729133758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 msgr2=0x55aa7e9e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce6e000640 1 --2- 172.21.15.26:0/1729133758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9e8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce6e000640 1 -- 172.21.15.26:0/1729133758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7e9e9000 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce6e000640 1 --2- 172.21.15.26:0/1729133758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea03660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0314e0 tx=0x7fce5c005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca60042021e0f153 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce5b400640 1 -- 172.21.15.26:0/1729133758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c035030 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce5b400640 1 -- 172.21.15.26:0/1729133758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c040070 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.940+0000 7fce5b400640 1 -- 172.21.15.26:0/1729133758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c031ae0 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.941+0000 7fce785ae740 1 -- 172.21.15.26:0/1729133758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 msgr2=0x55aa7ea03660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.941+0000 7fce785ae740 1 --2- 172.21.15.26:0/1729133758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea03660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0314e0 tx=0x7fce5c005cb0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.941+0000 7fce785ae740 1 -- 172.21.15.26:0/1729133758 shutdown_connections 2024-06-13T07:09:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.941+0000 7fce785ae740 1 --2- 172.21.15.26:0/1729133758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7e9e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.941+0000 7fce785ae740 1 --2- 172.21.15.26:0/1729133758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea03660 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.941+0000 7fce785ae740 1 --2- 172.21.15.26:0/1729133758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7e9eda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.941+0000 7fce785ae740 1 -- 172.21.15.26:0/1729133758 >> 172.21.15.26:0/1729133758 conn(0x55aa7e9fbce0 msgr2=0x55aa7e9ed320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.941+0000 7fce785ae740 1 -- 172.21.15.26:0/1729133758 shutdown_connections 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.942+0000 7fce785ae740 1 -- 172.21.15.26:0/1729133758 wait complete. 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.942+0000 7fce785ae740 1 -- start start 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.942+0000 7fce785ae740 1 Processor -- start 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce785ae740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7ea00990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce785ae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea00ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce785ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7ea01e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce785ae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce701aae70 con 0x55aa7e9f1dc0 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce785ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fce701ac2e0 con 0x55aa7e9f27b0 2024-06-13T07:09:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce785ae740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce701ab540 con 0x55aa7e9b9fb0 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce785ae740 10 client.0 _ll_register_callbacks cb 0x7fce60005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce6f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7ea01e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce6f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7ea01e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:58738/0 (socket says 172.21.15.26:58738) 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce6f400640 1 -- 172.21.15.26:0/1762414002 learned_addr learned my addr 172.21.15.26:0/1762414002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce6e000640 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea00ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce6f400640 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7ea00990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce6f400640 1 -- 172.21.15.26:0/1762414002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 msgr2=0x55aa7ea00990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.943+0000 7fce6f400640 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7ea00990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce6f400640 1 -- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 msgr2=0x55aa7ea00ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce6f400640 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea00ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce6f400640 1 -- 172.21.15.26:0/1762414002 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa7ea02480 con 0x55aa7e9f27b0 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce6e000640 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea00ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce6f400640 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7ea01e90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fce68011e00 tx=0x7fce6802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1fe22e45f537542 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce58c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:09:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce58c00640 1 -- 172.21.15.26:0/1762414002 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68004b00 con 0x55aa7e9f27b0 2024-06-13T07:09:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce785ae740 1 -- 172.21.15.26:0/1762414002 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5c040070 con 0x55aa7e9f27b0 2024-06-13T07:09:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce58c00640 1 -- 172.21.15.26:0/1762414002 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fce68004ca0 con 0x55aa7e9f27b0 2024-06-13T07:09:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce58c00640 1 -- 172.21.15.26:0/1762414002 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fce68004320 con 0x55aa7e9f27b0 2024-06-13T07:09:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce785ae740 10 client.26147 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.944+0000 7fce785ae740 1 -- 172.21.15.26:0/1762414002 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aa7ea03850 con 0x55aa7e9f27b0 2024-06-13T07:09:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce58c00640 1 -- 172.21.15.26:0/1762414002 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c040070 con 0x55aa7e9f27b0 2024-06-13T07:09:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce58c00640 1 -- 172.21.15.26:0/1762414002 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fce68025040 con 0x55aa7e9f27b0 2024-06-13T07:09:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce785ae740 10 client.26147 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce785ae740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce785ae740 20 client.26147 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce785ae740 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55aa7e9f7920 0x55aa7ea75310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce785ae740 4 client.26147 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce785ae740 1 -- 172.21.15.26:0/1762414002 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa7e95cea0 con 0x55aa7e9f7920 2024-06-13T07:09:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.945+0000 7fce6e000640 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55aa7e9f7920 0x55aa7ea75310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.946+0000 7fce6e000640 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55aa7e9f7920 0x55aa7ea75310 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb89631403a4531 server_cookie=4f136c8bcc2a5c2e in_seq=0 out_seq=0 2024-06-13T07:09:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:04.946+0000 7fce58c00640 10 client.26147 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce58c00640 1 -- 172.21.15.26:0/1762414002 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aa7e95cea0 con 0x55aa7e9f7920 2024-06-13T07:09:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce58c00640 10 client.26147 handle_command_reply: tid=0 2024-06-13T07:09:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 client.26147 shutdown 2024-06-13T07:09:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 -- 172.21.15.26:0/1762414002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 msgr2=0x55aa7ea01e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7ea01e90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fce68011e00 tx=0x7fce6802a040 comp rx=0 tx=0).stop 2024-06-13T07:09:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 -- 172.21.15.26:0/1762414002 shutdown_connections 2024-06-13T07:09:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55aa7e9f7920 0x55aa7ea75310 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aa7e9f27b0 0x55aa7ea01e90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce58c00640 0 client.26147 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa7e9f1dc0 0x55aa7ea00ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 --2- 172.21.15.26:0/1762414002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa7e9b9fb0 0x55aa7ea00990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.032+0000 7fce785ae740 1 -- 172.21.15.26:0/1762414002 >> 172.21.15.26:0/1762414002 conn(0x55aa7e9fbce0 msgr2=0x55aa7e9f0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.033+0000 7fce785ae740 1 -- 172.21.15.26:0/1762414002 shutdown_connections 2024-06-13T07:09:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.033+0000 7fce785ae740 1 -- 172.21.15.26:0/1762414002 wait complete. 2024-06-13T07:09:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.033+0000 7fce785ae740 20 client.26147 trim_cache size 0 max 16384 2024-06-13T07:09:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:09:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:09:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:09:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 -- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fce540a2b80 msgr2=0x7fce540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fce540a2b80 0x7fce540a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9c0 tx=0x7fce600d5530 comp rx=0 tx=0).stop 2024-06-13T07:09:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 -- 172.21.15.26:0/2325353307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce701000b0 msgr2=0x7fce70199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce701000b0 0x7fce70199d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9f0 tx=0x7fce6000edf0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 -- 172.21.15.26:0/2325353307 shutdown_connections 2024-06-13T07:09:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fce540a2b80 0x7fce540a5040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce701013b0 0x7fce701a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce70100a10 0x7fce7019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 --2- 172.21.15.26:0/2325353307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce701000b0 0x7fce70199d50 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9f0 tx=0x7fce6000edf0 comp rx=0 tx=0).stop 2024-06-13T07:09:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 -- 172.21.15.26:0/2325353307 >> 172.21.15.26:0/2325353307 conn(0x7fce700fbc90 msgr2=0x7fce700fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 -- 172.21.15.26:0/2325353307 shutdown_connections 2024-06-13T07:09:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:05.034+0000 7fce6fe00640 1 -- 172.21.15.26:0/2325353307 wait complete. 2024-06-13T07:09:05.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 378492 4.70 MB/sec execute 319 sec latency 78.824 ms 2024-06-13T07:09:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:05 smithi026 ceph-mon[32041]: pgmap v767: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:09:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:05 smithi135 ceph-mon[36276]: pgmap v767: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:09:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:05 smithi136 ceph-mon[37229]: pgmap v767: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:09:06.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 379559 4.72 MB/sec execute 320 sec latency 32.225 ms 2024-06-13T07:09:07.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 379953 4.71 MB/sec execute 321 sec latency 181.911 ms 2024-06-13T07:09:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:07 smithi026 ceph-mon[32041]: pgmap v768: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:09:08.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:07 smithi135 ceph-mon[36276]: pgmap v768: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:09:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:07 smithi136 ceph-mon[37229]: pgmap v768: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:09:08.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 380541 4.69 MB/sec execute 322 sec latency 37.504 ms 2024-06-13T07:09:09.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 381768 4.70 MB/sec execute 323 sec latency 23.078 ms 2024-06-13T07:09:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:10 smithi026 ceph-mon[32041]: pgmap v769: 129 pgs: 129 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-13T07:09:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:10 smithi135 ceph-mon[36276]: pgmap v769: 129 pgs: 129 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-13T07:09:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:10 smithi136 ceph-mon[37229]: pgmap v769: 129 pgs: 129 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-13T07:09:10.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 382770 4.70 MB/sec execute 324 sec latency 20.689 ms 2024-06-13T07:09:11.556 INFO:tasks.workunit.client.0.smithi026.stdout: 1 383492 4.71 MB/sec execute 325 sec latency 18.457 ms 2024-06-13T07:09:12.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:12 smithi135 ceph-mon[36276]: pgmap v770: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-06-13T07:09:12.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:12 smithi136 ceph-mon[37229]: pgmap v770: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-06-13T07:09:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:12 smithi026 ceph-mon[32041]: pgmap v770: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-06-13T07:09:12.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 384028 4.70 MB/sec execute 326 sec latency 66.121 ms 2024-06-13T07:09:13.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 385518 4.70 MB/sec execute 327 sec latency 19.865 ms 2024-06-13T07:09:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:14 smithi135 ceph-mon[36276]: pgmap v771: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-06-13T07:09:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:14 smithi136 ceph-mon[37229]: pgmap v771: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-06-13T07:09:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:14 smithi026 ceph-mon[32041]: pgmap v771: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-06-13T07:09:14.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 386485 4.71 MB/sec execute 328 sec latency 28.060 ms 2024-06-13T07:09:15.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 387065 4.71 MB/sec execute 329 sec latency 21.425 ms 2024-06-13T07:09:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:16 smithi135 ceph-mon[36276]: pgmap v772: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:09:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:16 smithi136 ceph-mon[37229]: pgmap v772: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:09:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:16 smithi026 ceph-mon[32041]: pgmap v772: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:09:16.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 387590 4.70 MB/sec execute 330 sec latency 27.649 ms 2024-06-13T07:09:17.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 389045 4.70 MB/sec execute 331 sec latency 33.502 ms 2024-06-13T07:09:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.188+0000 7fbf4f000640 1 -- 172.21.15.26:0/1738591503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48102340 msgr2=0x7fbf48104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.188+0000 7fbf4f000640 1 --2- 172.21.15.26:0/1738591503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48102340 0x7fbf48104730 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009900 tx=0x7fbf3c02a410 comp rx=0 tx=0).stop 2024-06-13T07:09:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.188+0000 7fbf4f000640 1 -- 172.21.15.26:0/1738591503 shutdown_connections 2024-06-13T07:09:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.188+0000 7fbf4f000640 1 --2- 172.21.15.26:0/1738591503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf481075a0 0x7fbf48109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.188+0000 7fbf4f000640 1 --2- 172.21.15.26:0/1738591503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf48104c70 0x7fbf48107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.188+0000 7fbf4f000640 1 --2- 172.21.15.26:0/1738591503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48102340 0x7fbf48104730 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.188+0000 7fbf4f000640 1 -- 172.21.15.26:0/1738591503 >> 172.21.15.26:0/1738591503 conn(0x7fbf480fbc90 msgr2=0x7fbf480fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.188+0000 7fbf4f000640 1 -- 172.21.15.26:0/1738591503 shutdown_connections 2024-06-13T07:09:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.189+0000 7fbf4f000640 1 -- 172.21.15.26:0/1738591503 wait complete. 2024-06-13T07:09:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.190+0000 7fbf4f000640 1 Processor -- start 2024-06-13T07:09:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.190+0000 7fbf4f000640 1 -- start start 2024-06-13T07:09:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.190+0000 7fbf4f000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf48102340 0x7fbf48199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.190+0000 7fbf4f000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48104c70 0x7fbf4819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.190+0000 7fbf4f000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf481075a0 0x7fbf481a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.190+0000 7fbf4f000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4810d500 con 0x7fbf481075a0 2024-06-13T07:09:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.190+0000 7fbf4f000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4810d380 con 0x7fbf48104c70 2024-06-13T07:09:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.190+0000 7fbf4f000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4810d680 con 0x7fbf48102340 2024-06-13T07:09:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48104c70 0x7fbf4819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf48102340 0x7fbf48199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4e600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf481075a0 0x7fbf481a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48104c70 0x7fbf4819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51494/0 (socket says 172.21.15.26:51494) 2024-06-13T07:09:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf48102340 0x7fbf48199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49652/0 (socket says 172.21.15.26:49652) 2024-06-13T07:09:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 -- 172.21.15.26:0/2697733699 learned_addr learned my addr 172.21.15.26:0/2697733699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 -- 172.21.15.26:0/2697733699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf48102340 msgr2=0x7fbf48199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf48102340 0x7fbf48199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 -- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf481075a0 msgr2=0x7fbf481a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf481075a0 0x7fbf481a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 -- 172.21.15.26:0/2697733699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf481a1990 con 0x7fbf48104c70 2024-06-13T07:09:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48104c70 0x7fbf4819a2b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400e9c0 tx=0x7fbf3400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf46000640 1 -- 172.21.15.26:0/2697733699 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400cd80 con 0x7fbf48104c70 2024-06-13T07:09:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf46000640 1 -- 172.21.15.26:0/2697733699 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf34005500 con 0x7fbf48104c70 2024-06-13T07:09:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.191+0000 7fbf46000640 1 -- 172.21.15.26:0/2697733699 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34010690 con 0x7fbf48104c70 2024-06-13T07:09:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.192+0000 7fbf4f000640 1 -- 172.21.15.26:0/2697733699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf481a1c80 con 0x7fbf48104c70 2024-06-13T07:09:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.192+0000 7fbf4f000640 1 -- 172.21.15.26:0/2697733699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf481a21d0 con 0x7fbf48104c70 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.196+0000 7fbf46000640 1 -- 172.21.15.26:0/2697733699 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf340046c0 con 0x7fbf48104c70 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.196+0000 7fbf53417740 1 Processor -- start 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.196+0000 7fbf53417740 1 -- start start 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.196+0000 7fbf53417740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.196+0000 7fbf53417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aa82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf53417740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa8920 0x55b6d4aaadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf53417740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b6d4a06420 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf53417740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b6d49adc50 con 0x55b6d4aa8920 2024-06-13T07:09:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf53417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b6d4a023e0 con 0x55b6d4aa37b0 2024-06-13T07:09:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aa82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aa82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49656/0 (socket says 172.21.15.26:49656) 2024-06-13T07:09:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa8920 0x55b6d4aaadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4dc00640 1 -- 172.21.15.26:0/4157088566 learned_addr learned my addr 172.21.15.26:0/4157088566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4e600640 1 --2- 172.21.15.26:0/4157088566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4dc00640 1 -- 172.21.15.26:0/4157088566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa8920 msgr2=0x55b6d4aaadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/4157088566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa8920 0x55b6d4aaadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4dc00640 1 -- 172.21.15.26:0/4157088566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa2dc0 msgr2=0x55b6d4aa31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/4157088566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.197+0000 7fbf4dc00640 1 -- 172.21.15.26:0/4157088566 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d471a210 con 0x55b6d4aa37b0 2024-06-13T07:09:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.198+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/4157088566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aa82b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c000c00 tx=0x7fbf3c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e0d3ca8d5e7d0f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.198+0000 7fbf46000640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbf300a2c00 0x7fbf300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.198+0000 7fbf3aa00640 1 -- 172.21.15.26:0/4157088566 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c035040 con 0x55b6d4aa37b0 2024-06-13T07:09:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.198+0000 7fbf46000640 1 -- 172.21.15.26:0/2697733699 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbf340cf790 con 0x7fbf48104c70 2024-06-13T07:09:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.198+0000 7fbf4e600640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbf300a2c00 0x7fbf300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.198+0000 7fbf3aa00640 1 -- 172.21.15.26:0/4157088566 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c005520 con 0x55b6d4aa37b0 2024-06-13T07:09:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.198+0000 7fbf3aa00640 1 -- 172.21.15.26:0/4157088566 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c031910 con 0x55b6d4aa37b0 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 -- 172.21.15.26:0/4157088566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 msgr2=0x55b6d4aa82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 --2- 172.21.15.26:0/4157088566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aa82b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c000c00 tx=0x7fbf3c004040 comp rx=0 tx=0).stop 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf4e600640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbf300a2c00 0x7fbf300a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55b6d4aa4010 tx=0x7fbf40009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 -- 172.21.15.26:0/4157088566 shutdown_connections 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 --2- 172.21.15.26:0/4157088566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa8920 0x55b6d4aaadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 --2- 172.21.15.26:0/4157088566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aa82b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 --2- 172.21.15.26:0/4157088566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 -- 172.21.15.26:0/4157088566 >> 172.21.15.26:0/4157088566 conn(0x55b6d4a6afb0 msgr2=0x55b6d4a9e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 -- 172.21.15.26:0/4157088566 shutdown_connections 2024-06-13T07:09:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.199+0000 7fbf53417740 1 -- 172.21.15.26:0/4157088566 wait complete. 2024-06-13T07:09:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.200+0000 7fbf53417740 1 -- start start 2024-06-13T07:09:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.200+0000 7fbf53417740 1 Processor -- start 2024-06-13T07:09:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf53417740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf53417740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf53417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 0x55b6d4ab4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf53417740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b6d4aab870 con 0x55b6d4aa37b0 2024-06-13T07:09:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf53417740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b6d49c10b0 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf53417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b6d4aab9f0 con 0x55b6d4aa8920 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf53417740 10 client.0 _ll_register_callbacks cb 0x55b6d49acf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51510/0 (socket says 172.21.15.26:51510) 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4dc00640 1 -- 172.21.15.26:0/2659561407 learned_addr learned my addr 172.21.15.26:0/2659561407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4e600640 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 0x55b6d4ab4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4dc00640 1 -- 172.21.15.26:0/2659561407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 msgr2=0x55b6d4ab4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 0x55b6d4ab4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4dc00640 1 -- 172.21.15.26:0/2659561407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 msgr2=0x55b6d4aad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.201+0000 7fbf4dc00640 1 -- 172.21.15.26:0/2659561407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d47e6620 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.202+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aad090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c02aa00 tx=0x7fbf3c004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f8b309a9834c09 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.202+0000 7fbf2f400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.202+0000 7fbf2f400640 1 -- 172.21.15.26:0/2659561407 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c035040 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.202+0000 7fbf2f400640 1 -- 172.21.15.26:0/2659561407 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbf3c004df0 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.202+0000 7fbf53417740 1 -- 172.21.15.26:0/2659561407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf3c005520 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf2f400640 1 -- 172.21.15.26:0/2659561407 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbf3c05a050 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf53417740 10 client.26159 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf53417740 1 -- 172.21.15.26:0/2659561407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b6d4ab7980 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf2f400640 1 -- 172.21.15.26:0/2659561407 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c005520 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf2f400640 1 -- 172.21.15.26:0/2659561407 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbf3c055030 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf53417740 10 client.26159 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf53417740 10 client.26159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf53417740 20 client.26159 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf53417740 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b6d4ab8cd0 0x55b6d4abb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf53417740 4 client.26159 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.203+0000 7fbf53417740 1 -- 172.21.15.26:0/2659561407 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6d4a13b50 con 0x55b6d4ab8cd0 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.204+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b6d4ab8cd0 0x55b6d4abb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.204+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b6d4ab8cd0 0x55b6d4abb0c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa16aa7763cc3fb5 server_cookie=29a876553f406199 in_seq=0 out_seq=0 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.204+0000 7fbf2f400640 10 client.26159 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.205+0000 7fbf2f400640 1 -- 172.21.15.26:0/2659561407 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b6d4a13b50 con 0x55b6d4ab8cd0 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.205+0000 7fbf2f400640 10 client.26159 handle_command_reply: tid=0 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.205+0000 7fbf53417740 1 client.26159 shutdown 2024-06-13T07:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.205+0000 7fbf53417740 1 -- 172.21.15.26:0/2659561407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 msgr2=0x55b6d4aad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.205+0000 7fbf53417740 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aad090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c02aa00 tx=0x7fbf3c004520 comp rx=0 tx=0).stop 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.205+0000 7fbf53417740 1 -- 172.21.15.26:0/2659561407 shutdown_connections 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.205+0000 7fbf53417740 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b6d4ab8cd0 0x55b6d4abb0c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.205+0000 7fbf53417740 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 0x55b6d4ab4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.206+0000 7fbf53417740 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4aad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.206+0000 7fbf53417740 1 --2- 172.21.15.26:0/2659561407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aad090 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.206+0000 7fbf2f400640 0 client.26159 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.206+0000 7fbf53417740 1 -- 172.21.15.26:0/2659561407 >> 172.21.15.26:0/2659561407 conn(0x55b6d4a6afb0 msgr2=0x55b6d4a9d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.206+0000 7fbf53417740 1 -- 172.21.15.26:0/2659561407 shutdown_connections 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.206+0000 7fbf53417740 1 -- 172.21.15.26:0/2659561407 wait complete. 2024-06-13T07:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.206+0000 7fbf53417740 20 client.26159 trim_cache size 0 max 16384 2024-06-13T07:09:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.225+0000 7fbf53417740 1 Processor -- start 2024-06-13T07:09:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.225+0000 7fbf53417740 1 -- start start 2024-06-13T07:09:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.225+0000 7fbf53417740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.225+0000 7fbf53417740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.225+0000 7fbf53417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 0x55b6d4aac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.225+0000 7fbf53417740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481af0c0 con 0x55b6d4aa37b0 2024-06-13T07:09:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.225+0000 7fbf53417740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481ac990 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.225+0000 7fbf53417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481aa7e0 con 0x55b6d4aa8920 2024-06-13T07:09:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51526/0 (socket says 172.21.15.26:51526) 2024-06-13T07:09:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4dc00640 1 -- 172.21.15.26:0/2743748977 learned_addr learned my addr 172.21.15.26:0/2743748977 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4e600640 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 0x55b6d4aac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4d200640 1 -- 172.21.15.26:0/2743748977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 msgr2=0x55b6d4aac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 0x55b6d4aac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4d200640 1 -- 172.21.15.26:0/2743748977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 msgr2=0x55b6d4aa1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4d200640 1 -- 172.21.15.26:0/2743748977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d4a9a000 con 0x55b6d4aa37b0 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.226+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.227+0000 7fbf4d200640 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab45d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf3409b8a0 tx=0x7fbf340d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5093fd76857c4101 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.227+0000 7fbf3a000640 1 -- 172.21.15.26:0/2743748977 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3409b6e0 con 0x55b6d4aa37b0 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.227+0000 7fbf3a000640 1 -- 172.21.15.26:0/2743748977 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3409c2a0 con 0x55b6d4aa37b0 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.227+0000 7fbf3a000640 1 -- 172.21.15.26:0/2743748977 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3409b500 con 0x55b6d4aa37b0 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 -- 172.21.15.26:0/2743748977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 msgr2=0x55b6d4ab45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab45d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf3409b8a0 tx=0x7fbf340d5530 comp rx=0 tx=0).stop 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 -- 172.21.15.26:0/2743748977 shutdown_connections 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa8920 0x55b6d4aac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab45d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 --2- 172.21.15.26:0/2743748977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4aa1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 -- 172.21.15.26:0/2743748977 >> 172.21.15.26:0/2743748977 conn(0x55b6d4a6afb0 msgr2=0x55b6d4aa1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 -- 172.21.15.26:0/2743748977 shutdown_connections 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 -- 172.21.15.26:0/2743748977 wait complete. 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.228+0000 7fbf53417740 1 -- start start 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf53417740 1 Processor -- start 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf53417740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4ab1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf53417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf53417740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa8920 0x55b6d4ab2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf53417740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481aae50 con 0x55b6d4aa8920 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf53417740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481ac2c0 con 0x55b6d4aa2dc0 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf53417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481ab520 con 0x55b6d4aa37b0 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf53417740 10 client.0 _ll_register_callbacks cb 0x7fbf40009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4ab1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf4e600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa8920 0x55b6d4ab2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49680/0 (socket says 172.21.15.26:49680) 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4ab1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51536/0 (socket says 172.21.15.26:51536) 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.229+0000 7fbf4d200640 1 -- 172.21.15.26:0/1649945662 learned_addr learned my addr 172.21.15.26:0/1649945662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf4d200640 1 -- 172.21.15.26:0/1649945662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 msgr2=0x55b6d4ab1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf4d200640 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4ab1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf4d200640 1 -- 172.21.15.26:0/1649945662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa8920 msgr2=0x55b6d4ab2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf4d200640 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa8920 0x55b6d4ab2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf4d200640 1 -- 172.21.15.26:0/1649945662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d4ab3240 con 0x55b6d4aa37b0 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf4d200640 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab1cb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400c6a0 tx=0x7fbf34005c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ee533b137dfb8d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf2f400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:09:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf2f400640 1 -- 172.21.15.26:0/1649945662 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3409b500 con 0x55b6d4aa37b0 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf2f400640 1 -- 172.21.15.26:0/1649945662 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbf34098210 con 0x55b6d4aa37b0 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.230+0000 7fbf53417740 1 -- 172.21.15.26:0/1649945662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf3409c2a0 con 0x55b6d4aa37b0 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf2f400640 1 -- 172.21.15.26:0/1649945662 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbf3409b6e0 con 0x55b6d4aa37b0 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf53417740 10 client.26203 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf53417740 1 -- 172.21.15.26:0/1649945662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b6d4ab4550 con 0x55b6d4aa37b0 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf2f400640 1 -- 172.21.15.26:0/1649945662 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3409c2a0 con 0x55b6d4aa37b0 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf2f400640 1 -- 172.21.15.26:0/1649945662 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbf340a4380 con 0x55b6d4aa37b0 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf53417740 10 client.26203 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf53417740 10 client.26203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf53417740 20 client.26203 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf53417740 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b6d4b23bd0 0x55b6d4b25ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf53417740 4 client.26203 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf53417740 1 -- 172.21.15.26:0/1649945662 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6d4a0dea0 con 0x55b6d4b23bd0 2024-06-13T07:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.231+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b6d4b23bd0 0x55b6d4b25ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.232+0000 7fbf4dc00640 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b6d4b23bd0 0x55b6d4b25ff0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b584992952de862 server_cookie=a8bf1731101d8172 in_seq=0 out_seq=0 2024-06-13T07:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.232+0000 7fbf2f400640 10 client.26203 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:18 smithi135 ceph-mon[36276]: pgmap v773: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-13T07:09:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.317+0000 7fbf2f400640 1 -- 172.21.15.26:0/1649945662 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b6d4a0dea0 con 0x55b6d4b23bd0 2024-06-13T07:09:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.317+0000 7fbf2f400640 10 client.26203 handle_command_reply: tid=0 2024-06-13T07:09:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.317+0000 7fbf53417740 1 client.26203 shutdown 2024-06-13T07:09:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 -- 172.21.15.26:0/1649945662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 msgr2=0x55b6d4ab1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab1cb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400c6a0 tx=0x7fbf34005c30 comp rx=0 tx=0).stop 2024-06-13T07:09:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 -- 172.21.15.26:0/1649945662 shutdown_connections 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b6d4b23bd0 0x55b6d4b25ff0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf2f400640 0 client.26203 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d4aa8920 0x55b6d4ab2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6d4aa37b0 0x55b6d4ab1cb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 --2- 172.21.15.26:0/1649945662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d4aa2dc0 0x55b6d4ab1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 -- 172.21.15.26:0/1649945662 >> 172.21.15.26:0/1649945662 conn(0x55b6d4a6afb0 msgr2=0x55b6d4a9d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 -- 172.21.15.26:0/1649945662 shutdown_connections 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 1 -- 172.21.15.26:0/1649945662 wait complete. 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.318+0000 7fbf53417740 20 client.26203 trim_cache size 0 max 16384 2024-06-13T07:09:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:09:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:09:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:09:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.319+0000 7fbf4f000640 1 -- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbf300a2c00 msgr2=0x7fbf300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.319+0000 7fbf4f000640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbf300a2c00 0x7fbf300a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55b6d4aa4010 tx=0x7fbf40009500 comp rx=0 tx=0).stop 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.319+0000 7fbf4f000640 1 -- 172.21.15.26:0/2697733699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48104c70 msgr2=0x7fbf4819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.319+0000 7fbf4f000640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48104c70 0x7fbf4819a2b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400e9c0 tx=0x7fbf3400ee80 comp rx=0 tx=0).stop 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.320+0000 7fbf4f000640 1 -- 172.21.15.26:0/2697733699 shutdown_connections 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.320+0000 7fbf4f000640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbf300a2c00 0x7fbf300a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.320+0000 7fbf4f000640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf48104c70 0x7fbf4819a2b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.320+0000 7fbf4f000640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf481075a0 0x7fbf481a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.320+0000 7fbf4f000640 1 --2- 172.21.15.26:0/2697733699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf48102340 0x7fbf48199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.320+0000 7fbf4f000640 1 -- 172.21.15.26:0/2697733699 >> 172.21.15.26:0/2697733699 conn(0x7fbf480fbc90 msgr2=0x7fbf48101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.320+0000 7fbf4f000640 1 -- 172.21.15.26:0/2697733699 shutdown_connections 2024-06-13T07:09:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:18.320+0000 7fbf4f000640 1 -- 172.21.15.26:0/2697733699 wait complete. 2024-06-13T07:09:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:18 smithi136 ceph-mon[37229]: pgmap v773: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-13T07:09:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:18 smithi026 ceph-mon[32041]: pgmap v773: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-13T07:09:18.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 390303 4.72 MB/sec execute 332 sec latency 39.512 ms 2024-06-13T07:09:19.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 390693 4.71 MB/sec execute 333 sec latency 26.333 ms 2024-06-13T07:09:20.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:20 smithi136 ceph-mon[37229]: pgmap v774: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:09:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:20 smithi026 ceph-mon[32041]: pgmap v774: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:09:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:20 smithi135 ceph-mon[36276]: pgmap v774: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:09:20.557 INFO:tasks.workunit.client.0.smithi026.stdout: 1 391399 4.70 MB/sec execute 334 sec latency 48.539 ms 2024-06-13T07:09:21.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 392615 4.70 MB/sec execute 335 sec latency 27.681 ms 2024-06-13T07:09:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:22 smithi136 ceph-mon[37229]: pgmap v775: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:09:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:22 smithi026 ceph-mon[32041]: pgmap v775: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:09:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:22 smithi135 ceph-mon[36276]: pgmap v775: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:09:22.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 393728 4.72 MB/sec execute 336 sec latency 44.994 ms 2024-06-13T07:09:23.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:09:23.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:09:23.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:09:23.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 394235 4.71 MB/sec execute 337 sec latency 22.139 ms 2024-06-13T07:09:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:24 smithi136 ceph-mon[37229]: pgmap v776: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:09:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:24 smithi026 ceph-mon[32041]: pgmap v776: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:09:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:24 smithi135 ceph-mon[36276]: pgmap v776: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:09:24.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 395018 4.70 MB/sec execute 338 sec latency 28.208 ms 2024-06-13T07:09:25.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:09:25.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:09:25.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:09:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:09:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:09:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:09:25.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:09:25.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:09:25.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:09:25.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 396253 4.71 MB/sec execute 339 sec latency 26.238 ms 2024-06-13T07:09:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:26 smithi026 ceph-mon[32041]: pgmap v777: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-13T07:09:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:26 smithi135 ceph-mon[36276]: pgmap v777: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-13T07:09:26.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 397394 4.72 MB/sec execute 340 sec latency 20.036 ms 2024-06-13T07:09:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:26 smithi136 ceph-mon[37229]: pgmap v777: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-13T07:09:27.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 397835 4.71 MB/sec execute 341 sec latency 67.821 ms 2024-06-13T07:09:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:28 smithi026 ceph-mon[32041]: pgmap v778: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:09:28.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:28 smithi135 ceph-mon[36276]: pgmap v778: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:09:28.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 398744 4.71 MB/sec execute 342 sec latency 35.971 ms 2024-06-13T07:09:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:28 smithi136 ceph-mon[37229]: pgmap v778: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:09:29.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 399905 4.71 MB/sec execute 343 sec latency 22.073 ms 2024-06-13T07:09:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:30 smithi026 ceph-mon[32041]: pgmap v779: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:09:30.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:30 smithi135 ceph-mon[36276]: pgmap v779: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:09:30.558 INFO:tasks.workunit.client.0.smithi026.stdout: 1 400994 4.72 MB/sec execute 344 sec latency 23.284 ms 2024-06-13T07:09:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:30 smithi136 ceph-mon[37229]: pgmap v779: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 -- 172.21.15.26:0/953221046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74105610 msgr2=0x7f8b7410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 --2- 172.21.15.26:0/953221046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74105610 0x7f8b7410bea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009ca0 tx=0x7f8b6c02a8c0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 -- 172.21.15.26:0/953221046 shutdown_connections 2024-06-13T07:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 --2- 172.21.15.26:0/953221046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b74104c50 0x7f8b741050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 --2- 172.21.15.26:0/953221046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74105610 0x7f8b7410bea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 --2- 172.21.15.26:0/953221046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b74103a60 0x7f8b74103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 -- 172.21.15.26:0/953221046 >> 172.21.15.26:0/953221046 conn(0x7f8b740fefd0 msgr2=0x7f8b74101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 -- 172.21.15.26:0/953221046 shutdown_connections 2024-06-13T07:09:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.476+0000 7f8b79a00640 1 -- 172.21.15.26:0/953221046 wait complete. 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.477+0000 7f8b79a00640 1 Processor -- start 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79a00640 1 -- start start 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74103a60 0x7f8b74199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b74104c50 0x7f8b7419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b74105610 0x7f8b741a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7410f6e0 con 0x7f8b74104c50 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7410f560 con 0x7f8b74105610 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7410f860 con 0x7f8b74103a60 2024-06-13T07:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b74105610 0x7f8b741a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74103a60 0x7f8b74199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b6be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b74104c50 0x7f8b7419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b74105610 0x7f8b741a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:57276/0 (socket says 172.21.15.26:57276) 2024-06-13T07:09:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74103a60 0x7f8b74199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55216/0 (socket says 172.21.15.26:55216) 2024-06-13T07:09:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.478+0000 7f8b79000640 1 -- 172.21.15.26:0/637114023 learned_addr learned my addr 172.21.15.26:0/637114023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.479+0000 7f8b79000640 1 -- 172.21.15.26:0/637114023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74103a60 msgr2=0x7f8b74199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.479+0000 7f8b79000640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74103a60 0x7f8b74199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.479+0000 7f8b79000640 1 -- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b74104c50 msgr2=0x7f8b7419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.479+0000 7f8b79000640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b74104c50 0x7f8b7419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.479+0000 7f8b79000640 1 -- 172.21.15.26:0/637114023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b741a1960 con 0x7f8b74105610 2024-06-13T07:09:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.479+0000 7f8b6be00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b74104c50 0x7f8b7419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.479+0000 7f8b73e00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74103a60 0x7f8b74199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.479+0000 7f8b79000640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b74105610 0x7f8b741a12f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c005e30 tx=0x7f8b6c031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.480+0000 7f8b71600640 1 -- 172.21.15.26:0/637114023 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c003b70 con 0x7f8b74105610 2024-06-13T07:09:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.480+0000 7f8b79a00640 1 -- 172.21.15.26:0/637114023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b741a1bf0 con 0x7f8b74105610 2024-06-13T07:09:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.480+0000 7f8b71600640 1 -- 172.21.15.26:0/637114023 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6c003d10 con 0x7f8b74105610 2024-06-13T07:09:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.480+0000 7f8b71600640 1 -- 172.21.15.26:0/637114023 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c02fb70 con 0x7f8b74105610 2024-06-13T07:09:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.480+0000 7f8b79a00640 1 -- 172.21.15.26:0/637114023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b741a2140 con 0x7f8b74105610 2024-06-13T07:09:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.483+0000 7f8b7de4a740 1 Processor -- start 2024-06-13T07:09:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.483+0000 7f8b71600640 1 -- 172.21.15.26:0/637114023 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b6c02fd10 con 0x7f8b74105610 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.483+0000 7f8b7de4a740 1 -- start start 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 0x55c991e582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 0x55c991e5adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c991db6420 con 0x55c991e537b0 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c991d5dc50 con 0x55c991e58920 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c991db23e0 con 0x55c991e52dc0 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b6be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b6be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55232/0 (socket says 172.21.15.26:55232) 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b6be00640 1 -- 172.21.15.26:0/997323941 learned_addr learned my addr 172.21.15.26:0/997323941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b6be00640 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 0x55c991e5adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.484+0000 7f8b73e00640 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 0x55c991e582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b6be00640 1 -- 172.21.15.26:0/997323941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 msgr2=0x55c991e5adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b6be00640 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 0x55c991e5adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b6be00640 1 -- 172.21.15.26:0/997323941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 msgr2=0x55c991e582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b6be00640 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 0x55c991e582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b6be00640 1 -- 172.21.15.26:0/997323941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c991aca210 con 0x55c991e52dc0 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b6be00640 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e531e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c004650 tx=0x7f8b5c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ac4d8f91824291 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b71600640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b580a2b80 0x7f8b580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b71600640 1 -- 172.21.15.26:0/637114023 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8b6c0f05e0 con 0x7f8b74105610 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b69600640 1 -- 172.21.15.26:0/997323941 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c00fc50 con 0x55c991e52dc0 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b69600640 1 -- 172.21.15.26:0/997323941 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c00fdf0 con 0x55c991e52dc0 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b69600640 1 -- 172.21.15.26:0/997323941 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c00e610 con 0x55c991e52dc0 2024-06-13T07:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.485+0000 7f8b73e00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b580a2b80 0x7f8b580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 -- 172.21.15.26:0/997323941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 msgr2=0x55c991e531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e531e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c004650 tx=0x7f8b5c009b80 comp rx=0 tx=0).stop 2024-06-13T07:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 -- 172.21.15.26:0/997323941 shutdown_connections 2024-06-13T07:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 0x55c991e5adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 0x55c991e582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/997323941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e531e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 -- 172.21.15.26:0/997323941 >> 172.21.15.26:0/997323941 conn(0x55c991e1afb0 msgr2=0x55c991e4e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b73e00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b580a2b80 0x7f8b580a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55c991e53bb0 tx=0x7f8b64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 -- 172.21.15.26:0/997323941 shutdown_connections 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.486+0000 7f8b7de4a740 1 -- 172.21.15.26:0/997323941 wait complete. 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.487+0000 7f8b7de4a740 1 -- start start 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.487+0000 7f8b7de4a740 1 Processor -- start 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e5d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 0x55c991e5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 0x55c991e646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c991e5b8d0 con 0x55c991e537b0 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c991d710b0 con 0x55c991e58920 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c991e5ba50 con 0x55c991e52dc0 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b7de4a740 10 client.0 _ll_register_callbacks cb 0x55c991d5cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e5d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e5d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55244/0 (socket says 172.21.15.26:55244) 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b73e00640 1 -- 172.21.15.26:0/4234145001 learned_addr learned my addr 172.21.15.26:0/4234145001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b6be00640 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 0x55c991e5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b79000640 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 0x55c991e646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b73e00640 1 -- 172.21.15.26:0/4234145001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 msgr2=0x55c991e646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b73e00640 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 0x55c991e646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b73e00640 1 -- 172.21.15.26:0/4234145001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 msgr2=0x55c991e5d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b73e00640 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 0x55c991e5d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.488+0000 7f8b73e00640 1 -- 172.21.15.26:0/4234145001 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c991b96620 con 0x55c991e52dc0 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.489+0000 7f8b73e00640 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e5d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8b64004650 tx=0x7f8b64004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c101870f92a9ac1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.489+0000 7f8b62000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.489+0000 7f8b62000640 1 -- 172.21.15.26:0/4234145001 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400f830 con 0x55c991e52dc0 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.489+0000 7f8b62000640 1 -- 172.21.15.26:0/4234145001 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8b6400f9d0 con 0x55c991e52dc0 2024-06-13T07:09:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.489+0000 7f8b7de4a740 1 -- 172.21.15.26:0/4234145001 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b5c00fdf0 con 0x55c991e52dc0 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.489+0000 7f8b62000640 1 -- 172.21.15.26:0/4234145001 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8b6400fbd0 con 0x55c991e52dc0 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b7de4a740 10 client.26215 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b7de4a740 1 -- 172.21.15.26:0/4234145001 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c991e67b10 con 0x55c991e52dc0 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b62000640 1 -- 172.21.15.26:0/4234145001 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c00fdf0 con 0x55c991e52dc0 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b62000640 1 -- 172.21.15.26:0/4234145001 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8b64027070 con 0x55c991e52dc0 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b7de4a740 10 client.26215 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b7de4a740 10 client.26215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b7de4a740 20 client.26215 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c991e69250 0x55c991e6b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b7de4a740 4 client.26215 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b7de4a740 1 -- 172.21.15.26:0/4234145001 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c991dc3b50 con 0x55c991e69250 2024-06-13T07:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.490+0000 7f8b6be00640 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c991e69250 0x55c991e6b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.491+0000 7f8b6be00640 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c991e69250 0x55c991e6b6c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e490a22814ef2f7 server_cookie=a9acbb4cefc9467e in_seq=0 out_seq=0 2024-06-13T07:09:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.491+0000 7f8b62000640 10 client.26215 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b62000640 1 -- 172.21.15.26:0/4234145001 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c991dc3b50 con 0x55c991e69250 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b62000640 10 client.26215 handle_command_reply: tid=0 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 client.26215 shutdown 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 -- 172.21.15.26:0/4234145001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 msgr2=0x55c991e5d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e5d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8b64004650 tx=0x7f8b64004730 comp rx=0 tx=0).stop 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 -- 172.21.15.26:0/4234145001 shutdown_connections 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c991e69250 0x55c991e6b6c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e58920 0x55c991e646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e537b0 0x55c991e5d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b62000640 0 client.26215 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/4234145001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e52dc0 0x55c991e5d150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 -- 172.21.15.26:0/4234145001 >> 172.21.15.26:0/4234145001 conn(0x55c991e1afb0 msgr2=0x55c991e4d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 -- 172.21.15.26:0/4234145001 shutdown_connections 2024-06-13T07:09:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.492+0000 7f8b7de4a740 1 -- 172.21.15.26:0/4234145001 wait complete. 2024-06-13T07:09:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.493+0000 7f8b7de4a740 20 client.26215 trim_cache size 0 max 16384 2024-06-13T07:09:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.521+0000 7f8b7de4a740 1 Processor -- start 2024-06-13T07:09:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.521+0000 7f8b7de4a740 1 -- start start 2024-06-13T07:09:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e1afb0 0x55c991e51e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e52dc0 0x55c991e64660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e537b0 0x55c991e5cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741af140 con 0x55c991e52dc0 2024-06-13T07:09:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741aca10 con 0x55c991e537b0 2024-06-13T07:09:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741aa860 con 0x55c991e1afb0 2024-06-13T07:09:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e1afb0 0x55c991e51e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e1afb0 0x55c991e51e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55256/0 (socket says 172.21.15.26:55256) 2024-06-13T07:09:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b73e00640 1 -- 172.21.15.26:0/1409289814 learned_addr learned my addr 172.21.15.26:0/1409289814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b79000640 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e537b0 0x55c991e5cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.522+0000 7f8b6be00640 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e52dc0 0x55c991e64660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b73e00640 1 -- 172.21.15.26:0/1409289814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e537b0 msgr2=0x55c991e5cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b73e00640 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e537b0 0x55c991e5cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b73e00640 1 -- 172.21.15.26:0/1409289814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e52dc0 msgr2=0x55c991e64660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b73e00640 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e52dc0 0x55c991e64660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b73e00640 1 -- 172.21.15.26:0/1409289814 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c991e57dc0 con 0x55c991e1afb0 2024-06-13T07:09:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b73e00640 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e1afb0 0x55c991e51e60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8b64000c00 tx=0x7f8b64012fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e81ffbe97747439 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b68c00640 1 -- 172.21.15.26:0/1409289814 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400fdf0 con 0x55c991e1afb0 2024-06-13T07:09:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b68c00640 1 -- 172.21.15.26:0/1409289814 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6400f830 con 0x55c991e1afb0 2024-06-13T07:09:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.523+0000 7f8b68c00640 1 -- 172.21.15.26:0/1409289814 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6401e6d0 con 0x55c991e1afb0 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 -- 172.21.15.26:0/1409289814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e1afb0 msgr2=0x55c991e51e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e1afb0 0x55c991e51e60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8b64000c00 tx=0x7f8b64012fd0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 -- 172.21.15.26:0/1409289814 shutdown_connections 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e537b0 0x55c991e5cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e52dc0 0x55c991e64660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/1409289814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e1afb0 0x55c991e51e60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 -- 172.21.15.26:0/1409289814 >> 172.21.15.26:0/1409289814 conn(0x55c991e543b0 msgr2=0x55c991e51750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 -- 172.21.15.26:0/1409289814 shutdown_connections 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.524+0000 7f8b7de4a740 1 -- 172.21.15.26:0/1409289814 wait complete. 2024-06-13T07:09:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.525+0000 7f8b7de4a740 1 -- start start 2024-06-13T07:09:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.525+0000 7f8b7de4a740 1 Processor -- start 2024-06-13T07:09:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e1afb0 0x55c991e61d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e52dc0 0x55c991e622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b7de4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e537b0 0x55c991e632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741aaed0 con 0x55c991e1afb0 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741ac340 con 0x55c991e52dc0 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b7de4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741ab5a0 con 0x55c991e537b0 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b7de4a740 10 client.0 _ll_register_callbacks cb 0x7f8b64009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e1afb0 0x55c991e61d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e1afb0 0x55c991e61d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54774/0 (socket says 172.21.15.26:54774) 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b73e00640 1 -- 172.21.15.26:0/993077637 learned_addr learned my addr 172.21.15.26:0/993077637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b79000640 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e537b0 0x55c991e632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.526+0000 7f8b6be00640 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e52dc0 0x55c991e622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.527+0000 7f8b73e00640 1 -- 172.21.15.26:0/993077637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e537b0 msgr2=0x55c991e632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.527+0000 7f8b73e00640 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e537b0 0x55c991e632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.527+0000 7f8b73e00640 1 -- 172.21.15.26:0/993077637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e52dc0 msgr2=0x55c991e622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.527+0000 7f8b73e00640 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e52dc0 0x55c991e622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.527+0000 7f8b73e00640 1 -- 172.21.15.26:0/993077637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c991e638b0 con 0x55c991e1afb0 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.527+0000 7f8b73e00640 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e1afb0 0x55c991e61d70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8b64029630 tx=0x7f8b640092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96cc7118353b41 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.527+0000 7f8b62000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.528+0000 7f8b62000640 1 -- 172.21.15.26:0/993077637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64004490 con 0x55c991e1afb0 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.528+0000 7f8b7de4a740 1 -- 172.21.15.26:0/993077637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b6400f830 con 0x55c991e1afb0 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.528+0000 7f8b62000640 1 -- 172.21.15.26:0/993077637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8b64040470 con 0x55c991e1afb0 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.528+0000 7f8b62000640 1 -- 172.21.15.26:0/993077637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8b64049050 con 0x55c991e1afb0 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.528+0000 7f8b7de4a740 10 client.16077 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.528+0000 7f8b7de4a740 1 -- 172.21.15.26:0/993077637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c991ed4000 con 0x55c991e1afb0 2024-06-13T07:09:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.528+0000 7f8b62000640 1 -- 172.21.15.26:0/993077637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6400f830 con 0x55c991e1afb0 2024-06-13T07:09:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.529+0000 7f8b62000640 1 -- 172.21.15.26:0/993077637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8b64027040 con 0x55c991e1afb0 2024-06-13T07:09:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.529+0000 7f8b7de4a740 10 client.16077 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.529+0000 7f8b7de4a740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.529+0000 7f8b7de4a740 20 client.16077 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.529+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c991e58920 0x55c991ed6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.529+0000 7f8b7de4a740 4 client.16077 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.529+0000 7f8b7de4a740 1 -- 172.21.15.26:0/993077637 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c991dbdea0 con 0x55c991e58920 2024-06-13T07:09:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.529+0000 7f8b6be00640 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c991e58920 0x55c991ed6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.530+0000 7f8b6be00640 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c991e58920 0x55c991ed6980 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fbfe286de62f23e server_cookie=a8613a34ece6ac3a in_seq=0 out_seq=0 2024-06-13T07:09:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.530+0000 7f8b62000640 10 client.16077 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:31.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 401413 4.71 MB/sec execute 345 sec latency 20.435 ms 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.630+0000 7f8b62000640 1 -- 172.21.15.26:0/993077637 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c991dbdea0 con 0x55c991e58920 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.630+0000 7f8b62000640 10 client.16077 handle_command_reply: tid=0 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.630+0000 7f8b7de4a740 1 client.16077 shutdown 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 -- 172.21.15.26:0/993077637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e1afb0 msgr2=0x55c991e61d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e1afb0 0x55c991e61d70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8b64029630 tx=0x7f8b640092b0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 -- 172.21.15.26:0/993077637 shutdown_connections 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c991e58920 0x55c991ed6980 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c991e52dc0 0x55c991e622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c991e537b0 0x55c991e632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b62000640 0 client.16077 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 --2- 172.21.15.26:0/993077637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c991e1afb0 0x55c991e61d70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 -- 172.21.15.26:0/993077637 >> 172.21.15.26:0/993077637 conn(0x55c991e543b0 msgr2=0x55c991e5ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 -- 172.21.15.26:0/993077637 shutdown_connections 2024-06-13T07:09:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 1 -- 172.21.15.26:0/993077637 wait complete. 2024-06-13T07:09:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.631+0000 7f8b7de4a740 20 client.16077 trim_cache size 0 max 16384 2024-06-13T07:09:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:09:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:09:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:09:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 -- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b580a2b80 msgr2=0x7f8b580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b580a2b80 0x7f8b580a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55c991e53bb0 tx=0x7f8b64005f90 comp rx=0 tx=0).stop 2024-06-13T07:09:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 -- 172.21.15.26:0/637114023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b74105610 msgr2=0x7f8b741a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b74105610 0x7f8b741a12f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c005e30 tx=0x7f8b6c031ee0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 -- 172.21.15.26:0/637114023 shutdown_connections 2024-06-13T07:09:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8b580a2b80 0x7f8b580a5040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b74105610 0x7f8b741a12f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b74104c50 0x7f8b7419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 --2- 172.21.15.26:0/637114023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b74103a60 0x7f8b74199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 -- 172.21.15.26:0/637114023 >> 172.21.15.26:0/637114023 conn(0x7f8b740fefd0 msgr2=0x7f8b74107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 -- 172.21.15.26:0/637114023 shutdown_connections 2024-06-13T07:09:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:31.632+0000 7f8b79a00640 1 -- 172.21.15.26:0/637114023 wait complete. 2024-06-13T07:09:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:32 smithi026 ceph-mon[32041]: pgmap v780: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-13T07:09:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:32 smithi135 ceph-mon[36276]: pgmap v780: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-13T07:09:32.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 402243 4.71 MB/sec execute 346 sec latency 54.423 ms 2024-06-13T07:09:32.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:32 smithi136 ceph-mon[37229]: pgmap v780: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-13T07:09:33.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 403635 4.72 MB/sec execute 347 sec latency 34.934 ms 2024-06-13T07:09:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:34 smithi026 ceph-mon[32041]: pgmap v781: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-06-13T07:09:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:34 smithi135 ceph-mon[36276]: pgmap v781: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-06-13T07:09:34.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 404629 4.73 MB/sec execute 348 sec latency 24.265 ms 2024-06-13T07:09:34.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:34 smithi136 ceph-mon[37229]: pgmap v781: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-06-13T07:09:35.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 404971 4.71 MB/sec execute 349 sec latency 31.293 ms 2024-06-13T07:09:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:36 smithi026 ceph-mon[32041]: pgmap v782: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-13T07:09:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:36 smithi135 ceph-mon[36276]: pgmap v782: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-13T07:09:36.559 INFO:tasks.workunit.client.0.smithi026.stdout: 1 405789 4.71 MB/sec execute 350 sec latency 31.713 ms 2024-06-13T07:09:36.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:36 smithi136 ceph-mon[37229]: pgmap v782: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-13T07:09:37.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 406892 4.71 MB/sec execute 351 sec latency 26.512 ms 2024-06-13T07:09:38.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:38 smithi026 ceph-mon[32041]: pgmap v783: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:09:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:38 smithi135 ceph-mon[36276]: pgmap v783: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:09:38.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 408157 4.73 MB/sec execute 352 sec latency 31.326 ms 2024-06-13T07:09:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:38 smithi136 ceph-mon[37229]: pgmap v783: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-13T07:09:39.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 408483 4.71 MB/sec execute 353 sec latency 28.193 ms 2024-06-13T07:09:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:40 smithi026 ceph-mon[32041]: pgmap v784: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:09:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:40 smithi135 ceph-mon[36276]: pgmap v784: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:09:40.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 409137 4.71 MB/sec execute 354 sec latency 28.281 ms 2024-06-13T07:09:40.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:40 smithi136 ceph-mon[37229]: pgmap v784: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:09:41.562 INFO:tasks.workunit.client.0.smithi026.stdout: 1 410356 4.71 MB/sec execute 355 sec latency 36.661 ms 2024-06-13T07:09:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:42 smithi026 ceph-mon[32041]: pgmap v785: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:09:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:42 smithi135 ceph-mon[36276]: pgmap v785: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:09:42.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 411297 4.71 MB/sec execute 356 sec latency 24.124 ms 2024-06-13T07:09:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:42 smithi136 ceph-mon[37229]: pgmap v785: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:09:43.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 411936 4.71 MB/sec execute 357 sec latency 64.626 ms 2024-06-13T07:09:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:44 smithi026 ceph-mon[32041]: pgmap v786: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:09:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:44 smithi135 ceph-mon[36276]: pgmap v786: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:09:44.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 412627 4.71 MB/sec execute 358 sec latency 34.488 ms 2024-06-13T07:09:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:44 smithi136 ceph-mon[37229]: pgmap v786: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:09:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 -- 172.21.15.26:0/2500536730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff4806a7d0 msgr2=0x7eff4810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 --2- 172.21.15.26:0/2500536730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff4806a7d0 0x7eff4810b4b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009900 tx=0x7eff3c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 -- 172.21.15.26:0/2500536730 shutdown_connections 2024-06-13T07:09:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 --2- 172.21.15.26:0/2500536730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff4810ba80 0x7eff4810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 --2- 172.21.15.26:0/2500536730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4810e500 0x7eff481109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 --2- 172.21.15.26:0/2500536730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff4806a7d0 0x7eff4810b4b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 -- 172.21.15.26:0/2500536730 >> 172.21.15.26:0/2500536730 conn(0x7eff4806ebd0 msgr2=0x7eff48071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 -- 172.21.15.26:0/2500536730 shutdown_connections 2024-06-13T07:09:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.800+0000 7eff4f600640 1 -- 172.21.15.26:0/2500536730 wait complete. 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.801+0000 7eff4f600640 1 Processor -- start 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4f600640 1 -- start start 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4f600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff4806a7d0 0x7eff481a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4f600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff4810ba80 0x7eff481a6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4f600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4810e500 0x7eff481ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4f600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7eff48113ce0 con 0x7eff4806a7d0 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4f600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7eff48113b60 con 0x7eff4810ba80 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4f600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7eff48113e60 con 0x7eff4810e500 2024-06-13T07:09:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff47e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff4810ba80 0x7eff481a6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff4806a7d0 0x7eff481a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff4d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4810e500 0x7eff481ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff47e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff4810ba80 0x7eff481a6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56664/0 (socket says 172.21.15.26:56664) 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.802+0000 7eff47e00640 1 -- 172.21.15.26:0/569832312 learned_addr learned my addr 172.21.15.26:0/569832312 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff4d600640 1 -- 172.21.15.26:0/569832312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff4810ba80 msgr2=0x7eff481a6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff4d600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff4810ba80 0x7eff481a6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff4d600640 1 -- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff4806a7d0 msgr2=0x7eff481a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff4d600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff4806a7d0 0x7eff481a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff4d600640 1 -- 172.21.15.26:0/569832312 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff481ae6b0 con 0x7eff4810e500 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff47e00640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff4810ba80 0x7eff481a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff4d600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4810e500 0x7eff481ae040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff40009fd0 tx=0x7eff4000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff45600640 1 -- 172.21.15.26:0/569832312 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4000ce20 con 0x7eff4810e500 2024-06-13T07:09:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff45600640 1 -- 172.21.15.26:0/569832312 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff40005020 con 0x7eff4810e500 2024-06-13T07:09:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff4f600640 1 -- 172.21.15.26:0/569832312 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff481ae9a0 con 0x7eff4810e500 2024-06-13T07:09:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.803+0000 7eff45600640 1 -- 172.21.15.26:0/569832312 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40018760 con 0x7eff4810e500 2024-06-13T07:09:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.804+0000 7eff4f600640 1 -- 172.21.15.26:0/569832312 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff481aeea0 con 0x7eff4810e500 2024-06-13T07:09:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.807+0000 7eff45600640 1 -- 172.21.15.26:0/569832312 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff400051d0 con 0x7eff4810e500 2024-06-13T07:09:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.808+0000 7eff525cb740 1 Processor -- start 2024-06-13T07:09:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.808+0000 7eff525cb740 1 -- start start 2024-06-13T07:09:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.808+0000 7eff525cb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 0x562b0f736020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.808+0000 7eff525cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f73b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.808+0000 7eff525cb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f73b760 0x562b0f73dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.808+0000 7eff525cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562b0f699420 con 0x562b0f7365f0 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.808+0000 7eff525cb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562b0f640c50 con 0x562b0f73b760 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.808+0000 7eff525cb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562b0f6953e0 con 0x562b0f735c00 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff45600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff300a2c30 0x7eff300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f73b760 0x562b0f73dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f73b760 0x562b0f73dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56674/0 (socket says 172.21.15.26:56674) 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 -- 172.21.15.26:0/2964832486 learned_addr learned my addr 172.21.15.26:0/2964832486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 0x562b0f736020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff4cc00640 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f73b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff45600640 1 -- 172.21.15.26:0/569832312 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7eff400cfb60 con 0x7eff4810e500 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff4d600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff300a2c30 0x7eff300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 -- 172.21.15.26:0/2964832486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 msgr2=0x562b0f736020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 0x562b0f736020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 -- 172.21.15.26:0/2964832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 msgr2=0x562b0f73b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f73b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 -- 172.21.15.26:0/2964832486 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0f3ad210 con 0x562b0f73b760 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff4cc00640 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f73b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.809+0000 7eff47e00640 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f73b760 0x562b0f73dbf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7eff34009fd0 tx=0x7eff34009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd21bd658fd3be8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.810+0000 7eff39c00640 1 -- 172.21.15.26:0/2964832486 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3400fc20 con 0x562b0f73b760 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.810+0000 7eff39c00640 1 -- 172.21.15.26:0/2964832486 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff3400fdc0 con 0x562b0f73b760 2024-06-13T07:09:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.810+0000 7eff39c00640 1 -- 172.21.15.26:0/2964832486 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34004a00 con 0x562b0f73b760 2024-06-13T07:09:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.810+0000 7eff4d600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff300a2c30 0x7eff300a50f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7eff4009bc70 tx=0x7eff4000c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 -- 172.21.15.26:0/2964832486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f73b760 msgr2=0x562b0f73dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f73b760 0x562b0f73dbf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7eff34009fd0 tx=0x7eff34009560 comp rx=0 tx=0).stop 2024-06-13T07:09:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 -- 172.21.15.26:0/2964832486 shutdown_connections 2024-06-13T07:09:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f73b760 0x562b0f73dbf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f73b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 --2- 172.21.15.26:0/2964832486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 0x562b0f736020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 -- 172.21.15.26:0/2964832486 >> 172.21.15.26:0/2964832486 conn(0x562b0f6fdfb0 msgr2=0x562b0f731680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 -- 172.21.15.26:0/2964832486 shutdown_connections 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.811+0000 7eff525cb740 1 -- 172.21.15.26:0/2964832486 wait complete. 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 1 -- start start 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 1 Processor -- start 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 0x562b0f73fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f7365f0 0x562b0f7403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 0x562b0f747400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562b0f73e710 con 0x562b0f73b760 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562b0f6540b0 con 0x562b0f7365f0 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562b0f73e890 con 0x562b0f735c00 2024-06-13T07:09:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.812+0000 7eff525cb740 10 client.0 _ll_register_callbacks cb 0x562b0f63ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f7365f0 0x562b0f7403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f7365f0 0x562b0f7403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56680/0 (socket says 172.21.15.26:56680) 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 -- 172.21.15.26:0/1227319927 learned_addr learned my addr 172.21.15.26:0/1227319927 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff47e00640 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 0x562b0f73fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff47e00640 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 0x562b0f747400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 -- 172.21.15.26:0/1227319927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 msgr2=0x562b0f73fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 0x562b0f73fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 -- 172.21.15.26:0/1227319927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 msgr2=0x562b0f747400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 0x562b0f747400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 -- 172.21.15.26:0/1227319927 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0f479620 con 0x562b0f7365f0 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff4cc00640 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f7365f0 0x562b0f7403c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7eff3c031440 tx=0x7eff3c0043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e11bccf9cbf0a5a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.813+0000 7eff2e000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.814+0000 7eff2e000640 1 -- 172.21.15.26:0/1227319927 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c035040 con 0x562b0f7365f0 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.814+0000 7eff2e000640 1 -- 172.21.15.26:0/1227319927 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7eff3c005450 con 0x562b0f7365f0 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.814+0000 7eff525cb740 1 -- 172.21.15.26:0/1227319927 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff3400fdc0 con 0x562b0f7365f0 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.814+0000 7eff2e000640 1 -- 172.21.15.26:0/1227319927 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff3c005650 con 0x562b0f7365f0 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.814+0000 7eff525cb740 10 client.26195 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.814+0000 7eff525cb740 1 -- 172.21.15.26:0/1227319927 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562b0f74a7e0 con 0x562b0f7365f0 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.814+0000 7eff2e000640 1 -- 172.21.15.26:0/1227319927 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff3400fdc0 con 0x562b0f7365f0 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.815+0000 7eff2e000640 1 -- 172.21.15.26:0/1227319927 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7eff3c031be0 con 0x562b0f7365f0 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.815+0000 7eff525cb740 10 client.26195 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.815+0000 7eff525cb740 10 client.26195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.815+0000 7eff525cb740 20 client.26195 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.815+0000 7eff525cb740 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b0f74bec0 0x562b0f74e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.815+0000 7eff525cb740 4 client.26195 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.815+0000 7eff525cb740 1 -- 172.21.15.26:0/1227319927 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b0f6a6b50 con 0x562b0f74bec0 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.815+0000 7eff47e00640 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b0f74bec0 0x562b0f74e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.816+0000 7eff2e000640 10 client.26195 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.816+0000 7eff47e00640 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b0f74bec0 0x562b0f74e2d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e182bb8f19799c server_cookie=8b82305972622104 in_seq=0 out_seq=0 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff2e000640 1 -- 172.21.15.26:0/1227319927 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b0f6a6b50 con 0x562b0f74bec0 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff2e000640 10 client.26195 handle_command_reply: tid=0 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 client.26195 shutdown 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 -- 172.21.15.26:0/1227319927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f7365f0 msgr2=0x562b0f7403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f7365f0 0x562b0f7403c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7eff3c031440 tx=0x7eff3c0043c0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 -- 172.21.15.26:0/1227319927 shutdown_connections 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b0f74bec0 0x562b0f74e2d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f7365f0 0x562b0f7403c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 0x562b0f747400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 --2- 172.21.15.26:0/1227319927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f735c00 0x562b0f73fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.817+0000 7eff525cb740 1 -- 172.21.15.26:0/1227319927 >> 172.21.15.26:0/1227319927 conn(0x562b0f6fdfb0 msgr2=0x562b0f730840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.818+0000 7eff2e000640 0 client.26195 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.818+0000 7eff525cb740 1 -- 172.21.15.26:0/1227319927 shutdown_connections 2024-06-13T07:09:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.818+0000 7eff525cb740 1 -- 172.21.15.26:0/1227319927 wait complete. 2024-06-13T07:09:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.818+0000 7eff525cb740 20 client.26195 trim_cache size 0 max 16384 2024-06-13T07:09:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.838+0000 7eff525cb740 1 Processor -- start 2024-06-13T07:09:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.838+0000 7eff525cb740 1 -- start start 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff525cb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f74e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff525cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f7473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff525cb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f73b760 0x562b0f73fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff525cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7eff481bbdd0 con 0x562b0f7365f0 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff525cb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7eff481b96a0 con 0x562b0f735c00 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff525cb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7eff481b74f0 con 0x562b0f73b760 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff47e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f74e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff47e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f74e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56690/0 (socket says 172.21.15.26:56690) 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff47e00640 1 -- 172.21.15.26:0/164866114 learned_addr learned my addr 172.21.15.26:0/164866114 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff47e00640 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f73b760 0x562b0f73fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.839+0000 7eff4cc00640 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f7473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.840+0000 7eff47e00640 1 -- 172.21.15.26:0/164866114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f73b760 msgr2=0x562b0f73fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.840+0000 7eff47e00640 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f73b760 0x562b0f73fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.840+0000 7eff47e00640 1 -- 172.21.15.26:0/164866114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 msgr2=0x562b0f7473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.840+0000 7eff47e00640 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f7473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.840+0000 7eff47e00640 1 -- 172.21.15.26:0/164866114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0f7369d0 con 0x562b0f735c00 2024-06-13T07:09:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.840+0000 7eff47e00640 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f74e2b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7eff34011c60 tx=0x7eff34009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0e4f63e0c344eaf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.840+0000 7eff39200640 1 -- 172.21.15.26:0/164866114 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34004220 con 0x562b0f735c00 2024-06-13T07:09:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.841+0000 7eff39200640 1 -- 172.21.15.26:0/164866114 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff34029040 con 0x562b0f735c00 2024-06-13T07:09:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.841+0000 7eff39200640 1 -- 172.21.15.26:0/164866114 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3402b550 con 0x562b0f735c00 2024-06-13T07:09:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 -- 172.21.15.26:0/164866114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 msgr2=0x562b0f74e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f74e2b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7eff34011c60 tx=0x7eff34009810 comp rx=0 tx=0).stop 2024-06-13T07:09:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 -- 172.21.15.26:0/164866114 shutdown_connections 2024-06-13T07:09:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f73b760 0x562b0f73fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f7365f0 0x562b0f7473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 --2- 172.21.15.26:0/164866114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f74e2b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 -- 172.21.15.26:0/164866114 >> 172.21.15.26:0/164866114 conn(0x562b0f6fdfb0 msgr2=0x562b0f731100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 -- 172.21.15.26:0/164866114 shutdown_connections 2024-06-13T07:09:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 -- 172.21.15.26:0/164866114 wait complete. 2024-06-13T07:09:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.842+0000 7eff525cb740 1 -- start start 2024-06-13T07:09:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff525cb740 1 Processor -- start 2024-06-13T07:09:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff525cb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f7441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff525cb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f7365f0 0x562b0f744700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff525cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 0x562b0f7b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff525cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7eff481b7b60 con 0x562b0f73b760 2024-06-13T07:09:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff525cb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7eff481b8fd0 con 0x562b0f735c00 2024-06-13T07:09:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff525cb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7eff481b8230 con 0x562b0f7365f0 2024-06-13T07:09:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff525cb740 10 client.0 _ll_register_callbacks cb 0x7eff400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f7365f0 0x562b0f744700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f7365f0 0x562b0f744700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40752/0 (socket says 172.21.15.26:40752) 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff4cc00640 1 -- 172.21.15.26:0/3456457761 learned_addr learned my addr 172.21.15.26:0/3456457761 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.843+0000 7eff47e00640 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f7441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff47e00640 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 0x562b0f7b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff4cc00640 1 -- 172.21.15.26:0/3456457761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 msgr2=0x562b0f7441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff4cc00640 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f7441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff4cc00640 1 -- 172.21.15.26:0/3456457761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 msgr2=0x562b0f7b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff4cc00640 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 0x562b0f7b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff4cc00640 1 -- 172.21.15.26:0/3456457761 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0f7b50e0 con 0x562b0f7365f0 2024-06-13T07:09:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff4cc00640 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f7365f0 0x562b0f744700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0037b0 tx=0x7eff3c003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7781024b914ac1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff2e000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:09:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.844+0000 7eff2e000640 1 -- 172.21.15.26:0/3456457761 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c056020 con 0x562b0f7365f0 2024-06-13T07:09:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff525cb740 1 -- 172.21.15.26:0/3456457761 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff34029040 con 0x562b0f7365f0 2024-06-13T07:09:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff2e000640 1 -- 172.21.15.26:0/3456457761 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7eff3c004770 con 0x562b0f7365f0 2024-06-13T07:09:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff2e000640 1 -- 172.21.15.26:0/3456457761 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff3c035070 con 0x562b0f7365f0 2024-06-13T07:09:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff525cb740 10 client.26239 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff525cb740 1 -- 172.21.15.26:0/3456457761 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562b0f7b6350 con 0x562b0f7365f0 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff2e000640 1 -- 172.21.15.26:0/3456457761 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff34029040 con 0x562b0f7365f0 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff2e000640 1 -- 172.21.15.26:0/3456457761 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7eff3c031b40 con 0x562b0f7365f0 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff525cb740 10 client.26239 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff525cb740 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff525cb740 20 client.26239 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.845+0000 7eff525cb740 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b0f7b6d80 0x562b0f7b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.846+0000 7eff525cb740 4 client.26239 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.846+0000 7eff525cb740 1 -- 172.21.15.26:0/3456457761 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b0f6a0ea0 con 0x562b0f7b6d80 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.846+0000 7eff47e00640 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b0f7b6d80 0x562b0f7b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.846+0000 7eff47e00640 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b0f7b6d80 0x562b0f7b91a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef463ce51b7a90c4 server_cookie=95be4f9b2bc1fa98 in_seq=0 out_seq=0 2024-06-13T07:09:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.846+0000 7eff2e000640 10 client.26239 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff2e000640 1 -- 172.21.15.26:0/3456457761 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562b0f6a0ea0 con 0x562b0f7b6d80 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff2e000640 10 client.26239 handle_command_reply: tid=0 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 client.26239 shutdown 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 -- 172.21.15.26:0/3456457761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f7365f0 msgr2=0x562b0f744700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f7365f0 0x562b0f744700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0037b0 tx=0x7eff3c003f00 comp rx=0 tx=0).stop 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 -- 172.21.15.26:0/3456457761 shutdown_connections 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b0f7b6d80 0x562b0f7b91a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b0f73b760 0x562b0f7b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b0f7365f0 0x562b0f744700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 --2- 172.21.15.26:0/3456457761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b0f735c00 0x562b0f7441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff2e000640 0 client.26239 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 -- 172.21.15.26:0/3456457761 >> 172.21.15.26:0/3456457761 conn(0x562b0f6fdfb0 msgr2=0x562b0f73fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 -- 172.21.15.26:0/3456457761 shutdown_connections 2024-06-13T07:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 1 -- 172.21.15.26:0/3456457761 wait complete. 2024-06-13T07:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.933+0000 7eff525cb740 20 client.26239 trim_cache size 0 max 16384 2024-06-13T07:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 -- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff300a2c30 msgr2=0x7eff300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff300a2c30 0x7eff300a50f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7eff4009bc70 tx=0x7eff4000c890 comp rx=0 tx=0).stop 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 -- 172.21.15.26:0/569832312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4810e500 msgr2=0x7eff481ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4810e500 0x7eff481ae040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff40009fd0 tx=0x7eff4000ef10 comp rx=0 tx=0).stop 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 -- 172.21.15.26:0/569832312 shutdown_connections 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff300a2c30 0x7eff300a50f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff4810ba80 0x7eff481a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4810e500 0x7eff481ae040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 --2- 172.21.15.26:0/569832312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff4806a7d0 0x7eff481a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 -- 172.21.15.26:0/569832312 >> 172.21.15.26:0/569832312 conn(0x7eff4806ebd0 msgr2=0x7eff48109a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 -- 172.21.15.26:0/569832312 shutdown_connections 2024-06-13T07:09:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:44.934+0000 7eff4f600640 1 -- 172.21.15.26:0/569832312 wait complete. 2024-06-13T07:09:45.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 413947 4.71 MB/sec execute 359 sec latency 55.143 ms 2024-06-13T07:09:46.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:46 smithi026 ceph-mon[32041]: pgmap v787: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:09:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:46 smithi135 ceph-mon[36276]: pgmap v787: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:09:46.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 415002 4.72 MB/sec execute 360 sec latency 34.252 ms 2024-06-13T07:09:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:46 smithi136 ceph-mon[37229]: pgmap v787: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:09:47.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 415460 4.71 MB/sec execute 361 sec latency 23.356 ms 2024-06-13T07:09:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:48 smithi026 ceph-mon[32041]: pgmap v788: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:09:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:48 smithi135 ceph-mon[36276]: pgmap v788: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:09:48.563 INFO:tasks.workunit.client.0.smithi026.stdout: 1 416164 4.71 MB/sec execute 362 sec latency 53.169 ms 2024-06-13T07:09:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:48 smithi136 ceph-mon[37229]: pgmap v788: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:09:49.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 417442 4.71 MB/sec execute 363 sec latency 39.633 ms 2024-06-13T07:09:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:50 smithi026 ceph-mon[32041]: pgmap v789: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:09:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:50 smithi135 ceph-mon[36276]: pgmap v789: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:09:50.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 418397 4.72 MB/sec execute 364 sec latency 25.231 ms 2024-06-13T07:09:50.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:50 smithi136 ceph-mon[37229]: pgmap v789: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:09:51.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 418986 4.72 MB/sec execute 365 sec latency 19.388 ms 2024-06-13T07:09:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:52 smithi026 ceph-mon[32041]: pgmap v790: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-06-13T07:09:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:52 smithi135 ceph-mon[36276]: pgmap v790: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-06-13T07:09:52.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 419507 4.70 MB/sec execute 366 sec latency 34.302 ms 2024-06-13T07:09:52.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:52 smithi136 ceph-mon[37229]: pgmap v790: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-06-13T07:09:53.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 420931 4.71 MB/sec execute 367 sec latency 23.408 ms 2024-06-13T07:09:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:54 smithi026 ceph-mon[32041]: pgmap v791: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-06-13T07:09:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:54 smithi135 ceph-mon[36276]: pgmap v791: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-06-13T07:09:54.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 421928 4.72 MB/sec execute 368 sec latency 25.347 ms 2024-06-13T07:09:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:54 smithi136 ceph-mon[37229]: pgmap v791: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-06-13T07:09:55.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 422499 4.72 MB/sec execute 369 sec latency 24.228 ms 2024-06-13T07:09:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:56 smithi135 ceph-mon[36276]: pgmap v792: 129 pgs: 129 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:09:56.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 423045 4.71 MB/sec execute 370 sec latency 24.521 ms 2024-06-13T07:09:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:56 smithi136 ceph-mon[37229]: pgmap v792: 129 pgs: 129 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:09:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:56 smithi026 ceph-mon[32041]: pgmap v792: 129 pgs: 129 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:09:57.564 INFO:tasks.workunit.client.0.smithi026.stdout: 1 424224 4.71 MB/sec execute 371 sec latency 26.860 ms 2024-06-13T07:09:57.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:57 smithi136 ceph-mon[37229]: pgmap v793: 129 pgs: 129 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:09:57.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:57 smithi026 ceph-mon[32041]: pgmap v793: 129 pgs: 129 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:09:57.805 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:57 smithi135 ceph-mon[36276]: pgmap v793: 129 pgs: 129 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:09:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.135+0000 7f24c7200640 1 -- 172.21.15.26:0/2480096050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 msgr2=0x7f24c010bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.135+0000 7f24c7200640 1 --2- 172.21.15.26:0/2480096050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 0x7f24c010bd30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009be0 tx=0x7f24b802a750 comp rx=0 tx=0).stop 2024-06-13T07:09:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.136+0000 7f24c7200640 1 -- 172.21.15.26:0/2480096050 shutdown_connections 2024-06-13T07:09:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.136+0000 7f24c7200640 1 --2- 172.21.15.26:0/2480096050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c00743a0 0x7f24c0074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.136+0000 7f24c7200640 1 --2- 172.21.15.26:0/2480096050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 0x7f24c010bd30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.136+0000 7f24c7200640 1 --2- 172.21.15.26:0/2480096050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c0075980 0x7f24c0073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.136+0000 7f24c7200640 1 -- 172.21.15.26:0/2480096050 >> 172.21.15.26:0/2480096050 conn(0x7f24c00fcf90 msgr2=0x7f24c00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.136+0000 7f24c7200640 1 -- 172.21.15.26:0/2480096050 shutdown_connections 2024-06-13T07:09:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.136+0000 7f24c7200640 1 -- 172.21.15.26:0/2480096050 wait complete. 2024-06-13T07:09:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.137+0000 7f24c7200640 1 Processor -- start 2024-06-13T07:09:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.137+0000 7f24c7200640 1 -- start start 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.137+0000 7f24c7200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c00743a0 0x7f24c0072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.137+0000 7f24c7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c0075980 0x7f24c0073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.137+0000 7f24c7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 0x7f24c0071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.137+0000 7f24c7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24c006be10 con 0x7f24c0075980 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.137+0000 7f24c7200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24c006bc90 con 0x7f24c00743a0 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.137+0000 7f24c7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f24c006bf90 con 0x7f24c0109890 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c00743a0 0x7f24c0072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c00743a0 0x7f24c0072ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39952/0 (socket says 172.21.15.26:39952) 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 -- 172.21.15.26:0/813522681 learned_addr learned my addr 172.21.15.26:0/813522681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24c5200640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 0x7f24c0071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bf400640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c0075980 0x7f24c0073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 -- 172.21.15.26:0/813522681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 msgr2=0x7f24c0071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 0x7f24c0071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 -- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c0075980 msgr2=0x7f24c0073430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c0075980 0x7f24c0073430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 -- 172.21.15.26:0/813522681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c0071c60 con 0x7f24c00743a0 2024-06-13T07:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24c5200640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 0x7f24c0071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bf400640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c0075980 0x7f24c0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.138+0000 7f24bfe00640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c00743a0 0x7f24c0072ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f24b000e9f0 tx=0x7f24b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.139+0000 7f24bcc00640 1 -- 172.21.15.26:0/813522681 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b000cd80 con 0x7f24c00743a0 2024-06-13T07:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.139+0000 7f24bcc00640 1 -- 172.21.15.26:0/813522681 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b0010910 con 0x7f24c00743a0 2024-06-13T07:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.139+0000 7f24bcc00640 1 -- 172.21.15.26:0/813522681 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0018990 con 0x7f24c00743a0 2024-06-13T07:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.139+0000 7f24c7200640 1 -- 172.21.15.26:0/813522681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c0071f50 con 0x7f24c00743a0 2024-06-13T07:09:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.139+0000 7f24c7200640 1 -- 172.21.15.26:0/813522681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c01aab10 con 0x7f24c00743a0 2024-06-13T07:09:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.142+0000 7f24c9fe5740 1 Processor -- start 2024-06-13T07:09:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.142+0000 7f24c9fe5740 1 -- start start 2024-06-13T07:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b85f0 0x5628951bd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951bd760 0x5628951bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56289511b420 con 0x5628951b7c00 2024-06-13T07:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5628950c2c50 con 0x5628951b85f0 2024-06-13T07:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5628951173e0 con 0x5628951bd760 2024-06-13T07:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24bcc00640 1 -- 172.21.15.26:0/813522681 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f24b0010ac0 con 0x7f24c00743a0 2024-06-13T07:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951bd760 0x5628951bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951bd760 0x5628951bfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41512/0 (socket says 172.21.15.26:41512) 2024-06-13T07:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24bf400640 1 -- 172.21.15.26:0/3305555011 learned_addr learned my addr 172.21.15.26:0/3305555011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.143+0000 7f24c5200640 1 --2- 172.21.15.26:0/3305555011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.144+0000 7f24bf400640 1 -- 172.21.15.26:0/3305555011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b85f0 msgr2=0x5628951bd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.144+0000 7f24bf400640 1 --2- 172.21.15.26:0/3305555011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b85f0 0x5628951bd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.144+0000 7f24bf400640 1 -- 172.21.15.26:0/3305555011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 msgr2=0x5628951b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.144+0000 7f24bf400640 1 --2- 172.21.15.26:0/3305555011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.144+0000 7f24bf400640 1 -- 172.21.15.26:0/3305555011 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562894e2f210 con 0x5628951bd760 2024-06-13T07:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.144+0000 7f24bf400640 1 --2- 172.21.15.26:0/3305555011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951bd760 0x5628951bfbf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009c00 tx=0x7f24ac010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e518150d8bbfbb88 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.144+0000 7f24b5600640 1 -- 172.21.15.26:0/3305555011 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac005020 con 0x5628951bd760 2024-06-13T07:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.145+0000 7f24b5600640 1 -- 172.21.15.26:0/3305555011 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24ac0047a0 con 0x5628951bd760 2024-06-13T07:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.145+0000 7f24b5600640 1 -- 172.21.15.26:0/3305555011 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac0126b0 con 0x5628951bd760 2024-06-13T07:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.145+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3305555011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951bd760 msgr2=0x5628951bfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.145+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3305555011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951bd760 0x5628951bfbf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009c00 tx=0x7f24ac010740 comp rx=0 tx=0).stop 2024-06-13T07:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3305555011 shutdown_connections 2024-06-13T07:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3305555011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b85f0 0x5628951bd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3305555011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951bd760 0x5628951bfbf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3305555011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3305555011 >> 172.21.15.26:0/3305555011 conn(0x56289517ffb0 msgr2=0x5628951b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24bcc00640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24a40a2cb0 0x7f24a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3305555011 shutdown_connections 2024-06-13T07:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24bcc00640 1 -- 172.21.15.26:0/813522681 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f24b00cf840 con 0x7f24c00743a0 2024-06-13T07:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3305555011 wait complete. 2024-06-13T07:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.146+0000 7f24bf400640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24a40a2cb0 0x7f24a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 1 -- start start 2024-06-13T07:09:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 1 Processor -- start 2024-06-13T07:09:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24bf400640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24a40a2cb0 0x7f24a40a5170 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0096c0 tx=0x7f24ac009460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951bd760 0x5628951d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5628951c0710 con 0x5628951b7c00 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5628950d60b0 con 0x5628951bd760 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5628951c0890 con 0x5628951b85f0 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.147+0000 7f24c9fe5740 10 client.0 _ll_register_callbacks cb 0x5628950c1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24c5200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951bd760 0x5628951d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24c5200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951bd760 0x5628951d1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39964/0 (socket says 172.21.15.26:39964) 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24c5200640 1 -- 172.21.15.26:0/3425348042 learned_addr learned my addr 172.21.15.26:0/3425348042 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24bfe00640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24bf400640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24c5200640 1 -- 172.21.15.26:0/3425348042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 msgr2=0x5628951cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24c5200640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24c5200640 1 -- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 msgr2=0x5628951ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24c5200640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24c5200640 1 -- 172.21.15.26:0/3425348042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562894efb620 con 0x5628951bd760 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24bfe00640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.148+0000 7f24bf400640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.149+0000 7f24c5200640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951bd760 0x5628951d1cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009be0 tx=0x7f24b80306d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce96376a95cae8ae server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.149+0000 7f24aa000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.149+0000 7f24aa000640 1 -- 172.21.15.26:0/3425348042 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8043030 con 0x5628951bd760 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.149+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3425348042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24ac0047a0 con 0x5628951bd760 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.149+0000 7f24aa000640 1 -- 172.21.15.26:0/3425348042 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f24b8003b80 con 0x5628951bd760 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.149+0000 7f24aa000640 1 -- 172.21.15.26:0/3425348042 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f24b8050050 con 0x5628951bd760 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.149+0000 7f24c9fe5740 10 client.26210 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.149+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3425348042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5628951d5100 con 0x5628951bd760 2024-06-13T07:09:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.150+0000 7f24aa000640 1 -- 172.21.15.26:0/3425348042 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24ac0047a0 con 0x5628951bd760 2024-06-13T07:09:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.150+0000 7f24aa000640 1 -- 172.21.15.26:0/3425348042 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f24b803c050 con 0x5628951bd760 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.150+0000 7f24c9fe5740 10 client.26210 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.150+0000 7f24c9fe5740 10 client.26210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.150+0000 7f24c9fe5740 20 client.26210 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.150+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5628951d67b0 0x5628951d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.150+0000 7f24c9fe5740 4 client.26210 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.150+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3425348042 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562895128b50 con 0x5628951d67b0 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.151+0000 7f24bfe00640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5628951d67b0 0x5628951d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.151+0000 7f24bfe00640 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5628951d67b0 0x5628951d8c40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab77a29bc8cd5085 server_cookie=c79eedc4079f63e8 in_seq=0 out_seq=0 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.151+0000 7f24aa000640 10 client.26210 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24aa000640 1 -- 172.21.15.26:0/3425348042 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562895128b50 con 0x5628951d67b0 2024-06-13T07:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24aa000640 10 client.26210 handle_command_reply: tid=0 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 client.26210 shutdown 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3425348042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951bd760 msgr2=0x5628951d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951bd760 0x5628951d1cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009be0 tx=0x7f24b80306d0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3425348042 shutdown_connections 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5628951d67b0 0x5628951d8c40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951bd760 0x5628951d1cd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/3425348042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951b7c00 0x5628951ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3425348042 >> 172.21.15.26:0/3425348042 conn(0x56289517ffb0 msgr2=0x5628951b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.153+0000 7f24aa000640 0 client.26210 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.154+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3425348042 shutdown_connections 2024-06-13T07:09:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.154+0000 7f24c9fe5740 1 -- 172.21.15.26:0/3425348042 wait complete. 2024-06-13T07:09:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.154+0000 7f24c9fe5740 20 client.26210 trim_cache size 0 max 16384 2024-06-13T07:09:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.173+0000 7f24c9fe5740 1 Processor -- start 2024-06-13T07:09:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.173+0000 7f24c9fe5740 1 -- start start 2024-06-13T07:09:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.173+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.173+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.173+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.173+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24c01b3630 con 0x5628951bd760 2024-06-13T07:09:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.173+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24c01b0f00 con 0x5628951b7c00 2024-06-13T07:09:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.173+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f24c01aed50 con 0x5628951b85f0 2024-06-13T07:09:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951d1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41522/0 (socket says 172.21.15.26:41522) 2024-06-13T07:09:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951c0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39980/0 (socket says 172.21.15.26:39980) 2024-06-13T07:09:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bf400640 1 -- 172.21.15.26:0/4043958496 learned_addr learned my addr 172.21.15.26:0/4043958496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24c5200640 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bfe00640 1 -- 172.21.15.26:0/4043958496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 msgr2=0x5628951d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bfe00640 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bfe00640 1 -- 172.21.15.26:0/4043958496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 msgr2=0x5628951ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bfe00640 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bfe00640 1 -- 172.21.15.26:0/4043958496 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628951b4330 con 0x5628951b7c00 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24c5200640 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bf400640 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.174+0000 7f24bfe00640 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951c0c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5628951b9060 tx=0x7f24b0018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3b39d701bb9309 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.175+0000 7f24b4c00640 1 -- 172.21.15.26:0/4043958496 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0099d50 con 0x5628951b7c00 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.175+0000 7f24b4c00640 1 -- 172.21.15.26:0/4043958496 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b0099fb0 con 0x5628951b7c00 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.175+0000 7f24b4c00640 1 -- 172.21.15.26:0/4043958496 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b009b100 con 0x5628951b7c00 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.175+0000 7f24c9fe5740 1 -- 172.21.15.26:0/4043958496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 msgr2=0x5628951c0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.175+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951c0c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5628951b9060 tx=0x7f24b0018420 comp rx=0 tx=0).stop 2024-06-13T07:09:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 -- 172.21.15.26:0/4043958496 shutdown_connections 2024-06-13T07:09:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/4043958496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951c0c90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 -- 172.21.15.26:0/4043958496 >> 172.21.15.26:0/4043958496 conn(0x56289517ffb0 msgr2=0x5628951b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 -- 172.21.15.26:0/4043958496 shutdown_connections 2024-06-13T07:09:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 -- 172.21.15.26:0/4043958496 wait complete. 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 -- start start 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.176+0000 7f24c9fe5740 1 Processor -- start 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24c9fe5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24c01af3c0 con 0x5628951bd760 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24c01b0830 con 0x5628951b7c00 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24c9fe5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f24c01afa90 con 0x5628951b85f0 2024-06-13T07:09:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24c9fe5740 10 client.0 _ll_register_callbacks cb 0x7f24b8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24bfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41524/0 (socket says 172.21.15.26:41524) 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24bf400640 1 -- 172.21.15.26:0/2772402893 learned_addr learned my addr 172.21.15.26:0/2772402893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.177+0000 7f24c5200640 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24bf400640 1 -- 172.21.15.26:0/2772402893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 msgr2=0x5628951cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24bf400640 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24bf400640 1 -- 172.21.15.26:0/2772402893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 msgr2=0x5628951d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24bf400640 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24bf400640 1 -- 172.21.15.26:0/2772402893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628951d0930 con 0x5628951b85f0 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24bfe00640 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24bf400640 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cf330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f24ac005fd0 tx=0x7f24ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eec6288a6c15a72 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24aa000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:09:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.178+0000 7f24aa000640 1 -- 172.21.15.26:0/2772402893 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac037060 con 0x5628951b85f0 2024-06-13T07:09:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.179+0000 7f24aa000640 1 -- 172.21.15.26:0/2772402893 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f24ac022030 con 0x5628951b85f0 2024-06-13T07:09:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.179+0000 7f24c9fe5740 1 -- 172.21.15.26:0/2772402893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b0099fb0 con 0x5628951b85f0 2024-06-13T07:09:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.179+0000 7f24aa000640 1 -- 172.21.15.26:0/2772402893 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f24ac01b890 con 0x5628951b85f0 2024-06-13T07:09:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.179+0000 7f24c9fe5740 10 client.26254 fetch_fsmap learned FSMap version 35 2024-06-13T07:09:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.179+0000 7f24c9fe5740 1 -- 172.21.15.26:0/2772402893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5628951d1ce0 con 0x5628951b85f0 2024-06-13T07:09:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.179+0000 7f24aa000640 1 -- 172.21.15.26:0/2772402893 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b0099fb0 con 0x5628951b85f0 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.179+0000 7f24aa000640 1 -- 172.21.15.26:0/2772402893 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f24ac00f040 con 0x5628951b85f0 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24c9fe5740 10 client.26254 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24c9fe5740 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24c9fe5740 20 client.26254 populate_metadata read hostname 'smithi026' 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5628952417b0 0x562895243bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24c9fe5740 4 client.26254 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24c9fe5740 1 -- 172.21.15.26:0/2772402893 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562895122ea0 con 0x5628952417b0 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24bfe00640 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5628952417b0 0x562895243bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24bfe00640 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5628952417b0 0x562895243bd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6287d825ff0d743 server_cookie=aeff407bb3ca96e4 in_seq=0 out_seq=0 2024-06-13T07:09:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.180+0000 7f24aa000640 10 client.26254 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.275+0000 7f24aa000640 1 -- 172.21.15.26:0/2772402893 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562895122ea0 con 0x5628952417b0 2024-06-13T07:09:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.275+0000 7f24aa000640 10 client.26254 handle_command_reply: tid=0 2024-06-13T07:09:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.275+0000 7f24c9fe5740 1 client.26254 shutdown 2024-06-13T07:09:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 -- 172.21.15.26:0/2772402893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 msgr2=0x5628951cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cf330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f24ac005fd0 tx=0x7f24ac009260 comp rx=0 tx=0).stop 2024-06-13T07:09:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 -- 172.21.15.26:0/2772402893 shutdown_connections 2024-06-13T07:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5628952417b0 0x562895243bd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5628951bd760 0x5628951d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628951b85f0 0x5628951cf330 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 --2- 172.21.15.26:0/2772402893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628951b7c00 0x5628951cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 -- 172.21.15.26:0/2772402893 >> 172.21.15.26:0/2772402893 conn(0x56289517ffb0 msgr2=0x5628951ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24aa000640 0 client.26254 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:09:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 -- 172.21.15.26:0/2772402893 shutdown_connections 2024-06-13T07:09:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 1 -- 172.21.15.26:0/2772402893 wait complete. 2024-06-13T07:09:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.276+0000 7f24c9fe5740 20 client.26254 trim_cache size 0 max 16384 2024-06-13T07:09:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:09:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:09:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 -- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24a40a2cb0 msgr2=0x7f24a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24a40a2cb0 0x7f24a40a5170 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0096c0 tx=0x7f24ac009460 comp rx=0 tx=0).stop 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 -- 172.21.15.26:0/813522681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c00743a0 msgr2=0x7f24c0072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c00743a0 0x7f24c0072ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f24b000e9f0 tx=0x7f24b000eeb0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 -- 172.21.15.26:0/813522681 shutdown_connections 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f24a40a2cb0 0x7f24a40a5170 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c0109890 0x7f24c0071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24c0075980 0x7f24c0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 --2- 172.21.15.26:0/813522681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c00743a0 0x7f24c0072ef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 -- 172.21.15.26:0/813522681 >> 172.21.15.26:0/813522681 conn(0x7f24c00fcf90 msgr2=0x7f24c0105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 -- 172.21.15.26:0/813522681 shutdown_connections 2024-06-13T07:09:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:09:58.277+0000 7f24c7200640 1 -- 172.21.15.26:0/813522681 wait complete. 2024-06-13T07:09:58.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 425338 4.71 MB/sec execute 372 sec latency 56.944 ms 2024-06-13T07:09:59.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 426107 4.72 MB/sec execute 373 sec latency 21.618 ms 2024-06-13T07:10:00.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:09:59 smithi136 ceph-mon[37229]: pgmap v794: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-13T07:10:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:09:59 smithi026 ceph-mon[32041]: pgmap v794: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-13T07:10:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:09:59.999+0000 7fc609800700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-06-13T07:10:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:09:59.999+0000 7fc609800700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-06-13T07:10:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:09:59.999+0000 7fc609800700 -1 log_channel(cluster) log [ERR] : mds.b(mds.2): Metadata damage detected 2024-06-13T07:10:00.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:09:59.999+0000 7fc609800700 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2024-06-13T07:10:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:09:59 smithi135 ceph-mon[36276]: pgmap v794: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-13T07:10:00.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 426861 4.71 MB/sec execute 374 sec latency 29.324 ms 2024-06-13T07:10:01.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:00 smithi136 ceph-mon[37229]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-06-13T07:10:01.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:00 smithi136 ceph-mon[37229]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-06-13T07:10:01.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:00 smithi136 ceph-mon[37229]: mds.b(mds.2): Metadata damage detected 2024-06-13T07:10:01.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:00 smithi136 ceph-mon[37229]: mds.i(mds.1): Metadata damage detected 2024-06-13T07:10:01.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:00 smithi026 ceph-mon[32041]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-06-13T07:10:01.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:00 smithi026 ceph-mon[32041]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-06-13T07:10:01.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:00 smithi026 ceph-mon[32041]: mds.b(mds.2): Metadata damage detected 2024-06-13T07:10:01.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:00 smithi026 ceph-mon[32041]: mds.i(mds.1): Metadata damage detected 2024-06-13T07:10:01.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:00 smithi135 ceph-mon[36276]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-06-13T07:10:01.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:00 smithi135 ceph-mon[36276]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-06-13T07:10:01.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:00 smithi135 ceph-mon[36276]: mds.b(mds.2): Metadata damage detected 2024-06-13T07:10:01.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:00 smithi135 ceph-mon[36276]: mds.i(mds.1): Metadata damage detected 2024-06-13T07:10:01.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 428228 4.71 MB/sec execute 375 sec latency 29.635 ms 2024-06-13T07:10:02.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:01 smithi136 ceph-mon[37229]: pgmap v795: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:10:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:01 smithi026 ceph-mon[32041]: pgmap v795: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:10:02.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:01 smithi135 ceph-mon[36276]: pgmap v795: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:10:02.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 429383 4.73 MB/sec execute 376 sec latency 17.995 ms 2024-06-13T07:10:03.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 429908 4.72 MB/sec execute 377 sec latency 33.124 ms 2024-06-13T07:10:04.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:03 smithi136 ceph-mon[37229]: pgmap v796: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:10:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:03 smithi026 ceph-mon[32041]: pgmap v796: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:10:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:03 smithi135 ceph-mon[36276]: pgmap v796: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:10:04.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 430998 4.72 MB/sec execute 378 sec latency 92.632 ms 2024-06-13T07:10:05.565 INFO:tasks.workunit.client.0.smithi026.stdout: 1 432388 4.73 MB/sec execute 379 sec latency 23.090 ms 2024-06-13T07:10:06.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:05 smithi136 ceph-mon[37229]: pgmap v797: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:10:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:05 smithi026 ceph-mon[32041]: pgmap v797: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:10:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:05 smithi135 ceph-mon[36276]: pgmap v797: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:10:06.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 433147 4.73 MB/sec execute 380 sec latency 21.987 ms 2024-06-13T07:10:07.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 433815 4.72 MB/sec execute 381 sec latency 35.162 ms 2024-06-13T07:10:08.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:07 smithi136 ceph-mon[37229]: pgmap v798: 129 pgs: 129 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-06-13T07:10:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:07 smithi026 ceph-mon[32041]: pgmap v798: 129 pgs: 129 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-06-13T07:10:08.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:07 smithi135 ceph-mon[36276]: pgmap v798: 129 pgs: 129 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-06-13T07:10:08.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 435155 4.72 MB/sec execute 382 sec latency 58.033 ms 2024-06-13T07:10:09.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 436293 4.74 MB/sec execute 383 sec latency 23.117 ms 2024-06-13T07:10:10.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:09 smithi136 ceph-mon[37229]: pgmap v799: 129 pgs: 129 active+clean; 105 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:10:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:09 smithi026 ceph-mon[32041]: pgmap v799: 129 pgs: 129 active+clean; 105 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:10:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:09 smithi135 ceph-mon[36276]: pgmap v799: 129 pgs: 129 active+clean; 105 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-13T07:10:10.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 436775 4.73 MB/sec execute 384 sec latency 18.172 ms 2024-06-13T07:10:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.442+0000 7fcbf8e00640 1 -- 172.21.15.26:0/3027064096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4102360 msgr2=0x7fcbf4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.442+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/3027064096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4102360 0x7fcbf4104750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009860 tx=0x7fcbe402a340 comp rx=0 tx=0).stop 2024-06-13T07:10:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.442+0000 7fcbf8e00640 1 -- 172.21.15.26:0/3027064096 shutdown_connections 2024-06-13T07:10:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.442+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/3027064096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf41075c0 0x7fcbf41099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.443+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/3027064096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf4104c90 0x7fcbf4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.443+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/3027064096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4102360 0x7fcbf4104750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.443+0000 7fcbf8e00640 1 -- 172.21.15.26:0/3027064096 >> 172.21.15.26:0/3027064096 conn(0x7fcbf40fbc90 msgr2=0x7fcbf40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.443+0000 7fcbf8e00640 1 -- 172.21.15.26:0/3027064096 shutdown_connections 2024-06-13T07:10:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.443+0000 7fcbf8e00640 1 -- 172.21.15.26:0/3027064096 wait complete. 2024-06-13T07:10:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.445+0000 7fcbf8e00640 1 Processor -- start 2024-06-13T07:10:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.445+0000 7fcbf8e00640 1 -- start start 2024-06-13T07:10:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf4102360 0x7fcbf4197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4104c90 0x7fcbf41980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf41075c0 0x7fcbf419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf8e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf410d4a0 con 0x7fcbf41075c0 2024-06-13T07:10:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf8e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf410d320 con 0x7fcbf4102360 2024-06-13T07:10:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf8e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf410d620 con 0x7fcbf4104c90 2024-06-13T07:10:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf4102360 0x7fcbf4197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf4102360 0x7fcbf4197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:60168/0 (socket says 172.21.15.26:60168) 2024-06-13T07:10:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf3400640 1 -- 172.21.15.26:0/2222743913 learned_addr learned my addr 172.21.15.26:0/2222743913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.446+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4104c90 0x7fcbf41980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.447+0000 7fcbf3e00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf41075c0 0x7fcbf419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.447+0000 7fcbf2a00640 1 -- 172.21.15.26:0/2222743913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf4102360 msgr2=0x7fcbf4197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.447+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf4102360 0x7fcbf4197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.447+0000 7fcbf2a00640 1 -- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf41075c0 msgr2=0x7fcbf419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.447+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf41075c0 0x7fcbf419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.447+0000 7fcbf2a00640 1 -- 172.21.15.26:0/2222743913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf41001d0 con 0x7fcbf4104c90 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.447+0000 7fcbf3e00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf41075c0 0x7fcbf419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.447+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4104c90 0x7fcbf41980b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0005fd0 tx=0x7fcbe000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.448+0000 7fcbebe00640 1 -- 172.21.15.26:0/2222743913 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000ccc0 con 0x7fcbf4104c90 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.448+0000 7fcbebe00640 1 -- 172.21.15.26:0/2222743913 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe000ee70 con 0x7fcbf4104c90 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.448+0000 7fcbebe00640 1 -- 172.21.15.26:0/2222743913 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0018840 con 0x7fcbf4104c90 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.448+0000 7fcbf8e00640 1 -- 172.21.15.26:0/2222743913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf41004c0 con 0x7fcbf4104c90 2024-06-13T07:10:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.448+0000 7fcbf8e00640 1 -- 172.21.15.26:0/2222743913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf41009c0 con 0x7fcbf4104c90 2024-06-13T07:10:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.452+0000 7fcbfd232740 1 Processor -- start 2024-06-13T07:10:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.452+0000 7fcbebe00640 1 -- 172.21.15.26:0/2222743913 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbe00189e0 con 0x7fcbf4104c90 2024-06-13T07:10:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.452+0000 7fcbfd232740 1 -- start start 2024-06-13T07:10:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.452+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256786d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x5625678722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 0x562567874db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbfd232740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5625677d0420 con 0x56256786d7b0 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbfd232740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562567777c50 con 0x562567872920 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbfd232740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5625677cc3e0 con 0x56256786cdc0 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256786d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256786d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39342/0 (socket says 172.21.15.26:39342) 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3e00640 1 -- 172.21.15.26:0/300631165 learned_addr learned my addr 172.21.15.26:0/300631165 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 0x562567874db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3400640 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x5625678722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3e00640 1 -- 172.21.15.26:0/300631165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 msgr2=0x562567874db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3e00640 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 0x562567874db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3e00640 1 -- 172.21.15.26:0/300631165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 msgr2=0x5625678722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3e00640 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x5625678722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.453+0000 7fcbf3e00640 1 -- 172.21.15.26:0/300631165 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625674e4210 con 0x56256786cdc0 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.454+0000 7fcbf3e00640 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256786d1e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009f90 tx=0x7fcbec00f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4d22ce1d1bc6c97 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.454+0000 7fcbe8c00640 1 -- 172.21.15.26:0/300631165 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec003b70 con 0x56256786cdc0 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.454+0000 7fcbe8c00640 1 -- 172.21.15.26:0/300631165 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbec003d10 con 0x56256786cdc0 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.454+0000 7fcbe8c00640 1 -- 172.21.15.26:0/300631165 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec011a60 con 0x56256786cdc0 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbebe00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbd80a2c00 0x7fcbd80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbebe00640 1 -- 172.21.15.26:0/2222743913 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcbe00cf860 con 0x7fcbf4104c90 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbfd232740 1 -- 172.21.15.26:0/300631165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 msgr2=0x56256786d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbfd232740 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256786d1e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009f90 tx=0x7fcbec00f7f0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbfd232740 1 -- 172.21.15.26:0/300631165 shutdown_connections 2024-06-13T07:10:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbfd232740 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 0x562567874db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbfd232740 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x5625678722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbfd232740 1 --2- 172.21.15.26:0/300631165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256786d1e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbf3400640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbd80a2c00 0x7fcbd80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.455+0000 7fcbfd232740 1 -- 172.21.15.26:0/300631165 >> 172.21.15.26:0/300631165 conn(0x562567834fb0 msgr2=0x562567868840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.456+0000 7fcbfd232740 1 -- 172.21.15.26:0/300631165 shutdown_connections 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.456+0000 7fcbfd232740 1 -- 172.21.15.26:0/300631165 wait complete. 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.456+0000 7fcbf3400640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbd80a2c00 0x7fcbd80a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40305b0 tx=0x7fcbe402c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.456+0000 7fcbfd232740 1 -- start start 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.457+0000 7fcbfd232740 1 Processor -- start 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.457+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x5625678770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.457+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x562567877630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.457+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 0x56256787e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.457+0000 7fcbfd232740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5625678758d0 con 0x56256786d7b0 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.457+0000 7fcbfd232740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56256778b0b0 con 0x562567872920 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.457+0000 7fcbfd232740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562567875a50 con 0x56256786cdc0 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.457+0000 7fcbfd232740 10 client.0 _ll_register_callbacks cb 0x562567776f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x5625678770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x5625678770f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39344/0 (socket says 172.21.15.26:39344) 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 -- 172.21.15.26:0/3727258477 learned_addr learned my addr 172.21.15.26:0/3727258477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3e00640 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 0x56256787e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x562567877630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 -- 172.21.15.26:0/3727258477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 msgr2=0x56256787e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 0x56256787e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 -- 172.21.15.26:0/3727258477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 msgr2=0x562567877630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x562567877630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 -- 172.21.15.26:0/3727258477 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625675b0620 con 0x56256786cdc0 2024-06-13T07:10:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.458+0000 7fcbf3400640 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x5625678770f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009250 tx=0x7fcbe402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba38873cc18e1ad7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.459+0000 7fcbdd600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:10:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.459+0000 7fcbdd600640 1 -- 172.21.15.26:0/3727258477 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4037040 con 0x56256786cdc0 2024-06-13T07:10:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.459+0000 7fcbdd600640 1 -- 172.21.15.26:0/3727258477 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcbe4005020 con 0x56256786cdc0 2024-06-13T07:10:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.459+0000 7fcbfd232740 1 -- 172.21.15.26:0/3727258477 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbec003d10 con 0x56256786cdc0 2024-06-13T07:10:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.459+0000 7fcbdd600640 1 -- 172.21.15.26:0/3727258477 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcbe4054050 con 0x56256786cdc0 2024-06-13T07:10:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.459+0000 7fcbfd232740 10 client.26266 fetch_fsmap learned FSMap version 35 2024-06-13T07:10:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.459+0000 7fcbfd232740 1 -- 172.21.15.26:0/3727258477 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625678819e0 con 0x56256786cdc0 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbdd600640 1 -- 172.21.15.26:0/3727258477 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbec003d10 con 0x56256786cdc0 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbdd600640 1 -- 172.21.15.26:0/3727258477 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcbe40046c0 con 0x56256786cdc0 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbfd232740 10 client.26266 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbfd232740 10 client.26266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbfd232740 20 client.26266 populate_metadata read hostname 'smithi026' 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbfd232740 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5625678830c0 0x5625678854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbfd232740 4 client.26266 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbfd232740 1 -- 172.21.15.26:0/3727258477 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625677ddb50 con 0x5625678830c0 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.460+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5625678830c0 0x5625678854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.461+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5625678830c0 0x5625678854d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b447682026ef3f5c server_cookie=51263b1e054eca9 in_seq=0 out_seq=0 2024-06-13T07:10:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.461+0000 7fcbdd600640 10 client.26266 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.463+0000 7fcbdd600640 1 -- 172.21.15.26:0/3727258477 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5625677ddb50 con 0x5625678830c0 2024-06-13T07:10:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.463+0000 7fcbdd600640 10 client.26266 handle_command_reply: tid=0 2024-06-13T07:10:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.463+0000 7fcbfd232740 1 client.26266 shutdown 2024-06-13T07:10:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 -- 172.21.15.26:0/3727258477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 msgr2=0x5625678770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x5625678770f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009250 tx=0x7fcbe402a340 comp rx=0 tx=0).stop 2024-06-13T07:10:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 -- 172.21.15.26:0/3727258477 shutdown_connections 2024-06-13T07:10:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5625678830c0 0x5625678854d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567872920 0x56256787e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x562567877630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 --2- 172.21.15.26:0/3727258477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x5625678770f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbdd600640 0 client.26266 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 -- 172.21.15.26:0/3727258477 >> 172.21.15.26:0/3727258477 conn(0x562567834fb0 msgr2=0x5625678679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 -- 172.21.15.26:0/3727258477 shutdown_connections 2024-06-13T07:10:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 1 -- 172.21.15.26:0/3727258477 wait complete. 2024-06-13T07:10:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.464+0000 7fcbfd232740 20 client.26266 trim_cache size 0 max 16384 2024-06-13T07:10:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.486+0000 7fcbfd232740 1 Processor -- start 2024-06-13T07:10:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.486+0000 7fcbfd232740 1 -- start start 2024-06-13T07:10:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.486+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x562567868120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.486+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786cdc0 0x56256787e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.486+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786d7b0 0x56256786be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.486+0000 7fcbfd232740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41af0c0 con 0x56256786cdc0 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.486+0000 7fcbfd232740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41ac990 con 0x562567834fb0 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.486+0000 7fcbfd232740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41aa830 con 0x56256786d7b0 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x562567868120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786d7b0 0x56256786be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x562567868120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:60190/0 (socket says 172.21.15.26:60190) 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786d7b0 0x56256786be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39356/0 (socket says 172.21.15.26:39356) 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 -- 172.21.15.26:0/2327172218 learned_addr learned my addr 172.21.15.26:0/2327172218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 -- 172.21.15.26:0/2327172218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786d7b0 msgr2=0x56256786be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786d7b0 0x56256786be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786cdc0 0x56256787e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 -- 172.21.15.26:0/2327172218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786cdc0 msgr2=0x56256787e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786cdc0 0x56256787e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 -- 172.21.15.26:0/2327172218 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256786d1a0 con 0x562567834fb0 2024-06-13T07:10:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786cdc0 0x56256787e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.487+0000 7fcbf3400640 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x562567868120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcbe402c010 tx=0x7fcbe4030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa35b4f9cf2652c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.488+0000 7fcbdfe00640 1 -- 172.21.15.26:0/2327172218 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe405e020 con 0x562567834fb0 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.488+0000 7fcbdfe00640 1 -- 172.21.15.26:0/2327172218 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe4009450 con 0x562567834fb0 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.488+0000 7fcbdfe00640 1 -- 172.21.15.26:0/2327172218 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4003a60 con 0x562567834fb0 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.488+0000 7fcbfd232740 1 -- 172.21.15.26:0/2327172218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 msgr2=0x562567868120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.488+0000 7fcbfd232740 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x562567868120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcbe402c010 tx=0x7fcbe4030340 comp rx=0 tx=0).stop 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 -- 172.21.15.26:0/2327172218 shutdown_connections 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786d7b0 0x56256786be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786cdc0 0x56256787e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 --2- 172.21.15.26:0/2327172218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x562567868120 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 -- 172.21.15.26:0/2327172218 >> 172.21.15.26:0/2327172218 conn(0x56256786e3b0 msgr2=0x56256786b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 -- 172.21.15.26:0/2327172218 shutdown_connections 2024-06-13T07:10:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 -- 172.21.15.26:0/2327172218 wait complete. 2024-06-13T07:10:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 -- start start 2024-06-13T07:10:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.489+0000 7fcbfd232740 1 Processor -- start 2024-06-13T07:10:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x56256787c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256787c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbfd232740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x56256787d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbfd232740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41aaea0 con 0x56256786d7b0 2024-06-13T07:10:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbfd232740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41ac2c0 con 0x562567834fb0 2024-06-13T07:10:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbfd232740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41ab520 con 0x56256786cdc0 2024-06-13T07:10:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbfd232740 10 client.0 _ll_register_callbacks cb 0x7fcbe0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256787c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x56256787c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256787c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39370/0 (socket says 172.21.15.26:39370) 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf2a00640 1 -- 172.21.15.26:0/158934110 learned_addr learned my addr 172.21.15.26:0/158934110 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf3e00640 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x56256787d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf2a00640 1 -- 172.21.15.26:0/158934110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 msgr2=0x56256787c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x56256787c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf2a00640 1 -- 172.21.15.26:0/158934110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 msgr2=0x56256787d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.490+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x56256787d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbf2a00640 1 -- 172.21.15.26:0/158934110 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256787df70 con 0x56256786cdc0 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbf2a00640 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256787c970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c6a0 tx=0x7fcbe00cce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c19277f7e7e1504 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbdd600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbdd600640 1 -- 172.21.15.26:0/158934110 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe009c3b0 con 0x56256786cdc0 2024-06-13T07:10:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbfd232740 1 -- 172.21.15.26:0/158934110 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe4009450 con 0x56256786cdc0 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbdd600640 1 -- 172.21.15.26:0/158934110 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcbe00cd060 con 0x56256786cdc0 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbdd600640 1 -- 172.21.15.26:0/158934110 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcbe009bcb0 con 0x56256786cdc0 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbfd232740 10 client.26272 fetch_fsmap learned FSMap version 35 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.491+0000 7fcbfd232740 1 -- 172.21.15.26:0/158934110 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625678ee4c0 con 0x56256786cdc0 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbdd600640 1 -- 172.21.15.26:0/158934110 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe4009450 con 0x56256786cdc0 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbdd600640 1 -- 172.21.15.26:0/158934110 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcbe00ba450 con 0x56256786cdc0 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbfd232740 10 client.26272 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbfd232740 10 client.26272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbfd232740 20 client.26272 populate_metadata read hostname 'smithi026' 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbfd232740 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562567872920 0x5625678f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbfd232740 4 client.26272 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbfd232740 1 -- 172.21.15.26:0/158934110 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625677d7ea0 con 0x562567872920 2024-06-13T07:10:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.492+0000 7fcbf3400640 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562567872920 0x5625678f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.493+0000 7fcbf3400640 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562567872920 0x5625678f0e70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6d9fc06e738749 server_cookie=efa52bc3cb27ba9c in_seq=0 out_seq=0 2024-06-13T07:10:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.493+0000 7fcbdd600640 10 client.26272 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:11.566 INFO:tasks.workunit.client.0.smithi026.stdout: 1 437551 4.72 MB/sec execute 385 sec latency 43.662 ms 2024-06-13T07:10:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.590+0000 7fcbdd600640 1 -- 172.21.15.26:0/158934110 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5625677d7ea0 con 0x562567872920 2024-06-13T07:10:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.590+0000 7fcbdd600640 10 client.26272 handle_command_reply: tid=0 2024-06-13T07:10:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.590+0000 7fcbfd232740 1 client.26272 shutdown 2024-06-13T07:10:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.590+0000 7fcbfd232740 1 -- 172.21.15.26:0/158934110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 msgr2=0x56256787c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.590+0000 7fcbfd232740 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256787c970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c6a0 tx=0x7fcbe00cce20 comp rx=0 tx=0).stop 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 1 -- 172.21.15.26:0/158934110 shutdown_connections 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562567872920 0x5625678f0e70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56256786d7b0 0x56256787d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56256786cdc0 0x56256787c970 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 1 --2- 172.21.15.26:0/158934110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562567834fb0 0x56256787c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 1 -- 172.21.15.26:0/158934110 >> 172.21.15.26:0/158934110 conn(0x56256786e3b0 msgr2=0x562567867110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbdd600640 0 client.26272 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 1 -- 172.21.15.26:0/158934110 shutdown_connections 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 1 -- 172.21.15.26:0/158934110 wait complete. 2024-06-13T07:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.591+0000 7fcbfd232740 20 client.26272 trim_cache size 0 max 16384 2024-06-13T07:10:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:10:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:10:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:10:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.592+0000 7fcbf8e00640 1 -- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbd80a2c00 msgr2=0x7fcbd80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.592+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbd80a2c00 0x7fcbd80a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40305b0 tx=0x7fcbe402c040 comp rx=0 tx=0).stop 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.592+0000 7fcbf8e00640 1 -- 172.21.15.26:0/2222743913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4104c90 msgr2=0x7fcbf41980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.592+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4104c90 0x7fcbf41980b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0005fd0 tx=0x7fcbe000ec20 comp rx=0 tx=0).stop 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.593+0000 7fcbf8e00640 1 -- 172.21.15.26:0/2222743913 shutdown_connections 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.593+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcbd80a2c00 0x7fcbd80a50c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.593+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbf41075c0 0x7fcbf419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.593+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf4104c90 0x7fcbf41980b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.593+0000 7fcbf8e00640 1 --2- 172.21.15.26:0/2222743913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf4102360 0x7fcbf4197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.593+0000 7fcbf8e00640 1 -- 172.21.15.26:0/2222743913 >> 172.21.15.26:0/2222743913 conn(0x7fcbf40fbc90 msgr2=0x7fcbf4102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.594+0000 7fcbf8e00640 1 -- 172.21.15.26:0/2222743913 shutdown_connections 2024-06-13T07:10:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:11.594+0000 7fcbf8e00640 1 -- 172.21.15.26:0/2222743913 wait complete. 2024-06-13T07:10:12.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:11 smithi136 ceph-mon[37229]: pgmap v800: 129 pgs: 129 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:10:12.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:11 smithi026 ceph-mon[32041]: pgmap v800: 129 pgs: 129 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:10:12.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:11 smithi135 ceph-mon[36276]: pgmap v800: 129 pgs: 129 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:10:12.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 438762 4.72 MB/sec execute 386 sec latency 41.213 ms 2024-06-13T07:10:13.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 439974 4.74 MB/sec execute 387 sec latency 41.734 ms 2024-06-13T07:10:14.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:13 smithi136 ceph-mon[37229]: pgmap v801: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:10:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:13 smithi026 ceph-mon[32041]: pgmap v801: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:10:14.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:13 smithi135 ceph-mon[36276]: pgmap v801: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-13T07:10:14.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 440437 4.73 MB/sec execute 388 sec latency 22.740 ms 2024-06-13T07:10:15.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 441351 4.73 MB/sec execute 389 sec latency 29.409 ms 2024-06-13T07:10:16.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:15 smithi136 ceph-mon[37229]: pgmap v802: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:10:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:15 smithi026 ceph-mon[32041]: pgmap v802: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:10:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:15 smithi135 ceph-mon[36276]: pgmap v802: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:10:16.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 442606 4.74 MB/sec execute 390 sec latency 23.328 ms 2024-06-13T07:10:17.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 443636 4.74 MB/sec execute 391 sec latency 26.360 ms 2024-06-13T07:10:18.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:17 smithi136 ceph-mon[37229]: pgmap v803: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-06-13T07:10:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:17 smithi026 ceph-mon[32041]: pgmap v803: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-06-13T07:10:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:17 smithi135 ceph-mon[36276]: pgmap v803: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-06-13T07:10:18.567 INFO:tasks.workunit.client.0.smithi026.stdout: 1 444022 4.73 MB/sec execute 392 sec latency 38.370 ms 2024-06-13T07:10:19.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 445266 4.73 MB/sec execute 393 sec latency 23.982 ms 2024-06-13T07:10:20.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:19 smithi136 ceph-mon[37229]: pgmap v804: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:10:20.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:19 smithi026 ceph-mon[32041]: pgmap v804: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:10:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:19 smithi135 ceph-mon[36276]: pgmap v804: 129 pgs: 129 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-06-13T07:10:20.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 446413 4.74 MB/sec execute 394 sec latency 22.465 ms 2024-06-13T07:10:21.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 447300 4.74 MB/sec execute 395 sec latency 17.864 ms 2024-06-13T07:10:22.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:21 smithi136 ceph-mon[37229]: pgmap v805: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.2 MiB/s wr, 197 op/s 2024-06-13T07:10:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:21 smithi026 ceph-mon[32041]: pgmap v805: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.2 MiB/s wr, 197 op/s 2024-06-13T07:10:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:21 smithi135 ceph-mon[36276]: pgmap v805: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.2 MiB/s wr, 197 op/s 2024-06-13T07:10:22.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 447756 4.73 MB/sec execute 396 sec latency 30.335 ms 2024-06-13T07:10:23.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 449150 4.74 MB/sec execute 397 sec latency 21.278 ms 2024-06-13T07:10:24.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:23 smithi136 ceph-mon[37229]: pgmap v806: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:10:24.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:23 smithi026 ceph-mon[32041]: pgmap v806: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:10:24.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:23 smithi135 ceph-mon[36276]: pgmap v806: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:10:24.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 450321 4.75 MB/sec execute 398 sec latency 38.247 ms 2024-06-13T07:10:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.748+0000 7fc0d9600640 1 -- 172.21.15.26:0/1960969934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d41013b0 msgr2=0x7fc0d4101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.748+0000 7fc0d9600640 1 --2- 172.21.15.26:0/1960969934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d41013b0 0x7fc0d4101830 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009ca0 tx=0x7fc0cc02a8c0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.748+0000 7fc0d9600640 1 -- 172.21.15.26:0/1960969934 shutdown_connections 2024-06-13T07:10:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.748+0000 7fc0d9600640 1 --2- 172.21.15.26:0/1960969934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d41013b0 0x7fc0d4101830 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.748+0000 7fc0d9600640 1 --2- 172.21.15.26:0/1960969934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0d4100a10 0x7fc0d4100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.748+0000 7fc0d9600640 1 --2- 172.21.15.26:0/1960969934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d41000b0 0x7fc0d41004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.748+0000 7fc0d9600640 1 -- 172.21.15.26:0/1960969934 >> 172.21.15.26:0/1960969934 conn(0x7fc0d40fbc90 msgr2=0x7fc0d40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.749+0000 7fc0d9600640 1 -- 172.21.15.26:0/1960969934 shutdown_connections 2024-06-13T07:10:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.749+0000 7fc0d9600640 1 -- 172.21.15.26:0/1960969934 wait complete. 2024-06-13T07:10:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d9600640 1 Processor -- start 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d9600640 1 -- start start 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d9600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d41000b0 0x7fc0d4199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d9600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0d4100a10 0x7fc0d419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d9600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d41013b0 0x7fc0d41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d9600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d410f6c0 con 0x7fc0d41000b0 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d9600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d410f540 con 0x7fc0d41013b0 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d9600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d410f840 con 0x7fc0d4100a10 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.750+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0d4100a10 0x7fc0d419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0d4100a10 0x7fc0d419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59828/0 (socket says 172.21.15.26:59828) 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d3400640 1 -- 172.21.15.26:0/2660280893 learned_addr learned my addr 172.21.15.26:0/2660280893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d41000b0 0x7fc0d4199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d41013b0 0x7fc0d41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d3400640 1 -- 172.21.15.26:0/2660280893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d41013b0 msgr2=0x7fc0d41a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d3400640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d41013b0 0x7fc0d41a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d3400640 1 -- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d41000b0 msgr2=0x7fc0d4199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d3400640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d41000b0 0x7fc0d4199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.751+0000 7fc0d3400640 1 -- 172.21.15.26:0/2660280893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d41a1900 con 0x7fc0d4100a10 2024-06-13T07:10:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.752+0000 7fc0d3400640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0d4100a10 0x7fc0d419a230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000e9f0 tx=0x7fc0c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.752+0000 7fc0d0c00640 1 -- 172.21.15.26:0/2660280893 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000cd80 con 0x7fc0d4100a10 2024-06-13T07:10:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.752+0000 7fc0d0c00640 1 -- 172.21.15.26:0/2660280893 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c0010910 con 0x7fc0d4100a10 2024-06-13T07:10:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.752+0000 7fc0d0c00640 1 -- 172.21.15.26:0/2660280893 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0018990 con 0x7fc0d4100a10 2024-06-13T07:10:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.752+0000 7fc0d9600640 1 -- 172.21.15.26:0/2660280893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d41a1bf0 con 0x7fc0d4100a10 2024-06-13T07:10:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.752+0000 7fc0d9600640 1 -- 172.21.15.26:0/2660280893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41a2190 con 0x7fc0d4100a10 2024-06-13T07:10:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0dd9f3740 1 Processor -- start 2024-06-13T07:10:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0dd9f3740 1 -- start start 2024-06-13T07:10:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0d0c00640 1 -- 172.21.15.26:0/2660280893 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc0c0010ac0 con 0x7fc0d4100a10 2024-06-13T07:10:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bb31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bb82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bbadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b2e2b16420 con 0x55b2e2bb37b0 2024-06-13T07:10:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2e2abdc50 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.757+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b2e2b123e0 con 0x55b2e2bb8920 2024-06-13T07:10:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d8c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bb31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d8c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bb31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49972/0 (socket says 172.21.15.26:49972) 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d8c00640 1 -- 172.21.15.26:0/3516583709 learned_addr learned my addr 172.21.15.26:0/3516583709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bb82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d3400640 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bbadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d8c00640 1 -- 172.21.15.26:0/3516583709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 msgr2=0x55b2e2bbadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bbadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d8c00640 1 -- 172.21.15.26:0/3516583709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 msgr2=0x55b2e2bb82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bb82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d8c00640 1 -- 172.21.15.26:0/3516583709 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e282a210 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.758+0000 7fc0d3400640 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bbadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.759+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bb82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.759+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bb31e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009ca0 tx=0x7fc0cc0093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5df45a33355b8c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.759+0000 7fc0c9200640 1 -- 172.21.15.26:0/3516583709 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc031650 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.759+0000 7fc0d0c00640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc0b80a2c00 0x7fc0b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.759+0000 7fc0c9200640 1 -- 172.21.15.26:0/3516583709 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0cc036030 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.759+0000 7fc0c9200640 1 -- 172.21.15.26:0/3516583709 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc004740 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.760+0000 7fc0d0c00640 1 -- 172.21.15.26:0/2660280893 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc0c00cf730 con 0x7fc0d4100a10 2024-06-13T07:10:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.760+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc0b80a2c00 0x7fc0b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.760+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/3516583709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 msgr2=0x55b2e2bb31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.760+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bb31e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009ca0 tx=0x7fc0cc0093c0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.761+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/3516583709 shutdown_connections 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.761+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bbadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.761+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bb82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.761+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/3516583709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bb31e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.761+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/3516583709 >> 172.21.15.26:0/3516583709 conn(0x55b2e2b7afb0 msgr2=0x55b2e2bae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.761+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/3516583709 shutdown_connections 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.761+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc0b80a2c00 0x7fc0b80a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55b2e2bb3bb0 tx=0x7fc0c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.761+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/3516583709 wait complete. 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.762+0000 7fc0dd9f3740 1 -- start start 2024-06-13T07:10:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.762+0000 7fc0dd9f3740 1 Processor -- start 2024-06-13T07:10:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.762+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.762+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bbd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.762+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b2e2bbb6f0 con 0x55b2e2bb8920 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2e2ad10b0 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b2e2bbb870 con 0x55b2e2bb37b0 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0dd9f3740 10 client.0 _ll_register_callbacks cb 0x55b2e2abcf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bbd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49986/0 (socket says 172.21.15.26:49986) 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3e00640 1 -- 172.21.15.26:0/2240252670 learned_addr learned my addr 172.21.15.26:0/2240252670 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3e00640 1 -- 172.21.15.26:0/2240252670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 msgr2=0x55b2e2bbd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bbd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3e00640 1 -- 172.21.15.26:0/2240252670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb8920 msgr2=0x55b2e2bc44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.763+0000 7fc0d3e00640 1 -- 172.21.15.26:0/2240252670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e28f6620 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.764+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbcf70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4004610 tx=0x7fc0c4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f366dd19ff7f29 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.764+0000 7fc0bd600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:10:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.764+0000 7fc0bd600640 1 -- 172.21.15.26:0/2240252670 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c401e680 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.764+0000 7fc0bd600640 1 -- 172.21.15.26:0/2240252670 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc0c401e820 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.764+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2240252670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0cc036030 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.764+0000 7fc0bd600640 1 -- 172.21.15.26:0/2240252670 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc0c401ea20 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.764+0000 7fc0dd9f3740 10 client.26246 fetch_fsmap learned FSMap version 35 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.764+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2240252670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b2e2bc7890 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.765+0000 7fc0bd600640 1 -- 172.21.15.26:0/2240252670 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0cc036030 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.765+0000 7fc0bd600640 1 -- 172.21.15.26:0/2240252670 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc0c4027070 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.765+0000 7fc0dd9f3740 10 client.26246 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.765+0000 7fc0dd9f3740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.765+0000 7fc0dd9f3740 20 client.26246 populate_metadata read hostname 'smithi026' 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.765+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b2e2bc8fd0 0x55b2e2bcb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.765+0000 7fc0dd9f3740 4 client.26246 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.765+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2240252670 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2e2b23b50 con 0x55b2e2bc8fd0 2024-06-13T07:10:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.766+0000 7fc0d3400640 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b2e2bc8fd0 0x55b2e2bcb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.767+0000 7fc0d3400640 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b2e2bc8fd0 0x55b2e2bcb3e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8928a6ef7331b88 server_cookie=6e8bc26378e0d4e6 in_seq=0 out_seq=0 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.767+0000 7fc0bd600640 10 client.26246 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0bd600640 1 -- 172.21.15.26:0/2240252670 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b2e2b23b50 con 0x55b2e2bc8fd0 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0bd600640 10 client.26246 handle_command_reply: tid=0 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 client.26246 shutdown 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2240252670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 msgr2=0x55b2e2bbcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbcf70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4004610 tx=0x7fc0c4004730 comp rx=0 tx=0).stop 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2240252670 shutdown_connections 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b2e2bc8fd0 0x55b2e2bcb3e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bbd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2240252670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbcf70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0bd600640 0 client.26246 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.769+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2240252670 >> 172.21.15.26:0/2240252670 conn(0x55b2e2b7afb0 msgr2=0x55b2e2bad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.770+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2240252670 shutdown_connections 2024-06-13T07:10:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.770+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2240252670 wait complete. 2024-06-13T07:10:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.770+0000 7fc0dd9f3740 20 client.26246 trim_cache size 0 max 16384 2024-06-13T07:10:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0dd9f3740 1 Processor -- start 2024-06-13T07:10:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0dd9f3740 1 -- start start 2024-06-13T07:10:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bb17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41aefb0 con 0x55b2e2bb37b0 2024-06-13T07:10:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41ac880 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41aa6d0 con 0x55b2e2bb8920 2024-06-13T07:10:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49996/0 (socket says 172.21.15.26:49996) 2024-06-13T07:10:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0d3e00640 1 -- 172.21.15.26:0/2664375180 learned_addr learned my addr 172.21.15.26:0/2664375180 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0d3400640 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.790+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bb17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.791+0000 7fc0d3e00640 1 -- 172.21.15.26:0/2664375180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 msgr2=0x55b2e2bb17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.791+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bb17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.791+0000 7fc0d3e00640 1 -- 172.21.15.26:0/2664375180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 msgr2=0x55b2e2bc44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.791+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.791+0000 7fc0d3e00640 1 -- 172.21.15.26:0/2664375180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e2bb31a0 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.791+0000 7fc0d3400640 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.791+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbd290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4000c00 tx=0x7fc0c4009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b4cba10a76f9ed server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.791+0000 7fc0bfe00640 1 -- 172.21.15.26:0/2664375180 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c403c070 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.792+0000 7fc0bfe00640 1 -- 172.21.15.26:0/2664375180 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c401e740 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.792+0000 7fc0bfe00640 1 -- 172.21.15.26:0/2664375180 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40296a0 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.792+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2664375180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 msgr2=0x55b2e2bbd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.792+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbd290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4000c00 tx=0x7fc0c4009dd0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2664375180 shutdown_connections 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb8920 0x55b2e2bb17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/2664375180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bbd290 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2664375180 >> 172.21.15.26:0/2664375180 conn(0x55b2e2b7afb0 msgr2=0x55b2e2bbcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2664375180 shutdown_connections 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/2664375180 wait complete. 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 -- start start 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.793+0000 7fc0dd9f3740 1 Processor -- start 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bc1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0dd9f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41aad40 con 0x55b2e2bb2dc0 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41ac1b0 con 0x55b2e2bb8920 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0dd9f3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41ab410 con 0x55b2e2bb37b0 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0dd9f3740 10 client.0 _ll_register_callbacks cb 0x7fc0c4009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bc1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0d8c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bc1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44848/0 (socket says 172.21.15.26:44848) 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.794+0000 7fc0d3e00640 1 -- 172.21.15.26:0/140732518 learned_addr learned my addr 172.21.15.26:0/140732518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0d8c00640 1 -- 172.21.15.26:0/140732518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 msgr2=0x55b2e2bc2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0d8c00640 1 -- 172.21.15.26:0/140732518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb2dc0 msgr2=0x55b2e2bc1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bc1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0d8c00640 1 -- 172.21.15.26:0/140732518 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e2bc3920 con 0x55b2e2bb8920 2024-06-13T07:10:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0d3400640 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bc1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0d8c00640 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc3330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc02aeb0 tx=0x7fc0cc031960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=188a9e2686ffc574 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0bd600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:10:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0bd600640 1 -- 172.21.15.26:0/140732518 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc054020 con 0x55b2e2bb8920 2024-06-13T07:10:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/140732518 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c401e740 con 0x55b2e2bb8920 2024-06-13T07:10:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.795+0000 7fc0bd600640 1 -- 172.21.15.26:0/140732518 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc0cc003b80 con 0x55b2e2bb8920 2024-06-13T07:10:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0bd600640 1 -- 172.21.15.26:0/140732518 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc0cc003d80 con 0x55b2e2bb8920 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0dd9f3740 10 client.26252 fetch_fsmap learned FSMap version 35 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/140732518 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b2e2c3c6e0 con 0x55b2e2bb8920 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0bd600640 1 -- 172.21.15.26:0/140732518 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c401e740 con 0x55b2e2bb8920 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0bd600640 1 -- 172.21.15.26:0/140732518 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc0cc043070 con 0x55b2e2bb8920 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0dd9f3740 10 client.26252 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0dd9f3740 10 client.26252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0dd9f3740 20 client.26252 populate_metadata read hostname 'smithi026' 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b2e2c3ce00 0x55b2e2c3f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0dd9f3740 4 client.26252 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.796+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/140732518 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2e2bb45e0 con 0x55b2e2c3ce00 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.797+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b2e2c3ce00 0x55b2e2c3f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.797+0000 7fc0d3e00640 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b2e2c3ce00 0x55b2e2c3f220 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b435775b738b18fd server_cookie=e585c3534ba1ac8f in_seq=0 out_seq=0 2024-06-13T07:10:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.797+0000 7fc0bd600640 10 client.26252 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0bd600640 1 -- 172.21.15.26:0/140732518 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b2e2bb45e0 con 0x55b2e2c3ce00 2024-06-13T07:10:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0bd600640 10 client.26252 handle_command_reply: tid=0 2024-06-13T07:10:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 client.26252 shutdown 2024-06-13T07:10:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/140732518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb8920 msgr2=0x55b2e2bc3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc3330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc02aeb0 tx=0x7fc0cc031960 comp rx=0 tx=0).stop 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/140732518 shutdown_connections 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b2e2c3ce00 0x55b2e2c3f220 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2e2bb8920 0x55b2e2bc3330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b2e2bb37b0 0x55b2e2bc2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0bd600640 0 client.26252 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 --2- 172.21.15.26:0/140732518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2e2bb2dc0 0x55b2e2bc1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.899+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/140732518 >> 172.21.15.26:0/140732518 conn(0x55b2e2b7afb0 msgr2=0x55b2e2bb17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.900+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/140732518 shutdown_connections 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.900+0000 7fc0dd9f3740 1 -- 172.21.15.26:0/140732518 wait complete. 2024-06-13T07:10:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.900+0000 7fc0dd9f3740 20 client.26252 trim_cache size 0 max 16384 2024-06-13T07:10:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:10:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:10:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:10:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.900+0000 7fc0d9600640 1 -- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc0b80a2c00 msgr2=0x7fc0b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.900+0000 7fc0d9600640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc0b80a2c00 0x7fc0b80a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55b2e2bb3bb0 tx=0x7fc0c4005f90 comp rx=0 tx=0).stop 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.900+0000 7fc0d9600640 1 -- 172.21.15.26:0/2660280893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0d4100a10 msgr2=0x7fc0d419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.900+0000 7fc0d9600640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0d4100a10 0x7fc0d419a230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000e9f0 tx=0x7fc0c000eeb0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.901+0000 7fc0d9600640 1 -- 172.21.15.26:0/2660280893 shutdown_connections 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.901+0000 7fc0d9600640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc0b80a2c00 0x7fc0b80a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.901+0000 7fc0d9600640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d41013b0 0x7fc0d41a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.901+0000 7fc0d9600640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0d4100a10 0x7fc0d419a230 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.901+0000 7fc0d9600640 1 --2- 172.21.15.26:0/2660280893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d41000b0 0x7fc0d4199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.901+0000 7fc0d9600640 1 -- 172.21.15.26:0/2660280893 >> 172.21.15.26:0/2660280893 conn(0x7fc0d40fbc90 msgr2=0x7fc0d40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.901+0000 7fc0d9600640 1 -- 172.21.15.26:0/2660280893 shutdown_connections 2024-06-13T07:10:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:24.901+0000 7fc0d9600640 1 -- 172.21.15.26:0/2660280893 wait complete. 2024-06-13T07:10:25.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:10:25.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:10:25.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:10:25.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 451042 4.74 MB/sec execute 399 sec latency 84.454 ms 2024-06-13T07:10:26.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:25 smithi026 ceph-mon[32041]: pgmap v807: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-06-13T07:10:26.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:10:26.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:10:26.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:10:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:25 smithi135 ceph-mon[36276]: pgmap v807: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-06-13T07:10:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:10:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:10:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:10:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:25 smithi136 ceph-mon[37229]: pgmap v807: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-06-13T07:10:26.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:10:26.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:10:26.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:10:26.568 INFO:tasks.workunit.client.0.smithi026.stdout: 1 452014 4.75 MB/sec execute 400 sec latency 27.181 ms 2024-06-13T07:10:27.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 453261 4.75 MB/sec execute 401 sec latency 19.523 ms 2024-06-13T07:10:28.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:27 smithi026 ceph-mon[32041]: pgmap v808: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:10:28.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:27 smithi135 ceph-mon[36276]: pgmap v808: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:10:28.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:27 smithi136 ceph-mon[37229]: pgmap v808: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:10:28.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 454265 4.76 MB/sec execute 402 sec latency 20.999 ms 2024-06-13T07:10:29.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 454633 4.75 MB/sec execute 403 sec latency 50.010 ms 2024-06-13T07:10:30.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:29 smithi026 ceph-mon[32041]: pgmap v809: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:10:30.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:29 smithi135 ceph-mon[36276]: pgmap v809: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:10:30.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:29 smithi136 ceph-mon[37229]: pgmap v809: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-06-13T07:10:30.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 455543 4.75 MB/sec execute 404 sec latency 27.605 ms 2024-06-13T07:10:31.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 456756 4.75 MB/sec execute 405 sec latency 24.911 ms 2024-06-13T07:10:32.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:31 smithi026 ceph-mon[32041]: pgmap v810: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-06-13T07:10:32.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:31 smithi135 ceph-mon[36276]: pgmap v810: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-06-13T07:10:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:31 smithi136 ceph-mon[37229]: pgmap v810: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-06-13T07:10:32.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 457786 4.76 MB/sec execute 406 sec latency 19.533 ms 2024-06-13T07:10:33.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 458152 4.75 MB/sec execute 407 sec latency 33.273 ms 2024-06-13T07:10:34.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:33 smithi026 ceph-mon[32041]: pgmap v811: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:10:34.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:33 smithi135 ceph-mon[36276]: pgmap v811: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:10:34.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:33 smithi136 ceph-mon[37229]: pgmap v811: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:10:34.569 INFO:tasks.workunit.client.0.smithi026.stdout: 1 458633 4.76 MB/sec execute 408 sec latency 52.631 ms 2024-06-13T07:10:35.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 459394 4.78 MB/sec execute 409 sec latency 28.453 ms 2024-06-13T07:10:36.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:35 smithi026 ceph-mon[32041]: pgmap v812: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:10:36.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:35 smithi135 ceph-mon[36276]: pgmap v812: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:10:36.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:35 smithi136 ceph-mon[37229]: pgmap v812: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-13T07:10:36.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 460750 4.78 MB/sec execute 410 sec latency 21.994 ms 2024-06-13T07:10:37.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 461734 4.79 MB/sec execute 411 sec latency 22.237 ms 2024-06-13T07:10:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 -- 172.21.15.26:0/489004790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898100930 msgr2=0x7fc898100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 --2- 172.21.15.26:0/489004790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898100930 0x7fc898100d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc888009900 tx=0x7fc88802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 -- 172.21.15.26:0/489004790 shutdown_connections 2024-06-13T07:10:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 --2- 172.21.15.26:0/489004790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8981024e0 0x7fc898108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 --2- 172.21.15.26:0/489004790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc898101b20 0x7fc898101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 --2- 172.21.15.26:0/489004790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898100930 0x7fc898100d50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 -- 172.21.15.26:0/489004790 >> 172.21.15.26:0/489004790 conn(0x7fc8980fbea0 msgr2=0x7fc8980fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 -- 172.21.15.26:0/489004790 shutdown_connections 2024-06-13T07:10:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.058+0000 7fc89d400640 1 -- 172.21.15.26:0/489004790 wait complete. 2024-06-13T07:10:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.059+0000 7fc89d400640 1 Processor -- start 2024-06-13T07:10:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.059+0000 7fc89d400640 1 -- start start 2024-06-13T07:10:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc89d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898100930 0x7fc898199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc89d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898101b20 0x7fc89819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc89d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8981024e0 0x7fc8981a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc89d400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc89810d6d0 con 0x7fc898101b20 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc89d400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc89810d550 con 0x7fc898100930 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc89d400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc89810d850 con 0x7fc8981024e0 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc896800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898100930 0x7fc898199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc896800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898100930 0x7fc898199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47306/0 (socket says 172.21.15.26:47306) 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc896800640 1 -- 172.21.15.26:0/2319272406 learned_addr learned my addr 172.21.15.26:0/2319272406 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc897200640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8981024e0 0x7fc8981a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.060+0000 7fc88fe00640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898101b20 0x7fc89819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc896800640 1 -- 172.21.15.26:0/2319272406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8981024e0 msgr2=0x7fc8981a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc896800640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8981024e0 0x7fc8981a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc896800640 1 -- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898101b20 msgr2=0x7fc89819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc896800640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898101b20 0x7fc89819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc896800640 1 -- 172.21.15.26:0/2319272406 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8981a1ad0 con 0x7fc898100930 2024-06-13T07:10:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc897200640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8981024e0 0x7fc8981a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc88fe00640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898101b20 0x7fc89819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc896800640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898100930 0x7fc898199eb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc888031610 tx=0x7fc888004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc88f400640 1 -- 172.21.15.26:0/2319272406 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88803f030 con 0x7fc898100930 2024-06-13T07:10:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc88f400640 1 -- 172.21.15.26:0/2319272406 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc888003ad0 con 0x7fc898100930 2024-06-13T07:10:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc88f400640 1 -- 172.21.15.26:0/2319272406 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888035070 con 0x7fc898100930 2024-06-13T07:10:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.061+0000 7fc89d400640 1 -- 172.21.15.26:0/2319272406 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8981a1d60 con 0x7fc898100930 2024-06-13T07:10:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.062+0000 7fc89d400640 1 -- 172.21.15.26:0/2319272406 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8981a21a0 con 0x7fc898100930 2024-06-13T07:10:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.065+0000 7fc88f400640 1 -- 172.21.15.26:0/2319272406 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc888042030 con 0x7fc898100930 2024-06-13T07:10:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.065+0000 7fc8a0206740 1 Processor -- start 2024-06-13T07:10:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc8a0206740 1 -- start start 2024-06-13T07:10:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc8a0206740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636bb74d420 con 0x5636bb7ea5f0 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc8a0206740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5636bb6f4c50 con 0x5636bb7ef760 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc8a0206740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636bb7493e0 con 0x5636bb7e9c00 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc897200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc897200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7ea020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:42586/0 (socket says 172.21.15.26:42586) 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc897200640 1 -- 172.21.15.26:0/167331993 learned_addr learned my addr 172.21.15.26:0/167331993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc896800640 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.066+0000 7fc88fe00640 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc897200640 1 -- 172.21.15.26:0/167331993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 msgr2=0x5636bb7f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc897200640 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc897200640 1 -- 172.21.15.26:0/167331993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 msgr2=0x5636bb7ef0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc897200640 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7ef0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc897200640 1 -- 172.21.15.26:0/167331993 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636bb461210 con 0x5636bb7e9c00 2024-06-13T07:10:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc896800640 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7ef0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc897200640 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7ea020 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8900117e0 tx=0x7fc890011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd40e402e972412f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc887400640 1 -- 172.21.15.26:0/167331993 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890011310 con 0x5636bb7e9c00 2024-06-13T07:10:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc887400640 1 -- 172.21.15.26:0/167331993 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890003e50 con 0x5636bb7e9c00 2024-06-13T07:10:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc887400640 1 -- 172.21.15.26:0/167331993 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8900045c0 con 0x5636bb7e9c00 2024-06-13T07:10:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.067+0000 7fc88f400640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc87c0a2c30 0x7fc87c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.068+0000 7fc88f400640 1 -- 172.21.15.26:0/2319272406 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc8880efbc0 con 0x7fc898100930 2024-06-13T07:10:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.068+0000 7fc896800640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc87c0a2c30 0x7fc87c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.068+0000 7fc8a0206740 1 -- 172.21.15.26:0/167331993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 msgr2=0x5636bb7ea020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.068+0000 7fc8a0206740 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7ea020 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8900117e0 tx=0x7fc890011ca0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc8a0206740 1 -- 172.21.15.26:0/167331993 shutdown_connections 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc8a0206740 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc8a0206740 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7ef0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc8a0206740 1 --2- 172.21.15.26:0/167331993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7ea020 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc8a0206740 1 -- 172.21.15.26:0/167331993 >> 172.21.15.26:0/167331993 conn(0x5636bb7b1fb0 msgr2=0x5636bb7e5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc896800640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc87c0a2c30 0x7fc87c0a50f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc888031440 tx=0x7fc88803b800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc8a0206740 1 -- 172.21.15.26:0/167331993 shutdown_connections 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc8a0206740 1 -- 172.21.15.26:0/167331993 wait complete. 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.069+0000 7fc8a0206740 1 -- start start 2024-06-13T07:10:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc8a0206740 1 Processor -- start 2024-06-13T07:10:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7fb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc8a0206740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636bb7f2710 con 0x5636bb7e9c00 2024-06-13T07:10:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc8a0206740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5636bb7080b0 con 0x5636bb7ef760 2024-06-13T07:10:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc8a0206740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636bb7f2890 con 0x5636bb7ea5f0 2024-06-13T07:10:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc8a0206740 10 client.0 _ll_register_callbacks cb 0x5636bb6f3f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc88fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc88fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:42588/0 (socket says 172.21.15.26:42588) 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc897200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.070+0000 7fc88fe00640 1 -- 172.21.15.26:0/1564864301 learned_addr learned my addr 172.21.15.26:0/1564864301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc897200640 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc88fe00640 1 -- 172.21.15.26:0/1564864301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 msgr2=0x5636bb7fb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc88fe00640 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7fb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc88fe00640 1 -- 172.21.15.26:0/1564864301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 msgr2=0x5636bb7f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc88fe00640 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc88fe00640 1 -- 172.21.15.26:0/1564864301 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636bb52d620 con 0x5636bb7ea5f0 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc897200640 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc88fe00640 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f44d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc88000bec0 tx=0x7fc8800117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e78d85fe6095f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc87be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:10:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.071+0000 7fc87be00640 1 -- 172.21.15.26:0/1564864301 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880005220 con 0x5636bb7ea5f0 2024-06-13T07:10:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc8a0206740 1 -- 172.21.15.26:0/1564864301 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc890003e50 con 0x5636bb7ea5f0 2024-06-13T07:10:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc87be00640 1 -- 172.21.15.26:0/1564864301 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc8800053c0 con 0x5636bb7ea5f0 2024-06-13T07:10:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc87be00640 1 -- 172.21.15.26:0/1564864301 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8800055c0 con 0x5636bb7ea5f0 2024-06-13T07:10:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc8a0206740 10 client.26299 fetch_fsmap learned FSMap version 35 2024-06-13T07:10:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc8a0206740 1 -- 172.21.15.26:0/1564864301 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5636bb7fe9a0 con 0x5636bb7ea5f0 2024-06-13T07:10:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc87be00640 1 -- 172.21.15.26:0/1564864301 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890003e50 con 0x5636bb7ea5f0 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc87be00640 1 -- 172.21.15.26:0/1564864301 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc88001b9a0 con 0x5636bb7ea5f0 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc8a0206740 10 client.26299 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc8a0206740 10 client.26299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc8a0206740 20 client.26299 populate_metadata read hostname 'smithi026' 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.072+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5636bb800020 0x5636bb802470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.073+0000 7fc8a0206740 4 client.26299 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.073+0000 7fc8a0206740 1 -- 172.21.15.26:0/1564864301 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636bb75ab50 con 0x5636bb800020 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.073+0000 7fc897200640 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5636bb800020 0x5636bb802470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.073+0000 7fc897200640 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5636bb800020 0x5636bb802470 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35373b4a254ebb46 server_cookie=6f5339b2c497012e in_seq=0 out_seq=0 2024-06-13T07:10:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.073+0000 7fc87be00640 10 client.26299 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc87be00640 1 -- 172.21.15.26:0/1564864301 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5636bb75ab50 con 0x5636bb800020 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc87be00640 10 client.26299 handle_command_reply: tid=0 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 client.26299 shutdown 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 -- 172.21.15.26:0/1564864301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 msgr2=0x5636bb7f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f44d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc88000bec0 tx=0x7fc8800117b0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 -- 172.21.15.26:0/1564864301 shutdown_connections 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5636bb800020 0x5636bb802470 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f44d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1564864301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc87be00640 0 client.26299 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 -- 172.21.15.26:0/1564864301 >> 172.21.15.26:0/1564864301 conn(0x5636bb7b1fb0 msgr2=0x5636bb7e4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 -- 172.21.15.26:0/1564864301 shutdown_connections 2024-06-13T07:10:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 1 -- 172.21.15.26:0/1564864301 wait complete. 2024-06-13T07:10:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.075+0000 7fc8a0206740 20 client.26299 trim_cache size 0 max 16384 2024-06-13T07:10:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.094+0000 7fc8a0206740 1 Processor -- start 2024-06-13T07:10:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.094+0000 7fc8a0206740 1 -- start start 2024-06-13T07:10:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.094+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.094+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.094+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.094+0000 7fc8a0206740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981af220 con 0x5636bb7ea5f0 2024-06-13T07:10:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.094+0000 7fc8a0206740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981acaf0 con 0x5636bb7ef760 2024-06-13T07:10:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.094+0000 7fc8a0206740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981aa940 con 0x5636bb7e9c00 2024-06-13T07:10:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7e8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47334/0 (socket says 172.21.15.26:47334) 2024-06-13T07:10:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 -- 172.21.15.26:0/1750153066 learned_addr learned my addr 172.21.15.26:0/1750153066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc88fe00640 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 -- 172.21.15.26:0/1750153066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 msgr2=0x5636bb7e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7e8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 -- 172.21.15.26:0/1750153066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 msgr2=0x5636bb7fb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7fb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc897200640 1 -- 172.21.15.26:0/1750153066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636bb7eb400 con 0x5636bb7ef760 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.095+0000 7fc88fe00640 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.096+0000 7fc897200640 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7e8b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc8900117b0 tx=0x7fc890005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afae7ff313ed16a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.096+0000 7fc886a00640 1 -- 172.21.15.26:0/1750153066 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890023040 con 0x5636bb7ef760 2024-06-13T07:10:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.096+0000 7fc886a00640 1 -- 172.21.15.26:0/1750153066 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8900050f0 con 0x5636bb7ef760 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.096+0000 7fc886a00640 1 -- 172.21.15.26:0/1750153066 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89002b5a0 con 0x5636bb7ef760 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 -- 172.21.15.26:0/1750153066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 msgr2=0x5636bb7e8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7e8b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc8900117b0 tx=0x7fc890005bd0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 -- 172.21.15.26:0/1750153066 shutdown_connections 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb7e8b50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7ea5f0 0x5636bb7fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1750153066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7e9c00 0x5636bb7e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 -- 172.21.15.26:0/1750153066 >> 172.21.15.26:0/1750153066 conn(0x5636bb7b1fb0 msgr2=0x5636bb7eb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 -- 172.21.15.26:0/1750153066 shutdown_connections 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 -- 172.21.15.26:0/1750153066 wait complete. 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.097+0000 7fc8a0206740 1 -- start start 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc8a0206740 1 Processor -- start 2024-06-13T07:10:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc8a0206740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb868d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc8a0206740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981aafb0 con 0x5636bb7e9c00 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc8a0206740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981ac420 con 0x5636bb7ef760 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc8a0206740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981ab680 con 0x5636bb7ea5f0 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc8a0206740 10 client.0 _ll_register_callbacks cb 0x7fc88000b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc897200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc897200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45596/0 (socket says 172.21.15.26:45596) 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc88fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc897200640 1 -- 172.21.15.26:0/1604343683 learned_addr learned my addr 172.21.15.26:0/1604343683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.098+0000 7fc897200640 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb868d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc88fe00640 1 -- 172.21.15.26:0/1604343683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 msgr2=0x5636bb868d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc88fe00640 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb868d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc88fe00640 1 -- 172.21.15.26:0/1604343683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 msgr2=0x5636bb7f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc88fe00640 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc88fe00640 1 -- 172.21.15.26:0/1604343683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636bb7f8d70 con 0x5636bb7ea5f0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc88fe00640 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f8730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc880000c00 tx=0x7fc88000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f15a278ce8ff60 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc87be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc87be00640 1 -- 172.21.15.26:0/1604343683 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880012930 con 0x5636bb7ea5f0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc87be00640 1 -- 172.21.15.26:0/1604343683 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc880012ad0 con 0x5636bb7ea5f0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.099+0000 7fc8a0206740 1 -- 172.21.15.26:0/1604343683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8900050f0 con 0x5636bb7ea5f0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc87be00640 1 -- 172.21.15.26:0/1604343683 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc880012cd0 con 0x5636bb7ea5f0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc8a0206740 10 client.26305 fetch_fsmap learned FSMap version 35 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc8a0206740 1 -- 172.21.15.26:0/1604343683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5636bb86a430 con 0x5636bb7ea5f0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc87be00640 1 -- 172.21.15.26:0/1604343683 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8900050f0 con 0x5636bb7ea5f0 2024-06-13T07:10:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc87be00640 1 -- 172.21.15.26:0/1604343683 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc88002c730 con 0x5636bb7ea5f0 2024-06-13T07:10:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc8a0206740 10 client.26305 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:10:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc8a0206740 10 client.26305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:10:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc8a0206740 20 client.26305 populate_metadata read hostname 'smithi026' 2024-06-13T07:10:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5636bb86adc0 0x5636bb86d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc8a0206740 4 client.26305 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.100+0000 7fc8a0206740 1 -- 172.21.15.26:0/1604343683 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636bb754ea0 con 0x5636bb86adc0 2024-06-13T07:10:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.101+0000 7fc897200640 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5636bb86adc0 0x5636bb86d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.101+0000 7fc897200640 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5636bb86adc0 0x5636bb86d1e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf5dabf4e03f876 server_cookie=2d2e8771dec32e99 in_seq=0 out_seq=0 2024-06-13T07:10:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.101+0000 7fc87be00640 10 client.26305 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc87be00640 1 -- 172.21.15.26:0/1604343683 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5636bb754ea0 con 0x5636bb86adc0 2024-06-13T07:10:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc87be00640 10 client.26305 handle_command_reply: tid=0 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 client.26305 shutdown 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 -- 172.21.15.26:0/1604343683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 msgr2=0x5636bb7f8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f8730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc880000c00 tx=0x7fc88000ba10 comp rx=0 tx=0).stop 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 -- 172.21.15.26:0/1604343683 shutdown_connections 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5636bb86adc0 0x5636bb86d1e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636bb7ef760 0x5636bb868d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636bb7ea5f0 0x5636bb7f8730 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 --2- 172.21.15.26:0/1604343683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636bb7e9c00 0x5636bb7f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc87be00640 0 client.26305 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.204+0000 7fc8a0206740 1 -- 172.21.15.26:0/1604343683 >> 172.21.15.26:0/1604343683 conn(0x5636bb7b1fb0 msgr2=0x5636bb7e8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.205+0000 7fc8a0206740 1 -- 172.21.15.26:0/1604343683 shutdown_connections 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.205+0000 7fc8a0206740 1 -- 172.21.15.26:0/1604343683 wait complete. 2024-06-13T07:10:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.205+0000 7fc8a0206740 20 client.26305 trim_cache size 0 max 16384 2024-06-13T07:10:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:10:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:10:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.205+0000 7fc89d400640 1 -- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc87c0a2c30 msgr2=0x7fc87c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.205+0000 7fc89d400640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc87c0a2c30 0x7fc87c0a50f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc888031440 tx=0x7fc88803b800 comp rx=0 tx=0).stop 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.205+0000 7fc89d400640 1 -- 172.21.15.26:0/2319272406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898100930 msgr2=0x7fc898199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.205+0000 7fc89d400640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898100930 0x7fc898199eb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc888031610 tx=0x7fc888004270 comp rx=0 tx=0).stop 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.206+0000 7fc89d400640 1 -- 172.21.15.26:0/2319272406 shutdown_connections 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.206+0000 7fc89d400640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc87c0a2c30 0x7fc87c0a50f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.206+0000 7fc89d400640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8981024e0 0x7fc8981a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.206+0000 7fc89d400640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc898101b20 0x7fc89819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.206+0000 7fc89d400640 1 --2- 172.21.15.26:0/2319272406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898100930 0x7fc898199eb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.206+0000 7fc89d400640 1 -- 172.21.15.26:0/2319272406 >> 172.21.15.26:0/2319272406 conn(0x7fc8980fbea0 msgr2=0x7fc898109730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.206+0000 7fc89d400640 1 -- 172.21.15.26:0/2319272406 shutdown_connections 2024-06-13T07:10:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:38.206+0000 7fc89d400640 1 -- 172.21.15.26:0/2319272406 wait complete. 2024-06-13T07:10:38.251 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:37 smithi026 ceph-mon[32041]: pgmap v813: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:10:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:37 smithi135 ceph-mon[36276]: pgmap v813: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:10:38.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:37 smithi136 ceph-mon[37229]: pgmap v813: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:10:38.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 462343 4.79 MB/sec execute 412 sec latency 80.791 ms 2024-06-13T07:10:39.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 462971 4.78 MB/sec execute 413 sec latency 55.594 ms 2024-06-13T07:10:40.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:39 smithi026 ceph-mon[32041]: pgmap v814: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-06-13T07:10:40.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:39 smithi135 ceph-mon[36276]: pgmap v814: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-06-13T07:10:40.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:39 smithi136 ceph-mon[37229]: pgmap v814: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-06-13T07:10:40.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 464310 4.78 MB/sec execute 414 sec latency 24.741 ms 2024-06-13T07:10:41.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 465391 4.79 MB/sec execute 415 sec latency 20.342 ms 2024-06-13T07:10:42.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:41 smithi026 ceph-mon[32041]: pgmap v815: 129 pgs: 129 active+clean; 92 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:10:42.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:41 smithi135 ceph-mon[36276]: pgmap v815: 129 pgs: 129 active+clean; 92 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:10:42.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:41 smithi136 ceph-mon[37229]: pgmap v815: 129 pgs: 129 active+clean; 92 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-06-13T07:10:42.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 465871 4.79 MB/sec execute 416 sec latency 22.178 ms 2024-06-13T07:10:43.570 INFO:tasks.workunit.client.0.smithi026.stdout: 1 466538 4.78 MB/sec execute 417 sec latency 29.909 ms 2024-06-13T07:10:44.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:43 smithi026 ceph-mon[32041]: pgmap v816: 129 pgs: 129 active+clean; 91 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-06-13T07:10:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:43 smithi135 ceph-mon[36276]: pgmap v816: 129 pgs: 129 active+clean; 91 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-06-13T07:10:44.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:43 smithi136 ceph-mon[37229]: pgmap v816: 129 pgs: 129 active+clean; 91 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-06-13T07:10:44.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 467350 4.78 MB/sec execute 418 sec latency 284.449 ms 2024-06-13T07:10:45.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 468652 4.78 MB/sec execute 419 sec latency 22.319 ms 2024-06-13T07:10:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:45 smithi026 ceph-mon[32041]: pgmap v817: 129 pgs: 129 active+clean; 92 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:10:46.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:45 smithi135 ceph-mon[36276]: pgmap v817: 129 pgs: 129 active+clean; 92 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:10:46.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:45 smithi136 ceph-mon[37229]: pgmap v817: 129 pgs: 129 active+clean; 92 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-06-13T07:10:46.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 469382 4.79 MB/sec execute 420 sec latency 26.112 ms 2024-06-13T07:10:47.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 470047 4.78 MB/sec execute 421 sec latency 28.515 ms 2024-06-13T07:10:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:47 smithi026 ceph-mon[32041]: pgmap v818: 129 pgs: 129 active+clean; 93 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-06-13T07:10:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:47 smithi135 ceph-mon[36276]: pgmap v818: 129 pgs: 129 active+clean; 93 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-06-13T07:10:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:47 smithi136 ceph-mon[37229]: pgmap v818: 129 pgs: 129 active+clean; 93 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-06-13T07:10:48.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 471404 4.78 MB/sec execute 422 sec latency 30.244 ms 2024-06-13T07:10:49.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 472518 4.79 MB/sec execute 423 sec latency 57.451 ms 2024-06-13T07:10:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:49 smithi026 ceph-mon[32041]: pgmap v819: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-06-13T07:10:50.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:49 smithi135 ceph-mon[36276]: pgmap v819: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-06-13T07:10:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:49 smithi136 ceph-mon[37229]: pgmap v819: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-06-13T07:10:50.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 473011 4.79 MB/sec execute 424 sec latency 31.101 ms 2024-06-13T07:10:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 -- 172.21.15.26:0/3837514467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 msgr2=0x7fab74104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 --2- 172.21.15.26:0/3837514467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 0x7fab74104710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fab64009900 tx=0x7fab6402a410 comp rx=0 tx=0).stop 2024-06-13T07:10:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 -- 172.21.15.26:0/3837514467 shutdown_connections 2024-06-13T07:10:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 --2- 172.21.15.26:0/3837514467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74107580 0x7fab74109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 --2- 172.21.15.26:0/3837514467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab74104c50 0x7fab74107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 --2- 172.21.15.26:0/3837514467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 0x7fab74104710 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 -- 172.21.15.26:0/3837514467 >> 172.21.15.26:0/3837514467 conn(0x7fab740fbc90 msgr2=0x7fab740fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 -- 172.21.15.26:0/3837514467 shutdown_connections 2024-06-13T07:10:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.356+0000 7fab79a00640 1 -- 172.21.15.26:0/3837514467 wait complete. 2024-06-13T07:10:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.358+0000 7fab79a00640 1 Processor -- start 2024-06-13T07:10:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.358+0000 7fab79a00640 1 -- start start 2024-06-13T07:10:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.358+0000 7fab79a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 0x7fab74197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.358+0000 7fab79a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab74104c50 0x7fab741980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.358+0000 7fab79a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74107580 0x7fab7419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.358+0000 7fab79a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410d510 con 0x7fab74107580 2024-06-13T07:10:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.358+0000 7fab79a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410d390 con 0x7fab74102320 2024-06-13T07:10:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.358+0000 7fab79a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410d690 con 0x7fab74104c50 2024-06-13T07:10:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 0x7fab74197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 0x7fab74197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59702/0 (socket says 172.21.15.26:59702) 2024-06-13T07:10:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 -- 172.21.15.26:0/4292433782 learned_addr learned my addr 172.21.15.26:0/4292433782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab79000640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74107580 0x7fab7419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab6be00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab74104c50 0x7fab741980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 -- 172.21.15.26:0/4292433782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab74104c50 msgr2=0x7fab741980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab74104c50 0x7fab741980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 -- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74107580 msgr2=0x7fab7419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74107580 0x7fab7419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 -- 172.21.15.26:0/4292433782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7419f7d0 con 0x7fab74102320 2024-06-13T07:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.359+0000 7fab73e00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 0x7fab74197bb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fab64031440 tx=0x7fab64032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.360+0000 7fab71600640 1 -- 172.21.15.26:0/4292433782 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6403f030 con 0x7fab74102320 2024-06-13T07:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.360+0000 7fab71600640 1 -- 172.21.15.26:0/4292433782 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab64003ce0 con 0x7fab74102320 2024-06-13T07:10:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.360+0000 7fab79a00640 1 -- 172.21.15.26:0/4292433782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7419fa60 con 0x7fab74102320 2024-06-13T07:10:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.360+0000 7fab71600640 1 -- 172.21.15.26:0/4292433782 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64035070 con 0x7fab74102320 2024-06-13T07:10:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.360+0000 7fab79a00640 1 -- 172.21.15.26:0/4292433782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7419ffa0 con 0x7fab74102320 2024-06-13T07:10:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.364+0000 7fab7de28740 1 Processor -- start 2024-06-13T07:10:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab7de28740 1 -- start start 2024-06-13T07:10:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab71600640 1 -- 172.21.15.26:0/4292433782 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fab64042030 con 0x7fab74102320 2024-06-13T07:10:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab7de28740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f4dc0 0x557c892f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab7de28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab7de28740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892fa920 0x557c892fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab7de28740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557c89258420 con 0x557c892fa920 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab7de28740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557c891ffc50 con 0x557c892f4dc0 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab7de28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557c892543e0 con 0x557c892f57b0 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab79000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f4dc0 0x557c892f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab79000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f4dc0 0x557c892f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59714/0 (socket says 172.21.15.26:59714) 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab79000640 1 -- 172.21.15.26:0/3435959519 learned_addr learned my addr 172.21.15.26:0/3435959519 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.365+0000 7fab6be00640 1 --2- 172.21.15.26:0/3435959519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892fa920 0x557c892fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab79000640 1 -- 172.21.15.26:0/3435959519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 msgr2=0x557c892fa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab79000640 1 --2- 172.21.15.26:0/3435959519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892fa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab79000640 1 -- 172.21.15.26:0/3435959519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892fa920 msgr2=0x557c892fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab79000640 1 --2- 172.21.15.26:0/3435959519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892fa920 0x557c892fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab79000640 1 -- 172.21.15.26:0/3435959519 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c88f6c210 con 0x557c892f4dc0 2024-06-13T07:10:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab79000640 1 --2- 172.21.15.26:0/3435959519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f4dc0 0x557c892f51e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0039f0 tx=0x7fab6c003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b081308b213db74 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab69600640 1 -- 172.21.15.26:0/3435959519 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c01a470 con 0x557c892f4dc0 2024-06-13T07:10:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab69600640 1 -- 172.21.15.26:0/3435959519 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6c015070 con 0x557c892f4dc0 2024-06-13T07:10:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.366+0000 7fab69600640 1 -- 172.21.15.26:0/3435959519 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c010900 con 0x557c892f4dc0 2024-06-13T07:10:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.367+0000 7fab71600640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fab580a2c00 0x7fab580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.367+0000 7fab71600640 1 -- 172.21.15.26:0/4292433782 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fab640ef910 con 0x7fab74102320 2024-06-13T07:10:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.367+0000 7fab7de28740 1 -- 172.21.15.26:0/3435959519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f4dc0 msgr2=0x557c892f51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.367+0000 7fab7de28740 1 --2- 172.21.15.26:0/3435959519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f4dc0 0x557c892f51e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0039f0 tx=0x7fab6c003eb0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.368+0000 7fab73e00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fab580a2c00 0x7fab580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.368+0000 7fab7de28740 1 -- 172.21.15.26:0/3435959519 shutdown_connections 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.368+0000 7fab7de28740 1 --2- 172.21.15.26:0/3435959519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892fa920 0x557c892fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.368+0000 7fab7de28740 1 --2- 172.21.15.26:0/3435959519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.368+0000 7fab7de28740 1 --2- 172.21.15.26:0/3435959519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f4dc0 0x557c892f51e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.368+0000 7fab7de28740 1 -- 172.21.15.26:0/3435959519 >> 172.21.15.26:0/3435959519 conn(0x557c892bcfb0 msgr2=0x557c892f0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.368+0000 7fab7de28740 1 -- 172.21.15.26:0/3435959519 shutdown_connections 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.368+0000 7fab7de28740 1 -- 172.21.15.26:0/3435959519 wait complete. 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.369+0000 7fab7de28740 1 -- start start 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.369+0000 7fab73e00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fab580a2c00 0x7fab580a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fab640ed390 tx=0x7fab640f5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.369+0000 7fab7de28740 1 Processor -- start 2024-06-13T07:10:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.369+0000 7fab7de28740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab7de28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab7de28740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 0x557c89306670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab7de28740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557c892fd8d0 con 0x557c892f4dc0 2024-06-13T07:10:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab7de28740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557c892130b0 con 0x557c892fa920 2024-06-13T07:10:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab7de28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557c892fda50 con 0x557c892f57b0 2024-06-13T07:10:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab7de28740 10 client.0 _ll_register_callbacks cb 0x557c891fef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab79000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab6be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab79000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45268/0 (socket says 172.21.15.26:45268) 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab79000640 1 -- 172.21.15.26:0/2868657039 learned_addr learned my addr 172.21.15.26:0/2868657039 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab6be00640 1 -- 172.21.15.26:0/2868657039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 msgr2=0x557c89306670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab6be00640 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 0x557c89306670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab6be00640 1 -- 172.21.15.26:0/2868657039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 msgr2=0x557c892ff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab6be00640 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab6be00640 1 -- 172.21.15.26:0/2868657039 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c89038620 con 0x557c892f57b0 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.370+0000 7fab79000640 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.371+0000 7fab6be00640 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892ff630 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab741990c0 tx=0x7fab5c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f20c297430e3510 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.371+0000 7fab62000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.371+0000 7fab62000640 1 -- 172.21.15.26:0/2868657039 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c004050 con 0x557c892f57b0 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.371+0000 7fab7de28740 1 -- 172.21.15.26:0/2868657039 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6c015070 con 0x557c892f57b0 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.371+0000 7fab62000640 1 -- 172.21.15.26:0/2868657039 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fab5c0041f0 con 0x557c892f57b0 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.371+0000 7fab62000640 1 -- 172.21.15.26:0/2868657039 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fab5c0043f0 con 0x557c892f57b0 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.371+0000 7fab7de28740 10 client.26317 fetch_fsmap learned FSMap version 35 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.371+0000 7fab7de28740 1 -- 172.21.15.26:0/2868657039 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557c89309b00 con 0x557c892f57b0 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.372+0000 7fab62000640 1 -- 172.21.15.26:0/2868657039 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6c015070 con 0x557c892f57b0 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.372+0000 7fab62000640 1 -- 172.21.15.26:0/2868657039 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fab5c003bc0 con 0x557c892f57b0 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.372+0000 7fab7de28740 10 client.26317 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.372+0000 7fab7de28740 10 client.26317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:10:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.372+0000 7fab7de28740 20 client.26317 populate_metadata read hostname 'smithi026' 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.372+0000 7fab7de28740 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557c8930b1f0 0x557c8930d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.372+0000 7fab7de28740 4 client.26317 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.372+0000 7fab7de28740 1 -- 172.21.15.26:0/2868657039 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c89265b50 con 0x557c8930b1f0 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.373+0000 7fab79000640 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557c8930b1f0 0x557c8930d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.373+0000 7fab79000640 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557c8930b1f0 0x557c8930d670 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b34bbd6f999c0e server_cookie=24cacf996adbf643 in_seq=0 out_seq=0 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.373+0000 7fab62000640 10 client.26317 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.375+0000 7fab62000640 1 -- 172.21.15.26:0/2868657039 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557c89265b50 con 0x557c8930b1f0 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.375+0000 7fab62000640 10 client.26317 handle_command_reply: tid=0 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.375+0000 7fab7de28740 1 client.26317 shutdown 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 -- 172.21.15.26:0/2868657039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 msgr2=0x557c892ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892ff630 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab741990c0 tx=0x7fab5c012b60 comp rx=0 tx=0).stop 2024-06-13T07:10:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 -- 172.21.15.26:0/2868657039 shutdown_connections 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557c8930b1f0 0x557c8930d670 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 0x557c89306670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f57b0 0x557c892ff630 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 --2- 172.21.15.26:0/2868657039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 -- 172.21.15.26:0/2868657039 >> 172.21.15.26:0/2868657039 conn(0x557c892bcfb0 msgr2=0x557c892ef9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab62000640 0 client.26317 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 -- 172.21.15.26:0/2868657039 shutdown_connections 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 1 -- 172.21.15.26:0/2868657039 wait complete. 2024-06-13T07:10:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.376+0000 7fab7de28740 20 client.26317 trim_cache size 0 max 16384 2024-06-13T07:10:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.395+0000 7fab7de28740 1 Processor -- start 2024-06-13T07:10:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.395+0000 7fab7de28740 1 -- start start 2024-06-13T07:10:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab7de28740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab7de28740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f57b0 0x557c892ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab7de28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892fa920 0x557c892f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab7de28740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab741aced0 con 0x557c892f4dc0 2024-06-13T07:10:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab7de28740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab741aa7a0 con 0x557c892f57b0 2024-06-13T07:10:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab7de28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab741a85f0 con 0x557c892fa920 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab79000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab6be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f57b0 0x557c892ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab6be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f57b0 0x557c892ff630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59730/0 (socket says 172.21.15.26:59730) 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab6be00640 1 -- 172.21.15.26:0/4099517203 learned_addr learned my addr 172.21.15.26:0/4099517203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.396+0000 7fab79000640 1 --2- 172.21.15.26:0/4099517203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892fa920 0x557c892f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab6be00640 1 -- 172.21.15.26:0/4099517203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892fa920 msgr2=0x557c892f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab6be00640 1 --2- 172.21.15.26:0/4099517203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892fa920 0x557c892f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab6be00640 1 -- 172.21.15.26:0/4099517203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 msgr2=0x557c892ff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab6be00640 1 --2- 172.21.15.26:0/4099517203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab6be00640 1 -- 172.21.15.26:0/4099517203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c892f65c0 con 0x557c892f57b0 2024-06-13T07:10:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab6be00640 1 --2- 172.21.15.26:0/4099517203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f57b0 0x557c892ff630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab5c000c00 tx=0x7fab5c009d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c06488e50f45047 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab68c00640 1 -- 172.21.15.26:0/4099517203 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c02d020 con 0x557c892f57b0 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab68c00640 1 -- 172.21.15.26:0/4099517203 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab5c003fb0 con 0x557c892f57b0 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.397+0000 7fab68c00640 1 -- 172.21.15.26:0/4099517203 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00f790 con 0x557c892f57b0 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.398+0000 7fab7de28740 1 -- 172.21.15.26:0/4099517203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f57b0 msgr2=0x557c892ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.398+0000 7fab7de28740 1 --2- 172.21.15.26:0/4099517203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f57b0 0x557c892ff630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab5c000c00 tx=0x7fab5c009d10 comp rx=0 tx=0).stop 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.398+0000 7fab7de28740 1 -- 172.21.15.26:0/4099517203 shutdown_connections 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.398+0000 7fab7de28740 1 --2- 172.21.15.26:0/4099517203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892f57b0 0x557c892ff630 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.398+0000 7fab7de28740 1 --2- 172.21.15.26:0/4099517203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892fa920 0x557c892f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.398+0000 7fab7de28740 1 --2- 172.21.15.26:0/4099517203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f4dc0 0x557c892ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.398+0000 7fab7de28740 1 -- 172.21.15.26:0/4099517203 >> 172.21.15.26:0/4099517203 conn(0x557c892bcfb0 msgr2=0x557c892f6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.398+0000 7fab7de28740 1 -- 172.21.15.26:0/4099517203 shutdown_connections 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.399+0000 7fab7de28740 1 -- 172.21.15.26:0/4099517203 wait complete. 2024-06-13T07:10:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.399+0000 7fab7de28740 1 -- start start 2024-06-13T07:10:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.399+0000 7fab7de28740 1 Processor -- start 2024-06-13T07:10:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab7de28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f4dc0 0x557c89370dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab7de28740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f57b0 0x557c89371300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab7de28740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 0x557c89378340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab7de28740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab741a8c60 con 0x557c892f57b0 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab7de28740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab741aa0d0 con 0x557c892fa920 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab7de28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab741a9330 con 0x557c892f4dc0 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab7de28740 10 client.0 _ll_register_callbacks cb 0x7fab5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab79000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f4dc0 0x557c89370dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab79000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f4dc0 0x557c89370dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58836/0 (socket says 172.21.15.26:58836) 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab79000640 1 -- 172.21.15.26:0/1514486843 learned_addr learned my addr 172.21.15.26:0/1514486843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab79000640 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 0x557c89378340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.400+0000 7fab6be00640 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f57b0 0x557c89371300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab79000640 1 -- 172.21.15.26:0/1514486843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 msgr2=0x557c89378340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab79000640 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 0x557c89378340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab79000640 1 -- 172.21.15.26:0/1514486843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f57b0 msgr2=0x557c89371300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab79000640 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f57b0 0x557c89371300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab79000640 1 -- 172.21.15.26:0/1514486843 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c89378930 con 0x557c892f4dc0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab79000640 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f4dc0 0x557c89370dc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006010 tx=0x7fab6c02b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4935dd68b9dfcb66 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab62000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab62000640 1 -- 172.21.15.26:0/1514486843 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c011640 con 0x557c892f4dc0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab7de28740 1 -- 172.21.15.26:0/1514486843 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5c003fb0 con 0x557c892f4dc0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.401+0000 7fab62000640 1 -- 172.21.15.26:0/1514486843 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fab6c0117e0 con 0x557c892f4dc0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab62000640 1 -- 172.21.15.26:0/1514486843 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fab6c0119e0 con 0x557c892f4dc0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab7de28740 10 client.26323 fetch_fsmap learned FSMap version 35 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab7de28740 1 -- 172.21.15.26:0/1514486843 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557c89379c90 con 0x557c892f4dc0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab62000640 1 -- 172.21.15.26:0/1514486843 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab5c003fb0 con 0x557c892f4dc0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab62000640 1 -- 172.21.15.26:0/1514486843 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fab6c026420 con 0x557c892f4dc0 2024-06-13T07:10:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab7de28740 10 client.26323 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:10:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab7de28740 10 client.26323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:10:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab7de28740 20 client.26323 populate_metadata read hostname 'smithi026' 2024-06-13T07:10:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab7de28740 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557c8937a370 0x557c8937c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab7de28740 4 client.26323 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.402+0000 7fab7de28740 1 -- 172.21.15.26:0/1514486843 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c8925fea0 con 0x557c8937a370 2024-06-13T07:10:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.403+0000 7fab6be00640 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557c8937a370 0x557c8937c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.403+0000 7fab6be00640 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557c8937a370 0x557c8937c790 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7463e0aa40d7f06b server_cookie=364858dfb6650bbb in_seq=0 out_seq=0 2024-06-13T07:10:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.403+0000 7fab62000640 10 client.26323 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab62000640 1 -- 172.21.15.26:0/1514486843 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557c8925fea0 con 0x557c8937a370 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab62000640 10 client.26323 handle_command_reply: tid=0 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 client.26323 shutdown 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 -- 172.21.15.26:0/1514486843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f4dc0 msgr2=0x557c89370dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f4dc0 0x557c89370dc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006010 tx=0x7fab6c02b040 comp rx=0 tx=0).stop 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 -- 172.21.15.26:0/1514486843 shutdown_connections 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557c8937a370 0x557c8937c790 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c892fa920 0x557c89378340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c892f57b0 0x557c89371300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab62000640 0 client.26323 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:10:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 --2- 172.21.15.26:0/1514486843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c892f4dc0 0x557c89370dc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.498+0000 7fab7de28740 1 -- 172.21.15.26:0/1514486843 >> 172.21.15.26:0/1514486843 conn(0x557c892bcfb0 msgr2=0x557c892ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.499+0000 7fab7de28740 1 -- 172.21.15.26:0/1514486843 shutdown_connections 2024-06-13T07:10:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.499+0000 7fab7de28740 1 -- 172.21.15.26:0/1514486843 wait complete. 2024-06-13T07:10:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.499+0000 7fab7de28740 20 client.26323 trim_cache size 0 max 16384 2024-06-13T07:10:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:10:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:10:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 -- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fab580a2c00 msgr2=0x7fab580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fab580a2c00 0x7fab580a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fab640ed390 tx=0x7fab640f5550 comp rx=0 tx=0).stop 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 -- 172.21.15.26:0/4292433782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 msgr2=0x7fab74197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 0x7fab74197bb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fab64031440 tx=0x7fab64032ce0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 -- 172.21.15.26:0/4292433782 shutdown_connections 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fab580a2c00 0x7fab580a50c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab74107580 0x7fab7419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab74104c50 0x7fab741980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 --2- 172.21.15.26:0/4292433782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab74102320 0x7fab74197bb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 -- 172.21.15.26:0/4292433782 >> 172.21.15.26:0/4292433782 conn(0x7fab740fbc90 msgr2=0x7fab74102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 -- 172.21.15.26:0/4292433782 shutdown_connections 2024-06-13T07:10:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:10:51.500+0000 7fab79a00640 1 -- 172.21.15.26:0/4292433782 wait complete. 2024-06-13T07:10:51.571 INFO:tasks.workunit.client.0.smithi026.stdout: 1 474011 4.79 MB/sec execute 425 sec latency 37.983 ms 2024-06-13T07:10:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:51 smithi026 ceph-mon[32041]: pgmap v820: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-13T07:10:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:51 smithi135 ceph-mon[36276]: pgmap v820: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-13T07:10:52.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:51 smithi136 ceph-mon[37229]: pgmap v820: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-13T07:10:52.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 475257 4.79 MB/sec execute 426 sec latency 22.197 ms 2024-06-13T07:10:53.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 476319 4.80 MB/sec execute 427 sec latency 23.083 ms 2024-06-13T07:10:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:53 smithi026 ceph-mon[32041]: pgmap v821: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:10:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:53 smithi135 ceph-mon[36276]: pgmap v821: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:10:54.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:53 smithi136 ceph-mon[37229]: pgmap v821: 129 pgs: 129 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:10:54.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 476809 4.79 MB/sec execute 428 sec latency 72.154 ms 2024-06-13T07:10:55.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 477940 4.79 MB/sec execute 429 sec latency 25.959 ms 2024-06-13T07:10:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:55 smithi026 ceph-mon[32041]: pgmap v822: 129 pgs: 129 active+clean; 91 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:10:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:55 smithi135 ceph-mon[36276]: pgmap v822: 129 pgs: 129 active+clean; 91 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:10:56.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:55 smithi136 ceph-mon[37229]: pgmap v822: 129 pgs: 129 active+clean; 91 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-13T07:10:56.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 479313 4.79 MB/sec execute 430 sec latency 22.365 ms 2024-06-13T07:10:57.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 480052 4.80 MB/sec execute 431 sec latency 22.106 ms 2024-06-13T07:10:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:57 smithi026 ceph-mon[32041]: pgmap v823: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-13T07:10:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:57 smithi135 ceph-mon[36276]: pgmap v823: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-13T07:10:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:57 smithi136 ceph-mon[37229]: pgmap v823: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-13T07:10:58.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 480647 4.79 MB/sec execute 432 sec latency 28.797 ms 2024-06-13T07:10:59.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 482004 4.79 MB/sec execute 433 sec latency 45.702 ms 2024-06-13T07:11:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:10:59 smithi026 ceph-mon[32041]: pgmap v824: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:11:00.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:10:59 smithi135 ceph-mon[36276]: pgmap v824: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:11:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:10:59 smithi136 ceph-mon[37229]: pgmap v824: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-13T07:11:00.572 INFO:tasks.workunit.client.0.smithi026.stdout: 1 483160 4.80 MB/sec execute 434 sec latency 48.918 ms 2024-06-13T07:11:01.573 INFO:tasks.workunit.client.0.smithi026.stdout: 1 483677 4.80 MB/sec execute 435 sec latency 21.458 ms 2024-06-13T07:11:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:01 smithi026 ceph-mon[32041]: pgmap v825: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2024-06-13T07:11:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:01 smithi135 ceph-mon[36276]: pgmap v825: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2024-06-13T07:11:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:01 smithi136 ceph-mon[37229]: pgmap v825: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2024-06-13T07:11:02.573 INFO:tasks.workunit.client.0.smithi026.stdout: 1 484425 4.79 MB/sec execute 436 sec latency 37.598 ms 2024-06-13T07:11:03.573 INFO:tasks.workunit.client.0.smithi026.stdout: 1 485697 4.79 MB/sec execute 437 sec latency 24.037 ms 2024-06-13T07:11:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:03 smithi026 ceph-mon[32041]: pgmap v826: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:11:04.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:03 smithi135 ceph-mon[36276]: pgmap v826: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:11:04.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:03 smithi136 ceph-mon[37229]: pgmap v826: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:11:04.573 INFO:tasks.workunit.client.0.smithi026.stdout: 1 486834 4.80 MB/sec execute 438 sec latency 19.234 ms 2024-06-13T07:11:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.700+0000 7fbe2ac00640 1 -- 172.21.15.26:0/1483089588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24109220 msgr2=0x7fbe241096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.700+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/1483089588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24109220 0x7fbe241096a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009900 tx=0x7fbe1002a440 comp rx=0 tx=0).stop 2024-06-13T07:11:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.701+0000 7fbe2ac00640 1 -- 172.21.15.26:0/1483089588 shutdown_connections 2024-06-13T07:11:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.701+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/1483089588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe24109be0 0x7fbe24110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.701+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/1483089588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24109220 0x7fbe241096a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.701+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/1483089588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe24108030 0x7fbe24108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.701+0000 7fbe2ac00640 1 -- 172.21.15.26:0/1483089588 >> 172.21.15.26:0/1483089588 conn(0x7fbe24076ff0 msgr2=0x7fbe24079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.701+0000 7fbe2ac00640 1 -- 172.21.15.26:0/1483089588 shutdown_connections 2024-06-13T07:11:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.701+0000 7fbe2ac00640 1 -- 172.21.15.26:0/1483089588 wait complete. 2024-06-13T07:11:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.702+0000 7fbe2ac00640 1 Processor -- start 2024-06-13T07:11:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe2ac00640 1 -- start start 2024-06-13T07:11:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe2ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24108030 0x7fbe2419e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe2ac00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe24109220 0x7fbe2419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe2ac00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe24109be0 0x7fbe241a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe2ac00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24113d10 con 0x7fbe24109220 2024-06-13T07:11:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe2ac00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24113b90 con 0x7fbe24109be0 2024-06-13T07:11:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe2ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24113e90 con 0x7fbe24108030 2024-06-13T07:11:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe28c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe24109be0 0x7fbe241a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe23400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe24109220 0x7fbe2419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24108030 0x7fbe2419e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe28c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe24109be0 0x7fbe241a57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43272/0 (socket says 172.21.15.26:43272) 2024-06-13T07:11:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.703+0000 7fbe28c00640 1 -- 172.21.15.26:0/968959993 learned_addr learned my addr 172.21.15.26:0/968959993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.704+0000 7fbe28c00640 1 -- 172.21.15.26:0/968959993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24108030 msgr2=0x7fbe2419e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.704+0000 7fbe28c00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24108030 0x7fbe2419e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.704+0000 7fbe28c00640 1 -- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe24109220 msgr2=0x7fbe2419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.704+0000 7fbe28c00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe24109220 0x7fbe2419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.704+0000 7fbe28c00640 1 -- 172.21.15.26:0/968959993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2407a320 con 0x7fbe24109be0 2024-06-13T07:11:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.704+0000 7fbe23e00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24108030 0x7fbe2419e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.705+0000 7fbe28c00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe24109be0 0x7fbe241a57a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c970 tx=0x7fbe1400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.705+0000 7fbe20c00640 1 -- 172.21.15.26:0/968959993 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14007bb0 con 0x7fbe24109be0 2024-06-13T07:11:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.705+0000 7fbe20c00640 1 -- 172.21.15.26:0/968959993 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe14007d50 con 0x7fbe24109be0 2024-06-13T07:11:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.705+0000 7fbe20c00640 1 -- 172.21.15.26:0/968959993 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14018800 con 0x7fbe24109be0 2024-06-13T07:11:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.705+0000 7fbe2ac00640 1 -- 172.21.15.26:0/968959993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2407a610 con 0x7fbe24109be0 2024-06-13T07:11:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.706+0000 7fbe2ac00640 1 -- 172.21.15.26:0/968959993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2407aae0 con 0x7fbe24109be0 2024-06-13T07:11:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe2db6f740 1 Processor -- start 2024-06-13T07:11:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe20c00640 1 -- 172.21.15.26:0/968959993 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbe140189e0 con 0x7fbe24109be0 2024-06-13T07:11:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe2db6f740 1 -- start start 2024-06-13T07:11:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 0x55bd716c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716c2760 0x55bd716c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd71620420 con 0x55bd716bd5f0 2024-06-13T07:11:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bd715c7c50 con 0x55bd716c2760 2024-06-13T07:11:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.711+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bd7161c3e0 con 0x55bd716bcc00 2024-06-13T07:11:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52070/0 (socket says 172.21.15.26:52070) 2024-06-13T07:11:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 -- 172.21.15.26:0/1735558691 learned_addr learned my addr 172.21.15.26:0/1735558691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23e00640 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 0x55bd716c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716c2760 0x55bd716c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 -- 172.21.15.26:0/1735558691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716c2760 msgr2=0x55bd716c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716c2760 0x55bd716c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 -- 172.21.15.26:0/1735558691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 msgr2=0x55bd716c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 0x55bd716c20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.712+0000 7fbe23400640 1 -- 172.21.15.26:0/1735558691 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd71334210 con 0x55bd716bcc00 2024-06-13T07:11:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.713+0000 7fbe23400640 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bd020 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbe10031470 tx=0x7fbe10005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebdc2065d23ba27 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.713+0000 7fbe19200640 1 -- 172.21.15.26:0/1735558691 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10035070 con 0x55bd716bcc00 2024-06-13T07:11:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.713+0000 7fbe19200640 1 -- 172.21.15.26:0/1735558691 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe100035f0 con 0x55bd716bcc00 2024-06-13T07:11:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.713+0000 7fbe19200640 1 -- 172.21.15.26:0/1735558691 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1003d6b0 con 0x55bd716bcc00 2024-06-13T07:11:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.713+0000 7fbe20c00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbe080a2d00 0x7fbe080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe20c00640 1 -- 172.21.15.26:0/968959993 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbe140cf390 con 0x7fbe24109be0 2024-06-13T07:11:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe23e00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbe080a2d00 0x7fbe080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1735558691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 msgr2=0x55bd716bd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bd020 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbe10031470 tx=0x7fbe10005fd0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1735558691 shutdown_connections 2024-06-13T07:11:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716c2760 0x55bd716c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 0x55bd716c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1735558691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bd020 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1735558691 >> 172.21.15.26:0/1735558691 conn(0x55bd71684fb0 msgr2=0x55bd716b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.714+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1735558691 shutdown_connections 2024-06-13T07:11:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.715+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1735558691 wait complete. 2024-06-13T07:11:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.715+0000 7fbe23e00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbe080a2d00 0x7fbe080a51c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55bd716bd9f0 tx=0x7fbe1c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.715+0000 7fbe2db6f740 1 -- start start 2024-06-13T07:11:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.715+0000 7fbe2db6f740 1 Processor -- start 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 0x55bd716c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716c2760 0x55bd716ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd716c56b0 con 0x55bd716bd5f0 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bd715db0b0 con 0x55bd716bcc00 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bd716c5830 con 0x55bd716c2760 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe2db6f740 10 client.0 _ll_register_callbacks cb 0x55bd715c6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe28c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716c2760 0x55bd716ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43294/0 (socket says 172.21.15.26:43294) 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe23e00640 1 -- 172.21.15.26:0/1229324718 learned_addr learned my addr 172.21.15.26:0/1229324718 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.716+0000 7fbe28c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716c2760 0x55bd716ce450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52078/0 (socket says 172.21.15.26:52078) 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe23400640 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 0x55bd716c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe23e00640 1 -- 172.21.15.26:0/1229324718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716c2760 msgr2=0x55bd716ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe23e00640 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716c2760 0x55bd716ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe23e00640 1 -- 172.21.15.26:0/1229324718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 msgr2=0x55bd716c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe23e00640 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 0x55bd716c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe23e00640 1 -- 172.21.15.26:0/1229324718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd71400620 con 0x55bd716bcc00 2024-06-13T07:11:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe23e00640 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716c6ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00ff70 tx=0x7fbe1c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddc48ea7afdc5a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe0d600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:11:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.717+0000 7fbe0d600640 1 -- 172.21.15.26:0/1229324718 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c004b20 con 0x55bd716bcc00 2024-06-13T07:11:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.718+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1229324718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe100035f0 con 0x55bd716bcc00 2024-06-13T07:11:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.718+0000 7fbe0d600640 1 -- 172.21.15.26:0/1229324718 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbe1c018030 con 0x55bd716bcc00 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.718+0000 7fbe0d600640 1 -- 172.21.15.26:0/1229324718 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c033050 con 0x55bd716bcc00 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.718+0000 7fbe2db6f740 10 client.26297 fetch_fsmap learned FSMap version 35 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.718+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1229324718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bd716d1880 con 0x55bd716bcc00 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.718+0000 7fbe0d600640 1 -- 172.21.15.26:0/1229324718 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe100035f0 con 0x55bd716bcc00 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.718+0000 7fbe0d600640 1 -- 172.21.15.26:0/1229324718 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbe1c028c00 con 0x55bd716bcc00 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.719+0000 7fbe2db6f740 10 client.26297 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.719+0000 7fbe2db6f740 10 client.26297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.719+0000 7fbe2db6f740 20 client.26297 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.719+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bd716d2f10 0x55bd716d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.719+0000 7fbe2db6f740 4 client.26297 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.719+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1229324718 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd7162db50 con 0x55bd716d2f10 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.719+0000 7fbe23400640 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bd716d2f10 0x55bd716d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.720+0000 7fbe23400640 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bd716d2f10 0x55bd716d53a0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b5aa4a6ca756afc server_cookie=b0e5aa192870b8de in_seq=0 out_seq=0 2024-06-13T07:11:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.720+0000 7fbe0d600640 10 client.26297 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.721+0000 7fbe0d600640 1 -- 172.21.15.26:0/1229324718 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bd7162db50 con 0x55bd716d2f10 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.721+0000 7fbe0d600640 10 client.26297 handle_command_reply: tid=0 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 client.26297 shutdown 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1229324718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 msgr2=0x55bd716c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716c6ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00ff70 tx=0x7fbe1c004ab0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1229324718 shutdown_connections 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bd716d2f10 0x55bd716d53a0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716c2760 0x55bd716ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716bd5f0 0x55bd716c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/1229324718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716c6ed0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe0d600640 0 client.26297 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1229324718 >> 172.21.15.26:0/1229324718 conn(0x55bd71684fb0 msgr2=0x55bd716b7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1229324718 shutdown_connections 2024-06-13T07:11:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 1 -- 172.21.15.26:0/1229324718 wait complete. 2024-06-13T07:11:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.722+0000 7fbe2db6f740 20 client.26297 trim_cache size 0 max 16384 2024-06-13T07:11:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.740+0000 7fbe2db6f740 1 Processor -- start 2024-06-13T07:11:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.740+0000 7fbe2db6f740 1 -- start start 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd71684fb0 0x55bd716ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241bbda0 con 0x55bd716c22a0 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b9670 con 0x55bd71684fb0 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b7600 con 0x55bd716bcc00 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe28c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34264/0 (socket says 172.21.15.26:34264) 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe23e00640 1 -- 172.21.15.26:0/724713687 learned_addr learned my addr 172.21.15.26:0/724713687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe28c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52092/0 (socket says 172.21.15.26:52092) 2024-06-13T07:11:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.741+0000 7fbe23400640 1 --2- 172.21.15.26:0/724713687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd71684fb0 0x55bd716ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe28c00640 1 -- 172.21.15.26:0/724713687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd71684fb0 msgr2=0x55bd716ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe28c00640 1 --2- 172.21.15.26:0/724713687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd71684fb0 0x55bd716ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe28c00640 1 -- 172.21.15.26:0/724713687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 msgr2=0x55bd716c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe28c00640 1 --2- 172.21.15.26:0/724713687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe28c00640 1 -- 172.21.15.26:0/724713687 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd716b9330 con 0x55bd716bcc00 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe28c00640 1 --2- 172.21.15.26:0/724713687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bbcd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbe1409b730 tx=0x7fbe1409bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c21b1c1dfd8f40 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe0fe00640 1 -- 172.21.15.26:0/724713687 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe140b4070 con 0x55bd716bcc00 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe0fe00640 1 -- 172.21.15.26:0/724713687 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe140cf170 con 0x55bd716bcc00 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.742+0000 7fbe0fe00640 1 -- 172.21.15.26:0/724713687 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400fa50 con 0x55bd716bcc00 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 -- 172.21.15.26:0/724713687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 msgr2=0x55bd716bbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/724713687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bbcd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbe1409b730 tx=0x7fbe1409bf60 comp rx=0 tx=0).stop 2024-06-13T07:11:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 -- 172.21.15.26:0/724713687 shutdown_connections 2024-06-13T07:11:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/724713687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd71684fb0 0x55bd716ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/724713687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd716bcc00 0x55bd716bbcd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/724713687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 -- 172.21.15.26:0/724713687 >> 172.21.15.26:0/724713687 conn(0x55bd716be1f0 msgr2=0x55bd716bb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 -- 172.21.15.26:0/724713687 shutdown_connections 2024-06-13T07:11:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.743+0000 7fbe2db6f740 1 -- 172.21.15.26:0/724713687 wait complete. 2024-06-13T07:11:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.744+0000 7fbe2db6f740 1 -- start start 2024-06-13T07:11:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.744+0000 7fbe2db6f740 1 Processor -- start 2024-06-13T07:11:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd71684fb0 0x55bd716cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe2db6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b7b30 con 0x55bd716c22a0 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b8fa0 con 0x55bd716bcc00 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe2db6f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b8200 con 0x55bd71684fb0 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe2db6f740 10 client.0 _ll_register_callbacks cb 0x7fbe1c00fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd71684fb0 0x55bd716cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd71684fb0 0x55bd716cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52104/0 (socket says 172.21.15.26:52104) 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23e00640 1 -- 172.21.15.26:0/974044515 learned_addr learned my addr 172.21.15.26:0/974044515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe28c00640 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23400640 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23e00640 1 -- 172.21.15.26:0/974044515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 msgr2=0x55bd716cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23e00640 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23e00640 1 -- 172.21.15.26:0/974044515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 msgr2=0x55bd716cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23e00640 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.745+0000 7fbe23e00640 1 -- 172.21.15.26:0/974044515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd716cd640 con 0x55bd71684fb0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe23e00640 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd71684fb0 0x55bd716cbb00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c01c610 tx=0x7fbe1c01dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f6a8c43ba94d08a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe0d600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe0d600640 1 -- 172.21.15.26:0/974044515 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c01c8a0 con 0x55bd71684fb0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe2db6f740 1 -- 172.21.15.26:0/974044515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bd71627ea0 con 0x55bd71684fb0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe0d600640 1 -- 172.21.15.26:0/974044515 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbe1c01ca40 con 0x55bd71684fb0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe0d600640 1 -- 172.21.15.26:0/974044515 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c01cc60 con 0x55bd71684fb0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe2db6f740 10 client.26341 fetch_fsmap learned FSMap version 35 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe2db6f740 1 -- 172.21.15.26:0/974044515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bd7173dcf0 con 0x55bd71684fb0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.746+0000 7fbe0d600640 1 -- 172.21.15.26:0/974044515 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bd71627ea0 con 0x55bd71684fb0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.747+0000 7fbe0d600640 1 -- 172.21.15.26:0/974044515 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbe1c037c10 con 0x55bd71684fb0 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.747+0000 7fbe2db6f740 10 client.26341 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.747+0000 7fbe2db6f740 10 client.26341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.747+0000 7fbe2db6f740 20 client.26341 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.747+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bd716bd5f0 0x55bd71740750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.747+0000 7fbe2db6f740 4 client.26341 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:11:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.747+0000 7fbe2db6f740 1 -- 172.21.15.26:0/974044515 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd715a57a0 con 0x55bd716bd5f0 2024-06-13T07:11:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.747+0000 7fbe23400640 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bd716bd5f0 0x55bd71740750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.748+0000 7fbe23400640 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bd716bd5f0 0x55bd71740750 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f190e0f6f748b6 server_cookie=c15dc55f611e014b in_seq=0 out_seq=0 2024-06-13T07:11:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.748+0000 7fbe0d600640 10 client.26341 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.854+0000 7fbe0d600640 1 -- 172.21.15.26:0/974044515 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bd715a57a0 con 0x55bd716bd5f0 2024-06-13T07:11:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.854+0000 7fbe0d600640 10 client.26341 handle_command_reply: tid=0 2024-06-13T07:11:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.854+0000 7fbe2db6f740 1 client.26341 shutdown 2024-06-13T07:11:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.854+0000 7fbe2db6f740 1 -- 172.21.15.26:0/974044515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd71684fb0 msgr2=0x55bd716cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.854+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd71684fb0 0x55bd716cbb00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c01c610 tx=0x7fbe1c01dcc0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.854+0000 7fbe2db6f740 1 -- 172.21.15.26:0/974044515 shutdown_connections 2024-06-13T07:11:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.854+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55bd716bd5f0 0x55bd71740750 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.855+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd716bcc00 0x55bd716cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.855+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd716c22a0 0x55bd716cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.855+0000 7fbe0d600640 0 client.26341 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.855+0000 7fbe2db6f740 1 --2- 172.21.15.26:0/974044515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bd71684fb0 0x55bd716cbb00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.855+0000 7fbe2db6f740 1 -- 172.21.15.26:0/974044515 >> 172.21.15.26:0/974044515 conn(0x55bd716be1f0 msgr2=0x55bd716c6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.855+0000 7fbe2db6f740 1 -- 172.21.15.26:0/974044515 shutdown_connections 2024-06-13T07:11:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.855+0000 7fbe2db6f740 1 -- 172.21.15.26:0/974044515 wait complete. 2024-06-13T07:11:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.855+0000 7fbe2db6f740 20 client.26341 trim_cache size 0 max 16384 2024-06-13T07:11:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:11:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:11:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 -- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbe080a2d00 msgr2=0x7fbe080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbe080a2d00 0x7fbe080a51c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55bd716bd9f0 tx=0x7fbe1c009500 comp rx=0 tx=0).stop 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 -- 172.21.15.26:0/968959993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe24109be0 msgr2=0x7fbe241a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe24109be0 0x7fbe241a57a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c970 tx=0x7fbe1400ce30 comp rx=0 tx=0).stop 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 -- 172.21.15.26:0/968959993 shutdown_connections 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbe080a2d00 0x7fbe080a51c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe24109be0 0x7fbe241a57a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe24109220 0x7fbe2419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 --2- 172.21.15.26:0/968959993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe24108030 0x7fbe2419e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 -- 172.21.15.26:0/968959993 >> 172.21.15.26:0/968959993 conn(0x7fbe24076ff0 msgr2=0x7fbe2410c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 -- 172.21.15.26:0/968959993 shutdown_connections 2024-06-13T07:11:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:04.856+0000 7fbe2ac00640 1 -- 172.21.15.26:0/968959993 wait complete. 2024-06-13T07:11:05.573 INFO:tasks.workunit.client.0.smithi026.stdout: 1 487301 4.80 MB/sec execute 439 sec latency 55.830 ms 2024-06-13T07:11:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:05 smithi026 ceph-mon[32041]: pgmap v827: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-13T07:11:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:05 smithi135 ceph-mon[36276]: pgmap v827: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-13T07:11:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:05 smithi136 ceph-mon[37229]: pgmap v827: 129 pgs: 129 active+clean; 88 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-13T07:11:06.573 INFO:tasks.workunit.client.0.smithi026.stdout: 1 488044 4.79 MB/sec execute 440 sec latency 31.213 ms 2024-06-13T07:11:07.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 489367 4.79 MB/sec execute 441 sec latency 29.863 ms 2024-06-13T07:11:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:07 smithi026 ceph-mon[32041]: pgmap v828: 129 pgs: 129 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-06-13T07:11:08.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:07 smithi135 ceph-mon[36276]: pgmap v828: 129 pgs: 129 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-06-13T07:11:08.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:07 smithi136 ceph-mon[37229]: pgmap v828: 129 pgs: 129 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-06-13T07:11:08.573 INFO:tasks.workunit.client.0.smithi026.stdout: 1 490428 4.80 MB/sec execute 442 sec latency 20.532 ms 2024-06-13T07:11:09.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 490784 4.80 MB/sec execute 443 sec latency 28.331 ms 2024-06-13T07:11:10.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:10 smithi026 ceph-mon[32041]: pgmap v829: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-06-13T07:11:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:10 smithi135 ceph-mon[36276]: pgmap v829: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-06-13T07:11:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:10 smithi136 ceph-mon[37229]: pgmap v829: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-06-13T07:11:10.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 491693 4.80 MB/sec execute 444 sec latency 42.980 ms 2024-06-13T07:11:11.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 492811 4.79 MB/sec execute 445 sec latency 26.734 ms 2024-06-13T07:11:12.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:12 smithi135 ceph-mon[36276]: pgmap v830: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-06-13T07:11:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:12 smithi136 ceph-mon[37229]: pgmap v830: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-06-13T07:11:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:12 smithi026 ceph-mon[32041]: pgmap v830: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-06-13T07:11:12.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 493924 4.80 MB/sec execute 446 sec latency 21.820 ms 2024-06-13T07:11:13.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 494317 4.80 MB/sec execute 447 sec latency 31.390 ms 2024-06-13T07:11:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:14 smithi135 ceph-mon[36276]: pgmap v831: 129 pgs: 129 active+clean; 110 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-06-13T07:11:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:14 smithi136 ceph-mon[37229]: pgmap v831: 129 pgs: 129 active+clean; 110 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-06-13T07:11:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:14 smithi026 ceph-mon[32041]: pgmap v831: 129 pgs: 129 active+clean; 110 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-06-13T07:11:14.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 494975 4.79 MB/sec execute 448 sec latency 32.129 ms 2024-06-13T07:11:15.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 496293 4.79 MB/sec execute 449 sec latency 50.486 ms 2024-06-13T07:11:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:16 smithi135 ceph-mon[36276]: pgmap v832: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 2.9 MiB/s wr, 183 op/s 2024-06-13T07:11:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:16 smithi136 ceph-mon[37229]: pgmap v832: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 2.9 MiB/s wr, 183 op/s 2024-06-13T07:11:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:16 smithi026 ceph-mon[32041]: pgmap v832: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 2.9 MiB/s wr, 183 op/s 2024-06-13T07:11:16.574 INFO:tasks.workunit.client.0.smithi026.stdout: 1 497238 4.80 MB/sec execute 450 sec latency 22.293 ms 2024-06-13T07:11:17.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 497783 4.80 MB/sec execute 451 sec latency 38.602 ms 2024-06-13T07:11:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.009+0000 7f3286a00640 1 -- 172.21.15.26:0/2424336172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32801024b0 msgr2=0x7f3280108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.009+0000 7f3286a00640 1 --2- 172.21.15.26:0/2424336172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32801024b0 0x7f3280108d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3278009ca0 tx=0x7f327802a8c0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.009+0000 7f3286a00640 1 -- 172.21.15.26:0/2424336172 shutdown_connections 2024-06-13T07:11:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.009+0000 7f3286a00640 1 --2- 172.21.15.26:0/2424336172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32801024b0 0x7f3280108d40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.009+0000 7f3286a00640 1 --2- 172.21.15.26:0/2424336172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3280101af0 0x7f3280101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.009+0000 7f3286a00640 1 --2- 172.21.15.26:0/2424336172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3280100900 0x7f3280100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.009+0000 7f3286a00640 1 -- 172.21.15.26:0/2424336172 >> 172.21.15.26:0/2424336172 conn(0x7f32800fbe70 msgr2=0x7f32800fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.009+0000 7f3286a00640 1 -- 172.21.15.26:0/2424336172 shutdown_connections 2024-06-13T07:11:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.010+0000 7f3286a00640 1 -- 172.21.15.26:0/2424336172 wait complete. 2024-06-13T07:11:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3286a00640 1 Processor -- start 2024-06-13T07:11:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3286a00640 1 -- start start 2024-06-13T07:11:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3286a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280100900 0x7f3280104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3286a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3280101af0 0x7f3280102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3286a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32801024b0 0x7f3280103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3286a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f328010c580 con 0x7f32801024b0 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3286a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f328010c400 con 0x7f3280101af0 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3286a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f328010c700 con 0x7f3280100900 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f3284a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32801024b0 0x7f3280103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.011+0000 7f327fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280100900 0x7f3280104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280100900 0x7f3280104890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39872/0 (socket says 172.21.15.26:39872) 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327fe00640 1 -- 172.21.15.26:0/1759866205 learned_addr learned my addr 172.21.15.26:0/1759866205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327f400640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3280101af0 0x7f3280102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327fe00640 1 -- 172.21.15.26:0/1759866205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3280101af0 msgr2=0x7f3280102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327fe00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3280101af0 0x7f3280102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327fe00640 1 -- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32801024b0 msgr2=0x7f3280103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327fe00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32801024b0 0x7f3280103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327fe00640 1 -- 172.21.15.26:0/1759866205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3280103ca0 con 0x7f3280100900 2024-06-13T07:11:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327fe00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280100900 0x7f3280104890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f327000e9c0 tx=0x7f327000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.012+0000 7f327cc00640 1 -- 172.21.15.26:0/1759866205 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327000cd80 con 0x7f3280100900 2024-06-13T07:11:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.013+0000 7f327cc00640 1 -- 172.21.15.26:0/1759866205 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32700035f0 con 0x7f3280100900 2024-06-13T07:11:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.013+0000 7f327cc00640 1 -- 172.21.15.26:0/1759866205 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270010690 con 0x7f3280100900 2024-06-13T07:11:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.013+0000 7f3286a00640 1 -- 172.21.15.26:0/1759866205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32801a1d40 con 0x7f3280100900 2024-06-13T07:11:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.013+0000 7f3286a00640 1 -- 172.21.15.26:0/1759866205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801a2180 con 0x7f3280100900 2024-06-13T07:11:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.016+0000 7f327cc00640 1 -- 172.21.15.26:0/1759866205 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32700037a0 con 0x7f3280100900 2024-06-13T07:11:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.016+0000 7f32897c7740 1 Processor -- start 2024-06-13T07:11:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.016+0000 7f32897c7740 1 -- start start 2024-06-13T07:11:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f32897c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684275f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f32897c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x5568427640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f32897c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x556842766bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f32897c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5568426c2420 con 0x556842764760 2024-06-13T07:11:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f32897c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556842669c50 con 0x55684275f5f0 2024-06-13T07:11:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f32897c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5568426be3e0 con 0x55684275ec00 2024-06-13T07:11:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f3284a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684275f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f3284a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684275f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39882/0 (socket says 172.21.15.26:39882) 2024-06-13T07:11:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f3284a00640 1 -- 172.21.15.26:0/1658860932 learned_addr learned my addr 172.21.15.26:0/1658860932 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f327f400640 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x556842766bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.017+0000 7f327fe00640 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x5568427640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3284a00640 1 -- 172.21.15.26:0/1658860932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 msgr2=0x5568427640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3284a00640 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x5568427640f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3284a00640 1 -- 172.21.15.26:0/1658860932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 msgr2=0x556842766bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3284a00640 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x556842766bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3284a00640 1 -- 172.21.15.26:0/1658860932 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568423d6210 con 0x55684275ec00 2024-06-13T07:11:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f327cc00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32640a2cb0 0x7f32640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3284a00640 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684275f020 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3278009ca0 tx=0x7f3278004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e12398784b65ec2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f327cc00640 1 -- 172.21.15.26:0/1759866205 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f32700cf7a0 con 0x7f3280100900 2024-06-13T07:11:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f327fe00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32640a2cb0 0x7f32640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3275200640 1 -- 172.21.15.26:0/1658860932 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780098c0 con 0x55684275ec00 2024-06-13T07:11:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3275200640 1 -- 172.21.15.26:0/1658860932 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3278036040 con 0x55684275ec00 2024-06-13T07:11:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.018+0000 7f3275200640 1 -- 172.21.15.26:0/1658860932 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278004290 con 0x55684275ec00 2024-06-13T07:11:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.019+0000 7f32897c7740 1 -- 172.21.15.26:0/1658860932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 msgr2=0x55684275f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.019+0000 7f32897c7740 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684275f020 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3278009ca0 tx=0x7f3278004a40 comp rx=0 tx=0).stop 2024-06-13T07:11:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.019+0000 7f32897c7740 1 -- 172.21.15.26:0/1658860932 shutdown_connections 2024-06-13T07:11:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.019+0000 7f32897c7740 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x5568427640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.019+0000 7f32897c7740 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x556842766bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.019+0000 7f32897c7740 1 --2- 172.21.15.26:0/1658860932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684275f020 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.019+0000 7f327fe00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32640a2cb0 0x7f32640a5170 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f32700a42f0 tx=0x7f32700045f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.019+0000 7f32897c7740 1 -- 172.21.15.26:0/1658860932 >> 172.21.15.26:0/1658860932 conn(0x556842726fb0 msgr2=0x55684275a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.020+0000 7f32897c7740 1 -- 172.21.15.26:0/1658860932 shutdown_connections 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.020+0000 7f32897c7740 1 -- 172.21.15.26:0/1658860932 wait complete. 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.020+0000 7f32897c7740 1 -- start start 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.020+0000 7f32897c7740 1 Processor -- start 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f32897c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f32897c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f32897c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f32897c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5568427674d0 con 0x556842764760 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f32897c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55684267d0b0 con 0x55684275ec00 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f32897c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556842767650 con 0x55684275f5f0 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f32897c7740 10 client.0 _ll_register_callbacks cb 0x556842668f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f3284a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f327f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.021+0000 7f3284a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f327f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39886/0 (socket says 172.21.15.26:39886) 2024-06-13T07:11:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f327f400640 1 -- 172.21.15.26:0/1002834441 learned_addr learned my addr 172.21.15.26:0/1002834441 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f3284a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49982/0 (socket says 172.21.15.26:49982) 2024-06-13T07:11:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f327f400640 1 -- 172.21.15.26:0/1002834441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275ec00 msgr2=0x556842768d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f327f400640 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f327f400640 1 -- 172.21.15.26:0/1002834441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 msgr2=0x5568427702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f327f400640 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f327f400640 1 -- 172.21.15.26:0/1002834441 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568424a2620 con 0x55684275f5f0 2024-06-13T07:11:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f3284a00640 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.022+0000 7f327f400640 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f326c012700 tx=0x7f326c012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e6f21b02a5e75fc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.023+0000 7f3269600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:11:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.025+0000 7f3269600640 1 -- 172.21.15.26:0/1002834441 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c004030 con 0x55684275f5f0 2024-06-13T07:11:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.025+0000 7f3269600640 1 -- 172.21.15.26:0/1002834441 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f326c0041d0 con 0x55684275f5f0 2024-06-13T07:11:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.025+0000 7f32897c7740 1 -- 172.21.15.26:0/1002834441 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3278036040 con 0x55684275f5f0 2024-06-13T07:11:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.026+0000 7f3269600640 1 -- 172.21.15.26:0/1002834441 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f326c025050 con 0x55684275f5f0 2024-06-13T07:11:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.026+0000 7f32897c7740 10 client.26353 fetch_fsmap learned FSMap version 35 2024-06-13T07:11:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.026+0000 7f32897c7740 1 -- 172.21.15.26:0/1002834441 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556842773730 con 0x55684275f5f0 2024-06-13T07:11:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.026+0000 7f3269600640 1 -- 172.21.15.26:0/1002834441 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3278036040 con 0x55684275f5f0 2024-06-13T07:11:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.027+0000 7f3269600640 1 -- 172.21.15.26:0/1002834441 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f326c003a40 con 0x55684275f5f0 2024-06-13T07:11:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.027+0000 7f32897c7740 10 client.26353 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:11:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.027+0000 7f32897c7740 10 client.26353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.027+0000 7f32897c7740 20 client.26353 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.027+0000 7f32897c7740 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556842774e50 0x556842777240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.027+0000 7f32897c7740 4 client.26353 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.027+0000 7f32897c7740 1 -- 172.21.15.26:0/1002834441 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568426cfb50 con 0x556842774e50 2024-06-13T07:11:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.027+0000 7f3284a00640 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556842774e50 0x556842777240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.028+0000 7f3284a00640 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556842774e50 0x556842777240 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0e10a544964037 server_cookie=95b34ef98740e5b8 in_seq=0 out_seq=0 2024-06-13T07:11:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.028+0000 7f3269600640 10 client.26353 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f3269600640 1 -- 172.21.15.26:0/1002834441 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5568426cfb50 con 0x556842774e50 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f3269600640 10 client.26353 handle_command_reply: tid=0 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 client.26353 shutdown 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 -- 172.21.15.26:0/1002834441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 msgr2=0x556842769260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f326c012700 tx=0x7f326c012bc0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 -- 172.21.15.26:0/1002834441 shutdown_connections 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x556842774e50 0x556842777240 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 --2- 172.21.15.26:0/1002834441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f32897c7740 1 -- 172.21.15.26:0/1002834441 >> 172.21.15.26:0/1002834441 conn(0x556842726fb0 msgr2=0x556842759840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.031+0000 7f3269600640 0 client.26353 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.032+0000 7f32897c7740 1 -- 172.21.15.26:0/1002834441 shutdown_connections 2024-06-13T07:11:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.032+0000 7f32897c7740 1 -- 172.21.15.26:0/1002834441 wait complete. 2024-06-13T07:11:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.032+0000 7f32897c7740 20 client.26353 trim_cache size 0 max 16384 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f32897c7740 1 Processor -- start 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f32897c7740 1 -- start start 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f32897c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f32897c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f32897c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556842764760 0x556842770260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f32897c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32801af1a0 con 0x55684275ec00 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f32897c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32801aca70 con 0x556842764760 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f32897c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32801aa8c0 con 0x55684275f5f0 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f327f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f3284a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f327f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39888/0 (socket says 172.21.15.26:39888) 2024-06-13T07:11:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.056+0000 7f327f400640 1 -- 172.21.15.26:0/1263627697 learned_addr learned my addr 172.21.15.26:0/1263627697 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f3284a00640 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556842764760 0x556842770260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f327f400640 1 -- 172.21.15.26:0/1263627697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556842764760 msgr2=0x556842770260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f327f400640 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556842764760 0x556842770260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f327f400640 1 -- 172.21.15.26:0/1263627697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55684275ec00 msgr2=0x556842768d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f327f400640 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f327f400640 1 -- 172.21.15.26:0/1263627697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568427775b0 con 0x55684275f5f0 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f3284a00640 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556842764760 0x556842770260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f327f400640 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f326c00a010 tx=0x7f326c010690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f63ada694acaedb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.057+0000 7f326be00640 1 -- 172.21.15.26:0/1263627697 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c02d020 con 0x55684275f5f0 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.058+0000 7f326be00640 1 -- 172.21.15.26:0/1263627697 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f326c005550 con 0x55684275f5f0 2024-06-13T07:11:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.058+0000 7f326be00640 1 -- 172.21.15.26:0/1263627697 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c022040 con 0x55684275f5f0 2024-06-13T07:11:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 -- 172.21.15.26:0/1263627697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 msgr2=0x556842769260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f326c00a010 tx=0x7f326c010690 comp rx=0 tx=0).stop 2024-06-13T07:11:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 -- 172.21.15.26:0/1263627697 shutdown_connections 2024-06-13T07:11:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556842764760 0x556842770260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275f5f0 0x556842769260 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 --2- 172.21.15.26:0/1263627697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55684275ec00 0x556842768d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 -- 172.21.15.26:0/1263627697 >> 172.21.15.26:0/1263627697 conn(0x556842726fb0 msgr2=0x556842768340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 -- 172.21.15.26:0/1263627697 shutdown_connections 2024-06-13T07:11:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.059+0000 7f32897c7740 1 -- 172.21.15.26:0/1263627697 wait complete. 2024-06-13T07:11:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.060+0000 7f32897c7740 1 -- start start 2024-06-13T07:11:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.060+0000 7f32897c7740 1 Processor -- start 2024-06-13T07:11:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.060+0000 7f32897c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684276d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f32897c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x55684276d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f32897c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f32897c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32801aaf30 con 0x556842764760 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f32897c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32801ac3a0 con 0x55684275f5f0 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f32897c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32801ab600 con 0x55684275ec00 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f32897c7740 10 client.0 _ll_register_callbacks cb 0x7f326c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f327f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x55684276d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f327f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x55684276d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50000/0 (socket says 172.21.15.26:50000) 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f327f400640 1 -- 172.21.15.26:0/1749327904 learned_addr learned my addr 172.21.15.26:0/1749327904 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f3284a00640 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f3284a00640 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684276d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f327f400640 1 -- 172.21.15.26:0/1749327904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 msgr2=0x55684276d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f327f400640 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684276d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f327f400640 1 -- 172.21.15.26:0/1749327904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 msgr2=0x5568427ddaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f327f400640 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427ddaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.061+0000 7f327f400640 1 -- 172.21.15.26:0/1749327904 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556842760400 con 0x55684275f5f0 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.062+0000 7f327f400640 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x55684276d860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f326c009d70 tx=0x7f326c005220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4813c184f1fb585 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.062+0000 7f3269600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.062+0000 7f3269600640 1 -- 172.21.15.26:0/1749327904 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c02d070 con 0x55684275f5f0 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.062+0000 7f32897c7740 1 -- 172.21.15.26:0/1749327904 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f326c005550 con 0x55684275f5f0 2024-06-13T07:11:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.062+0000 7f3269600640 1 -- 172.21.15.26:0/1749327904 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f326c010a60 con 0x55684275f5f0 2024-06-13T07:11:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f3269600640 1 -- 172.21.15.26:0/1749327904 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f326c039050 con 0x55684275f5f0 2024-06-13T07:11:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f32897c7740 10 client.26318 fetch_fsmap learned FSMap version 35 2024-06-13T07:11:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f32897c7740 1 -- 172.21.15.26:0/1749327904 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5568427df130 con 0x55684275f5f0 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f3269600640 1 -- 172.21.15.26:0/1749327904 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f326c005550 con 0x55684275f5f0 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f3269600640 1 -- 172.21.15.26:0/1749327904 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f326c00f930 con 0x55684275f5f0 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f32897c7740 10 client.26318 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f32897c7740 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f32897c7740 20 client.26318 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f32897c7740 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5568427df9f0 0x5568427e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f32897c7740 4 client.26318 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.063+0000 7f32897c7740 1 -- 172.21.15.26:0/1749327904 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568426c9ea0 con 0x5568427df9f0 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.064+0000 7f3284a00640 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5568427df9f0 0x5568427e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.064+0000 7f3284a00640 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5568427df9f0 0x5568427e1e10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4400084f2c44202f server_cookie=d4b35eacc54f4fad in_seq=0 out_seq=0 2024-06-13T07:11:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.064+0000 7f3269600640 10 client.26318 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.166+0000 7f3269600640 1 -- 172.21.15.26:0/1749327904 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5568426c9ea0 con 0x5568427df9f0 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.166+0000 7f3269600640 10 client.26318 handle_command_reply: tid=0 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.166+0000 7f32897c7740 1 client.26318 shutdown 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 -- 172.21.15.26:0/1749327904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 msgr2=0x55684276d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x55684276d860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f326c009d70 tx=0x7f326c005220 comp rx=0 tx=0).stop 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 -- 172.21.15.26:0/1749327904 shutdown_connections 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5568427df9f0 0x5568427e1e10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55684275f5f0 0x55684276d860 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556842764760 0x5568427ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 --2- 172.21.15.26:0/1749327904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55684275ec00 0x55684276d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 -- 172.21.15.26:0/1749327904 >> 172.21.15.26:0/1749327904 conn(0x556842726fb0 msgr2=0x556842758fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f3269600640 0 client.26318 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 -- 172.21.15.26:0/1749327904 shutdown_connections 2024-06-13T07:11:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 1 -- 172.21.15.26:0/1749327904 wait complete. 2024-06-13T07:11:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.167+0000 7f32897c7740 20 client.26318 trim_cache size 0 max 16384 2024-06-13T07:11:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:11:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:11:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 -- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32640a2cb0 msgr2=0x7f32640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32640a2cb0 0x7f32640a5170 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f32700a42f0 tx=0x7f32700045f0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 -- 172.21.15.26:0/1759866205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280100900 msgr2=0x7f3280104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280100900 0x7f3280104890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f327000e9c0 tx=0x7f327000ee80 comp rx=0 tx=0).stop 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 -- 172.21.15.26:0/1759866205 shutdown_connections 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32640a2cb0 0x7f32640a5170 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3280101af0 0x7f3280102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32801024b0 0x7f3280103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 --2- 172.21.15.26:0/1759866205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280100900 0x7f3280104890 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 -- 172.21.15.26:0/1759866205 >> 172.21.15.26:0/1759866205 conn(0x7f32800fbe70 msgr2=0x7f32800fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 -- 172.21.15.26:0/1759866205 shutdown_connections 2024-06-13T07:11:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:18.168+0000 7f3286a00640 1 -- 172.21.15.26:0/1759866205 wait complete. 2024-06-13T07:11:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:18 smithi135 ceph-mon[36276]: pgmap v833: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-06-13T07:11:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:18 smithi136 ceph-mon[37229]: pgmap v833: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-06-13T07:11:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:18 smithi026 ceph-mon[32041]: pgmap v833: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-06-13T07:11:18.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 498378 4.79 MB/sec execute 452 sec latency 25.270 ms 2024-06-13T07:11:19.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 499737 4.79 MB/sec execute 453 sec latency 35.381 ms 2024-06-13T07:11:20.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:20 smithi135 ceph-mon[36276]: pgmap v834: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-06-13T07:11:20.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:20 smithi136 ceph-mon[37229]: pgmap v834: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-06-13T07:11:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:20 smithi026 ceph-mon[32041]: pgmap v834: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-06-13T07:11:20.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 500749 4.80 MB/sec execute 454 sec latency 39.358 ms 2024-06-13T07:11:21.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 501325 4.80 MB/sec execute 455 sec latency 25.587 ms 2024-06-13T07:11:22.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:22 smithi135 ceph-mon[36276]: pgmap v835: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-06-13T07:11:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:22 smithi136 ceph-mon[37229]: pgmap v835: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-06-13T07:11:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:22 smithi026 ceph-mon[32041]: pgmap v835: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-06-13T07:11:22.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 501815 4.79 MB/sec execute 456 sec latency 49.398 ms 2024-06-13T07:11:23.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 502920 4.79 MB/sec execute 457 sec latency 21.827 ms 2024-06-13T07:11:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:24 smithi136 ceph-mon[37229]: pgmap v836: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-06-13T07:11:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:24 smithi026 ceph-mon[32041]: pgmap v836: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-06-13T07:11:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:24 smithi135 ceph-mon[36276]: pgmap v836: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-06-13T07:11:24.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 503954 4.79 MB/sec execute 458 sec latency 25.003 ms 2024-06-13T07:11:25.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 504816 4.79 MB/sec execute 459 sec latency 31.381 ms 2024-06-13T07:11:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:26 smithi136 ceph-mon[37229]: pgmap v837: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-13T07:11:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:26 smithi026 ceph-mon[32041]: pgmap v837: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-13T07:11:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:26 smithi135 ceph-mon[36276]: pgmap v837: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-13T07:11:26.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 505365 4.79 MB/sec execute 460 sec latency 23.860 ms 2024-06-13T07:11:27.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:27 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:11:27.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:27 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:11:27.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:27 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:11:27.575 INFO:tasks.workunit.client.0.smithi026.stdout: 1 506868 4.79 MB/sec execute 461 sec latency 24.735 ms 2024-06-13T07:11:28.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:28 smithi136 ceph-mon[37229]: pgmap v838: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:11:28.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:11:28.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:11:28.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:11:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:28 smithi026 ceph-mon[32041]: pgmap v838: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:11:28.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:11:28.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:11:28.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:11:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:28 smithi135 ceph-mon[36276]: pgmap v838: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:11:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:11:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:11:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:11:28.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 507783 4.80 MB/sec execute 462 sec latency 23.592 ms 2024-06-13T07:11:29.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 508402 4.80 MB/sec execute 463 sec latency 36.466 ms 2024-06-13T07:11:30.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:30 smithi136 ceph-mon[37229]: pgmap v839: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:11:30.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:30 smithi026 ceph-mon[32041]: pgmap v839: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:11:30.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:30 smithi135 ceph-mon[36276]: pgmap v839: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:11:30.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 508935 4.79 MB/sec execute 464 sec latency 33.282 ms 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 -- 172.21.15.26:0/3295937602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac101a00 msgr2=0x7f6dac101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 --2- 172.21.15.26:0/3295937602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac101a00 0x7f6dac101e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009ca0 tx=0x7f6da002a8c0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 -- 172.21.15.26:0/3295937602 shutdown_connections 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 --2- 172.21.15.26:0/3295937602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dac1035b0 0x7f6dac109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 --2- 172.21.15.26:0/3295937602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dac102bf0 0x7f6dac103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 --2- 172.21.15.26:0/3295937602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac101a00 0x7f6dac101e20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 -- 172.21.15.26:0/3295937602 >> 172.21.15.26:0/3295937602 conn(0x7f6dac0fcf90 msgr2=0x7f6dac0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 -- 172.21.15.26:0/3295937602 shutdown_connections 2024-06-13T07:11:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.355+0000 7f6db4600640 1 -- 172.21.15.26:0/3295937602 wait complete. 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.356+0000 7f6db4600640 1 Processor -- start 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.356+0000 7f6db4600640 1 -- start start 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db4600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dac101a00 0x7f6dac199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db4600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dac102bf0 0x7f6dac19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db4600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac1035b0 0x7f6dac1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db4600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac10d670 con 0x7f6dac101a00 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db4600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac10d4f0 con 0x7f6dac1035b0 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db4600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac10d7f0 con 0x7f6dac102bf0 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dac101a00 0x7f6dac199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dac101a00 0x7f6dac199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36804/0 (socket says 172.21.15.26:36804) 2024-06-13T07:11:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db1c00640 1 -- 172.21.15.26:0/3064715211 learned_addr learned my addr 172.21.15.26:0/3064715211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db2600640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac1035b0 0x7f6dac1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.357+0000 7f6db1200640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dac102bf0 0x7f6dac19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.358+0000 7f6db1c00640 1 -- 172.21.15.26:0/3064715211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dac102bf0 msgr2=0x7f6dac19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.358+0000 7f6db1c00640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dac102bf0 0x7f6dac19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.358+0000 7f6db1c00640 1 -- 172.21.15.26:0/3064715211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac1035b0 msgr2=0x7f6dac1a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.358+0000 7f6db1c00640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac1035b0 0x7f6dac1a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.358+0000 7f6db1c00640 1 -- 172.21.15.26:0/3064715211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac1a1b30 con 0x7f6dac101a00 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.358+0000 7f6db2600640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac1035b0 0x7f6dac1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.358+0000 7f6db1c00640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dac101a00 0x7f6dac199f60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009bf0 tx=0x7f6da0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.358+0000 7f6daa000640 1 -- 172.21.15.26:0/3064715211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da002fa20 con 0x7f6dac101a00 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.359+0000 7f6db4600640 1 -- 172.21.15.26:0/3064715211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac1a1dc0 con 0x7f6dac101a00 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.359+0000 7f6daa000640 1 -- 172.21.15.26:0/3064715211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da002fbc0 con 0x7f6dac101a00 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.359+0000 7f6daa000640 1 -- 172.21.15.26:0/3064715211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00466f0 con 0x7f6dac101a00 2024-06-13T07:11:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.359+0000 7f6db4600640 1 -- 172.21.15.26:0/3064715211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac1a2230 con 0x7f6dac101a00 2024-06-13T07:11:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db738c740 1 Processor -- start 2024-06-13T07:11:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db738c740 1 -- start start 2024-06-13T07:11:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6daa000640 1 -- 172.21.15.26:0/3064715211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6da002fd70 con 0x7f6dac101a00 2024-06-13T07:11:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db738c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556cccbc00 0x55556cccc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db738c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db738c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556ccd1760 0x55556ccd3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db738c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55556cc2f420 con 0x55556ccd1760 2024-06-13T07:11:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db738c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55556cbd6c50 con 0x55556cccc5f0 2024-06-13T07:11:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db738c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55556cc2b3e0 con 0x55556cccbc00 2024-06-13T07:11:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db2600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556cccbc00 0x55556cccc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db1c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db2600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556cccbc00 0x55556cccc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55260/0 (socket says 172.21.15.26:55260) 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.363+0000 7f6db2600640 1 -- 172.21.15.26:0/4167514275 learned_addr learned my addr 172.21.15.26:0/4167514275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6db1200640 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556ccd1760 0x55556ccd3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6db2600640 1 -- 172.21.15.26:0/4167514275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 msgr2=0x55556ccd10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6db2600640 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6db2600640 1 -- 172.21.15.26:0/4167514275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556ccd1760 msgr2=0x55556ccd3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6db2600640 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556ccd1760 0x55556ccd3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6db2600640 1 -- 172.21.15.26:0/4167514275 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556c943210 con 0x55556cccbc00 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6db1200640 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556ccd1760 0x55556ccd3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6db2600640 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556cccbc00 0x55556cccc020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6da4011440 tx=0x7f6da4011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf9b91109e58e10 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6d9ea00640 1 -- 172.21.15.26:0/4167514275 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4004020 con 0x55556cccbc00 2024-06-13T07:11:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6d9ea00640 1 -- 172.21.15.26:0/4167514275 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da40041c0 con 0x55556cccbc00 2024-06-13T07:11:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.364+0000 7f6d9ea00640 1 -- 172.21.15.26:0/4167514275 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da400f940 con 0x55556cccbc00 2024-06-13T07:11:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.365+0000 7f6daa000640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d940a2c30 0x7f6d940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.365+0000 7f6daa000640 1 -- 172.21.15.26:0/3064715211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6da00f0810 con 0x7f6dac101a00 2024-06-13T07:11:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.365+0000 7f6db1c00640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d940a2c30 0x7f6d940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.365+0000 7f6db738c740 1 -- 172.21.15.26:0/4167514275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556cccbc00 msgr2=0x55556cccc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.365+0000 7f6db738c740 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556cccbc00 0x55556cccc020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6da4011440 tx=0x7f6da4011900 comp rx=0 tx=0).stop 2024-06-13T07:11:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db738c740 1 -- 172.21.15.26:0/4167514275 shutdown_connections 2024-06-13T07:11:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db738c740 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db738c740 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556ccd1760 0x55556ccd3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db738c740 1 --2- 172.21.15.26:0/4167514275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556cccbc00 0x55556cccc020 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db738c740 1 -- 172.21.15.26:0/4167514275 >> 172.21.15.26:0/4167514275 conn(0x55556cc93fb0 msgr2=0x55556ccc7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db738c740 1 -- 172.21.15.26:0/4167514275 shutdown_connections 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db738c740 1 -- 172.21.15.26:0/4167514275 wait complete. 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db1c00640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d940a2c30 0x7f6d940a50f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6da00ee290 tx=0x7f6da00f6550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.366+0000 7f6db738c740 1 -- start start 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.367+0000 7f6db738c740 1 Processor -- start 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.367+0000 7f6db738c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccd5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.367+0000 7f6db738c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.367+0000 7f6db738c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccdd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.367+0000 7f6db738c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55556ccd4710 con 0x55556cccbc00 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.367+0000 7f6db738c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55556cbea0b0 con 0x55556cccc5f0 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.367+0000 7f6db738c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55556ccd4890 con 0x55556ccd1760 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.367+0000 7f6db738c740 10 client.0 _ll_register_callbacks cb 0x55556cbd5f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db2600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccd5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db1200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db2600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccd5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36820/0 (socket says 172.21.15.26:36820) 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db2600640 1 -- 172.21.15.26:0/3369890563 learned_addr learned my addr 172.21.15.26:0/3369890563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db2600640 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccdd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db1200640 1 -- 172.21.15.26:0/3369890563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 msgr2=0x55556ccdd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db1200640 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccdd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db1200640 1 -- 172.21.15.26:0/3369890563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 msgr2=0x55556ccd5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db1200640 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccd5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db1200640 1 -- 172.21.15.26:0/3369890563 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556ca0f620 con 0x55556cccc5f0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.368+0000 7f6db2600640 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccdd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6db1200640 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd6510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6dac19b420 tx=0x7f6d98012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9011e8b24f42fa6c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6d93400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6d93400640 1 -- 172.21.15.26:0/3369890563 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004050 con 0x55556cccc5f0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6d93400640 1 -- 172.21.15.26:0/3369890563 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6d980041f0 con 0x55556cccc5f0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6db738c740 1 -- 172.21.15.26:0/3369890563 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da40041c0 con 0x55556cccc5f0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6d93400640 1 -- 172.21.15.26:0/3369890563 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d980043f0 con 0x55556cccc5f0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6db738c740 10 client.26330 fetch_fsmap learned FSMap version 35 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6db738c740 1 -- 172.21.15.26:0/3369890563 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55556cce0930 con 0x55556cccc5f0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.369+0000 7f6d93400640 1 -- 172.21.15.26:0/3369890563 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da40041c0 con 0x55556cccc5f0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.370+0000 7f6d93400640 1 -- 172.21.15.26:0/3369890563 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6d98010430 con 0x55556cccc5f0 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.370+0000 7f6db738c740 10 client.26330 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:11:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.370+0000 7f6db738c740 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.370+0000 7f6db738c740 20 client.26330 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.370+0000 7f6db738c740 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55556cce2000 0x55556cce4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.370+0000 7f6db738c740 4 client.26330 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.370+0000 7f6db738c740 1 -- 172.21.15.26:0/3369890563 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55556cc3cb50 con 0x55556cce2000 2024-06-13T07:11:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.370+0000 7f6db2600640 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55556cce2000 0x55556cce4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.371+0000 7f6d93400640 10 client.26330 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.371+0000 7f6db2600640 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55556cce2000 0x55556cce4430 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0d3a293496cf72 server_cookie=21a09b92e80995a3 in_seq=0 out_seq=0 2024-06-13T07:11:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.371+0000 7f6d93400640 1 -- 172.21.15.26:0/3369890563 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55556cc3cb50 con 0x55556cce2000 2024-06-13T07:11:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.371+0000 7f6d93400640 10 client.26330 handle_command_reply: tid=0 2024-06-13T07:11:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 client.26330 shutdown 2024-06-13T07:11:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 -- 172.21.15.26:0/3369890563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 msgr2=0x55556ccd6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd6510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6dac19b420 tx=0x7f6d98012b60 comp rx=0 tx=0).stop 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 -- 172.21.15.26:0/3369890563 shutdown_connections 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55556cce2000 0x55556cce4430 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd6510 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccdd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 --2- 172.21.15.26:0/3369890563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccd5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 -- 172.21.15.26:0/3369890563 >> 172.21.15.26:0/3369890563 conn(0x55556cc93fb0 msgr2=0x55556ccc6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6d93400640 0 client.26330 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 -- 172.21.15.26:0/3369890563 shutdown_connections 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 1 -- 172.21.15.26:0/3369890563 wait complete. 2024-06-13T07:11:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.372+0000 7f6db738c740 20 client.26330 trim_cache size 0 max 16384 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.392+0000 7f6db738c740 1 Processor -- start 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db738c740 1 -- start start 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db738c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccbc00 0x55556ccc7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db738c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccc5f0 0x55556ccdd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db738c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccc7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db738c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1af240 con 0x55556cccc5f0 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db738c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1acb10 con 0x55556cccbc00 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db738c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1aa960 con 0x55556ccd1760 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db2600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccbc00 0x55556ccc7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccc5f0 0x55556ccdd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db2600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccbc00 0x55556ccc7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49636/0 (socket says 172.21.15.26:49636) 2024-06-13T07:11:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db2600640 1 -- 172.21.15.26:0/657040614 learned_addr learned my addr 172.21.15.26:0/657040614 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.393+0000 7f6db2600640 1 --2- 172.21.15.26:0/657040614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccc7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6db2600640 1 -- 172.21.15.26:0/657040614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 msgr2=0x55556ccc7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6db2600640 1 --2- 172.21.15.26:0/657040614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccc7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6db2600640 1 -- 172.21.15.26:0/657040614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccc5f0 msgr2=0x55556ccdd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6db2600640 1 --2- 172.21.15.26:0/657040614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccc5f0 0x55556ccdd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6db2600640 1 -- 172.21.15.26:0/657040614 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556ccc8330 con 0x55556cccbc00 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6db2600640 1 --2- 172.21.15.26:0/657040614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccbc00 0x55556ccc7100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005f30 tx=0x7f6da4003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac56c9baba537d60 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6d9e000640 1 -- 172.21.15.26:0/657040614 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da401d020 con 0x55556cccbc00 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6d9e000640 1 -- 172.21.15.26:0/657040614 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da4005040 con 0x55556cccbc00 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.394+0000 7f6d9e000640 1 -- 172.21.15.26:0/657040614 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da400f8c0 con 0x55556cccbc00 2024-06-13T07:11:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.395+0000 7f6db738c740 1 -- 172.21.15.26:0/657040614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccbc00 msgr2=0x55556ccc7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.395+0000 7f6db738c740 1 --2- 172.21.15.26:0/657040614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccbc00 0x55556ccc7100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005f30 tx=0x7f6da4003d20 comp rx=0 tx=0).stop 2024-06-13T07:11:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.395+0000 7f6db738c740 1 -- 172.21.15.26:0/657040614 shutdown_connections 2024-06-13T07:11:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.395+0000 7f6db738c740 1 --2- 172.21.15.26:0/657040614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccc7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.395+0000 7f6db738c740 1 --2- 172.21.15.26:0/657040614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccc5f0 0x55556ccdd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.395+0000 7f6db738c740 1 --2- 172.21.15.26:0/657040614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccbc00 0x55556ccc7100 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.395+0000 7f6db738c740 1 -- 172.21.15.26:0/657040614 >> 172.21.15.26:0/657040614 conn(0x55556cc93fb0 msgr2=0x55556cccd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.395+0000 7f6db738c740 1 -- 172.21.15.26:0/657040614 shutdown_connections 2024-06-13T07:11:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.396+0000 7f6db738c740 1 -- 172.21.15.26:0/657040614 wait complete. 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.396+0000 7f6db738c740 1 -- start start 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.396+0000 7f6db738c740 1 Processor -- start 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db738c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccdd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db738c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db738c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccda180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db738c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1aafd0 con 0x55556cccbc00 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db738c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1ac440 con 0x55556cccc5f0 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db738c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1ab6a0 con 0x55556ccd1760 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db738c740 10 client.0 _ll_register_callbacks cb 0x7f6d98009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db1200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db1200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49650/0 (socket says 172.21.15.26:49650) 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db2600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccdd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db1200640 1 -- 172.21.15.26:0/2750151826 learned_addr learned my addr 172.21.15.26:0/2750151826 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db2600640 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccda180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db1200640 1 -- 172.21.15.26:0/2750151826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 msgr2=0x55556ccda180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db1200640 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccda180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db1200640 1 -- 172.21.15.26:0/2750151826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 msgr2=0x55556ccdd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db1200640 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccdd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.397+0000 7f6db1200640 1 -- 172.21.15.26:0/2750151826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556ccda710 con 0x55556cccc5f0 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.398+0000 7f6db1200640 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd9c40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009830 tx=0x7f6d98005610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2909112d8069ed0b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.398+0000 7f6d93400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.398+0000 7f6d93400640 1 -- 172.21.15.26:0/2750151826 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9802e070 con 0x55556cccc5f0 2024-06-13T07:11:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.398+0000 7f6d93400640 1 -- 172.21.15.26:0/2750151826 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6d9800f9a0 con 0x55556cccc5f0 2024-06-13T07:11:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.398+0000 7f6db738c740 1 -- 172.21.15.26:0/2750151826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da4005040 con 0x55556cccc5f0 2024-06-13T07:11:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.398+0000 7f6d93400640 1 -- 172.21.15.26:0/2750151826 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d9800fba0 con 0x55556cccc5f0 2024-06-13T07:11:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6db738c740 10 client.26336 fetch_fsmap learned FSMap version 35 2024-06-13T07:11:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6db738c740 1 -- 172.21.15.26:0/2750151826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55556cd4be20 con 0x55556cccc5f0 2024-06-13T07:11:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6d93400640 1 -- 172.21.15.26:0/2750151826 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da4005040 con 0x55556cccc5f0 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6d93400640 1 -- 172.21.15.26:0/2750151826 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6d98033ca0 con 0x55556cccc5f0 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6db738c740 10 client.26336 fetch_fsmap finished waiting for FSMap version 35 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6db738c740 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6db738c740 20 client.26336 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6db738c740 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55556cd4c800 0x55556cd4ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6db738c740 4 client.26336 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.399+0000 7f6db738c740 1 -- 172.21.15.26:0/2750151826 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55556cc36ea0 con 0x55556cd4c800 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.400+0000 7f6db2600640 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55556cd4c800 0x55556cd4ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.400+0000 7f6db2600640 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55556cd4c800 0x55556cd4ec20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa478ab0d4a9245f server_cookie=49842fbae4a3cf10 in_seq=0 out_seq=0 2024-06-13T07:11:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.400+0000 7f6d93400640 10 client.26336 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6d93400640 1 -- 172.21.15.26:0/2750151826 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55556cc36ea0 con 0x55556cd4c800 2024-06-13T07:11:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6d93400640 10 client.26336 handle_command_reply: tid=0 2024-06-13T07:11:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 client.26336 shutdown 2024-06-13T07:11:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 -- 172.21.15.26:0/2750151826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 msgr2=0x55556ccd9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd9c40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009830 tx=0x7f6d98005610 comp rx=0 tx=0).stop 2024-06-13T07:11:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 -- 172.21.15.26:0/2750151826 shutdown_connections 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55556cd4c800 0x55556cd4ec20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55556cccc5f0 0x55556ccd9c40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55556ccd1760 0x55556ccda180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 --2- 172.21.15.26:0/2750151826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556cccbc00 0x55556ccdd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6d93400640 0 client.26336 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 -- 172.21.15.26:0/2750151826 >> 172.21.15.26:0/2750151826 conn(0x55556cc93fb0 msgr2=0x55556ccc7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 -- 172.21.15.26:0/2750151826 shutdown_connections 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 1 -- 172.21.15.26:0/2750151826 wait complete. 2024-06-13T07:11:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.485+0000 7f6db738c740 20 client.26336 trim_cache size 0 max 16384 2024-06-13T07:11:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:11:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:11:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 -- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d940a2c30 msgr2=0x7f6d940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d940a2c30 0x7f6d940a50f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6da00ee290 tx=0x7f6da00f6550 comp rx=0 tx=0).stop 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 -- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dac101a00 msgr2=0x7f6dac199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dac101a00 0x7f6dac199f60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009bf0 tx=0x7f6da0031ee0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 -- 172.21.15.26:0/3064715211 shutdown_connections 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6d940a2c30 0x7f6d940a50f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac1035b0 0x7f6dac1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dac102bf0 0x7f6dac19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 --2- 172.21.15.26:0/3064715211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dac101a00 0x7f6dac199f60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 -- 172.21.15.26:0/3064715211 >> 172.21.15.26:0/3064715211 conn(0x7f6dac0fcf90 msgr2=0x7f6dac073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 -- 172.21.15.26:0/3064715211 shutdown_connections 2024-06-13T07:11:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:31.486+0000 7f6db4600640 1 -- 172.21.15.26:0/3064715211 wait complete. 2024-06-13T07:11:31.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 510065 4.79 MB/sec execute 465 sec latency 79.688 ms 2024-06-13T07:11:32.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:32 smithi136 ceph-mon[37229]: pgmap v840: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-13T07:11:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:32 smithi026 ceph-mon[32041]: pgmap v840: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-13T07:11:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:32 smithi135 ceph-mon[36276]: pgmap v840: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-13T07:11:32.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 511200 4.79 MB/sec execute 466 sec latency 30.321 ms 2024-06-13T07:11:33.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 511882 4.79 MB/sec execute 467 sec latency 28.615 ms 2024-06-13T07:11:34.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:34 smithi136 ceph-mon[37229]: pgmap v841: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:11:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:34 smithi026 ceph-mon[32041]: pgmap v841: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:11:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:34 smithi135 ceph-mon[36276]: pgmap v841: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:11:34.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 512313 4.79 MB/sec execute 468 sec latency 32.831 ms 2024-06-13T07:11:35.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 513248 4.79 MB/sec execute 469 sec latency 39.276 ms 2024-06-13T07:11:36.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:36 smithi136 ceph-mon[37229]: pgmap v842: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:11:36.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:36 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:36 smithi026 ceph-mon[32041]: pgmap v842: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:11:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:36 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:36 smithi135 ceph-mon[36276]: pgmap v842: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-13T07:11:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:36 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:36.576 INFO:tasks.workunit.client.0.smithi026.stdout: 1 514510 4.79 MB/sec execute 470 sec latency 29.998 ms 2024-06-13T07:11:37.577 INFO:tasks.workunit.client.0.smithi026.stdout: 1 515082 4.79 MB/sec execute 471 sec latency 432.951 ms 2024-06-13T07:11:38.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:38 smithi136 ceph-mon[37229]: pgmap v843: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-13T07:11:38.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:38 smithi026 ceph-mon[32041]: pgmap v843: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-13T07:11:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:38 smithi135 ceph-mon[36276]: pgmap v843: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-13T07:11:38.577 INFO:tasks.workunit.client.0.smithi026.stdout: 1 515512 4.79 MB/sec execute 472 sec latency 27.683 ms 2024-06-13T07:11:39.577 INFO:tasks.workunit.client.0.smithi026.stdout: 1 515990 4.78 MB/sec execute 473 sec latency 35.286 ms 2024-06-13T07:11:40.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:40 smithi136 ceph-mon[37229]: pgmap v844: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:11:40.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:40 smithi026 ceph-mon[32041]: pgmap v844: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:11:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:40 smithi135 ceph-mon[36276]: pgmap v844: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-06-13T07:11:40.577 INFO:tasks.workunit.client.0.smithi026.stdout: 1 516828 4.78 MB/sec execute 474 sec latency 43.033 ms 2024-06-13T07:11:41.577 INFO:tasks.workunit.client.0.smithi026.stdout: 1 518054 4.78 MB/sec execute 475 sec latency 26.291 ms 2024-06-13T07:11:42.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:42 smithi136 ceph-mon[37229]: pgmap v845: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:11:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:42 smithi026 ceph-mon[32041]: pgmap v845: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:11:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:42 smithi135 ceph-mon[36276]: pgmap v845: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:11:42.577 INFO:tasks.workunit.client.0.smithi026.stdout: 1 518916 4.78 MB/sec execute 476 sec latency 27.158 ms 2024-06-13T07:11:43.577 INFO:tasks.workunit.client.0.smithi026.stdout: 1 519275 4.77 MB/sec execute 477 sec latency 38.954 ms 2024-06-13T07:11:44.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:44 smithi136 ceph-mon[37229]: pgmap v846: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-13T07:11:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:44 smithi026 ceph-mon[32041]: pgmap v846: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-13T07:11:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:44 smithi135 ceph-mon[36276]: pgmap v846: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-13T07:11:44.577 INFO:tasks.workunit.client.0.smithi026.stdout: 1 520076 4.78 MB/sec execute 478 sec latency 26.566 ms 2024-06-13T07:11:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.646+0000 7f4a19400640 1 -- 172.21.15.26:0/3458844688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14104c90 msgr2=0x7f4a14107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.646+0000 7f4a19400640 1 --2- 172.21.15.26:0/3458844688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14104c90 0x7f4a14107080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009900 tx=0x7f49fc02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.646+0000 7f4a19400640 1 -- 172.21.15.26:0/3458844688 shutdown_connections 2024-06-13T07:11:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.646+0000 7f4a19400640 1 --2- 172.21.15.26:0/3458844688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a141075c0 0x7f4a141099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.646+0000 7f4a19400640 1 --2- 172.21.15.26:0/3458844688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14104c90 0x7f4a14107080 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.646+0000 7f4a19400640 1 --2- 172.21.15.26:0/3458844688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a14102360 0x7f4a14104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.646+0000 7f4a19400640 1 -- 172.21.15.26:0/3458844688 >> 172.21.15.26:0/3458844688 conn(0x7f4a140fbc90 msgr2=0x7f4a140fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.646+0000 7f4a19400640 1 -- 172.21.15.26:0/3458844688 shutdown_connections 2024-06-13T07:11:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.647+0000 7f4a19400640 1 -- 172.21.15.26:0/3458844688 wait complete. 2024-06-13T07:11:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.648+0000 7f4a19400640 1 Processor -- start 2024-06-13T07:11:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.648+0000 7f4a19400640 1 -- start start 2024-06-13T07:11:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.648+0000 7f4a19400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14102360 0x7f4a14197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.648+0000 7f4a19400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a14104c90 0x7f4a141980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.648+0000 7f4a19400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141075c0 0x7f4a1419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a19400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1410d500 con 0x7f4a141075c0 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a19400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1410d380 con 0x7f4a14104c90 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a19400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1410d680 con 0x7f4a14102360 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a0be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a14104c90 0x7f4a141980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14102360 0x7f4a14197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a0be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a14104c90 0x7f4a141980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59738/0 (socket says 172.21.15.26:59738) 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a0be00640 1 -- 172.21.15.26:0/3933235088 learned_addr learned my addr 172.21.15.26:0/3933235088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a18a00640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141075c0 0x7f4a1419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a0be00640 1 -- 172.21.15.26:0/3933235088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14102360 msgr2=0x7f4a14197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a0be00640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14102360 0x7f4a14197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a0be00640 1 -- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141075c0 msgr2=0x7f4a1419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a0be00640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141075c0 0x7f4a1419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a0be00640 1 -- 172.21.15.26:0/3933235088 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1419f790 con 0x7f4a14104c90 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a13e00640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14102360 0x7f4a14197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.649+0000 7f4a18a00640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141075c0 0x7f4a1419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.650+0000 7f4a0be00640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a14104c90 0x7f4a141980b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f49fc031440 tx=0x7f49fc032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.650+0000 7f4a11600640 1 -- 172.21.15.26:0/3933235088 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc03f030 con 0x7f4a14104c90 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.650+0000 7f4a11600640 1 -- 172.21.15.26:0/3933235088 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49fc003ce0 con 0x7f4a14104c90 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.650+0000 7f4a19400640 1 -- 172.21.15.26:0/3933235088 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a14100150 con 0x7f4a14104c90 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.650+0000 7f4a11600640 1 -- 172.21.15.26:0/3933235088 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc035070 con 0x7f4a14104c90 2024-06-13T07:11:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.650+0000 7f4a19400640 1 -- 172.21.15.26:0/3933235088 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a14100640 con 0x7f4a14104c90 2024-06-13T07:11:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.654+0000 7f4a1d82f740 1 Processor -- start 2024-06-13T07:11:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.654+0000 7f4a1d82f740 1 -- start start 2024-06-13T07:11:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.654+0000 7f4a11600640 1 -- 172.21.15.26:0/3933235088 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f49fc042030 con 0x7f4a14104c90 2024-06-13T07:11:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad69e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e5ad601420 con 0x55e5ad69ddc0 2024-06-13T07:11:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e5ad5a8c50 con 0x55e5ad69e7b0 2024-06-13T07:11:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e5ad5fd3e0 con 0x55e5ad6a3920 2024-06-13T07:11:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a0be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a18a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad69e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59752/0 (socket says 172.21.15.26:59752) 2024-06-13T07:11:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a13e00640 1 -- 172.21.15.26:0/3714463719 learned_addr learned my addr 172.21.15.26:0/3714463719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a13e00640 1 -- 172.21.15.26:0/3714463719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 msgr2=0x55e5ad6a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a13e00640 1 --2- 172.21.15.26:0/3714463719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a13e00640 1 -- 172.21.15.26:0/3714463719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 msgr2=0x55e5ad69e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a13e00640 1 --2- 172.21.15.26:0/3714463719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad69e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.655+0000 7f4a13e00640 1 -- 172.21.15.26:0/3714463719 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ad315210 con 0x55e5ad69e7b0 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.656+0000 7f4a13e00640 1 --2- 172.21.15.26:0/3714463719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a32b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009880 tx=0x7f4a04005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8bc43fdfd3a103a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.656+0000 7f4a09200640 1 -- 172.21.15.26:0/3714463719 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0401e020 con 0x55e5ad69e7b0 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.656+0000 7f4a09200640 1 -- 172.21.15.26:0/3714463719 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0401a400 con 0x55e5ad69e7b0 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.656+0000 7f4a09200640 1 -- 172.21.15.26:0/3714463719 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a040195c0 con 0x55e5ad69e7b0 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.656+0000 7f4a11600640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f49f80a2b80 0x7f49f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a11600640 1 -- 172.21.15.26:0/3933235088 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f49fc0ef990 con 0x7f4a14104c90 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a18a00640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f49f80a2b80 0x7f49f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3714463719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69e7b0 msgr2=0x55e5ad6a32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3714463719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a32b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009880 tx=0x7f4a04005f90 comp rx=0 tx=0).stop 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3714463719 shutdown_connections 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3714463719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a32b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3714463719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3714463719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad69e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3714463719 >> 172.21.15.26:0/3714463719 conn(0x55e5ad665fb0 msgr2=0x55e5ad699840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3714463719 shutdown_connections 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.657+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3714463719 wait complete. 2024-06-13T07:11:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.658+0000 7f4a18a00640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f49f80a2b80 0x7f49f80a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55e5ad69f010 tx=0x7f4a0c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.658+0000 7f4a1d82f740 1 -- start start 2024-06-13T07:11:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.658+0000 7f4a1d82f740 1 Processor -- start 2024-06-13T07:11:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.658+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e5ad6a68d0 con 0x55e5ad69e7b0 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e5ad5bc0b0 con 0x55e5ad6a3920 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e5ad6a6a50 con 0x55e5ad69ddc0 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a1d82f740 10 client.0 _ll_register_callbacks cb 0x55e5ad5a7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:32950/0 (socket says 172.21.15.26:32950) 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a18a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a13e00640 1 -- 172.21.15.26:0/2430766353 learned_addr learned my addr 172.21.15.26:0/2430766353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a0be00640 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a13e00640 1 -- 172.21.15.26:0/2430766353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 msgr2=0x55e5ad6af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a13e00640 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a13e00640 1 -- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 msgr2=0x55e5ad6a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a13e00640 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a13e00640 1 -- 172.21.15.26:0/2430766353 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ad3e1620 con 0x55e5ad69ddc0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.659+0000 7f4a0be00640 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.660+0000 7f4a13e00640 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a8100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a04012010 tx=0x7f4a0400bad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0867a79778ea7f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.660+0000 7f4a01600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.660+0000 7f4a01600640 1 -- 172.21.15.26:0/2430766353 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0401e070 con 0x55e5ad69ddc0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.660+0000 7f4a01600640 1 -- 172.21.15.26:0/2430766353 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4a04004230 con 0x55e5ad69ddc0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.660+0000 7f4a1d82f740 1 -- 172.21.15.26:0/2430766353 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a0401a400 con 0x55e5ad69ddc0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a01600640 1 -- 172.21.15.26:0/2430766353 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4a04035050 con 0x55e5ad69ddc0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a1d82f740 10 client.26389 fetch_fsmap learned FSMap version 36 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a1d82f740 1 -- 172.21.15.26:0/2430766353 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e5ad6b29f0 con 0x55e5ad69ddc0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a01600640 1 -- 172.21.15.26:0/2430766353 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0401a400 con 0x55e5ad69ddc0 2024-06-13T07:11:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a01600640 1 -- 172.21.15.26:0/2430766353 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f4a04004750 con 0x55e5ad69ddc0 2024-06-13T07:11:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a1d82f740 10 client.26389 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:11:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a1d82f740 10 client.26389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a1d82f740 20 client.26389 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e5ad6b3ed0 0x55e5ad6b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a1d82f740 4 client.26389 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.661+0000 7f4a1d82f740 1 -- 172.21.15.26:0/2430766353 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ad60eb50 con 0x55e5ad6b3ed0 2024-06-13T07:11:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.662+0000 7f4a0be00640 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e5ad6b3ed0 0x55e5ad6b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.662+0000 7f4a0be00640 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e5ad6b3ed0 0x55e5ad6b62c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe0044ca8f4b9a5c server_cookie=d8c7153f096ed00f in_seq=0 out_seq=0 2024-06-13T07:11:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.662+0000 7f4a01600640 10 client.26389 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.663+0000 7f4a01600640 1 -- 172.21.15.26:0/2430766353 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e5ad60eb50 con 0x55e5ad6b3ed0 2024-06-13T07:11:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.663+0000 7f4a01600640 10 client.26389 handle_command_reply: tid=0 2024-06-13T07:11:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.663+0000 7f4a1d82f740 1 client.26389 shutdown 2024-06-13T07:11:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.663+0000 7f4a1d82f740 1 -- 172.21.15.26:0/2430766353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69ddc0 msgr2=0x55e5ad6a8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.663+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a8100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a04012010 tx=0x7f4a0400bad0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 1 -- 172.21.15.26:0/2430766353 shutdown_connections 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e5ad6b3ed0 0x55e5ad6b62c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/2430766353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a8100 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a01600640 0 client.26389 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 1 -- 172.21.15.26:0/2430766353 >> 172.21.15.26:0/2430766353 conn(0x55e5ad665fb0 msgr2=0x55e5ad6989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 1 -- 172.21.15.26:0/2430766353 shutdown_connections 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 1 -- 172.21.15.26:0/2430766353 wait complete. 2024-06-13T07:11:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.664+0000 7f4a1d82f740 20 client.26389 trim_cache size 0 max 16384 2024-06-13T07:11:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.685+0000 7f4a1d82f740 1 Processor -- start 2024-06-13T07:11:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.685+0000 7f4a1d82f740 1 -- start start 2024-06-13T07:11:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141aacf0 con 0x55e5ad69ddc0 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141a85c0 con 0x55e5ad6a3920 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141a6410 con 0x55e5ad69e7b0 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a18a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a0be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a18a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59756/0 (socket says 172.21.15.26:59756) 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.686+0000 7f4a18a00640 1 -- 172.21.15.26:0/1730397098 learned_addr learned my addr 172.21.15.26:0/1730397098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a0be00640 1 -- 172.21.15.26:0/1730397098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 msgr2=0x55e5ad6a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a0be00640 1 --2- 172.21.15.26:0/1730397098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a0be00640 1 -- 172.21.15.26:0/1730397098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 msgr2=0x55e5ad6a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a0be00640 1 --2- 172.21.15.26:0/1730397098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a0be00640 1 -- 172.21.15.26:0/1730397098 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ad69e5b0 con 0x55e5ad69e7b0 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a13e00640 1 --2- 172.21.15.26:0/1730397098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a0be00640 1 --2- 172.21.15.26:0/1730397098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6af640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f49fc000c00 tx=0x7f49fc0f5530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ff91a45969ed2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a03e00640 1 -- 172.21.15.26:0/1730397098 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0f5020 con 0x55e5ad69e7b0 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a03e00640 1 -- 172.21.15.26:0/1730397098 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49fc0f1e50 con 0x55e5ad69e7b0 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.687+0000 7f4a03e00640 1 -- 172.21.15.26:0/1730397098 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0bf030 con 0x55e5ad69e7b0 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 -- 172.21.15.26:0/1730397098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69e7b0 msgr2=0x55e5ad6af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/1730397098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6af640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f49fc000c00 tx=0x7f49fc0f5530 comp rx=0 tx=0).stop 2024-06-13T07:11:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 -- 172.21.15.26:0/1730397098 shutdown_connections 2024-06-13T07:11:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/1730397098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad6a3920 0x55e5ad6a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/1730397098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6af640 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/1730397098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 -- 172.21.15.26:0/1730397098 >> 172.21.15.26:0/1730397098 conn(0x55e5ad665fb0 msgr2=0x55e5ad69cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 -- 172.21.15.26:0/1730397098 shutdown_connections 2024-06-13T07:11:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.688+0000 7f4a1d82f740 1 -- 172.21.15.26:0/1730397098 wait complete. 2024-06-13T07:11:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 1 -- start start 2024-06-13T07:11:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 1 Processor -- start 2024-06-13T07:11:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 0x55e5ad6adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141a6a80 con 0x55e5ad69e7b0 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141a7ef0 con 0x55e5ad69ddc0 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141a7150 con 0x55e5ad6a3920 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.689+0000 7f4a1d82f740 10 client.0 _ll_register_callbacks cb 0x7f4a0401af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a18a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 0x55e5ad6adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59760/0 (socket says 172.21.15.26:59760) 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 -- 172.21.15.26:0/3205816247 learned_addr learned my addr 172.21.15.26:0/3205816247 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a0be00640 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 -- 172.21.15.26:0/3205816247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 msgr2=0x55e5ad6adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 0x55e5ad6adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 -- 172.21.15.26:0/3205816247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 msgr2=0x55e5ad6acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 -- 172.21.15.26:0/3205816247 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ad6ae3d0 con 0x55e5ad69ddc0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a13e00640 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6ac890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a0401a900 tx=0x7f4a0401ae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e924d0e93b8f5fa2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.690+0000 7f4a01600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a01600640 1 -- 172.21.15.26:0/3205816247 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0401e070 con 0x55e5ad69ddc0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3205816247 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49fc0f1e50 con 0x55e5ad69ddc0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a01600640 1 -- 172.21.15.26:0/3205816247 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4a04010440 con 0x55e5ad69ddc0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a01600640 1 -- 172.21.15.26:0/3205816247 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4a0401a400 con 0x55e5ad69ddc0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a1d82f740 10 client.26354 fetch_fsmap learned FSMap version 36 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3205816247 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e5ad6af6e0 con 0x55e5ad69ddc0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a01600640 1 -- 172.21.15.26:0/3205816247 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49fc0f1e50 con 0x55e5ad69ddc0 2024-06-13T07:11:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a01600640 1 -- 172.21.15.26:0/3205816247 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f4a04004d60 con 0x55e5ad69ddc0 2024-06-13T07:11:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.691+0000 7f4a1d82f740 10 client.26354 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:11:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.692+0000 7f4a1d82f740 10 client.26354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.692+0000 7f4a1d82f740 20 client.26354 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.692+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e5ad727790 0x55e5ad729b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.692+0000 7f4a1d82f740 4 client.26354 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:11:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.692+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3205816247 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5ad608ea0 con 0x55e5ad727790 2024-06-13T07:11:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.692+0000 7f4a0be00640 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e5ad727790 0x55e5ad729b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.692+0000 7f4a0be00640 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e5ad727790 0x55e5ad729b80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02462ac2905df4c server_cookie=5a7317f10a86a20a in_seq=0 out_seq=0 2024-06-13T07:11:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.692+0000 7f4a01600640 10 client.26354 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a01600640 1 -- 172.21.15.26:0/3205816247 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e5ad608ea0 con 0x55e5ad727790 2024-06-13T07:11:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a01600640 10 client.26354 handle_command_reply: tid=0 2024-06-13T07:11:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 client.26354 shutdown 2024-06-13T07:11:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3205816247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69ddc0 msgr2=0x55e5ad6ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6ac890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a0401a900 tx=0x7f4a0401ae00 comp rx=0 tx=0).stop 2024-06-13T07:11:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3205816247 shutdown_connections 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55e5ad727790 0x55e5ad729b80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5ad6a3920 0x55e5ad6adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5ad69e7b0 0x55e5ad6acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 --2- 172.21.15.26:0/3205816247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5ad69ddc0 0x55e5ad6ac890 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3205816247 >> 172.21.15.26:0/3205816247 conn(0x55e5ad665fb0 msgr2=0x55e5ad698110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.774+0000 7f4a01600640 0 client.26354 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.775+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3205816247 shutdown_connections 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.775+0000 7f4a1d82f740 1 -- 172.21.15.26:0/3205816247 wait complete. 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.775+0000 7f4a1d82f740 20 client.26354 trim_cache size 0 max 16384 2024-06-13T07:11:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:11:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:11:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:11:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.775+0000 7f4a19400640 1 -- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f49f80a2b80 msgr2=0x7f49f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.775+0000 7f4a19400640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f49f80a2b80 0x7f49f80a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55e5ad69f010 tx=0x7f4a0c009500 comp rx=0 tx=0).stop 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 -- 172.21.15.26:0/3933235088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a14104c90 msgr2=0x7f4a141980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a14104c90 0x7f4a141980b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f49fc031440 tx=0x7f49fc032ce0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 -- 172.21.15.26:0/3933235088 shutdown_connections 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f49f80a2b80 0x7f49f80a5040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a14104c90 0x7f4a141980b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141075c0 0x7f4a1419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 --2- 172.21.15.26:0/3933235088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a14102360 0x7f4a14197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 -- 172.21.15.26:0/3933235088 >> 172.21.15.26:0/3933235088 conn(0x7f4a140fbc90 msgr2=0x7f4a14102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 -- 172.21.15.26:0/3933235088 shutdown_connections 2024-06-13T07:11:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:44.776+0000 7f4a19400640 1 -- 172.21.15.26:0/3933235088 wait complete. 2024-06-13T07:11:45.578 INFO:tasks.workunit.client.0.smithi026.stdout: 1 521164 4.77 MB/sec execute 479 sec latency 41.481 ms 2024-06-13T07:11:46.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:46 smithi136 ceph-mon[37229]: pgmap v847: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-06-13T07:11:46.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:46 smithi026 ceph-mon[32041]: pgmap v847: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-06-13T07:11:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:46 smithi135 ceph-mon[36276]: pgmap v847: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-06-13T07:11:46.578 INFO:tasks.workunit.client.0.smithi026.stdout: 1 522249 4.78 MB/sec execute 480 sec latency 64.969 ms 2024-06-13T07:11:47.579 INFO:tasks.workunit.client.0.smithi026.stdout: 1 522664 4.77 MB/sec execute 481 sec latency 31.397 ms 2024-06-13T07:11:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:48 smithi136 ceph-mon[37229]: pgmap v848: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-13T07:11:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:48 smithi026 ceph-mon[32041]: pgmap v848: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-13T07:11:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:48 smithi135 ceph-mon[36276]: pgmap v848: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-13T07:11:48.578 INFO:tasks.workunit.client.0.smithi026.stdout: 1 523273 4.77 MB/sec execute 482 sec latency 31.604 ms 2024-06-13T07:11:49.579 INFO:tasks.workunit.client.0.smithi026.stdout: 1 524543 4.77 MB/sec execute 483 sec latency 33.190 ms 2024-06-13T07:11:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:50 smithi136 ceph-mon[37229]: pgmap v849: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:11:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:50 smithi026 ceph-mon[32041]: pgmap v849: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:11:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:50 smithi135 ceph-mon[36276]: pgmap v849: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-06-13T07:11:50.579 INFO:tasks.workunit.client.0.smithi026.stdout: 1 525236 4.77 MB/sec execute 484 sec latency 26.067 ms 2024-06-13T07:11:51.579 INFO:tasks.workunit.client.0.smithi026.stdout: 1 526089 4.77 MB/sec execute 485 sec latency 74.755 ms 2024-06-13T07:11:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:52 smithi136 ceph-mon[37229]: pgmap v850: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-13T07:11:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:52 smithi026 ceph-mon[32041]: pgmap v850: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-13T07:11:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:52 smithi135 ceph-mon[36276]: pgmap v850: 129 pgs: 129 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-13T07:11:52.579 INFO:tasks.workunit.client.0.smithi026.stdout: 1 526555 4.77 MB/sec execute 486 sec latency 33.599 ms 2024-06-13T07:11:53.579 INFO:tasks.workunit.client.0.smithi026.stdout: 1 527456 4.77 MB/sec execute 487 sec latency 26.427 ms 2024-06-13T07:11:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:54 smithi026 ceph-mon[32041]: pgmap v851: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-13T07:11:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:54 smithi135 ceph-mon[36276]: pgmap v851: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-13T07:11:54.579 INFO:tasks.workunit.client.0.smithi026.stdout: 1 528531 4.77 MB/sec execute 488 sec latency 56.019 ms 2024-06-13T07:11:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:54 smithi136 ceph-mon[37229]: pgmap v851: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-13T07:11:55.579 INFO:tasks.workunit.client.0.smithi026.stdout: 1 529446 4.77 MB/sec execute 489 sec latency 22.257 ms 2024-06-13T07:11:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:56 smithi026 ceph-mon[32041]: pgmap v852: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:11:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:56 smithi135 ceph-mon[36276]: pgmap v852: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:11:56.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 529842 4.77 MB/sec execute 490 sec latency 73.390 ms 2024-06-13T07:11:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:56 smithi136 ceph-mon[37229]: pgmap v852: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-06-13T07:11:57.580 INFO:tasks.workunit.client.0.smithi026.stdout: 1 530557 4.76 MB/sec execute 491 sec latency 53.122 ms 2024-06-13T07:11:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 -- 172.21.15.26:0/2716402112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808105610 msgr2=0x7f980810bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 --2- 172.21.15.26:0/2716402112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808105610 0x7f980810bea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9800009900 tx=0x7f980002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 -- 172.21.15.26:0/2716402112 shutdown_connections 2024-06-13T07:11:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 --2- 172.21.15.26:0/2716402112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808105610 0x7f980810bea0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 --2- 172.21.15.26:0/2716402112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9808104c50 0x7f98081050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 --2- 172.21.15.26:0/2716402112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9808103a60 0x7f9808103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 -- 172.21.15.26:0/2716402112 >> 172.21.15.26:0/2716402112 conn(0x7f98080fefd0 msgr2=0x7f9808101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 -- 172.21.15.26:0/2716402112 shutdown_connections 2024-06-13T07:11:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.940+0000 7f9807200640 1 -- 172.21.15.26:0/2716402112 wait complete. 2024-06-13T07:11:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9807200640 1 Processor -- start 2024-06-13T07:11:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9807200640 1 -- start start 2024-06-13T07:11:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9807200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808103a60 0x7f9808199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9807200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9808104c50 0x7f980819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9807200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9808105610 0x7f98081a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9807200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f980810f730 con 0x7f9808105610 2024-06-13T07:11:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9807200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f980810f5b0 con 0x7f9808103a60 2024-06-13T07:11:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9807200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f980810f8b0 con 0x7f9808104c50 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9805e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808103a60 0x7f9808199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9805400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9808104c50 0x7f980819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9805e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808103a60 0x7f9808199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46204/0 (socket says 172.21.15.26:46204) 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9805e00640 1 -- 172.21.15.26:0/3302439203 learned_addr learned my addr 172.21.15.26:0/3302439203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.942+0000 7f9805400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9808104c50 0x7f980819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51860/0 (socket says 172.21.15.26:51860) 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.943+0000 7f9806800640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9808105610 0x7f98081a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.943+0000 7f9805e00640 1 -- 172.21.15.26:0/3302439203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9808104c50 msgr2=0x7f980819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.943+0000 7f9805e00640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9808104c50 0x7f980819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.943+0000 7f9805e00640 1 -- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9808105610 msgr2=0x7f98081a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.943+0000 7f9805e00640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9808105610 0x7f98081a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.943+0000 7f9805e00640 1 -- 172.21.15.26:0/3302439203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98081a1940 con 0x7f9808103a60 2024-06-13T07:11:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.943+0000 7f9805400640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9808104c50 0x7f980819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.943+0000 7f9805e00640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808103a60 0x7f9808199d30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f97f800eb70 tx=0x7f97f800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.944+0000 7f97fea00640 1 -- 172.21.15.26:0/3302439203 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8003b70 con 0x7f9808103a60 2024-06-13T07:11:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.944+0000 7f97fea00640 1 -- 172.21.15.26:0/3302439203 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f800ee70 con 0x7f9808103a60 2024-06-13T07:11:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.944+0000 7f9807200640 1 -- 172.21.15.26:0/3302439203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98081a1c30 con 0x7f9808103a60 2024-06-13T07:11:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.944+0000 7f97fea00640 1 -- 172.21.15.26:0/3302439203 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8010830 con 0x7f9808103a60 2024-06-13T07:11:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.944+0000 7f9807200640 1 -- 172.21.15.26:0/3302439203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98081a2130 con 0x7f9808103a60 2024-06-13T07:11:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.947+0000 7f97fea00640 1 -- 172.21.15.26:0/3302439203 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97f8004020 con 0x7f9808103a60 2024-06-13T07:11:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.948+0000 7f980fbf0740 1 Processor -- start 2024-06-13T07:11:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.948+0000 7f980fbf0740 1 -- start start 2024-06-13T07:11:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.948+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695920dc0 0x5646959211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f97fea00640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97ec0a2b80 0x7f97ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9806800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695920dc0 0x5646959211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x5646959262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695926920 0x564695928db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f980fbf0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564695884420 con 0x5646959217b0 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f980fbf0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56469582bc50 con 0x564695920dc0 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f980fbf0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5646958803e0 con 0x564695926920 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9806800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695920dc0 0x5646959211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46208/0 (socket says 172.21.15.26:46208) 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9806800640 1 -- 172.21.15.26:0/2681387471 learned_addr learned my addr 172.21.15.26:0/2681387471 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f97fea00640 1 -- 172.21.15.26:0/3302439203 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f97f80cfbb0 con 0x7f9808103a60 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9805e00640 1 --2- 172.21.15.26:0/2681387471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x5646959262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9805400640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97ec0a2b80 0x7f97ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9806800640 1 -- 172.21.15.26:0/2681387471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695926920 msgr2=0x564695928db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9806800640 1 --2- 172.21.15.26:0/2681387471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695926920 0x564695928db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9806800640 1 -- 172.21.15.26:0/2681387471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 msgr2=0x5646959262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9806800640 1 --2- 172.21.15.26:0/2681387471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x5646959262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.949+0000 7f9806800640 1 -- 172.21.15.26:0/2681387471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564695598210 con 0x564695920dc0 2024-06-13T07:11:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.950+0000 7f9806800640 1 --2- 172.21.15.26:0/2681387471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695920dc0 0x5646959211e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f980002a9e0 tx=0x7f9800005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12f42f1c6eee92 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.950+0000 7f97f7400640 1 -- 172.21.15.26:0/2681387471 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800040070 con 0x564695920dc0 2024-06-13T07:11:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.950+0000 7f9805400640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97ec0a2b80 0x7f97ec0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009fc0 tx=0x7f97f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.950+0000 7f97f7400640 1 -- 172.21.15.26:0/2681387471 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9800004400 con 0x564695920dc0 2024-06-13T07:11:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.950+0000 7f97f7400640 1 -- 172.21.15.26:0/2681387471 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800035040 con 0x564695920dc0 2024-06-13T07:11:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 -- 172.21.15.26:0/2681387471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695920dc0 msgr2=0x5646959211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 --2- 172.21.15.26:0/2681387471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695920dc0 0x5646959211e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f980002a9e0 tx=0x7f9800005b90 comp rx=0 tx=0).stop 2024-06-13T07:11:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 -- 172.21.15.26:0/2681387471 shutdown_connections 2024-06-13T07:11:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 --2- 172.21.15.26:0/2681387471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695926920 0x564695928db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 --2- 172.21.15.26:0/2681387471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x5646959262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 --2- 172.21.15.26:0/2681387471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695920dc0 0x5646959211e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 -- 172.21.15.26:0/2681387471 >> 172.21.15.26:0/2681387471 conn(0x5646958e8fb0 msgr2=0x56469591c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 -- 172.21.15.26:0/2681387471 shutdown_connections 2024-06-13T07:11:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.951+0000 7f980fbf0740 1 -- 172.21.15.26:0/2681387471 wait complete. 2024-06-13T07:11:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.952+0000 7f980fbf0740 1 -- start start 2024-06-13T07:11:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.952+0000 7f980fbf0740 1 Processor -- start 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646959217b0 0x56469592b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695926920 0x564695932650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f980fbf0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564695929870 con 0x564695926920 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f980fbf0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56469583f0b0 con 0x5646959217b0 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f980fbf0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5646959299f0 con 0x564695920dc0 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f980fbf0740 10 client.0 _ll_register_callbacks cb 0x56469582af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646959217b0 0x56469592b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646959217b0 0x56469592b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46216/0 (socket says 172.21.15.26:46216) 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805400640 1 -- 172.21.15.26:0/1927434635 learned_addr learned my addr 172.21.15.26:0/1927434635 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9806800640 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695926920 0x564695932650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805e00640 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805400640 1 -- 172.21.15.26:0/1927434635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 msgr2=0x56469592b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805400640 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805400640 1 -- 172.21.15.26:0/1927434635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695926920 msgr2=0x564695932650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805400640 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695926920 0x564695932650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805400640 1 -- 172.21.15.26:0/1927434635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564695664620 con 0x5646959217b0 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.953+0000 7f9805e00640 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.954+0000 7f9805400640 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646959217b0 0x56469592b610 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f97f0003760 tx=0x7f97f0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212fc38b08df4c85 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.954+0000 7f97ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.954+0000 7f97ebe00640 1 -- 172.21.15.26:0/1927434635 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f000f740 con 0x5646959217b0 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.954+0000 7f97ebe00640 1 -- 172.21.15.26:0/1927434635 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f97f000f8e0 con 0x5646959217b0 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.954+0000 7f980fbf0740 1 -- 172.21.15.26:0/1927434635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9800004400 con 0x5646959217b0 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.954+0000 7f97ebe00640 1 -- 172.21.15.26:0/1927434635 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f97f000fae0 con 0x5646959217b0 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.954+0000 7f980fbf0740 10 client.26366 fetch_fsmap learned FSMap version 36 2024-06-13T07:11:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.954+0000 7f980fbf0740 1 -- 172.21.15.26:0/1927434635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5646959359a0 con 0x5646959217b0 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f97ebe00640 1 -- 172.21.15.26:0/1927434635 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9800004400 con 0x5646959217b0 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f97ebe00640 1 -- 172.21.15.26:0/1927434635 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f97f001a070 con 0x5646959217b0 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f980fbf0740 10 client.26366 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f980fbf0740 10 client.26366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f980fbf0740 20 client.26366 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564695937150 0x5646959395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f980fbf0740 4 client.26366 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f980fbf0740 1 -- 172.21.15.26:0/1927434635 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564695891b50 con 0x564695937150 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.955+0000 7f9805e00640 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564695937150 0x5646959395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.956+0000 7f9805e00640 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564695937150 0x5646959395a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48398f3a54ef6572 server_cookie=864424d6ae8ebec5 in_seq=0 out_seq=0 2024-06-13T07:11:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.956+0000 7f97ebe00640 10 client.26366 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f97ebe00640 1 -- 172.21.15.26:0/1927434635 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564695891b50 con 0x564695937150 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f97ebe00640 10 client.26366 handle_command_reply: tid=0 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 client.26366 shutdown 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 -- 172.21.15.26:0/1927434635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646959217b0 msgr2=0x56469592b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646959217b0 0x56469592b610 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f97f0003760 tx=0x7f97f0004730 comp rx=0 tx=0).stop 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 -- 172.21.15.26:0/1927434635 shutdown_connections 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564695937150 0x5646959395a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646959217b0 0x56469592b610 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695926920 0x564695932650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1927434635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f97ebe00640 0 client.26366 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 -- 172.21.15.26:0/1927434635 >> 172.21.15.26:0/1927434635 conn(0x5646958e8fb0 msgr2=0x56469591b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 -- 172.21.15.26:0/1927434635 shutdown_connections 2024-06-13T07:11:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 1 -- 172.21.15.26:0/1927434635 wait complete. 2024-06-13T07:11:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.957+0000 7f980fbf0740 20 client.26366 trim_cache size 0 max 16384 2024-06-13T07:11:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.977+0000 7f980fbf0740 1 Processor -- start 2024-06-13T07:11:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.977+0000 7f980fbf0740 1 -- start start 2024-06-13T07:11:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.977+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695920dc0 0x564695926160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.977+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646959217b0 0x564695932610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.977+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x56469591c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.977+0000 7f980fbf0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98081af0e0 con 0x564695920dc0 2024-06-13T07:11:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.977+0000 7f980fbf0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ac9b0 con 0x564695926920 2024-06-13T07:11:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.977+0000 7f980fbf0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98081aa800 con 0x5646959217b0 2024-06-13T07:11:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9806800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x56469591c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646959217b0 0x564695932610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9806800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x56469591c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46228/0 (socket says 172.21.15.26:46228) 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9806800640 1 -- 172.21.15.26:0/1740678860 learned_addr learned my addr 172.21.15.26:0/1740678860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805e00640 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695920dc0 0x564695926160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805400640 1 -- 172.21.15.26:0/1740678860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 msgr2=0x56469591c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805400640 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x56469591c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805400640 1 -- 172.21.15.26:0/1740678860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695920dc0 msgr2=0x564695926160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805400640 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695920dc0 0x564695926160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805400640 1 -- 172.21.15.26:0/1740678860 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646959233a0 con 0x5646959217b0 2024-06-13T07:11:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9806800640 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x56469591c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805e00640 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695920dc0 0x564695926160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.978+0000 7f9805400640 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646959217b0 0x564695932610 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f97f001c4f0 tx=0x7f97f0011d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c66237169a97cbfc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.979+0000 7f97f6a00640 1 -- 172.21.15.26:0/1740678860 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0038020 con 0x5646959217b0 2024-06-13T07:11:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.979+0000 7f97f6a00640 1 -- 172.21.15.26:0/1740678860 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f000fe80 con 0x5646959217b0 2024-06-13T07:11:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.979+0000 7f97f6a00640 1 -- 172.21.15.26:0/1740678860 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f00346a0 con 0x5646959217b0 2024-06-13T07:11:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.979+0000 7f980fbf0740 1 -- 172.21.15.26:0/1740678860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646959217b0 msgr2=0x564695932610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.979+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646959217b0 0x564695932610 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f97f001c4f0 tx=0x7f97f0011d80 comp rx=0 tx=0).stop 2024-06-13T07:11:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 -- 172.21.15.26:0/1740678860 shutdown_connections 2024-06-13T07:11:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x56469591c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646959217b0 0x564695932610 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 --2- 172.21.15.26:0/1740678860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564695920dc0 0x564695926160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 -- 172.21.15.26:0/1740678860 >> 172.21.15.26:0/1740678860 conn(0x5646958e8fb0 msgr2=0x564695922040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 -- 172.21.15.26:0/1740678860 shutdown_connections 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 -- 172.21.15.26:0/1740678860 wait complete. 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 -- start start 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.980+0000 7f980fbf0740 1 Processor -- start 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x56469592fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f980fbf0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x564695930cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f980fbf0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98081aae70 con 0x5646959217b0 2024-06-13T07:11:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f980fbf0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ac2e0 con 0x564695926920 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f980fbf0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ab540 con 0x564695920dc0 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f980fbf0740 10 client.0 _ll_register_callbacks cb 0x7f97f00035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f9805e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f9806800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x564695930cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f9805e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51896/0 (socket says 172.21.15.26:51896) 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f9806800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x564695930cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46234/0 (socket says 172.21.15.26:46234) 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f9805e00640 1 -- 172.21.15.26:0/675923062 learned_addr learned my addr 172.21.15.26:0/675923062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.981+0000 7f9805400640 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x56469592fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f9805e00640 1 -- 172.21.15.26:0/675923062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 msgr2=0x564695930cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f9805e00640 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x564695930cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f9805e00640 1 -- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 msgr2=0x56469592fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f9805e00640 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x56469592fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f9805e00640 1 -- 172.21.15.26:0/675923062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646959312e0 con 0x564695920dc0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f9805400640 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x56469592fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f9805e00640 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592f7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f97f8098900 tx=0x7f97f809a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c2974577a861efc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f97ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f97ebe00640 1 -- 172.21.15.26:0/675923062 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8065020 con 0x564695920dc0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f97ebe00640 1 -- 172.21.15.26:0/675923062 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f97f8004310 con 0x564695920dc0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f980fbf0740 1 -- 172.21.15.26:0/675923062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56469588bea0 con 0x564695920dc0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f97ebe00640 1 -- 172.21.15.26:0/675923062 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f97f809bfc0 con 0x564695920dc0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f980fbf0740 10 client.26410 fetch_fsmap learned FSMap version 36 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.982+0000 7f980fbf0740 1 -- 172.21.15.26:0/675923062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564695932660 con 0x564695920dc0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f97ebe00640 1 -- 172.21.15.26:0/675923062 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56469588bea0 con 0x564695920dc0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f97ebe00640 1 -- 172.21.15.26:0/675923062 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f97f809e040 con 0x564695920dc0 2024-06-13T07:11:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f980fbf0740 10 client.26410 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:11:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f980fbf0740 10 client.26410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:11:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f980fbf0740 20 client.26410 populate_metadata read hostname 'smithi026' 2024-06-13T07:11:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f980fbf0740 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5646959a23e0 0x5646959a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f980fbf0740 4 client.26410 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:11:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f980fbf0740 1 -- 172.21.15.26:0/675923062 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646958097a0 con 0x5646959a23e0 2024-06-13T07:11:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.983+0000 7f9805400640 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5646959a23e0 0x5646959a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.984+0000 7f9805400640 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5646959a23e0 0x5646959a4800 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d605ef22e9c6aa server_cookie=55cf7b277a002e89 in_seq=0 out_seq=0 2024-06-13T07:11:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:57.984+0000 7f97ebe00640 10 client.26410 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f97ebe00640 1 -- 172.21.15.26:0/675923062 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5646958097a0 con 0x5646959a23e0 2024-06-13T07:11:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f97ebe00640 10 client.26410 handle_command_reply: tid=0 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 client.26410 shutdown 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 -- 172.21.15.26:0/675923062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 msgr2=0x56469592f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592f7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f97f8098900 tx=0x7f97f809a1c0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 -- 172.21.15.26:0/675923062 shutdown_connections 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5646959a23e0 0x5646959a4800 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564695926920 0x564695930cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5646959217b0 0x56469592fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 --2- 172.21.15.26:0/675923062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564695920dc0 0x56469592f7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f97ebe00640 0 client.26410 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.066+0000 7f980fbf0740 1 -- 172.21.15.26:0/675923062 >> 172.21.15.26:0/675923062 conn(0x5646958e8fb0 msgr2=0x564695926090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.067+0000 7f980fbf0740 1 -- 172.21.15.26:0/675923062 shutdown_connections 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.067+0000 7f980fbf0740 1 -- 172.21.15.26:0/675923062 wait complete. 2024-06-13T07:11:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.067+0000 7f980fbf0740 20 client.26410 trim_cache size 0 max 16384 2024-06-13T07:11:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:11:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:11:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 -- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97ec0a2b80 msgr2=0x7f97ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97ec0a2b80 0x7f97ec0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009fc0 tx=0x7f97f0005f90 comp rx=0 tx=0).stop 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 -- 172.21.15.26:0/3302439203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808103a60 msgr2=0x7f9808199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808103a60 0x7f9808199d30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f97f800eb70 tx=0x7f97f800c6a0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 -- 172.21.15.26:0/3302439203 shutdown_connections 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97ec0a2b80 0x7f97ec0a5040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9808105610 0x7f98081a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9808104c50 0x7f980819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 --2- 172.21.15.26:0/3302439203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9808103a60 0x7f9808199d30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 -- 172.21.15.26:0/3302439203 >> 172.21.15.26:0/3302439203 conn(0x7f98080fefd0 msgr2=0x7f9808107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 -- 172.21.15.26:0/3302439203 shutdown_connections 2024-06-13T07:11:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:11:58.068+0000 7f9807200640 1 -- 172.21.15.26:0/3302439203 wait complete. 2024-06-13T07:11:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:11:58 smithi026 ceph-mon[32041]: pgmap v853: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-13T07:11:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:11:58 smithi135 ceph-mon[36276]: pgmap v853: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-13T07:11:58.580 INFO:tasks.workunit.client.0.smithi026.stdout: 1 531796 4.76 MB/sec execute 492 sec latency 29.129 ms 2024-06-13T07:11:58.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:11:58 smithi136 ceph-mon[37229]: pgmap v853: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-13T07:11:59.580 INFO:tasks.workunit.client.0.smithi026.stdout: 1 532709 4.77 MB/sec execute 493 sec latency 27.401 ms 2024-06-13T07:12:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:00 smithi026 ceph-mon[32041]: pgmap v854: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:12:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:00 smithi135 ceph-mon[36276]: pgmap v854: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:12:00.580 INFO:tasks.workunit.client.0.smithi026.stdout: 1 533260 4.77 MB/sec execute 494 sec latency 24.666 ms 2024-06-13T07:12:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:00 smithi136 ceph-mon[37229]: pgmap v854: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:12:01.580 INFO:tasks.workunit.client.0.smithi026.stdout: 1 533831 4.76 MB/sec execute 495 sec latency 56.521 ms 2024-06-13T07:12:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:02 smithi026 ceph-mon[32041]: pgmap v855: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-06-13T07:12:02.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:02 smithi135 ceph-mon[36276]: pgmap v855: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-06-13T07:12:02.580 INFO:tasks.workunit.client.0.smithi026.stdout: 1 535222 4.76 MB/sec execute 496 sec latency 43.932 ms 2024-06-13T07:12:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:02 smithi136 ceph-mon[37229]: pgmap v855: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-06-13T07:12:03.580 INFO:tasks.workunit.client.0.smithi026.stdout: 1 536075 4.76 MB/sec execute 497 sec latency 68.834 ms 2024-06-13T07:12:04.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:04 smithi026 ceph-mon[32041]: pgmap v856: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-06-13T07:12:04.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:04 smithi135 ceph-mon[36276]: pgmap v856: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-06-13T07:12:04.580 INFO:tasks.workunit.client.0.smithi026.stdout: 1 536771 4.77 MB/sec execute 498 sec latency 23.158 ms 2024-06-13T07:12:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:04 smithi136 ceph-mon[37229]: pgmap v856: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-06-13T07:12:05.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 537296 4.76 MB/sec execute 499 sec latency 46.662 ms 2024-06-13T07:12:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:06 smithi026 ceph-mon[32041]: pgmap v857: 129 pgs: 129 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:12:06.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:06 smithi135 ceph-mon[36276]: pgmap v857: 129 pgs: 129 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:12:06.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 538631 4.76 MB/sec execute 500 sec latency 25.930 ms 2024-06-13T07:12:06.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:06 smithi136 ceph-mon[37229]: pgmap v857: 129 pgs: 129 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-13T07:12:07.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 539639 4.76 MB/sec execute 501 sec latency 23.266 ms 2024-06-13T07:12:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:08 smithi026 ceph-mon[32041]: pgmap v858: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-06-13T07:12:08.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:08 smithi135 ceph-mon[36276]: pgmap v858: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-06-13T07:12:08.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 540338 4.77 MB/sec execute 502 sec latency 29.233 ms 2024-06-13T07:12:08.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:08 smithi136 ceph-mon[37229]: pgmap v858: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-06-13T07:12:09.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 540866 4.76 MB/sec execute 503 sec latency 27.493 ms 2024-06-13T07:12:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:10 smithi026 ceph-mon[32041]: pgmap v859: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.2 MiB/s wr, 169 op/s 2024-06-13T07:12:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:10 smithi135 ceph-mon[36276]: pgmap v859: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.2 MiB/s wr, 169 op/s 2024-06-13T07:12:10.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 541856 4.76 MB/sec execute 504 sec latency 24.932 ms 2024-06-13T07:12:10.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:10 smithi136 ceph-mon[37229]: pgmap v859: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.2 MiB/s wr, 169 op/s 2024-06-13T07:12:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 -- 172.21.15.26:0/353881519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 msgr2=0x7f1c8c1144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 --2- 172.21.15.26:0/353881519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 0x7f1c8c1144d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009860 tx=0x7f1c7c02a360 comp rx=0 tx=0).stop 2024-06-13T07:12:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 -- 172.21.15.26:0/353881519 shutdown_connections 2024-06-13T07:12:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 --2- 172.21.15.26:0/353881519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 0x7f1c8c1144d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 --2- 172.21.15.26:0/353881519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c8c10aef0 0x7f1c8c10b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 --2- 172.21.15.26:0/353881519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c8c0737d0 0x7f1c8c10a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 -- 172.21.15.26:0/353881519 >> 172.21.15.26:0/353881519 conn(0x7f1c8c06ea00 msgr2=0x7f1c8c070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 -- 172.21.15.26:0/353881519 shutdown_connections 2024-06-13T07:12:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.268+0000 7f1c90e00640 1 -- 172.21.15.26:0/353881519 wait complete. 2024-06-13T07:12:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.270+0000 7f1c90e00640 1 Processor -- start 2024-06-13T07:12:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.270+0000 7f1c90e00640 1 -- start start 2024-06-13T07:12:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.270+0000 7f1c90e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c8c0737d0 0x7f1c8c1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.270+0000 7f1c90e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c8c10aef0 0x7f1c8c1a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.270+0000 7f1c90e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 0x7f1c8c1ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.270+0000 7f1c90e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c117ea0 con 0x7f1c8c0737d0 2024-06-13T07:12:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.270+0000 7f1c90e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c117d20 con 0x7f1c8c10aef0 2024-06-13T07:12:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.270+0000 7f1c90e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c118020 con 0x7f1c8c1120e0 2024-06-13T07:12:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c8c0737d0 0x7f1c8c1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c8c10aef0 0x7f1c8c1a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 0x7f1c8c1ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c8c10aef0 0x7f1c8c1a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36146/0 (socket says 172.21.15.26:36146) 2024-06-13T07:12:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c8c0737d0 0x7f1c8c1a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35242/0 (socket says 172.21.15.26:35242) 2024-06-13T07:12:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8aa00640 1 -- 172.21.15.26:0/869379050 learned_addr learned my addr 172.21.15.26:0/869379050 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8aa00640 1 -- 172.21.15.26:0/869379050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 msgr2=0x7f1c8c1ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8aa00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 0x7f1c8c1ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8aa00640 1 -- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c8c0737d0 msgr2=0x7f1c8c1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8aa00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c8c0737d0 0x7f1c8c1a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8aa00640 1 -- 172.21.15.26:0/869379050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c1ae4d0 con 0x7f1c8c10aef0 2024-06-13T07:12:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.271+0000 7f1c8be00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 0x7f1c8c1ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.272+0000 7f1c8aa00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c8c10aef0 0x7f1c8c1a6df0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c920 tx=0x7f1c7800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.272+0000 7f1c83e00640 1 -- 172.21.15.26:0/869379050 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78007c70 con 0x7f1c8c10aef0 2024-06-13T07:12:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.272+0000 7f1c83e00640 1 -- 172.21.15.26:0/869379050 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c78007e10 con 0x7f1c8c10aef0 2024-06-13T07:12:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.272+0000 7f1c90e00640 1 -- 172.21.15.26:0/869379050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8c1ae7c0 con 0x7f1c8c10aef0 2024-06-13T07:12:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.272+0000 7f1c83e00640 1 -- 172.21.15.26:0/869379050 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78018890 con 0x7f1c8c10aef0 2024-06-13T07:12:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.272+0000 7f1c90e00640 1 -- 172.21.15.26:0/869379050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8c1aed10 con 0x7f1c8c10aef0 2024-06-13T07:12:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.277+0000 7f1c9538e740 1 Processor -- start 2024-06-13T07:12:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.277+0000 7f1c9538e740 1 -- start start 2024-06-13T07:12:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb37e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb385db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c9538e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558ebb2e1420 con 0x558ebb37ddc0 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c9538e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558ebb288c50 con 0x558ebb37e7b0 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c9538e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ebb2dd3e0 con 0x558ebb383920 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c83e00640 1 -- 172.21.15.26:0/869379050 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c78018a70 con 0x7f1c8c10aef0 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3832b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36158/0 (socket says 172.21.15.26:36158) 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8b400640 1 -- 172.21.15.26:0/3319833147 learned_addr learned my addr 172.21.15.26:0/3319833147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8aa00640 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb385db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8be00640 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb37e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8b400640 1 -- 172.21.15.26:0/3319833147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 msgr2=0x558ebb385db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8b400640 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb385db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8b400640 1 -- 172.21.15.26:0/3319833147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 msgr2=0x558ebb37e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8b400640 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb37e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8b400640 1 -- 172.21.15.26:0/3319833147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ebaff5210 con 0x558ebb37e7b0 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.278+0000 7f1c8be00640 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb37e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.279+0000 7f1c8b400640 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3832b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e550 tx=0x7f1c840095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=325cc62d883b489c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.279+0000 7f1c80c00640 1 -- 172.21.15.26:0/3319833147 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84004b20 con 0x558ebb37e7b0 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.279+0000 7f1c80c00640 1 -- 172.21.15.26:0/3319833147 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c84003e50 con 0x558ebb37e7b0 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.279+0000 7f1c80c00640 1 -- 172.21.15.26:0/3319833147 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84004020 con 0x558ebb37e7b0 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 -- 172.21.15.26:0/3319833147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 msgr2=0x558ebb3832b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3832b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e550 tx=0x7f1c840095c0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 -- 172.21.15.26:0/3319833147 shutdown_connections 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3832b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb385db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 --2- 172.21.15.26:0/3319833147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb37e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 -- 172.21.15.26:0/3319833147 >> 172.21.15.26:0/3319833147 conn(0x558ebb345fb0 msgr2=0x558ebb379840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 -- 172.21.15.26:0/3319833147 shutdown_connections 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c9538e740 1 -- 172.21.15.26:0/3319833147 wait complete. 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c83e00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1c700a2b10 0x7f1c700a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.280+0000 7f1c83e00640 1 -- 172.21.15.26:0/869379050 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1c780cf530 con 0x7f1c8c10aef0 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c9538e740 1 -- start start 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c8b400640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1c700a2b10 0x7f1c700a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c9538e740 1 Processor -- start 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb37ddc0 0x558ebb388090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb38f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c9538e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558ebb3868d0 con 0x558ebb383920 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c9538e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558ebb29c0b0 con 0x558ebb37e7b0 2024-06-13T07:12:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.281+0000 7f1c9538e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ebb386a50 con 0x558ebb37ddc0 2024-06-13T07:12:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c9538e740 10 client.0 _ll_register_callbacks cb 0x558ebb287f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:12:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb38f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3885d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36170/0 (socket says 172.21.15.26:36170) 2024-06-13T07:12:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb38f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35260/0 (socket says 172.21.15.26:35260) 2024-06-13T07:12:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8aa00640 1 -- 172.21.15.26:0/775300799 learned_addr learned my addr 172.21.15.26:0/775300799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8aa00640 1 -- 172.21.15.26:0/775300799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb37ddc0 msgr2=0x558ebb388090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:12:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8aa00640 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb37ddc0 0x558ebb388090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8aa00640 1 -- 172.21.15.26:0/775300799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 msgr2=0x558ebb38f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8aa00640 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb38f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8aa00640 1 -- 172.21.15.26:0/775300799 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebb0c1620 con 0x558ebb37e7b0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.282+0000 7f1c8b400640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1c700a2b10 0x7f1c700a4fd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e520 tx=0x7f1c8400ef90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.283+0000 7f1c8aa00640 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3885d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1c78098310 tx=0x7f1c7809b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b11c3770a2a1b78 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.283+0000 7f1c75600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.283+0000 7f1c75600640 1 -- 172.21.15.26:0/775300799 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7809b940 con 0x558ebb37e7b0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.283+0000 7f1c75600640 1 -- 172.21.15.26:0/775300799 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1c780d2d40 con 0x558ebb37e7b0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.283+0000 7f1c9538e740 1 -- 172.21.15.26:0/775300799 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c84003e50 con 0x558ebb37e7b0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.284+0000 7f1c75600640 1 -- 172.21.15.26:0/775300799 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1c7809fcf0 con 0x558ebb37e7b0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.284+0000 7f1c9538e740 10 client.26384 fetch_fsmap learned FSMap version 36 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.284+0000 7f1c9538e740 1 -- 172.21.15.26:0/775300799 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558ebb392930 con 0x558ebb37e7b0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.284+0000 7f1c75600640 1 -- 172.21.15.26:0/775300799 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c84003e50 con 0x558ebb37e7b0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.284+0000 7f1c75600640 1 -- 172.21.15.26:0/775300799 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f1c780b4070 con 0x558ebb37e7b0 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.284+0000 7f1c9538e740 10 client.26384 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:12:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.284+0000 7f1c9538e740 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:12:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.285+0000 7f1c9538e740 20 client.26384 populate_metadata read hostname 'smithi026' 2024-06-13T07:12:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.285+0000 7f1c9538e740 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558ebb394020 0x558ebb396460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.285+0000 7f1c9538e740 4 client.26384 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:12:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.285+0000 7f1c9538e740 1 -- 172.21.15.26:0/775300799 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ebb2eeb50 con 0x558ebb394020 2024-06-13T07:12:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.285+0000 7f1c8b400640 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558ebb394020 0x558ebb396460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.286+0000 7f1c75600640 10 client.26384 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.286+0000 7f1c8b400640 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558ebb394020 0x558ebb396460 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71694b29a9e9c05 server_cookie=bf208709e483f821 in_seq=0 out_seq=0 2024-06-13T07:12:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c75600640 1 -- 172.21.15.26:0/775300799 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558ebb2eeb50 con 0x558ebb394020 2024-06-13T07:12:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c75600640 10 client.26384 handle_command_reply: tid=0 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 client.26384 shutdown 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 -- 172.21.15.26:0/775300799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 msgr2=0x558ebb3885d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3885d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1c78098310 tx=0x7f1c7809b480 comp rx=0 tx=0).stop 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 -- 172.21.15.26:0/775300799 shutdown_connections 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558ebb394020 0x558ebb396460 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb3885d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb38f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 --2- 172.21.15.26:0/775300799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb37ddc0 0x558ebb388090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c9538e740 1 -- 172.21.15.26:0/775300799 >> 172.21.15.26:0/775300799 conn(0x558ebb345fb0 msgr2=0x558ebb3789a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.287+0000 7f1c75600640 0 client.26384 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.288+0000 7f1c9538e740 1 -- 172.21.15.26:0/775300799 shutdown_connections 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.288+0000 7f1c9538e740 1 -- 172.21.15.26:0/775300799 wait complete. 2024-06-13T07:12:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.288+0000 7f1c9538e740 20 client.26384 trim_cache size 0 max 16384 2024-06-13T07:12:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.314+0000 7f1c9538e740 1 Processor -- start 2024-06-13T07:12:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.314+0000 7f1c9538e740 1 -- start start 2024-06-13T07:12:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.314+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb387360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.314+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.314+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb3878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.314+0000 7f1c9538e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1bbcd0 con 0x558ebb37ddc0 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.314+0000 7f1c9538e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b95a0 con 0x558ebb37e7b0 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.314+0000 7f1c9538e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b73f0 con 0x558ebb383920 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb3878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb3878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36406/0 (socket says 172.21.15.26:36406) 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8be00640 1 -- 172.21.15.26:0/437211583 learned_addr learned my addr 172.21.15.26:0/437211583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8aa00640 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8b400640 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb387360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8be00640 1 -- 172.21.15.26:0/437211583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 msgr2=0x558ebb38f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8be00640 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8be00640 1 -- 172.21.15.26:0/437211583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 msgr2=0x558ebb387360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8be00640 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb387360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8be00640 1 -- 172.21.15.26:0/437211583 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ebb375000 con 0x558ebb383920 2024-06-13T07:12:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.315+0000 7f1c8b400640 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb387360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:12:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.316+0000 7f1c8be00640 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb3878a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0057c0 tx=0x7f1c7c0305f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2162cfab3a2268 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.316+0000 7f1c77e00640 1 -- 172.21.15.26:0/437211583 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c037040 con 0x558ebb383920 2024-06-13T07:12:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.316+0000 7f1c77e00640 1 -- 172.21.15.26:0/437211583 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c004030 con 0x558ebb383920 2024-06-13T07:12:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.316+0000 7f1c77e00640 1 -- 172.21.15.26:0/437211583 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0035f0 con 0x558ebb383920 2024-06-13T07:12:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.317+0000 7f1c9538e740 1 -- 172.21.15.26:0/437211583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 msgr2=0x558ebb3878a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.317+0000 7f1c9538e740 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb3878a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0057c0 tx=0x7f1c7c0305f0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.318+0000 7f1c9538e740 1 -- 172.21.15.26:0/437211583 shutdown_connections 2024-06-13T07:12:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.318+0000 7f1c9538e740 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.318+0000 7f1c9538e740 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb383920 0x558ebb3878a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.318+0000 7f1c9538e740 1 --2- 172.21.15.26:0/437211583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb37ddc0 0x558ebb387360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.318+0000 7f1c9538e740 1 -- 172.21.15.26:0/437211583 >> 172.21.15.26:0/437211583 conn(0x558ebb345fb0 msgr2=0x558ebb37fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.318+0000 7f1c9538e740 1 -- 172.21.15.26:0/437211583 shutdown_connections 2024-06-13T07:12:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.319+0000 7f1c9538e740 1 -- 172.21.15.26:0/437211583 wait complete. 2024-06-13T07:12:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.319+0000 7f1c9538e740 1 -- start start 2024-06-13T07:12:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.319+0000 7f1c9538e740 1 Processor -- start 2024-06-13T07:12:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.321+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb37ddc0 0x558ebb38c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.321+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.321+0000 7f1c9538e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb3fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c9538e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b7a60 con 0x558ebb383920 2024-06-13T07:12:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c9538e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b8ed0 con 0x558ebb37e7b0 2024-06-13T07:12:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c9538e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b8130 con 0x558ebb37ddc0 2024-06-13T07:12:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c9538e740 10 client.0 _ll_register_callbacks cb 0x7f1c78097be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:12:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c8be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb3fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c8be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb3fcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35272/0 (socket says 172.21.15.26:35272) 2024-06-13T07:12:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c8be00640 1 -- 172.21.15.26:0/562831642 learned_addr learned my addr 172.21.15.26:0/562831642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.322+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36202/0 (socket says 172.21.15.26:36202) 2024-06-13T07:12:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.323+0000 7f1c8be00640 1 -- 172.21.15.26:0/562831642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb37ddc0 msgr2=0x558ebb38c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.323+0000 7f1c8be00640 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb37ddc0 0x558ebb38c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.323+0000 7f1c8be00640 1 -- 172.21.15.26:0/562831642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 msgr2=0x558ebb38c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.323+0000 7f1c8be00640 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.323+0000 7f1c8be00640 1 -- 172.21.15.26:0/562831642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebb38f6f0 con 0x558ebb383920 2024-06-13T07:12:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.324+0000 7f1c8be00640 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb3fcd30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c030520 tx=0x7f1c7c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745b8282f63e8277 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.324+0000 7f1c75600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:12:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.325+0000 7f1c75600640 1 -- 172.21.15.26:0/562831642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c04d070 con 0x558ebb383920 2024-06-13T07:12:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.325+0000 7f1c9538e740 1 -- 172.21.15.26:0/562831642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7c004030 con 0x558ebb383920 2024-06-13T07:12:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.325+0000 7f1c75600640 1 -- 172.21.15.26:0/562831642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1c7c0043f0 con 0x558ebb383920 2024-06-13T07:12:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.326+0000 7f1c75600640 1 -- 172.21.15.26:0/562831642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1c7c059050 con 0x558ebb383920 2024-06-13T07:12:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.326+0000 7f1c9538e740 10 client.16272 fetch_fsmap learned FSMap version 36 2024-06-13T07:12:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.326+0000 7f1c9538e740 1 -- 172.21.15.26:0/562831642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558ebb3fe1b0 con 0x558ebb383920 2024-06-13T07:12:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.327+0000 7f1c75600640 1 -- 172.21.15.26:0/562831642 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c004030 con 0x558ebb383920 2024-06-13T07:12:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.329+0000 7f1c75600640 1 -- 172.21.15.26:0/562831642 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f1c7c003ce0 con 0x558ebb383920 2024-06-13T07:12:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.329+0000 7f1c9538e740 10 client.16272 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:12:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.329+0000 7f1c9538e740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:12:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.329+0000 7f1c9538e740 20 client.16272 populate_metadata read hostname 'smithi026' 2024-06-13T07:12:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.329+0000 7f1c9538e740 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558ebb3feb10 0x558ebb400f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.329+0000 7f1c9538e740 4 client.16272 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:12:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.329+0000 7f1c9538e740 1 -- 172.21.15.26:0/562831642 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ebb2e8ea0 con 0x558ebb3feb10 2024-06-13T07:12:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.330+0000 7f1c8b400640 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558ebb3feb10 0x558ebb400f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.331+0000 7f1c8b400640 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558ebb3feb10 0x558ebb400f30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f5c6b1b5b866a45 server_cookie=f8bc8abae1819c55 in_seq=0 out_seq=0 2024-06-13T07:12:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.331+0000 7f1c75600640 10 client.16272 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c75600640 1 -- 172.21.15.26:0/562831642 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558ebb2e8ea0 con 0x558ebb3feb10 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c75600640 10 client.16272 handle_command_reply: tid=0 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 client.16272 shutdown 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 -- 172.21.15.26:0/562831642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 msgr2=0x558ebb3fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb3fcd30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c030520 tx=0x7f1c7c0328a0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 -- 172.21.15.26:0/562831642 shutdown_connections 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558ebb3feb10 0x558ebb400f30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558ebb37e7b0 0x558ebb38c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ebb383920 0x558ebb3fcd30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 --2- 172.21.15.26:0/562831642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ebb37ddc0 0x558ebb38c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c75600640 0 client.16272 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.420+0000 7f1c9538e740 1 -- 172.21.15.26:0/562831642 >> 172.21.15.26:0/562831642 conn(0x558ebb345fb0 msgr2=0x558ebb387360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.421+0000 7f1c9538e740 1 -- 172.21.15.26:0/562831642 shutdown_connections 2024-06-13T07:12:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.421+0000 7f1c9538e740 1 -- 172.21.15.26:0/562831642 wait complete. 2024-06-13T07:12:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.421+0000 7f1c9538e740 20 client.16272 trim_cache size 0 max 16384 2024-06-13T07:12:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:12:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:12:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:12:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 -- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1c700a2b10 msgr2=0x7f1c700a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1c700a2b10 0x7f1c700a4fd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e520 tx=0x7f1c8400ef90 comp rx=0 tx=0).stop 2024-06-13T07:12:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 -- 172.21.15.26:0/869379050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c8c10aef0 msgr2=0x7f1c8c1a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c8c10aef0 0x7f1c8c1a6df0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c920 tx=0x7f1c7800cde0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 -- 172.21.15.26:0/869379050 shutdown_connections 2024-06-13T07:12:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1c700a2b10 0x7f1c700a4fd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c8c10aef0 0x7f1c8c1a6df0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c8c1120e0 0x7f1c8c1ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 --2- 172.21.15.26:0/869379050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c8c0737d0 0x7f1c8c1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 -- 172.21.15.26:0/869379050 >> 172.21.15.26:0/869379050 conn(0x7f1c8c06ea00 msgr2=0x7f1c8c108e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 -- 172.21.15.26:0/869379050 shutdown_connections 2024-06-13T07:12:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:11.422+0000 7f1c90e00640 1 -- 172.21.15.26:0/869379050 wait complete. 2024-06-13T07:12:11.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 542979 4.76 MB/sec execute 505 sec latency 32.677 ms 2024-06-13T07:12:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:12 smithi026 ceph-mon[32041]: pgmap v860: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-06-13T07:12:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:12 smithi135 ceph-mon[36276]: pgmap v860: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-06-13T07:12:12.581 INFO:tasks.workunit.client.0.smithi026.stdout: 1 543843 4.77 MB/sec execute 506 sec latency 23.193 ms 2024-06-13T07:12:12.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:12 smithi136 ceph-mon[37229]: pgmap v860: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-06-13T07:12:13.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 544326 4.76 MB/sec execute 507 sec latency 28.519 ms 2024-06-13T07:12:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:14 smithi026 ceph-mon[32041]: pgmap v861: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-13T07:12:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:14 smithi135 ceph-mon[36276]: pgmap v861: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-13T07:12:14.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 545346 4.76 MB/sec execute 508 sec latency 35.022 ms 2024-06-13T07:12:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:14 smithi136 ceph-mon[37229]: pgmap v861: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-13T07:12:15.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 546503 4.76 MB/sec execute 509 sec latency 18.997 ms 2024-06-13T07:12:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:16 smithi026 ceph-mon[32041]: pgmap v862: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:12:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:16 smithi135 ceph-mon[36276]: pgmap v862: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:12:16.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 547338 4.77 MB/sec execute 510 sec latency 49.055 ms 2024-06-13T07:12:16.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:16 smithi136 ceph-mon[37229]: pgmap v862: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-13T07:12:17.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 547859 4.76 MB/sec execute 511 sec latency 23.458 ms 2024-06-13T07:12:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:18 smithi026 ceph-mon[32041]: pgmap v863: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-13T07:12:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:18 smithi135 ceph-mon[36276]: pgmap v863: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-13T07:12:18.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 548896 4.76 MB/sec execute 512 sec latency 24.709 ms 2024-06-13T07:12:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:18 smithi136 ceph-mon[37229]: pgmap v863: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-13T07:12:19.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 550067 4.76 MB/sec execute 513 sec latency 27.800 ms 2024-06-13T07:12:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:20 smithi026 ceph-mon[32041]: pgmap v864: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-13T07:12:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:20 smithi135 ceph-mon[36276]: pgmap v864: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-13T07:12:20.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 550883 4.77 MB/sec execute 514 sec latency 21.587 ms 2024-06-13T07:12:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:20 smithi136 ceph-mon[37229]: pgmap v864: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-13T07:12:21.582 INFO:tasks.workunit.client.0.smithi026.stdout: 1 551346 4.76 MB/sec execute 515 sec latency 36.132 ms 2024-06-13T07:12:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:22 smithi026 ceph-mon[32041]: pgmap v865: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:12:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:22 smithi135 ceph-mon[36276]: pgmap v865: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:12:22.583 INFO:tasks.workunit.client.0.smithi026.stdout: 1 552507 4.76 MB/sec execute 516 sec latency 34.951 ms 2024-06-13T07:12:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:22 smithi136 ceph-mon[37229]: pgmap v865: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:12:23.583 INFO:tasks.workunit.client.0.smithi026.stdout: 1 553611 4.76 MB/sec execute 517 sec latency 20.936 ms 2024-06-13T07:12:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:24 smithi026 ceph-mon[32041]: pgmap v866: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-13T07:12:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:24 smithi135 ceph-mon[36276]: pgmap v866: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-13T07:12:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1362500000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20101830 msgr2=0x7f6b20101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1362500000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20101830 0x7f6b20101c50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009900 tx=0x7f6b1002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1362500000 shutdown_connections 2024-06-13T07:12:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1362500000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b201033e0 0x7f6b20109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1362500000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b20102a20 0x7f6b20102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1362500000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20101830 0x7f6b20101c50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1362500000 >> 172.21.15.26:0/1362500000 conn(0x7f6b200fcda0 msgr2=0x7f6b200ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1362500000 shutdown_connections 2024-06-13T07:12:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.577+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1362500000 wait complete. 2024-06-13T07:12:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.578+0000 7f6b1fe00640 1 Processor -- start 2024-06-13T07:12:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.578+0000 7f6b1fe00640 1 -- start start 2024-06-13T07:12:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b20101830 0x7f6b20199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b20102a20 0x7f6b2019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b201033e0 0x7f6b201a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010d480 con 0x7f6b201033e0 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1fe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010d300 con 0x7f6b20102a20 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010d600 con 0x7f6b20101830 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b20101830 0x7f6b20199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b20101830 0x7f6b20199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51468/0 (socket says 172.21.15.26:51468) 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b20102a20 0x7f6b2019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1ea00640 1 -- 172.21.15.26:0/1959059712 learned_addr learned my addr 172.21.15.26:0/1959059712 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.579+0000 7f6b1f400640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b201033e0 0x7f6b201a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b1ea00640 1 -- 172.21.15.26:0/1959059712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b20102a20 msgr2=0x7f6b2019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b20102a20 0x7f6b2019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b1ea00640 1 -- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b201033e0 msgr2=0x7f6b201a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b201033e0 0x7f6b201a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b1ea00640 1 -- 172.21.15.26:0/1959059712 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b201a1980 con 0x7f6b20101830 2024-06-13T07:12:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b20102a20 0x7f6b2019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b20101830 0x7f6b20199d60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6b10031610 tx=0x7f6b10004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b17400640 1 -- 172.21.15.26:0/1959059712 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1003f030 con 0x7f6b20101830 2024-06-13T07:12:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.580+0000 7f6b17400640 1 -- 172.21.15.26:0/1959059712 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b10003ad0 con 0x7f6b20101830 2024-06-13T07:12:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.581+0000 7f6b17400640 1 -- 172.21.15.26:0/1959059712 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10035070 con 0x7f6b20101830 2024-06-13T07:12:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.581+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1959059712 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b201a1c10 con 0x7f6b20101830 2024-06-13T07:12:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.581+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1959059712 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b201a20b0 con 0x7f6b20101830 2024-06-13T07:12:24.583 INFO:tasks.workunit.client.0.smithi026.stdout: 1 554446 4.77 MB/sec execute 518 sec latency 21.980 ms 2024-06-13T07:12:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.585+0000 7f6b289d3740 1 Processor -- start 2024-06-13T07:12:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.585+0000 7f6b17400640 1 -- 172.21.15.26:0/1959059712 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b10042030 con 0x7f6b20101830 2024-06-13T07:12:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b289d3740 1 -- start start 2024-06-13T07:12:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b39bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b289d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55877b2f7420 con 0x55877b3947b0 2024-06-13T07:12:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b289d3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55877b29ec50 con 0x55877b393dc0 2024-06-13T07:12:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b289d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55877b2f33e0 con 0x55877b399920 2024-06-13T07:12:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b1e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b39bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b1f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b1e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b39bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51484/0 (socket says 172.21.15.26:51484) 2024-06-13T07:12:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b1f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3941e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36276/0 (socket says 172.21.15.26:36276) 2024-06-13T07:12:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b1e000640 1 -- 172.21.15.26:0/503814553 learned_addr learned my addr 172.21.15.26:0/503814553 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.586+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b1e000640 1 -- 172.21.15.26:0/503814553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 msgr2=0x55877b3941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b1e000640 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b1e000640 1 -- 172.21.15.26:0/503814553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 msgr2=0x55877b3992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b1e000640 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b1e000640 1 -- 172.21.15.26:0/503814553 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877b00b210 con 0x55877b399920 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b1f400640 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b1e000640 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b39bdb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0114f0 tx=0x7f6b0c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7049750c0fd226 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b0be00640 1 -- 172.21.15.26:0/503814553 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c01e070 con 0x55877b399920 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.587+0000 7f6b17400640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b040a2c00 0x7f6b040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.588+0000 7f6b0be00640 1 -- 172.21.15.26:0/503814553 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c00faa0 con 0x55877b399920 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.588+0000 7f6b0be00640 1 -- 172.21.15.26:0/503814553 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c019c40 con 0x55877b399920 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.588+0000 7f6b17400640 1 -- 172.21.15.26:0/1959059712 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6b100ef960 con 0x7f6b20101830 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.588+0000 7f6b1f400640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b040a2c00 0x7f6b040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.588+0000 7f6b289d3740 1 -- 172.21.15.26:0/503814553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 msgr2=0x55877b39bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.588+0000 7f6b289d3740 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b39bdb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0114f0 tx=0x7f6b0c0119b0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b289d3740 1 -- 172.21.15.26:0/503814553 shutdown_connections 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b289d3740 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b39bdb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b289d3740 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b289d3740 1 --2- 172.21.15.26:0/503814553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b289d3740 1 -- 172.21.15.26:0/503814553 >> 172.21.15.26:0/503814553 conn(0x55877b35bfb0 msgr2=0x55877b38f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b1f400640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b040a2c00 0x7f6b040a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55877b395010 tx=0x7f6b18009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b289d3740 1 -- 172.21.15.26:0/503814553 shutdown_connections 2024-06-13T07:12:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b289d3740 1 -- 172.21.15.26:0/503814553 wait complete. 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.589+0000 7f6b289d3740 1 -- start start 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b289d3740 1 Processor -- start 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b39e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b39e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b289d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55877b39c8d0 con 0x55877b3947b0 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b289d3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55877b2b20b0 con 0x55877b393dc0 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b289d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55877b39ca50 con 0x55877b399920 2024-06-13T07:12:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b289d3740 10 client.0 _ll_register_callbacks cb 0x55877b29df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:12:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.590+0000 7f6b1ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b39e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b39e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36290/0 (socket says 172.21.15.26:36290) 2024-06-13T07:12:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1ea00640 1 -- 172.21.15.26:0/1816049073 learned_addr learned my addr 172.21.15.26:0/1816049073 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1f400640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b39e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1ea00640 1 -- 172.21.15.26:0/1816049073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 msgr2=0x55877b3a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1ea00640 1 -- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 msgr2=0x55877b39e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b39e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1ea00640 1 -- 172.21.15.26:0/1816049073 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877b0d7620 con 0x55877b393dc0 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b39e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1f400640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b39e120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b100ed070 tx=0x7f6b100b8220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2055c0e53ab1d8d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.591+0000 7f6b08c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b08c00640 1 -- 172.21.15.26:0/1816049073 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10049070 con 0x55877b393dc0 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b08c00640 1 -- 172.21.15.26:0/1816049073 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6b1003b460 con 0x55877b393dc0 2024-06-13T07:12:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 1 -- 172.21.15.26:0/1816049073 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b0c00faa0 con 0x55877b393dc0 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b08c00640 1 -- 172.21.15.26:0/1816049073 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6b1003be40 con 0x55877b393dc0 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 10 client.26402 fetch_fsmap learned FSMap version 36 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 1 -- 172.21.15.26:0/1816049073 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55877b3a8a10 con 0x55877b393dc0 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b08c00640 1 -- 172.21.15.26:0/1816049073 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c00faa0 con 0x55877b393dc0 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b08c00640 1 -- 172.21.15.26:0/1816049073 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f6b100d1070 con 0x55877b393dc0 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 10 client.26402 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 10 client.26402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 20 client.26402 populate_metadata read hostname 'smithi026' 2024-06-13T07:12:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877b3aa1f0 0x55877b3ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 4 client.26402 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:12:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.592+0000 7f6b289d3740 1 -- 172.21.15.26:0/1816049073 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55877b304b50 con 0x55877b3aa1f0 2024-06-13T07:12:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.593+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877b3aa1f0 0x55877b3ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.593+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877b3aa1f0 0x55877b3ac6b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08d210480b37a5 server_cookie=2098c8ae9adbf2ad in_seq=0 out_seq=0 2024-06-13T07:12:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.593+0000 7f6b08c00640 10 client.26402 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.594+0000 7f6b08c00640 1 -- 172.21.15.26:0/1816049073 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55877b304b50 con 0x55877b3aa1f0 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.594+0000 7f6b08c00640 10 client.26402 handle_command_reply: tid=0 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.594+0000 7f6b289d3740 1 client.26402 shutdown 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 -- 172.21.15.26:0/1816049073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 msgr2=0x55877b39e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b39e120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b100ed070 tx=0x7f6b100b8220 comp rx=0 tx=0).stop 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 -- 172.21.15.26:0/1816049073 shutdown_connections 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877b3aa1f0 0x55877b3ac6b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b39e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b08c00640 0 client.26402 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1816049073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b39e120 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 -- 172.21.15.26:0/1816049073 >> 172.21.15.26:0/1816049073 conn(0x55877b35bfb0 msgr2=0x55877b38e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 -- 172.21.15.26:0/1816049073 shutdown_connections 2024-06-13T07:12:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 1 -- 172.21.15.26:0/1816049073 wait complete. 2024-06-13T07:12:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.595+0000 7f6b289d3740 20 client.26402 trim_cache size 0 max 16384 2024-06-13T07:12:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.617+0000 7f6b289d3740 1 Processor -- start 2024-06-13T07:12:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.617+0000 7f6b289d3740 1 -- start start 2024-06-13T07:12:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.617+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b393dc0 0x55877b392e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.617+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b3947b0 0x55877b3a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.617+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b399920 0x55877b39d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.617+0000 7f6b289d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201af010 con 0x55877b399920 2024-06-13T07:12:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.617+0000 7f6b289d3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ac8e0 con 0x55877b3947b0 2024-06-13T07:12:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.617+0000 7f6b289d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201aa730 con 0x55877b393dc0 2024-06-13T07:12:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b3947b0 0x55877b3a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b399920 0x55877b39d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b393dc0 0x55877b392e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b3947b0 0x55877b3a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36292/0 (socket says 172.21.15.26:36292) 2024-06-13T07:12:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 -- 172.21.15.26:0/1319624135 learned_addr learned my addr 172.21.15.26:0/1319624135 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 -- 172.21.15.26:0/1319624135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b393dc0 msgr2=0x55877b392e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1319624135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b393dc0 0x55877b392e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 -- 172.21.15.26:0/1319624135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b399920 msgr2=0x55877b39d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1319624135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b399920 0x55877b39d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 -- 172.21.15.26:0/1319624135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877b38b000 con 0x55877b3947b0 2024-06-13T07:12:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.618+0000 7f6b1e000640 1 --2- 172.21.15.26:0/1319624135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b3947b0 0x55877b3a5660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55877b3906f0 tx=0x7f6b0c011240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f03ca8787bc9531 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.619+0000 7f6b0b400640 1 -- 172.21.15.26:0/1319624135 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c01e040 con 0x55877b3947b0 2024-06-13T07:12:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.619+0000 7f6b0b400640 1 -- 172.21.15.26:0/1319624135 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c029070 con 0x55877b3947b0 2024-06-13T07:12:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.619+0000 7f6b0b400640 1 -- 172.21.15.26:0/1319624135 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c005320 con 0x55877b3947b0 2024-06-13T07:12:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 -- 172.21.15.26:0/1319624135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b3947b0 msgr2=0x55877b3a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1319624135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b3947b0 0x55877b3a5660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55877b3906f0 tx=0x7f6b0c011240 comp rx=0 tx=0).stop 2024-06-13T07:12:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 -- 172.21.15.26:0/1319624135 shutdown_connections 2024-06-13T07:12:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1319624135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b3947b0 0x55877b3a5660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1319624135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b399920 0x55877b39d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 --2- 172.21.15.26:0/1319624135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b393dc0 0x55877b392e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 -- 172.21.15.26:0/1319624135 >> 172.21.15.26:0/1319624135 conn(0x55877b35bfb0 msgr2=0x55877b392730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 -- 172.21.15.26:0/1319624135 shutdown_connections 2024-06-13T07:12:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 -- 172.21.15.26:0/1319624135 wait complete. 2024-06-13T07:12:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.620+0000 7f6b289d3740 1 -- start start 2024-06-13T07:12:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.621+0000 7f6b289d3740 1 Processor -- start 2024-06-13T07:12:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.621+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.621+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.621+0000 7f6b289d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.621+0000 7f6b289d3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201aada0 con 0x55877b3947b0 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.621+0000 7f6b289d3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ac210 con 0x55877b393dc0 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.621+0000 7f6b289d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ab470 con 0x55877b399920 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.621+0000 7f6b289d3740 10 client.0 _ll_register_callbacks cb 0x7f6b10031ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51516/0 (socket says 172.21.15.26:51516) 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 -- 172.21.15.26:0/4182034798 learned_addr learned my addr 172.21.15.26:0/4182034798 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 -- 172.21.15.26:0/4182034798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 msgr2=0x55877b3a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 -- 172.21.15.26:0/4182034798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 msgr2=0x55877b3a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3a2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 -- 172.21.15.26:0/4182034798 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877b3a4590 con 0x55877b399920 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1e000640 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b1f400640 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a3fa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b18003f20 tx=0x7f6b18004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc0584cdaccc5bb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.622+0000 7f6b08c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.623+0000 7f6b08c00640 1 -- 172.21.15.26:0/4182034798 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b180045c0 con 0x55877b399920 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.623+0000 7f6b08c00640 1 -- 172.21.15.26:0/4182034798 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6b18004760 con 0x55877b399920 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.623+0000 7f6b289d3740 1 -- 172.21.15.26:0/4182034798 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b0c029070 con 0x55877b399920 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.623+0000 7f6b08c00640 1 -- 172.21.15.26:0/4182034798 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6b18034050 con 0x55877b399920 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.623+0000 7f6b289d3740 10 client.26440 fetch_fsmap learned FSMap version 36 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.623+0000 7f6b289d3740 1 -- 172.21.15.26:0/4182034798 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55877b3a5910 con 0x55877b399920 2024-06-13T07:12:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.623+0000 7f6b08c00640 1 -- 172.21.15.26:0/4182034798 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c029070 con 0x55877b399920 2024-06-13T07:12:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.623+0000 7f6b08c00640 1 -- 172.21.15.26:0/4182034798 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f6b18005770 con 0x55877b399920 2024-06-13T07:12:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.624+0000 7f6b289d3740 10 client.26440 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:12:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.624+0000 7f6b289d3740 10 client.26440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:12:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.624+0000 7f6b289d3740 20 client.26440 populate_metadata read hostname 'smithi026' 2024-06-13T07:12:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.624+0000 7f6b289d3740 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877b4157d0 0x55877b417bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.624+0000 7f6b289d3740 4 client.26440 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:12:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.624+0000 7f6b289d3740 1 -- 172.21.15.26:0/4182034798 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55877b2feea0 con 0x55877b4157d0 2024-06-13T07:12:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.624+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877b4157d0 0x55877b417bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:24 smithi136 ceph-mon[37229]: pgmap v866: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-13T07:12:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.625+0000 7f6b1ea00640 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877b4157d0 0x55877b417bf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8834a35c509a5c server_cookie=266de3b9d74c6ad6 in_seq=0 out_seq=0 2024-06-13T07:12:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.625+0000 7f6b08c00640 10 client.26440 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b08c00640 1 -- 172.21.15.26:0/4182034798 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55877b2feea0 con 0x55877b4157d0 2024-06-13T07:12:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b08c00640 10 client.26440 handle_command_reply: tid=0 2024-06-13T07:12:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 client.26440 shutdown 2024-06-13T07:12:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 -- 172.21.15.26:0/4182034798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 msgr2=0x55877b3a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a3fa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b18003f20 tx=0x7f6b18004a40 comp rx=0 tx=0).stop 2024-06-13T07:12:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 -- 172.21.15.26:0/4182034798 shutdown_connections 2024-06-13T07:12:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55877b4157d0 0x55877b417bf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55877b399920 0x55877b3a3fa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877b3947b0 0x55877b3a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 --2- 172.21.15.26:0/4182034798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55877b393dc0 0x55877b3a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b08c00640 0 client.26440 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 -- 172.21.15.26:0/4182034798 >> 172.21.15.26:0/4182034798 conn(0x55877b35bfb0 msgr2=0x55877b38e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 -- 172.21.15.26:0/4182034798 shutdown_connections 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.708+0000 7f6b289d3740 1 -- 172.21.15.26:0/4182034798 wait complete. 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.709+0000 7f6b289d3740 20 client.26440 trim_cache size 0 max 16384 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:12:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:12:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:12:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.709+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b040a2c00 msgr2=0x7f6b040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.709+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b040a2c00 0x7f6b040a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55877b395010 tx=0x7f6b18009500 comp rx=0 tx=0).stop 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.709+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1959059712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b20101830 msgr2=0x7f6b20199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.709+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b20101830 0x7f6b20199d60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6b10031610 tx=0x7f6b10004270 comp rx=0 tx=0).stop 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.710+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1959059712 shutdown_connections 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.710+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b040a2c00 0x7f6b040a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.710+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b20102a20 0x7f6b2019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.710+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b201033e0 0x7f6b201a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.710+0000 7f6b1fe00640 1 --2- 172.21.15.26:0/1959059712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b20101830 0x7f6b20199d60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.710+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1959059712 >> 172.21.15.26:0/1959059712 conn(0x7f6b200fcda0 msgr2=0x7f6b20073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.710+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1959059712 shutdown_connections 2024-06-13T07:12:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:24.710+0000 7f6b1fe00640 1 -- 172.21.15.26:0/1959059712 wait complete. 2024-06-13T07:12:25.583 INFO:tasks.workunit.client.0.smithi026.stdout: 1 554987 4.76 MB/sec execute 519 sec latency 50.767 ms 2024-06-13T07:12:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:26 smithi026 ceph-mon[32041]: pgmap v867: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-13T07:12:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:26 smithi135 ceph-mon[36276]: pgmap v867: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-13T07:12:26.583 INFO:tasks.workunit.client.0.smithi026.stdout: 1 555953 4.76 MB/sec execute 520 sec latency 45.749 ms 2024-06-13T07:12:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:26 smithi136 ceph-mon[37229]: pgmap v867: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-13T07:12:27.583 INFO:tasks.workunit.client.0.smithi026.stdout: 1 557328 4.76 MB/sec execute 521 sec latency 31.801 ms 2024-06-13T07:12:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:28 smithi026 ceph-mon[32041]: pgmap v868: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-13T07:12:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:28 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:12:28.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:28 smithi135 ceph-mon[36276]: pgmap v868: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-13T07:12:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:28 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:12:28.583 INFO:tasks.workunit.client.0.smithi026.stdout: 1 558088 4.77 MB/sec execute 522 sec latency 27.791 ms 2024-06-13T07:12:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:28 smithi136 ceph-mon[37229]: pgmap v868: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-13T07:12:28.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:28 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:12:29.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:12:29.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:12:29.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:29 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:12:29.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:12:29.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:12:29.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:29 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:12:29.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 558777 4.76 MB/sec execute 523 sec latency 19.402 ms 2024-06-13T07:12:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:12:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:12:29.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:29 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:12:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:30 smithi026 ceph-mon[32041]: pgmap v869: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-13T07:12:30.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:30 smithi135 ceph-mon[36276]: pgmap v869: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-13T07:12:30.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 560140 4.76 MB/sec execute 524 sec latency 26.223 ms 2024-06-13T07:12:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:30 smithi136 ceph-mon[37229]: pgmap v869: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-13T07:12:31.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 561201 4.77 MB/sec execute 525 sec latency 20.565 ms 2024-06-13T07:12:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:32 smithi026 ceph-mon[32041]: pgmap v870: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:12:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:32 smithi135 ceph-mon[36276]: pgmap v870: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:12:32.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 561720 4.77 MB/sec execute 526 sec latency 36.802 ms 2024-06-13T07:12:32.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:32 smithi136 ceph-mon[37229]: pgmap v870: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-13T07:12:33.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 562528 4.76 MB/sec execute 527 sec latency 33.820 ms 2024-06-13T07:12:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:34 smithi135 ceph-mon[36276]: pgmap v871: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-13T07:12:34.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 563696 4.76 MB/sec execute 528 sec latency 45.415 ms 2024-06-13T07:12:34.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:34 smithi136 ceph-mon[37229]: pgmap v871: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-13T07:12:34.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:34 smithi026 ceph-mon[32041]: pgmap v871: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-13T07:12:35.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 564761 4.77 MB/sec execute 529 sec latency 19.623 ms 2024-06-13T07:12:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:36 smithi135 ceph-mon[36276]: pgmap v872: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:12:36.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 565265 4.77 MB/sec execute 530 sec latency 29.594 ms 2024-06-13T07:12:36.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:36 smithi136 ceph-mon[37229]: pgmap v872: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:12:36.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:36 smithi026 ceph-mon[32041]: pgmap v872: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.2 MiB/s wr, 177 op/s 2024-06-13T07:12:37.584 INFO:tasks.workunit.client.0.smithi026.stdout: 1 566282 4.77 MB/sec execute 531 sec latency 34.036 ms 2024-06-13T07:12:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.872+0000 7fb7da600640 1 -- 172.21.15.26:0/3122887467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4107e50 msgr2=0x7fb7d4108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.872+0000 7fb7da600640 1 --2- 172.21.15.26:0/3122887467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4107e50 0x7fb7d4108270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009900 tx=0x7fb7c802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.872+0000 7fb7da600640 1 -- 172.21.15.26:0/3122887467 shutdown_connections 2024-06-13T07:12:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.872+0000 7fb7da600640 1 --2- 172.21.15.26:0/3122887467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d4108fa0 0x7fb7d4109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.872+0000 7fb7da600640 1 --2- 172.21.15.26:0/3122887467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7d4109960 0x7fb7d4110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.872+0000 7fb7da600640 1 --2- 172.21.15.26:0/3122887467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4107e50 0x7fb7d4108270 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.872+0000 7fb7da600640 1 -- 172.21.15.26:0/3122887467 >> 172.21.15.26:0/3122887467 conn(0x7fb7d4076e30 msgr2=0x7fb7d4079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.872+0000 7fb7da600640 1 -- 172.21.15.26:0/3122887467 shutdown_connections 2024-06-13T07:12:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.873+0000 7fb7da600640 1 -- 172.21.15.26:0/3122887467 wait complete. 2024-06-13T07:12:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7da600640 1 Processor -- start 2024-06-13T07:12:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7da600640 1 -- start start 2024-06-13T07:12:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7da600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7d4107e50 0x7fb7d419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7da600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d4108fa0 0x7fb7d419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7da600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4109960 0x7fb7d41a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7da600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113ad0 con 0x7fb7d4107e50 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7da600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113950 con 0x7fb7d4108fa0 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7da600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113c50 con 0x7fb7d4109960 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7d9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4109960 0x7fb7d41a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7d9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7d4107e50 0x7fb7d419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d4108fa0 0x7fb7d419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7d9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4109960 0x7fb7d41a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54778/0 (socket says 172.21.15.26:54778) 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.874+0000 7fb7d9c00640 1 -- 172.21.15.26:0/365406273 learned_addr learned my addr 172.21.15.26:0/365406273 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.875+0000 7fb7d9c00640 1 -- 172.21.15.26:0/365406273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d4108fa0 msgr2=0x7fb7d419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.875+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d4108fa0 0x7fb7d419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.875+0000 7fb7d9c00640 1 -- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7d4107e50 msgr2=0x7fb7d419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.875+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7d4107e50 0x7fb7d419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.875+0000 7fb7d9c00640 1 -- 172.21.15.26:0/365406273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d41a5cd0 con 0x7fb7d4109960 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.875+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4109960 0x7fb7d41a5660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00cce0 tx=0x7fb7cc00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.876+0000 7fb7d1600640 1 -- 172.21.15.26:0/365406273 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00f450 con 0x7fb7d4109960 2024-06-13T07:12:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.876+0000 7fb7d1600640 1 -- 172.21.15.26:0/365406273 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc00fad0 con 0x7fb7d4109960 2024-06-13T07:12:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.876+0000 7fb7d1600640 1 -- 172.21.15.26:0/365406273 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc018c10 con 0x7fb7d4109960 2024-06-13T07:12:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.876+0000 7fb7da600640 1 -- 172.21.15.26:0/365406273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d41a5fc0 con 0x7fb7d4109960 2024-06-13T07:12:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.876+0000 7fb7da600640 1 -- 172.21.15.26:0/365406273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41a6480 con 0x7fb7d4109960 2024-06-13T07:12:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.879+0000 7fb7d1600640 1 -- 172.21.15.26:0/365406273 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb7cc00f5f0 con 0x7fb7d4109960 2024-06-13T07:12:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.880+0000 7fb7dea03740 1 Processor -- start 2024-06-13T07:12:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.880+0000 7fb7dea03740 1 -- start start 2024-06-13T07:12:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.880+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.880+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.880+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 0x5569ef5d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.880+0000 7fb7dea03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5569ef52d420 con 0x5569ef5c9dc0 2024-06-13T07:12:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.880+0000 7fb7dea03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5569ef4d4c50 con 0x5569ef5ca7b0 2024-06-13T07:12:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.880+0000 7fb7dea03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5569ef5293e0 con 0x5569ef5cf920 2024-06-13T07:12:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.881+0000 7fb7d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.881+0000 7fb7d9200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.881+0000 7fb7d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40538/0 (socket says 172.21.15.26:40538) 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.881+0000 7fb7d3e00640 1 -- 172.21.15.26:0/1953927653 learned_addr learned my addr 172.21.15.26:0/1953927653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.881+0000 7fb7d3e00640 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 0x5569ef5d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d3e00640 1 -- 172.21.15.26:0/1953927653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 msgr2=0x5569ef5d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d3e00640 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 0x5569ef5d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d3e00640 1 -- 172.21.15.26:0/1953927653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 msgr2=0x5569ef5cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d3e00640 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d3e00640 1 -- 172.21.15.26:0/1953927653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ef241210 con 0x5569ef5c9dc0 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d9200640 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d1600640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7bc0a2c00 0x7fb7bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d3e00640 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5ca1e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000be60 tx=0x7fb7c0004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd3b9fcad63ad23 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.882+0000 7fb7d1600640 1 -- 172.21.15.26:0/365406273 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb7cc0cf990 con 0x7fb7d4109960 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.883+0000 7fb7d9200640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7bc0a2c00 0x7fb7bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.883+0000 7fb7c6000640 1 -- 172.21.15.26:0/1953927653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0012450 con 0x5569ef5c9dc0 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.883+0000 7fb7c6000640 1 -- 172.21.15.26:0/1953927653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c0012ad0 con 0x5569ef5c9dc0 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.883+0000 7fb7c6000640 1 -- 172.21.15.26:0/1953927653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0010920 con 0x5569ef5c9dc0 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 -- 172.21.15.26:0/1953927653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 msgr2=0x5569ef5ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5ca1e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000be60 tx=0x7fb7c0004120 comp rx=0 tx=0).stop 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7d9200640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7bc0a2c00 0x7fb7bc0a50c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80314e0 tx=0x7fb7c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 -- 172.21.15.26:0/1953927653 shutdown_connections 2024-06-13T07:12:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 0x5569ef5d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 --2- 172.21.15.26:0/1953927653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5ca1e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 -- 172.21.15.26:0/1953927653 >> 172.21.15.26:0/1953927653 conn(0x5569ef591fb0 msgr2=0x5569ef5c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 -- 172.21.15.26:0/1953927653 shutdown_connections 2024-06-13T07:12:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.884+0000 7fb7dea03740 1 -- 172.21.15.26:0/1953927653 wait complete. 2024-06-13T07:12:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 1 -- start start 2024-06-13T07:12:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 1 Processor -- start 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5d4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 0x5569ef5db640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5569ef5d2870 con 0x5569ef5c9dc0 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5569ef4e80b0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5569ef5d29f0 con 0x5569ef5cf920 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.885+0000 7fb7dea03740 10 client.0 _ll_register_callbacks cb 0x5569ef4d3f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5d4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5d4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54112/0 (socket says 172.21.15.26:54112) 2024-06-13T07:12:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 -- 172.21.15.26:0/3224689003 learned_addr learned my addr 172.21.15.26:0/3224689003 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 0x5569ef5db640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 -- 172.21.15.26:0/3224689003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 msgr2=0x5569ef5db640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 0x5569ef5db640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 -- 172.21.15.26:0/3224689003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 msgr2=0x5569ef5d40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5d40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 -- 172.21.15.26:0/3224689003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ef30d620 con 0x5569ef5ca7b0 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7d3e00640 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5d4600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c010 tx=0x7fb7c000b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5635299c9faa7bc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.886+0000 7fb7baa00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.887+0000 7fb7baa00640 1 -- 172.21.15.26:0/3224689003 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0026020 con 0x5569ef5ca7b0 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.887+0000 7fb7baa00640 1 -- 172.21.15.26:0/3224689003 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb7c0005240 con 0x5569ef5ca7b0 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.887+0000 7fb7dea03740 1 -- 172.21.15.26:0/3224689003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c0012ad0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.887+0000 7fb7baa00640 1 -- 172.21.15.26:0/3224689003 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb7c0031050 con 0x5569ef5ca7b0 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.887+0000 7fb7dea03740 10 client.26420 fetch_fsmap learned FSMap version 36 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.887+0000 7fb7dea03740 1 -- 172.21.15.26:0/3224689003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5569ef5de9b0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.887+0000 7fb7baa00640 1 -- 172.21.15.26:0/3224689003 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c0012ad0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.888+0000 7fb7baa00640 1 -- 172.21.15.26:0/3224689003 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c001f460 con 0x5569ef5ca7b0 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.888+0000 7fb7dea03740 10 client.26420 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.888+0000 7fb7dea03740 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.888+0000 7fb7dea03740 20 client.26420 populate_metadata read hostname 'smithi026' 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.888+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5569ef5dfec0 0x5569ef5e2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.888+0000 7fb7dea03740 4 client.26420 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.888+0000 7fb7dea03740 1 -- 172.21.15.26:0/3224689003 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569ef53ab50 con 0x5569ef5dfec0 2024-06-13T07:12:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.888+0000 7fb7d9200640 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5569ef5dfec0 0x5569ef5e2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.889+0000 7fb7d9200640 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5569ef5dfec0 0x5569ef5e2300 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce87cdc5b190efed server_cookie=bf8266dd1657e629 in_seq=0 out_seq=0 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.889+0000 7fb7baa00640 10 client.26420 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7baa00640 1 -- 172.21.15.26:0/3224689003 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569ef53ab50 con 0x5569ef5dfec0 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7baa00640 10 client.26420 handle_command_reply: tid=0 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 client.26420 shutdown 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 -- 172.21.15.26:0/3224689003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 msgr2=0x5569ef5d4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5d4600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c010 tx=0x7fb7c000b6a0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 -- 172.21.15.26:0/3224689003 shutdown_connections 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5569ef5dfec0 0x5569ef5e2300 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5d4600 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef5cf920 0x5569ef5db640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3224689003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5d40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7baa00640 0 client.26420 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 -- 172.21.15.26:0/3224689003 >> 172.21.15.26:0/3224689003 conn(0x5569ef591fb0 msgr2=0x5569ef5c49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 -- 172.21.15.26:0/3224689003 shutdown_connections 2024-06-13T07:12:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 1 -- 172.21.15.26:0/3224689003 wait complete. 2024-06-13T07:12:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.890+0000 7fb7dea03740 20 client.26420 trim_cache size 0 max 16384 2024-06-13T07:12:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.913+0000 7fb7dea03740 1 Processor -- start 2024-06-13T07:12:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.913+0000 7fb7dea03740 1 -- start start 2024-06-13T07:12:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.914+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.914+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.914+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.914+0000 7fb7dea03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41b33b0 con 0x5569ef5c9dc0 2024-06-13T07:12:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.914+0000 7fb7dea03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41b0c80 con 0x5569ef5ca7b0 2024-06-13T07:12:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.914+0000 7fb7dea03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41aead0 con 0x5569ef591fb0 2024-06-13T07:12:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.915+0000 7fb7d9c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.915+0000 7fb7d9c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54116/0 (socket says 172.21.15.26:54116) 2024-06-13T07:12:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.915+0000 7fb7d9c00640 1 -- 172.21.15.26:0/3247255038 learned_addr learned my addr 172.21.15.26:0/3247255038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.915+0000 7fb7d9200640 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.915+0000 7fb7d3e00640 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.916+0000 7fb7d9c00640 1 -- 172.21.15.26:0/3247255038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 msgr2=0x5569ef5c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.916+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.916+0000 7fb7d9c00640 1 -- 172.21.15.26:0/3247255038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 msgr2=0x5569ef5db600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.916+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5db600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.916+0000 7fb7d9c00640 1 -- 172.21.15.26:0/3247255038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ef5bfdb0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.916+0000 7fb7d9200640 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:12:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.916+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5c8780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00ccb0 tx=0x7fb7cc09b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2e0c4f539e082e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.916+0000 7fb7c5600640 1 -- 172.21.15.26:0/3247255038 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0b9070 con 0x5569ef5ca7b0 2024-06-13T07:12:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.917+0000 7fb7c5600640 1 -- 172.21.15.26:0/3247255038 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc0a0150 con 0x5569ef5ca7b0 2024-06-13T07:12:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.917+0000 7fb7c5600640 1 -- 172.21.15.26:0/3247255038 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc09f070 con 0x5569ef5ca7b0 2024-06-13T07:12:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.918+0000 7fb7dea03740 1 -- 172.21.15.26:0/3247255038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 msgr2=0x5569ef5c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.918+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5c8780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00ccb0 tx=0x7fb7cc09b700 comp rx=0 tx=0).stop 2024-06-13T07:12:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.919+0000 7fb7dea03740 1 -- 172.21.15.26:0/3247255038 shutdown_connections 2024-06-13T07:12:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.919+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5c8780 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.919+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.919+0000 7fb7dea03740 1 --2- 172.21.15.26:0/3247255038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.919+0000 7fb7dea03740 1 -- 172.21.15.26:0/3247255038 >> 172.21.15.26:0/3247255038 conn(0x5569ef5d3c80 msgr2=0x5569ef5c52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.919+0000 7fb7dea03740 1 -- 172.21.15.26:0/3247255038 shutdown_connections 2024-06-13T07:12:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.919+0000 7fb7dea03740 1 -- 172.21.15.26:0/3247255038 wait complete. 2024-06-13T07:12:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.920+0000 7fb7dea03740 1 -- start start 2024-06-13T07:12:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.920+0000 7fb7dea03740 1 Processor -- start 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7dea03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5da160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7dea03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41af140 con 0x5569ef5c9dc0 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7dea03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41b05b0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7dea03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41af810 con 0x5569ef591fb0 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7dea03740 10 client.0 _ll_register_callbacks cb 0x7fb7c8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7d9c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5da160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7d9c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5da160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54122/0 (socket says 172.21.15.26:54122) 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7d9c00640 1 -- 172.21.15.26:0/2561072444 learned_addr learned my addr 172.21.15.26:0/2561072444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7d9200640 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.921+0000 7fb7d9c00640 1 -- 172.21.15.26:0/2561072444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 msgr2=0x5569ef5d8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5d8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7d9c00640 1 -- 172.21.15.26:0/2561072444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 msgr2=0x5569ef5d9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5d9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7d9c00640 1 -- 172.21.15.26:0/2561072444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ef5da750 con 0x5569ef5ca7b0 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7d9200640 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7d9c00640 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5da160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0186f0 tx=0x7fb7cc00f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03c56445199c5dc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7baa00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7baa00640 1 -- 172.21.15.26:0/2561072444 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0b9070 con 0x5569ef5ca7b0 2024-06-13T07:12:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7dea03740 1 -- 172.21.15.26:0/2561072444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7cc0a0150 con 0x5569ef5ca7b0 2024-06-13T07:12:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7baa00640 1 -- 172.21.15.26:0/2561072444 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb7cc09b1f0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7baa00640 1 -- 172.21.15.26:0/2561072444 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb7cc09f070 con 0x5569ef5ca7b0 2024-06-13T07:12:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7dea03740 10 client.26426 fetch_fsmap learned FSMap version 36 2024-06-13T07:12:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.922+0000 7fb7dea03740 1 -- 172.21.15.26:0/2561072444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5569ef64aba0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.923+0000 7fb7baa00640 1 -- 172.21.15.26:0/2561072444 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc0a0150 con 0x5569ef5ca7b0 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.923+0000 7fb7baa00640 1 -- 172.21.15.26:0/2561072444 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fb7cc0a12e0 con 0x5569ef5ca7b0 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.923+0000 7fb7dea03740 10 client.26426 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.923+0000 7fb7dea03740 10 client.26426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.923+0000 7fb7dea03740 20 client.26426 populate_metadata read hostname 'smithi026' 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.923+0000 7fb7dea03740 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5569ef5cf920 0x5569ef64d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.923+0000 7fb7dea03740 4 client.26426 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.923+0000 7fb7dea03740 1 -- 172.21.15.26:0/2561072444 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569ef534ea0 con 0x5569ef5cf920 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.924+0000 7fb7d9200640 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5569ef5cf920 0x5569ef64d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.925+0000 7fb7baa00640 10 client.26426 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:37.925+0000 7fb7d9200640 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5569ef5cf920 0x5569ef64d4d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f654f624ae87e482 server_cookie=1e8fd359eeba8704 in_seq=0 out_seq=0 2024-06-13T07:12:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7baa00640 1 -- 172.21.15.26:0/2561072444 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5569ef534ea0 con 0x5569ef5cf920 2024-06-13T07:12:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7baa00640 10 client.26426 handle_command_reply: tid=0 2024-06-13T07:12:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 client.26426 shutdown 2024-06-13T07:12:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 -- 172.21.15.26:0/2561072444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 msgr2=0x5569ef5da160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5da160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0186f0 tx=0x7fb7cc00f960 comp rx=0 tx=0).stop 2024-06-13T07:12:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 -- 172.21.15.26:0/2561072444 shutdown_connections 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5569ef5cf920 0x5569ef64d4d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5569ef5ca7b0 0x5569ef5da160 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5569ef5c9dc0 0x5569ef5d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 --2- 172.21.15.26:0/2561072444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5569ef591fb0 0x5569ef5d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7dea03740 1 -- 172.21.15.26:0/2561072444 >> 172.21.15.26:0/2561072444 conn(0x5569ef5d3c80 msgr2=0x5569ef5c4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.005+0000 7fb7baa00640 0 client.26426 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.006+0000 7fb7dea03740 1 -- 172.21.15.26:0/2561072444 shutdown_connections 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.006+0000 7fb7dea03740 1 -- 172.21.15.26:0/2561072444 wait complete. 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.006+0000 7fb7dea03740 20 client.26426 trim_cache size 0 max 16384 2024-06-13T07:12:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:12:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:12:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:12:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 -- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7bc0a2c00 msgr2=0x7fb7bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7bc0a2c00 0x7fb7bc0a50c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80314e0 tx=0x7fb7c8000bc0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 -- 172.21.15.26:0/365406273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4109960 msgr2=0x7fb7d41a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4109960 0x7fb7d41a5660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00cce0 tx=0x7fb7cc00a620 comp rx=0 tx=0).stop 2024-06-13T07:12:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 -- 172.21.15.26:0/365406273 shutdown_connections 2024-06-13T07:12:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb7bc0a2c00 0x7fb7bc0a50c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d4108fa0 0x7fb7d419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7d4109960 0x7fb7d41a5660 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 --2- 172.21.15.26:0/365406273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7d4107e50 0x7fb7d419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 -- 172.21.15.26:0/365406273 >> 172.21.15.26:0/365406273 conn(0x7fb7d4076e30 msgr2=0x7fb7d410c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 -- 172.21.15.26:0/365406273 shutdown_connections 2024-06-13T07:12:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:38.007+0000 7fb7da600640 1 -- 172.21.15.26:0/365406273 wait complete. 2024-06-13T07:12:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:38 smithi135 ceph-mon[36276]: pgmap v873: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-06-13T07:12:38.585 INFO:tasks.workunit.client.0.smithi026.stdout: 1 567674 4.77 MB/sec execute 532 sec latency 40.736 ms 2024-06-13T07:12:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:38 smithi136 ceph-mon[37229]: pgmap v873: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-06-13T07:12:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:38 smithi026 ceph-mon[32041]: pgmap v873: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-06-13T07:12:39.585 INFO:tasks.workunit.client.0.smithi026.stdout: 1 568565 4.78 MB/sec execute 533 sec latency 27.654 ms 2024-06-13T07:12:40.585 INFO:tasks.workunit.client.0.smithi026.stdout: 1 568947 4.77 MB/sec execute 534 sec latency 30.779 ms 2024-06-13T07:12:40.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:40 smithi136 ceph-mon[37229]: pgmap v874: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-06-13T07:12:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:40 smithi026 ceph-mon[32041]: pgmap v874: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-06-13T07:12:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:40 smithi135 ceph-mon[36276]: pgmap v874: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-06-13T07:12:41.585 INFO:tasks.workunit.client.0.smithi026.stdout: 1 569853 4.77 MB/sec execute 535 sec latency 19.916 ms 2024-06-13T07:12:42.585 INFO:tasks.workunit.client.0.smithi026.stdout: 1 571309 4.77 MB/sec execute 536 sec latency 59.042 ms 2024-06-13T07:12:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:42 smithi136 ceph-mon[37229]: pgmap v875: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:12:42.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:42 smithi026 ceph-mon[32041]: pgmap v875: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:12:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:42 smithi135 ceph-mon[36276]: pgmap v875: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-13T07:12:43.585 INFO:tasks.workunit.client.0.smithi026.stdout: 1 572164 4.78 MB/sec execute 537 sec latency 19.907 ms 2024-06-13T07:12:43.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:43 smithi136 ceph-mon[37229]: pgmap v876: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-13T07:12:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:43 smithi026 ceph-mon[32041]: pgmap v876: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-13T07:12:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:43 smithi135 ceph-mon[36276]: pgmap v876: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-13T07:12:44.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 572668 4.77 MB/sec execute 538 sec latency 32.134 ms 2024-06-13T07:12:45.585 INFO:tasks.workunit.client.0.smithi026.stdout: 1 573579 4.77 MB/sec execute 539 sec latency 22.156 ms 2024-06-13T07:12:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:45 smithi026 ceph-mon[32041]: pgmap v877: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:12:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:45 smithi135 ceph-mon[36276]: pgmap v877: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:12:46.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:45 smithi136 ceph-mon[37229]: pgmap v877: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-13T07:12:46.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 574770 4.77 MB/sec execute 540 sec latency 21.537 ms 2024-06-13T07:12:47.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 575719 4.78 MB/sec execute 541 sec latency 33.534 ms 2024-06-13T07:12:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:47 smithi026 ceph-mon[32041]: pgmap v878: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:12:48.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:47 smithi135 ceph-mon[36276]: pgmap v878: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:12:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:47 smithi136 ceph-mon[37229]: pgmap v878: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-13T07:12:48.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 576241 4.77 MB/sec execute 542 sec latency 28.784 ms 2024-06-13T07:12:49.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 577260 4.77 MB/sec execute 543 sec latency 33.718 ms 2024-06-13T07:12:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:49 smithi026 ceph-mon[32041]: pgmap v879: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-06-13T07:12:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:49 smithi135 ceph-mon[36276]: pgmap v879: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-06-13T07:12:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:49 smithi136 ceph-mon[37229]: pgmap v879: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-06-13T07:12:50.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 578410 4.77 MB/sec execute 544 sec latency 22.239 ms 2024-06-13T07:12:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 -- 172.21.15.26:0/21282828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 msgr2=0x7efedc073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 --2- 172.21.15.26:0/21282828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 0x7efedc073e30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efed0009900 tx=0x7efed002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 -- 172.21.15.26:0/21282828 shutdown_connections 2024-06-13T07:12:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 --2- 172.21.15.26:0/21282828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc10de10 0x7efedc110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 --2- 172.21.15.26:0/21282828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efedc074400 0x7efedc074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 --2- 172.21.15.26:0/21282828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 0x7efedc073e30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 -- 172.21.15.26:0/21282828 >> 172.21.15.26:0/21282828 conn(0x7efedc0fcdb0 msgr2=0x7efedc0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 -- 172.21.15.26:0/21282828 shutdown_connections 2024-06-13T07:12:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.158+0000 7efee2000640 1 -- 172.21.15.26:0/21282828 wait complete. 2024-06-13T07:12:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.160+0000 7efee2000640 1 Processor -- start 2024-06-13T07:12:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.160+0000 7efee2000640 1 -- start start 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.160+0000 7efee2000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc074400 0x7efedc19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.160+0000 7efee2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 0x7efedc19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efee2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efedc10de10 0x7efedc1a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efee2000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113b00 con 0x7efedc10de10 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efee2000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113980 con 0x7efedc074400 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efee2000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113c80 con 0x7efedc0759e0 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efedbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 0x7efedc19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efedbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 0x7efedc19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37132/0 (socket says 172.21.15.26:37132) 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efee0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc074400 0x7efedc19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efedbe00640 1 -- 172.21.15.26:0/660809879 learned_addr learned my addr 172.21.15.26:0/660809879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efee1600640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efedc10de10 0x7efedc1a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efedbe00640 1 -- 172.21.15.26:0/660809879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc074400 msgr2=0x7efedc19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efedbe00640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc074400 0x7efedc19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efedbe00640 1 -- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efedc10de10 msgr2=0x7efedc1a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efedbe00640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efedc10de10 0x7efedc1a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efedbe00640 1 -- 172.21.15.26:0/660809879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc1a5d50 con 0x7efedc0759e0 2024-06-13T07:12:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.161+0000 7efee0c00640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc074400 0x7efedc19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.162+0000 7efedbe00640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 0x7efedc19e6c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efec800e9f0 tx=0x7efec800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.162+0000 7efed9600640 1 -- 172.21.15.26:0/660809879 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec800cd80 con 0x7efedc0759e0 2024-06-13T07:12:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.162+0000 7efee2000640 1 -- 172.21.15.26:0/660809879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc1a5fe0 con 0x7efedc0759e0 2024-06-13T07:12:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.162+0000 7efed9600640 1 -- 172.21.15.26:0/660809879 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec8010910 con 0x7efedc0759e0 2024-06-13T07:12:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.162+0000 7efed9600640 1 -- 172.21.15.26:0/660809879 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80188f0 con 0x7efedc0759e0 2024-06-13T07:12:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.162+0000 7efee2000640 1 -- 172.21.15.26:0/660809879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc1a6550 con 0x7efedc0759e0 2024-06-13T07:12:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee6577740 1 Processor -- start 2024-06-13T07:12:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee6577740 1 -- start start 2024-06-13T07:12:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee6577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8373d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee6577740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a837422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee6577740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a83744db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee6577740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558a836a0420 con 0x558a8373d7b0 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee6577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a83647c50 con 0x558a8373cdc0 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee6577740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558a8369c3e0 con 0x558a83742920 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efed9600640 1 -- 172.21.15.26:0/660809879 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efec8010ac0 con 0x7efedc0759e0 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8373d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.166+0000 7efee1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8373d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50860/0 (socket says 172.21.15.26:50860) 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efee1600640 1 -- 172.21.15.26:0/3669940876 learned_addr learned my addr 172.21.15.26:0/3669940876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efee0c00640 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a837422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efedbe00640 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a83744db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efee1600640 1 -- 172.21.15.26:0/3669940876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 msgr2=0x558a83744db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efee1600640 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a83744db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efee1600640 1 -- 172.21.15.26:0/3669940876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 msgr2=0x558a837422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efee1600640 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a837422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efee1600640 1 -- 172.21.15.26:0/3669940876 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a833b4210 con 0x558a8373cdc0 2024-06-13T07:12:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.167+0000 7efee1600640 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8373d1e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30b531e6cd1151a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.168+0000 7efecdc00640 1 -- 172.21.15.26:0/3669940876 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4004d20 con 0x558a8373cdc0 2024-06-13T07:12:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.168+0000 7efecdc00640 1 -- 172.21.15.26:0/3669940876 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed4004050 con 0x558a8373cdc0 2024-06-13T07:12:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.168+0000 7efecdc00640 1 -- 172.21.15.26:0/3669940876 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4005260 con 0x558a8373cdc0 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.168+0000 7efed9600640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efec40a2b80 0x7efec40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.168+0000 7efed9600640 1 -- 172.21.15.26:0/660809879 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7efec80cf840 con 0x7efedc0759e0 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee0c00640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efec40a2b80 0x7efec40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 -- 172.21.15.26:0/3669940876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 msgr2=0x558a8373d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8373d1e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed40118d0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 -- 172.21.15.26:0/3669940876 shutdown_connections 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a83744db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a837422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 --2- 172.21.15.26:0/3669940876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8373d1e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 -- 172.21.15.26:0/3669940876 >> 172.21.15.26:0/3669940876 conn(0x558a83704fb0 msgr2=0x558a83738840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 -- 172.21.15.26:0/3669940876 shutdown_connections 2024-06-13T07:12:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.169+0000 7efee6577740 1 -- 172.21.15.26:0/3669940876 wait complete. 2024-06-13T07:12:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.170+0000 7efee0c00640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efec40a2b80 0x7efec40a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efed0005ab0 tx=0x7efed0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.170+0000 7efee6577740 1 -- start start 2024-06-13T07:12:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.170+0000 7efee6577740 1 Processor -- start 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.170+0000 7efee6577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a837470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee6577740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a83747610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee6577740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee6577740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558a837458d0 con 0x558a8373d7b0 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee6577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a8365b0b0 con 0x558a8373cdc0 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee6577740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558a83745a50 con 0x558a83742920 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee6577740 10 client.0 _ll_register_callbacks cb 0x558a83646f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a837470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a837470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50862/0 (socket says 172.21.15.26:50862) 2024-06-13T07:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee0c00640 1 -- 172.21.15.26:0/1674793559 learned_addr learned my addr 172.21.15.26:0/1674793559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efedbe00640 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a83747610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee1600640 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee0c00640 1 -- 172.21.15.26:0/1674793559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 msgr2=0x558a8374e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee0c00640 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee0c00640 1 -- 172.21.15.26:0/1674793559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 msgr2=0x558a83747610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee0c00640 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a83747610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee0c00640 1 -- 172.21.15.26:0/1674793559 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a83480620 con 0x558a8373cdc0 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.171+0000 7efee1600640 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.172+0000 7efee0c00640 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a837470d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efed0000c00 tx=0x7efed0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46f48297a1ace5b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.172+0000 7efec2000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.172+0000 7efec2000640 1 -- 172.21.15.26:0/1674793559 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed004d070 con 0x558a8373cdc0 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.172+0000 7efec2000640 1 -- 172.21.15.26:0/1674793559 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7efed0031840 con 0x558a8373cdc0 2024-06-13T07:12:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.172+0000 7efee6577740 1 -- 172.21.15.26:0/1674793559 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed4004050 con 0x558a8373cdc0 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.172+0000 7efec2000640 1 -- 172.21.15.26:0/1674793559 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efed0059050 con 0x558a8373cdc0 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.172+0000 7efee6577740 10 client.26438 fetch_fsmap learned FSMap version 36 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.172+0000 7efee6577740 1 -- 172.21.15.26:0/1674793559 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558a83751a30 con 0x558a8373cdc0 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efec2000640 1 -- 172.21.15.26:0/1674793559 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed4004050 con 0x558a8373cdc0 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efec2000640 1 -- 172.21.15.26:0/1674793559 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7efed0003b70 con 0x558a8373cdc0 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efee6577740 10 client.26438 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efee6577740 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efee6577740 20 client.26438 populate_metadata read hostname 'smithi026' 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efee6577740 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a837530c0 0x558a83755580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efee6577740 4 client.26438 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efee6577740 1 -- 172.21.15.26:0/1674793559 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a836adb50 con 0x558a837530c0 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.173+0000 7efedbe00640 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a837530c0 0x558a83755580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.174+0000 7efedbe00640 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a837530c0 0x558a83755580 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e5219aa8706f28 server_cookie=c21c207e00c53605 in_seq=0 out_seq=0 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.174+0000 7efec2000640 10 client.26438 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efec2000640 1 -- 172.21.15.26:0/1674793559 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558a836adb50 con 0x558a837530c0 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efec2000640 10 client.26438 handle_command_reply: tid=0 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 client.26438 shutdown 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 -- 172.21.15.26:0/1674793559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 msgr2=0x558a837470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a837470d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efed0000c00 tx=0x7efed0009260 comp rx=0 tx=0).stop 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 -- 172.21.15.26:0/1674793559 shutdown_connections 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a837530c0 0x558a83755580 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a83747610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efec2000640 0 client.26438 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 --2- 172.21.15.26:0/1674793559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a837470d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 -- 172.21.15.26:0/1674793559 >> 172.21.15.26:0/1674793559 conn(0x558a83704fb0 msgr2=0x558a837379a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 -- 172.21.15.26:0/1674793559 shutdown_connections 2024-06-13T07:12:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.175+0000 7efee6577740 1 -- 172.21.15.26:0/1674793559 wait complete. 2024-06-13T07:12:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.176+0000 7efee6577740 20 client.26438 trim_cache size 0 max 16384 2024-06-13T07:12:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.194+0000 7efee6577740 1 Processor -- start 2024-06-13T07:12:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.194+0000 7efee6577740 1 -- start start 2024-06-13T07:12:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.194+0000 7efee6577740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a8373cdc0 0x558a837473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.194+0000 7efee6577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373d7b0 0x558a8374e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.194+0000 7efee6577740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a83742920 0x558a8373b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.194+0000 7efee6577740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1b34a0 con 0x558a83742920 2024-06-13T07:12:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.194+0000 7efee6577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1b0d70 con 0x558a8373d7b0 2024-06-13T07:12:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.194+0000 7efee6577740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1aebc0 con 0x558a8373cdc0 2024-06-13T07:12:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efedbe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373d7b0 0x558a8374e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a8373cdc0 0x558a837473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a8373cdc0 0x558a837473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37154/0 (socket says 172.21.15.26:37154) 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee0c00640 1 -- 172.21.15.26:0/3058621369 learned_addr learned my addr 172.21.15.26:0/3058621369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efedbe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373d7b0 0x558a8374e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50876/0 (socket says 172.21.15.26:50876) 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee1600640 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a83742920 0x558a8373b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee0c00640 1 -- 172.21.15.26:0/3058621369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373d7b0 msgr2=0x558a8374e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee0c00640 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373d7b0 0x558a8374e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee0c00640 1 -- 172.21.15.26:0/3058621369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a83742920 msgr2=0x558a8373b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee0c00640 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a83742920 0x558a8373b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efee0c00640 1 -- 172.21.15.26:0/3058621369 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a83741d80 con 0x558a8373cdc0 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.195+0000 7efedbe00640 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373d7b0 0x558a8374e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.196+0000 7efee0c00640 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a8373cdc0 0x558a837473f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efed00096c0 tx=0x7efed0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=979489bcd36dad1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.196+0000 7efecd200640 1 -- 172.21.15.26:0/3058621369 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed004d070 con 0x558a8373cdc0 2024-06-13T07:12:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.196+0000 7efecd200640 1 -- 172.21.15.26:0/3058621369 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00044d0 con 0x558a8373cdc0 2024-06-13T07:12:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.196+0000 7efecd200640 1 -- 172.21.15.26:0/3058621369 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0035040 con 0x558a8373cdc0 2024-06-13T07:12:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 -- 172.21.15.26:0/3058621369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a8373cdc0 msgr2=0x558a837473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a8373cdc0 0x558a837473f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efed00096c0 tx=0x7efed0009780 comp rx=0 tx=0).stop 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 -- 172.21.15.26:0/3058621369 shutdown_connections 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373d7b0 0x558a8374e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a83742920 0x558a8373b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 --2- 172.21.15.26:0/3058621369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a8373cdc0 0x558a837473f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 -- 172.21.15.26:0/3058621369 >> 172.21.15.26:0/3058621369 conn(0x558a83704fb0 msgr2=0x558a83746ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 -- 172.21.15.26:0/3058621369 shutdown_connections 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 -- 172.21.15.26:0/3058621369 wait complete. 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.197+0000 7efee6577740 1 -- start start 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee6577740 1 Processor -- start 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee6577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8374bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee6577740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a8374c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee6577740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee6577740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1af230 con 0x558a8373d7b0 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee6577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1b06a0 con 0x558a8373cdc0 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee6577740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1af900 con 0x558a83742920 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee6577740 10 client.0 _ll_register_callbacks cb 0x7efed40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8374bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efedbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a8374c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.198+0000 7efee1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8374bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50888/0 (socket says 172.21.15.26:50888) 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37166/0 (socket says 172.21.15.26:37166) 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee1600640 1 -- 172.21.15.26:0/1421159114 learned_addr learned my addr 172.21.15.26:0/1421159114 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efedbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a8374c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56024/0 (socket says 172.21.15.26:56024) 2024-06-13T07:12:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee0c00640 1 -- 172.21.15.26:0/1421159114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 msgr2=0x558a8374d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee0c00640 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee0c00640 1 -- 172.21.15.26:0/1421159114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 msgr2=0x558a8374c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee0c00640 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a8374c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee0c00640 1 -- 172.21.15.26:0/1421159114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8374d920 con 0x558a8373cdc0 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee1600640 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efee0c00640 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8374bde0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efed00315a0 tx=0x7efed0032cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4509b9d5ef15a2d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.199+0000 7efec2000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.200+0000 7efec2000640 1 -- 172.21.15.26:0/1421159114 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed006a020 con 0x558a8373cdc0 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.200+0000 7efec2000640 1 -- 172.21.15.26:0/1421159114 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7efed0003a00 con 0x558a8373cdc0 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.200+0000 7efee6577740 1 -- 172.21.15.26:0/1421159114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed00044d0 con 0x558a8373cdc0 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.200+0000 7efec2000640 1 -- 172.21.15.26:0/1421159114 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efed0072050 con 0x558a8373cdc0 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.200+0000 7efee6577740 10 client.26444 fetch_fsmap learned FSMap version 36 2024-06-13T07:12:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.200+0000 7efee6577740 1 -- 172.21.15.26:0/1421159114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558a837bdf20 con 0x558a8373cdc0 2024-06-13T07:12:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.200+0000 7efec2000640 1 -- 172.21.15.26:0/1421159114 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00044d0 con 0x558a8373cdc0 2024-06-13T07:12:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.201+0000 7efec2000640 1 -- 172.21.15.26:0/1421159114 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7efed003db90 con 0x558a8373cdc0 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.201+0000 7efee6577740 10 client.26444 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.201+0000 7efee6577740 10 client.26444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.201+0000 7efee6577740 20 client.26444 populate_metadata read hostname 'smithi026' 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.201+0000 7efee6577740 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a837be830 0x558a837c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.201+0000 7efee6577740 4 client.26444 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.201+0000 7efee6577740 1 -- 172.21.15.26:0/1421159114 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a836a7ea0 con 0x558a837be830 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.201+0000 7efedbe00640 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a837be830 0x558a837c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.202+0000 7efedbe00640 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a837be830 0x558a837c0c50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3061996fedbb5f4d server_cookie=600035c0db92b5dc in_seq=0 out_seq=0 2024-06-13T07:12:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.202+0000 7efec2000640 10 client.26444 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.305+0000 7efec2000640 1 -- 172.21.15.26:0/1421159114 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558a836a7ea0 con 0x558a837be830 2024-06-13T07:12:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.305+0000 7efec2000640 10 client.26444 handle_command_reply: tid=0 2024-06-13T07:12:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.305+0000 7efee6577740 1 client.26444 shutdown 2024-06-13T07:12:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.305+0000 7efee6577740 1 -- 172.21.15.26:0/1421159114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 msgr2=0x558a8374bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.305+0000 7efee6577740 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8374bde0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efed00315a0 tx=0x7efed0032cf0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 1 -- 172.21.15.26:0/1421159114 shutdown_connections 2024-06-13T07:12:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a837be830 0x558a837c0c50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a83742920 0x558a8374d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a8373d7b0 0x558a8374c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 1 --2- 172.21.15.26:0/1421159114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8373cdc0 0x558a8374bde0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efec2000640 0 client.26444 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:12:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 1 -- 172.21.15.26:0/1421159114 >> 172.21.15.26:0/1421159114 conn(0x558a83704fb0 msgr2=0x558a83737110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 1 -- 172.21.15.26:0/1421159114 shutdown_connections 2024-06-13T07:12:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 1 -- 172.21.15.26:0/1421159114 wait complete. 2024-06-13T07:12:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.306+0000 7efee6577740 20 client.26444 trim_cache size 0 max 16384 2024-06-13T07:12:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:12:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:12:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:12:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 -- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efec40a2b80 msgr2=0x7efec40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efec40a2b80 0x7efec40a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efed0005ab0 tx=0x7efed0005a20 comp rx=0 tx=0).stop 2024-06-13T07:12:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 -- 172.21.15.26:0/660809879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 msgr2=0x7efedc19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 0x7efedc19e6c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efec800e9f0 tx=0x7efec800eeb0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 -- 172.21.15.26:0/660809879 shutdown_connections 2024-06-13T07:12:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7efec40a2b80 0x7efec40a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efedc10de10 0x7efedc1a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efedc0759e0 0x7efedc19e6c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 --2- 172.21.15.26:0/660809879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc074400 0x7efedc19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 -- 172.21.15.26:0/660809879 >> 172.21.15.26:0/660809879 conn(0x7efedc0fcdb0 msgr2=0x7efedc1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 -- 172.21.15.26:0/660809879 shutdown_connections 2024-06-13T07:12:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:12:51.307+0000 7efee2000640 1 -- 172.21.15.26:0/660809879 wait complete. 2024-06-13T07:12:51.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 579273 4.78 MB/sec execute 545 sec latency 43.730 ms 2024-06-13T07:12:52.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:51 smithi136 ceph-mon[37229]: pgmap v880: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-06-13T07:12:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:51 smithi026 ceph-mon[32041]: pgmap v880: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-06-13T07:12:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:51 smithi135 ceph-mon[36276]: pgmap v880: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-06-13T07:12:52.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 579864 4.77 MB/sec execute 546 sec latency 32.711 ms 2024-06-13T07:12:53.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 581161 4.77 MB/sec execute 547 sec latency 27.397 ms 2024-06-13T07:12:54.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:53 smithi026 ceph-mon[32041]: pgmap v881: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-06-13T07:12:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:53 smithi135 ceph-mon[36276]: pgmap v881: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-06-13T07:12:54.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:53 smithi136 ceph-mon[37229]: pgmap v881: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-06-13T07:12:54.586 INFO:tasks.workunit.client.0.smithi026.stdout: 1 581970 4.77 MB/sec execute 548 sec latency 25.960 ms 2024-06-13T07:12:55.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 582776 4.78 MB/sec execute 549 sec latency 24.439 ms 2024-06-13T07:12:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:55 smithi026 ceph-mon[32041]: pgmap v882: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-06-13T07:12:56.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:55 smithi135 ceph-mon[36276]: pgmap v882: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-06-13T07:12:56.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:55 smithi136 ceph-mon[37229]: pgmap v882: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-06-13T07:12:56.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 583188 4.77 MB/sec execute 550 sec latency 33.849 ms 2024-06-13T07:12:57.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 584049 4.77 MB/sec execute 551 sec latency 42.640 ms 2024-06-13T07:12:58.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:57 smithi026 ceph-mon[32041]: pgmap v883: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-06-13T07:12:58.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:57 smithi135 ceph-mon[36276]: pgmap v883: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-06-13T07:12:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:57 smithi136 ceph-mon[37229]: pgmap v883: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-06-13T07:12:58.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 585430 4.77 MB/sec execute 552 sec latency 26.255 ms 2024-06-13T07:12:59.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 586283 4.78 MB/sec execute 553 sec latency 51.142 ms 2024-06-13T07:13:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:12:59 smithi026 ceph-mon[32041]: pgmap v884: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:13:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:12:59 smithi135 ceph-mon[36276]: pgmap v884: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:13:00.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:12:59 smithi136 ceph-mon[37229]: pgmap v884: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-06-13T07:13:00.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 586624 4.77 MB/sec execute 554 sec latency 31.118 ms 2024-06-13T07:13:01.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 587323 4.76 MB/sec execute 555 sec latency 39.733 ms 2024-06-13T07:13:02.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:01 smithi026 ceph-mon[32041]: pgmap v885: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 997 KiB/s wr, 150 op/s 2024-06-13T07:13:02.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:01 smithi135 ceph-mon[36276]: pgmap v885: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 997 KiB/s wr, 150 op/s 2024-06-13T07:13:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:01 smithi136 ceph-mon[37229]: pgmap v885: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 997 KiB/s wr, 150 op/s 2024-06-13T07:13:02.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 588407 4.76 MB/sec execute 556 sec latency 65.346 ms 2024-06-13T07:13:03.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 589239 4.77 MB/sec execute 557 sec latency 328.584 ms 2024-06-13T07:13:04.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:03 smithi026 ceph-mon[32041]: pgmap v886: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:13:04.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:03 smithi135 ceph-mon[36276]: pgmap v886: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:13:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:03 smithi136 ceph-mon[37229]: pgmap v886: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-06-13T07:13:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.458+0000 7ff3ab200640 1 -- 172.21.15.26:0/3567084614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a4103c00 msgr2=0x7ff3a4104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.458+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3567084614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a4103c00 0x7ff3a4104020 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff394009900 tx=0x7ff39402a410 comp rx=0 tx=0).stop 2024-06-13T07:13:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.459+0000 7ff3ab200640 1 -- 172.21.15.26:0/3567084614 shutdown_connections 2024-06-13T07:13:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.459+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3567084614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a41057b0 0x7ff3a410c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.459+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3567084614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a4104df0 0x7ff3a4105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.459+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3567084614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a4103c00 0x7ff3a4104020 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.459+0000 7ff3ab200640 1 -- 172.21.15.26:0/3567084614 >> 172.21.15.26:0/3567084614 conn(0x7ff3a40ff190 msgr2=0x7ff3a41015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.459+0000 7ff3ab200640 1 -- 172.21.15.26:0/3567084614 shutdown_connections 2024-06-13T07:13:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.459+0000 7ff3ab200640 1 -- 172.21.15.26:0/3567084614 wait complete. 2024-06-13T07:13:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.460+0000 7ff3ab200640 1 Processor -- start 2024-06-13T07:13:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.460+0000 7ff3ab200640 1 -- start start 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.460+0000 7ff3ab200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a4103c00 0x7ff3a4199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.460+0000 7ff3ab200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a4104df0 0x7ff3a419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.460+0000 7ff3ab200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a41057b0 0x7ff3a41a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.460+0000 7ff3ab200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a410f880 con 0x7ff3a4103c00 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.460+0000 7ff3ab200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a410f700 con 0x7ff3a4104df0 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.460+0000 7ff3ab200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a410fa00 con 0x7ff3a41057b0 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a4103c00 0x7ff3a4199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a41057b0 0x7ff3a41a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a4103c00 0x7ff3a4199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53266/0 (socket says 172.21.15.26:53266) 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a3e00640 1 -- 172.21.15.26:0/3799281858 learned_addr learned my addr 172.21.15.26:0/3799281858 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff39b400640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a4104df0 0x7ff3a419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a9200640 1 -- 172.21.15.26:0/3799281858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a4104df0 msgr2=0x7ff3a419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a9200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a4104df0 0x7ff3a419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a9200640 1 -- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a4103c00 msgr2=0x7ff3a4199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a9200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a4103c00 0x7ff3a4199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a9200640 1 -- 172.21.15.26:0/3799281858 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a41a1a90 con 0x7ff3a41057b0 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.461+0000 7ff3a9200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a41057b0 0x7ff3a41a1420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009fd0 tx=0x7ff39c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.462+0000 7ff3a1600640 1 -- 172.21.15.26:0/3799281858 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00ce20 con 0x7ff3a41057b0 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.462+0000 7ff3a1600640 1 -- 172.21.15.26:0/3799281858 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c005020 con 0x7ff3a41057b0 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.462+0000 7ff3ab200640 1 -- 172.21.15.26:0/3799281858 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a41a1d80 con 0x7ff3a41057b0 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.462+0000 7ff3a1600640 1 -- 172.21.15.26:0/3799281858 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c010720 con 0x7ff3a41057b0 2024-06-13T07:13:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.462+0000 7ff3ab200640 1 -- 172.21.15.26:0/3799281858 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a41a22a0 con 0x7ff3a41057b0 2024-06-13T07:13:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.465+0000 7ff3a1600640 1 -- 172.21.15.26:0/3799281858 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff39c005540 con 0x7ff3a41057b0 2024-06-13T07:13:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.465+0000 7ff3ae05f740 1 Processor -- start 2024-06-13T07:13:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.466+0000 7ff3ae05f740 1 -- start start 2024-06-13T07:13:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.466+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e22020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.466+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 0x55dab3e270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.466+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.466+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dab3d85420 con 0x55dab3e225f0 2024-06-13T07:13:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.466+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dab3d2cc50 con 0x55dab3e27760 2024-06-13T07:13:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.466+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dab3d813e0 con 0x55dab3e21c00 2024-06-13T07:13:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e22020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 0x55dab3e270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e22020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60966/0 (socket says 172.21.15.26:60966) 2024-06-13T07:13:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 -- 172.21.15.26:0/180680758 learned_addr learned my addr 172.21.15.26:0/180680758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 --2- 172.21.15.26:0/180680758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 -- 172.21.15.26:0/180680758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 msgr2=0x55dab3e22020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 --2- 172.21.15.26:0/180680758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e22020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 -- 172.21.15.26:0/180680758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 msgr2=0x55dab3e270f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 --2- 172.21.15.26:0/180680758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 0x55dab3e270f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 -- 172.21.15.26:0/180680758 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab3a99210 con 0x55dab3e27760 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.467+0000 7ff39b400640 1 --2- 172.21.15.26:0/180680758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e29bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009fd0 tx=0x7ff38c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ae77883399c2f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.468+0000 7ff399600640 1 -- 172.21.15.26:0/180680758 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c011d70 con 0x55dab3e27760 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.468+0000 7ff399600640 1 -- 172.21.15.26:0/180680758 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c00f790 con 0x55dab3e27760 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.468+0000 7ff399600640 1 -- 172.21.15.26:0/180680758 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c003850 con 0x55dab3e27760 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.468+0000 7ff3a1600640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff3880a2cb0 0x7ff3880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.468+0000 7ff3a1600640 1 -- 172.21.15.26:0/3799281858 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff39c0cfa20 con 0x7ff3a41057b0 2024-06-13T07:13:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.468+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff3880a2cb0 0x7ff3880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 -- 172.21.15.26:0/180680758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 msgr2=0x55dab3e29bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/180680758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e29bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009fd0 tx=0x7ff38c005f90 comp rx=0 tx=0).stop 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 -- 172.21.15.26:0/180680758 shutdown_connections 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/180680758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e29bf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/180680758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 0x55dab3e270f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/180680758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e22020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 -- 172.21.15.26:0/180680758 >> 172.21.15.26:0/180680758 conn(0x55dab3de9fb0 msgr2=0x55dab3e1d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 -- 172.21.15.26:0/180680758 shutdown_connections 2024-06-13T07:13:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3ae05f740 1 -- 172.21.15.26:0/180680758 wait complete. 2024-06-13T07:13:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.469+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff3880a2cb0 0x7ff3880a5170 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff394000c00 tx=0x7ff3940046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.470+0000 7ff3ae05f740 1 -- start start 2024-06-13T07:13:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.470+0000 7ff3ae05f740 1 Processor -- start 2024-06-13T07:13:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e21c00 0x55dab3e34720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e225f0 0x55dab3e34c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dab3e2a710 con 0x55dab3e27760 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dab3d400b0 con 0x55dab3e21c00 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dab3e2a890 con 0x55dab3e225f0 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3ae05f740 10 client.0 _ll_register_callbacks cb 0x55dab3d2bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e21c00 0x55dab3e34720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff39b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e225f0 0x55dab3e34c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff39b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e225f0 0x55dab3e34c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60970/0 (socket says 172.21.15.26:60970) 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff39b400640 1 -- 172.21.15.26:0/3939462914 learned_addr learned my addr 172.21.15.26:0/3939462914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e21c00 0x55dab3e34720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:55334/0 (socket says 172.21.15.26:55334) 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3a9200640 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff39b400640 1 -- 172.21.15.26:0/3939462914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e21c00 msgr2=0x55dab3e34720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff39b400640 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e21c00 0x55dab3e34720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff39b400640 1 -- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 msgr2=0x55dab3e3bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff39b400640 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff39b400640 1 -- 172.21.15.26:0/3939462914 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab3b65620 con 0x55dab3e225f0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3a9200640 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.471+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e21c00 0x55dab3e34720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.472+0000 7ff39b400640 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e225f0 0x55dab3e34c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff38c004470 tx=0x7ff38c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e1d33ba05ea787d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.472+0000 7ff392000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.472+0000 7ff392000640 1 -- 172.21.15.26:0/3939462914 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c005070 con 0x55dab3e225f0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.472+0000 7ff392000640 1 -- 172.21.15.26:0/3939462914 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff38c005210 con 0x55dab3e225f0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.472+0000 7ff3ae05f740 1 -- 172.21.15.26:0/3939462914 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38c00f790 con 0x55dab3e225f0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.472+0000 7ff392000640 1 -- 172.21.15.26:0/3939462914 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff38c005410 con 0x55dab3e225f0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3ae05f740 10 client.26485 fetch_fsmap learned FSMap version 36 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3ae05f740 1 -- 172.21.15.26:0/3939462914 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dab3e3f010 con 0x55dab3e225f0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff392000640 1 -- 172.21.15.26:0/3939462914 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c030070 con 0x55dab3e225f0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff392000640 1 -- 172.21.15.26:0/3939462914 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7ff38c0035f0 con 0x55dab3e225f0 2024-06-13T07:13:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3ae05f740 10 client.26485 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:13:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3ae05f740 10 client.26485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3ae05f740 20 client.26485 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dab3e40550 0x55dab3e42990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3ae05f740 4 client.26485 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:13:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3ae05f740 1 -- 172.21.15.26:0/3939462914 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dab3d92b50 con 0x55dab3e40550 2024-06-13T07:13:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.473+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dab3e40550 0x55dab3e42990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.474+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dab3e40550 0x55dab3e42990 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b435a8538cb8d3e server_cookie=9b3b8088d1fbad60 in_seq=0 out_seq=0 2024-06-13T07:13:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.474+0000 7ff392000640 10 client.26485 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff392000640 1 -- 172.21.15.26:0/3939462914 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dab3d92b50 con 0x55dab3e40550 2024-06-13T07:13:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff392000640 10 client.26485 handle_command_reply: tid=0 2024-06-13T07:13:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 client.26485 shutdown 2024-06-13T07:13:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 -- 172.21.15.26:0/3939462914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e225f0 msgr2=0x55dab3e34c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e225f0 0x55dab3e34c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff38c004470 tx=0x7ff38c005850 comp rx=0 tx=0).stop 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 -- 172.21.15.26:0/3939462914 shutdown_connections 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dab3e40550 0x55dab3e42990 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e225f0 0x55dab3e34c60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/3939462914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e21c00 0x55dab3e34720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 -- 172.21.15.26:0/3939462914 >> 172.21.15.26:0/3939462914 conn(0x55dab3de9fb0 msgr2=0x55dab3e1c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff392000640 0 client.26485 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.475+0000 7ff3ae05f740 1 -- 172.21.15.26:0/3939462914 shutdown_connections 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.476+0000 7ff3ae05f740 1 -- 172.21.15.26:0/3939462914 wait complete. 2024-06-13T07:13:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.476+0000 7ff3ae05f740 20 client.26485 trim_cache size 0 max 16384 2024-06-13T07:13:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.495+0000 7ff3ae05f740 1 Processor -- start 2024-06-13T07:13:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.495+0000 7ff3ae05f740 1 -- start start 2024-06-13T07:13:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e225f0 0x55dab3e23e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41aada0 con 0x55dab3e27760 2024-06-13T07:13:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41a8670 con 0x55dab3e225f0 2024-06-13T07:13:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41a64c0 con 0x55dab3e21c00 2024-06-13T07:13:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e238e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60980/0 (socket says 172.21.15.26:60980) 2024-06-13T07:13:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a3e00640 1 -- 172.21.15.26:0/255802284 learned_addr learned my addr 172.21.15.26:0/255802284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a9200640 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff39b400640 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e225f0 0x55dab3e23e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a3e00640 1 -- 172.21.15.26:0/255802284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e225f0 msgr2=0x55dab3e23e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e225f0 0x55dab3e23e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a3e00640 1 -- 172.21.15.26:0/255802284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 msgr2=0x55dab3e3bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a3e00640 1 -- 172.21.15.26:0/255802284 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab3e229d0 con 0x55dab3e21c00 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.496+0000 7ff3a9200640 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.497+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e238e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff394054fd0 tx=0x7ff39403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52dd7e224b937912 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.497+0000 7ff398c00640 1 -- 172.21.15.26:0/255802284 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394035040 con 0x55dab3e21c00 2024-06-13T07:13:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.497+0000 7ff398c00640 1 -- 172.21.15.26:0/255802284 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39405a070 con 0x55dab3e21c00 2024-06-13T07:13:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.497+0000 7ff398c00640 1 -- 172.21.15.26:0/255802284 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394054c30 con 0x55dab3e21c00 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 -- 172.21.15.26:0/255802284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 msgr2=0x55dab3e238e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e238e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff394054fd0 tx=0x7ff39403f000 comp rx=0 tx=0).stop 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 -- 172.21.15.26:0/255802284 shutdown_connections 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e225f0 0x55dab3e23e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e27760 0x55dab3e3bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/255802284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e238e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 -- 172.21.15.26:0/255802284 >> 172.21.15.26:0/255802284 conn(0x55dab3de9fb0 msgr2=0x55dab3e26f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 -- 172.21.15.26:0/255802284 shutdown_connections 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 -- 172.21.15.26:0/255802284 wait complete. 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.498+0000 7ff3ae05f740 1 -- start start 2024-06-13T07:13:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3ae05f740 1 Processor -- start 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e3b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 0x55dab3e38390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3ae05f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41a6b30 con 0x55dab3e225f0 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41a7fa0 con 0x55dab3e27760 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3ae05f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41a7200 con 0x55dab3e21c00 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3ae05f740 10 client.0 _ll_register_callbacks cb 0x7ff39c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3a9200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e3b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff39b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 0x55dab3e38390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e3b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60990/0 (socket says 172.21.15.26:60990) 2024-06-13T07:13:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.499+0000 7ff3a3e00640 1 -- 172.21.15.26:0/1474061686 learned_addr learned my addr 172.21.15.26:0/1474061686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff3a9200640 1 -- 172.21.15.26:0/1474061686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 msgr2=0x55dab3e3b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff3a9200640 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e3b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff3a9200640 1 -- 172.21.15.26:0/1474061686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 msgr2=0x55dab3e38390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff3a9200640 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 0x55dab3e38390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff3a9200640 1 -- 172.21.15.26:0/1474061686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab3e38e90 con 0x55dab3e27760 2024-06-13T07:13:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e3b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff3a9200640 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e388d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00c760 tx=0x7ff39c09c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fafc932ccc566598 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff392000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff392000640 1 -- 172.21.15.26:0/1474061686 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0051d0 con 0x55dab3e27760 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff3ae05f740 1 -- 172.21.15.26:0/1474061686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff39405a070 con 0x55dab3e27760 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.500+0000 7ff392000640 1 -- 172.21.15.26:0/1474061686 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff39c09f040 con 0x55dab3e27760 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff392000640 1 -- 172.21.15.26:0/1474061686 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff39c0a4610 con 0x55dab3e27760 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff3ae05f740 10 client.26462 fetch_fsmap learned FSMap version 36 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff3ae05f740 1 -- 172.21.15.26:0/1474061686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dab3eaa330 con 0x55dab3e27760 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff392000640 1 -- 172.21.15.26:0/1474061686 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39405a070 con 0x55dab3e27760 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff392000640 1 -- 172.21.15.26:0/1474061686 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7ff39c083020 con 0x55dab3e27760 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff3ae05f740 10 client.26462 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff3ae05f740 10 client.26462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff3ae05f740 20 client.26462 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dab3eaacb0 0x55dab3ead0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff3ae05f740 4 client.26462 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:13:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.501+0000 7ff3ae05f740 1 -- 172.21.15.26:0/1474061686 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dab3d8cea0 con 0x55dab3eaacb0 2024-06-13T07:13:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.502+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dab3eaacb0 0x55dab3ead0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.502+0000 7ff3a3e00640 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dab3eaacb0 0x55dab3ead0d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=931a082ed7b8cd05 server_cookie=d9893e2c9c806fa3 in_seq=0 out_seq=0 2024-06-13T07:13:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.502+0000 7ff392000640 10 client.26462 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:04.587 INFO:tasks.workunit.client.0.smithi026.stdout: 1 589904 4.77 MB/sec execute 558 sec latency 44.819 ms 2024-06-13T07:13:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff392000640 1 -- 172.21.15.26:0/1474061686 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dab3d8cea0 con 0x55dab3eaacb0 2024-06-13T07:13:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff392000640 10 client.26462 handle_command_reply: tid=0 2024-06-13T07:13:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 client.26462 shutdown 2024-06-13T07:13:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 -- 172.21.15.26:0/1474061686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 msgr2=0x55dab3e388d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e388d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00c760 tx=0x7ff39c09c670 comp rx=0 tx=0).stop 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 -- 172.21.15.26:0/1474061686 shutdown_connections 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55dab3eaacb0 0x55dab3ead0d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dab3e27760 0x55dab3e388d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dab3e225f0 0x55dab3e38390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 --2- 172.21.15.26:0/1474061686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dab3e21c00 0x55dab3e3b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff392000640 0 client.26462 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 -- 172.21.15.26:0/1474061686 >> 172.21.15.26:0/1474061686 conn(0x55dab3de9fb0 msgr2=0x55dab3e34650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 -- 172.21.15.26:0/1474061686 shutdown_connections 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.636+0000 7ff3ae05f740 1 -- 172.21.15.26:0/1474061686 wait complete. 2024-06-13T07:13:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.637+0000 7ff3ae05f740 20 client.26462 trim_cache size 0 max 16384 2024-06-13T07:13:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:13:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:13:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:13:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.637+0000 7ff3ab200640 1 -- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff3880a2cb0 msgr2=0x7ff3880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.637+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff3880a2cb0 0x7ff3880a5170 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff394000c00 tx=0x7ff3940046c0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.637+0000 7ff3ab200640 1 -- 172.21.15.26:0/3799281858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a41057b0 msgr2=0x7ff3a41a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.637+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a41057b0 0x7ff3a41a1420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009fd0 tx=0x7ff39c00ef10 comp rx=0 tx=0).stop 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.638+0000 7ff3ab200640 1 -- 172.21.15.26:0/3799281858 shutdown_connections 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.638+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff3880a2cb0 0x7ff3880a5170 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.638+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a4104df0 0x7ff3a419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.638+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3a41057b0 0x7ff3a41a1420 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.638+0000 7ff3ab200640 1 --2- 172.21.15.26:0/3799281858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a4103c00 0x7ff3a4199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.638+0000 7ff3ab200640 1 -- 172.21.15.26:0/3799281858 >> 172.21.15.26:0/3799281858 conn(0x7ff3a40ff190 msgr2=0x7ff3a41080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.638+0000 7ff3ab200640 1 -- 172.21.15.26:0/3799281858 shutdown_connections 2024-06-13T07:13:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:04.638+0000 7ff3ab200640 1 -- 172.21.15.26:0/3799281858 wait complete. 2024-06-13T07:13:05.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 590360 4.76 MB/sec execute 559 sec latency 30.765 ms 2024-06-13T07:13:06.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:05 smithi026 ceph-mon[32041]: pgmap v887: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:13:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:05 smithi135 ceph-mon[36276]: pgmap v887: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:13:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:05 smithi136 ceph-mon[37229]: pgmap v887: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-13T07:13:06.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 591269 4.76 MB/sec execute 560 sec latency 25.489 ms 2024-06-13T07:13:07.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 592230 4.76 MB/sec execute 561 sec latency 50.271 ms 2024-06-13T07:13:08.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:07 smithi026 ceph-mon[32041]: pgmap v888: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-13T07:13:08.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:07 smithi135 ceph-mon[36276]: pgmap v888: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-13T07:13:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:07 smithi136 ceph-mon[37229]: pgmap v888: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-13T07:13:08.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 593390 4.77 MB/sec execute 562 sec latency 27.362 ms 2024-06-13T07:13:09.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 593738 4.76 MB/sec execute 563 sec latency 45.858 ms 2024-06-13T07:13:10.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:09 smithi026 ceph-mon[32041]: pgmap v889: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-13T07:13:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:09 smithi135 ceph-mon[36276]: pgmap v889: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-13T07:13:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:09 smithi136 ceph-mon[37229]: pgmap v889: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-13T07:13:10.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 594549 4.76 MB/sec execute 564 sec latency 23.741 ms 2024-06-13T07:13:11.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 595595 4.75 MB/sec execute 565 sec latency 27.044 ms 2024-06-13T07:13:12.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:11 smithi026 ceph-mon[32041]: pgmap v890: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-13T07:13:12.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:11 smithi135 ceph-mon[36276]: pgmap v890: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-13T07:13:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:11 smithi136 ceph-mon[37229]: pgmap v890: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-13T07:13:12.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 596521 4.76 MB/sec execute 566 sec latency 26.102 ms 2024-06-13T07:13:13.588 INFO:tasks.workunit.client.0.smithi026.stdout: 1 597144 4.76 MB/sec execute 567 sec latency 38.626 ms 2024-06-13T07:13:14.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:13 smithi026 ceph-mon[32041]: pgmap v891: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-06-13T07:13:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:13 smithi135 ceph-mon[36276]: pgmap v891: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-06-13T07:13:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:13 smithi136 ceph-mon[37229]: pgmap v891: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-06-13T07:13:14.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 597866 4.75 MB/sec execute 568 sec latency 31.584 ms 2024-06-13T07:13:15.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 599050 4.75 MB/sec execute 569 sec latency 27.824 ms 2024-06-13T07:13:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:15 smithi026 ceph-mon[32041]: pgmap v892: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-13T07:13:16.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:15 smithi135 ceph-mon[36276]: pgmap v892: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-13T07:13:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:15 smithi136 ceph-mon[37229]: pgmap v892: 129 pgs: 129 active+clean; 127 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-13T07:13:16.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 599725 4.76 MB/sec execute 570 sec latency 26.487 ms 2024-06-13T07:13:17.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 600526 4.76 MB/sec execute 571 sec latency 24.543 ms 2024-06-13T07:13:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.830+0000 7f7ed6800640 1 -- 172.21.15.26:0/1635636254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0105610 msgr2=0x7f7ed010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.830+0000 7f7ed6800640 1 --2- 172.21.15.26:0/1635636254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0105610 0x7f7ed010bea0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009ca0 tx=0x7f7ec802a8c0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.831+0000 7f7ed6800640 1 -- 172.21.15.26:0/1635636254 shutdown_connections 2024-06-13T07:13:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.831+0000 7f7ed6800640 1 --2- 172.21.15.26:0/1635636254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ed0104c50 0x7f7ed01050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.831+0000 7f7ed6800640 1 --2- 172.21.15.26:0/1635636254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0105610 0x7f7ed010bea0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.831+0000 7f7ed6800640 1 --2- 172.21.15.26:0/1635636254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed0103a60 0x7f7ed0103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.831+0000 7f7ed6800640 1 -- 172.21.15.26:0/1635636254 >> 172.21.15.26:0/1635636254 conn(0x7f7ed00fefd0 msgr2=0x7f7ed0101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.831+0000 7f7ed6800640 1 -- 172.21.15.26:0/1635636254 shutdown_connections 2024-06-13T07:13:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.831+0000 7f7ed6800640 1 -- 172.21.15.26:0/1635636254 wait complete. 2024-06-13T07:13:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.832+0000 7f7ed6800640 1 Processor -- start 2024-06-13T07:13:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed6800640 1 -- start start 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed0103a60 0x7f7ed0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0104c50 0x7f7ed019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed6800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ed0105610 0x7f7ed01a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed6800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010f6e0 con 0x7f7ed0103a60 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed6800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010f560 con 0x7f7ed0105610 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed6800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010f860 con 0x7f7ed0104c50 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0104c50 0x7f7ed019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0104c50 0x7f7ed019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35212/0 (socket says 172.21.15.26:35212) 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ed0105610 0x7f7ed01a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed4a00640 1 -- 172.21.15.26:0/3024322597 learned_addr learned my addr 172.21.15.26:0/3024322597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.833+0000 7f7ed5400640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed0103a60 0x7f7ed0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ed4a00640 1 -- 172.21.15.26:0/3024322597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ed0105610 msgr2=0x7f7ed01a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ed0105610 0x7f7ed01a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ed4a00640 1 -- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed0103a60 msgr2=0x7f7ed0199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed0103a60 0x7f7ed0199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ed4a00640 1 -- 172.21.15.26:0/3024322597 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed01a1960 con 0x7f7ed0104c50 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ed5400640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed0103a60 0x7f7ed0199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0104c50 0x7f7ed019a280 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00e9f0 tx=0x7f7ebc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ece000640 1 -- 172.21.15.26:0/3024322597 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00cd80 con 0x7f7ed0104c50 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ed6800640 1 -- 172.21.15.26:0/3024322597 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed01a1c50 con 0x7f7ed0104c50 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.834+0000 7f7ece000640 1 -- 172.21.15.26:0/3024322597 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc005500 con 0x7f7ed0104c50 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.835+0000 7f7ed6800640 1 -- 172.21.15.26:0/3024322597 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01a21a0 con 0x7f7ed0104c50 2024-06-13T07:13:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.835+0000 7f7ece000640 1 -- 172.21.15.26:0/3024322597 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc010690 con 0x7f7ed0104c50 2024-06-13T07:13:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.838+0000 7f7edad78740 1 Processor -- start 2024-06-13T07:13:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.838+0000 7f7edad78740 1 -- start start 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7edad78740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e1331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7edad78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e1382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7edad78740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e13adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7edad78740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55de9e096420 con 0x55de9e138920 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7edad78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55de9e03dc50 con 0x55de9e1337b0 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7edad78740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55de9e0923e0 con 0x55de9e132dc0 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7ece000640 1 -- 172.21.15.26:0/3024322597 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ebc0046c0 con 0x7f7ed0104c50 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e1331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e1331e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35222/0 (socket says 172.21.15.26:35222) 2024-06-13T07:13:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7ed5e00640 1 -- 172.21.15.26:0/4208885686 learned_addr learned my addr 172.21.15.26:0/4208885686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e13adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.839+0000 7f7ed5400640 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e1382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.840+0000 7f7ed5e00640 1 -- 172.21.15.26:0/4208885686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 msgr2=0x55de9e1382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.840+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e1382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.840+0000 7f7ed5e00640 1 -- 172.21.15.26:0/4208885686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 msgr2=0x55de9e13adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.840+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e13adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.840+0000 7f7ed5e00640 1 -- 172.21.15.26:0/4208885686 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9ddaa210 con 0x55de9e132dc0 2024-06-13T07:13:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.840+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e1331e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8035000 tx=0x7f7ec8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7bd1868c04c213 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.840+0000 7f7ece000640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7eb80a2c00 0x7f7eb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.841+0000 7f7ec2600640 1 -- 172.21.15.26:0/4208885686 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8009800 con 0x55de9e132dc0 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.841+0000 7f7ec2600640 1 -- 172.21.15.26:0/4208885686 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8036040 con 0x55de9e132dc0 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.841+0000 7f7ece000640 1 -- 172.21.15.26:0/3024322597 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7ebc0cf8c0 con 0x7f7ed0104c50 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.841+0000 7f7ec2600640 1 -- 172.21.15.26:0/4208885686 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8004290 con 0x55de9e132dc0 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.841+0000 7f7ed5400640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7eb80a2c00 0x7f7eb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7edad78740 1 -- 172.21.15.26:0/4208885686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 msgr2=0x55de9e1331e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7edad78740 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e1331e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8035000 tx=0x7f7ec8005dc0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7edad78740 1 -- 172.21.15.26:0/4208885686 shutdown_connections 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7edad78740 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e1382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7edad78740 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e13adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7edad78740 1 --2- 172.21.15.26:0/4208885686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e1331e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7edad78740 1 -- 172.21.15.26:0/4208885686 >> 172.21.15.26:0/4208885686 conn(0x55de9e0fafb0 msgr2=0x55de9e12e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7edad78740 1 -- 172.21.15.26:0/4208885686 shutdown_connections 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.842+0000 7f7ed5400640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7eb80a2c00 0x7f7eb80a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55de9e133bb0 tx=0x7f7ec4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.843+0000 7f7edad78740 1 -- 172.21.15.26:0/4208885686 wait complete. 2024-06-13T07:13:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.843+0000 7f7edad78740 1 -- start start 2024-06-13T07:13:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.843+0000 7f7edad78740 1 Processor -- start 2024-06-13T07:13:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7edad78740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e13cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7edad78740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e1337b0 0x55de9e13d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7edad78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e138920 0x55de9e144490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7edad78740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55de9e13b690 con 0x55de9e1337b0 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7edad78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55de9e0510b0 con 0x55de9e138920 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7edad78740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55de9e13b810 con 0x55de9e132dc0 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7edad78740 10 client.0 _ll_register_callbacks cb 0x55de9e03cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7ed5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e13cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7ed5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e13cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35234/0 (socket says 172.21.15.26:35234) 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7ed5400640 1 -- 172.21.15.26:0/1208724480 learned_addr learned my addr 172.21.15.26:0/1208724480 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e1337b0 0x55de9e13d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.844+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e138920 0x55de9e144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7ed5400640 1 -- 172.21.15.26:0/1208724480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e138920 msgr2=0x55de9e144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7ed5400640 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e138920 0x55de9e144490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7ed5400640 1 -- 172.21.15.26:0/1208724480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e1337b0 msgr2=0x55de9e13d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7ed5400640 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e1337b0 0x55de9e13d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7ed5400640 1 -- 172.21.15.26:0/1208724480 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9de76620 con 0x55de9e132dc0 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e138920 0x55de9e144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7ed5400640 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e13cf10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4011c10 tx=0x7f7ec40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c6aabc16cef709 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1208724480 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4010d30 con 0x55de9e132dc0 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1208724480 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7ec4028400 con 0x55de9e132dc0 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.845+0000 7f7edad78740 1 -- 172.21.15.26:0/1208724480 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec8036040 con 0x55de9e132dc0 2024-06-13T07:13:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1208724480 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ec4031050 con 0x55de9e132dc0 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7edad78740 10 client.26503 fetch_fsmap learned FSMap version 36 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7edad78740 1 -- 172.21.15.26:0/1208724480 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55de9e1478c0 con 0x55de9e132dc0 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1208724480 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8036040 con 0x55de9e132dc0 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1208724480 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f7ec4020750 con 0x55de9e132dc0 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7edad78740 10 client.26503 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7edad78740 10 client.26503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7edad78740 20 client.26503 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7edad78740 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55de9e148fb0 0x55de9e14b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7edad78740 4 client.26503 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.846+0000 7f7edad78740 1 -- 172.21.15.26:0/1208724480 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de9e0a3b50 con 0x55de9e148fb0 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.847+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55de9e148fb0 0x55de9e14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.847+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55de9e148fb0 0x55de9e14b440 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff37f27f676fb3c server_cookie=b53c66fb4c2b3e4b in_seq=0 out_seq=0 2024-06-13T07:13:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.847+0000 7f7eb6a00640 10 client.26503 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.848+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1208724480 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55de9e0a3b50 con 0x55de9e148fb0 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.848+0000 7f7eb6a00640 10 client.26503 handle_command_reply: tid=0 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.848+0000 7f7edad78740 1 client.26503 shutdown 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 -- 172.21.15.26:0/1208724480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 msgr2=0x55de9e13cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e13cf10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4011c10 tx=0x7f7ec40046c0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 -- 172.21.15.26:0/1208724480 shutdown_connections 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55de9e148fb0 0x55de9e14b440 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e138920 0x55de9e144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e1337b0 0x55de9e13d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7eb6a00640 0 client.26503 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 --2- 172.21.15.26:0/1208724480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e13cf10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 -- 172.21.15.26:0/1208724480 >> 172.21.15.26:0/1208724480 conn(0x55de9e0fafb0 msgr2=0x55de9e12d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 -- 172.21.15.26:0/1208724480 shutdown_connections 2024-06-13T07:13:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 1 -- 172.21.15.26:0/1208724480 wait complete. 2024-06-13T07:13:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.849+0000 7f7edad78740 20 client.26503 trim_cache size 0 max 16384 2024-06-13T07:13:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.871+0000 7f7edad78740 1 Processor -- start 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.871+0000 7f7edad78740 1 -- start start 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.871+0000 7f7edad78740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e132dc0 0x55de9e13d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.871+0000 7f7edad78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e144450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.871+0000 7f7edad78740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e138920 0x55de9e131700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.871+0000 7f7edad78740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01aefd0 con 0x55de9e132dc0 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.871+0000 7f7edad78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01ac8a0 con 0x55de9e1337b0 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.871+0000 7f7edad78740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01aa6f0 con 0x55de9e138920 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e138920 0x55de9e131700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed4a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e138920 0x55de9e131700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35240/0 (socket says 172.21.15.26:35240) 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 -- 172.21.15.26:0/1335288300 learned_addr learned my addr 172.21.15.26:0/1335288300 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5400640 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e132dc0 0x55de9e13d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 -- 172.21.15.26:0/1335288300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 msgr2=0x55de9e144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 -- 172.21.15.26:0/1335288300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e132dc0 msgr2=0x55de9e13d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e132dc0 0x55de9e13d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 -- 172.21.15.26:0/1335288300 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9e1331a0 con 0x55de9e138920 2024-06-13T07:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.872+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e138920 0x55de9e131700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8005e30 tx=0x7f7ec802af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a8118e4193773c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.873+0000 7f7ec1c00640 1 -- 172.21.15.26:0/1335288300 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8031760 con 0x55de9e138920 2024-06-13T07:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.873+0000 7f7ec1c00640 1 -- 172.21.15.26:0/1335288300 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8036040 con 0x55de9e138920 2024-06-13T07:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.873+0000 7f7ec1c00640 1 -- 172.21.15.26:0/1335288300 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80046d0 con 0x55de9e138920 2024-06-13T07:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 -- 172.21.15.26:0/1335288300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e138920 msgr2=0x55de9e131700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e138920 0x55de9e131700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8005e30 tx=0x7f7ec802af40 comp rx=0 tx=0).stop 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 -- 172.21.15.26:0/1335288300 shutdown_connections 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e138920 0x55de9e131700 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 --2- 172.21.15.26:0/1335288300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e132dc0 0x55de9e13d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 -- 172.21.15.26:0/1335288300 >> 172.21.15.26:0/1335288300 conn(0x55de9e0fafb0 msgr2=0x55de9e13cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 -- 172.21.15.26:0/1335288300 shutdown_connections 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.874+0000 7f7edad78740 1 -- 172.21.15.26:0/1335288300 wait complete. 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.875+0000 7f7edad78740 1 -- start start 2024-06-13T07:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.875+0000 7f7edad78740 1 Processor -- start 2024-06-13T07:13:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.875+0000 7f7edad78740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e141710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.875+0000 7f7edad78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e141c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.875+0000 7f7edad78740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e142b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.875+0000 7f7edad78740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01aad60 con 0x55de9e138920 2024-06-13T07:13:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.875+0000 7f7edad78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01ac1d0 con 0x55de9e1337b0 2024-06-13T07:13:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.875+0000 7f7edad78740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01ab430 con 0x55de9e132dc0 2024-06-13T07:13:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7edad78740 10 client.0 _ll_register_callbacks cb 0x7f7ec4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e141710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35252/0 (socket says 172.21.15.26:35252) 2024-06-13T07:13:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed4a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed5400640 1 -- 172.21.15.26:0/1747660320 learned_addr learned my addr 172.21.15.26:0/1747660320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed4a00640 1 -- 172.21.15.26:0/1747660320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 msgr2=0x55de9e141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed4a00640 1 -- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 msgr2=0x55de9e142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed4a00640 1 -- 172.21.15.26:0/1747660320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9e143140 con 0x55de9e1337b0 2024-06-13T07:13:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed5400640 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.876+0000 7f7ed5e00640 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7ed4a00640 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e141c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0ccf70 tx=0x7f7ebc0d54b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5828233605a030a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:13:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1747660320 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc09c3d0 con 0x55de9e1337b0 2024-06-13T07:13:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7edad78740 1 -- 172.21.15.26:0/1747660320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec8036040 con 0x55de9e1337b0 2024-06-13T07:13:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1747660320 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7ebc098340 con 0x55de9e1337b0 2024-06-13T07:13:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1747660320 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ebc09bcd0 con 0x55de9e1337b0 2024-06-13T07:13:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7edad78740 10 client.26480 fetch_fsmap learned FSMap version 36 2024-06-13T07:13:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7edad78740 1 -- 172.21.15.26:0/1747660320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55de9e1444c0 con 0x55de9e1337b0 2024-06-13T07:13:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1747660320 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8036040 con 0x55de9e1337b0 2024-06-13T07:13:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1747660320 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f7ebc09b120 con 0x55de9e1337b0 2024-06-13T07:13:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7edad78740 10 client.26480 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:13:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7edad78740 10 client.26480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7edad78740 20 client.26480 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.877+0000 7f7edad78740 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55de9e1bc940 0x55de9e1bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.878+0000 7f7edad78740 4 client.26480 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:13:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.878+0000 7f7edad78740 1 -- 172.21.15.26:0/1747660320 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de9e09dea0 con 0x55de9e1bc940 2024-06-13T07:13:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.878+0000 7f7ed5400640 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55de9e1bc940 0x55de9e1bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.878+0000 7f7eb6a00640 10 client.26480 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.878+0000 7f7ed5400640 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55de9e1bc940 0x55de9e1bed60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745ade502208f2b8 server_cookie=6bc2ede7e61833d8 in_seq=0 out_seq=0 2024-06-13T07:13:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7eb6a00640 1 -- 172.21.15.26:0/1747660320 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55de9e09dea0 con 0x55de9e1bc940 2024-06-13T07:13:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7eb6a00640 10 client.26480 handle_command_reply: tid=0 2024-06-13T07:13:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 client.26480 shutdown 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 -- 172.21.15.26:0/1747660320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 msgr2=0x55de9e141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e141c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0ccf70 tx=0x7f7ebc0d54b0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 -- 172.21.15.26:0/1747660320 shutdown_connections 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55de9e1bc940 0x55de9e1bed60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de9e1337b0 0x55de9e141c50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de9e138920 0x55de9e142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 --2- 172.21.15.26:0/1747660320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de9e132dc0 0x55de9e141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7eb6a00640 0 client.26480 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 -- 172.21.15.26:0/1747660320 >> 172.21.15.26:0/1747660320 conn(0x55de9e0fafb0 msgr2=0x55de9e131700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.978+0000 7f7edad78740 1 -- 172.21.15.26:0/1747660320 shutdown_connections 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.979+0000 7f7edad78740 1 -- 172.21.15.26:0/1747660320 wait complete. 2024-06-13T07:13:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.979+0000 7f7edad78740 20 client.26480 trim_cache size 0 max 16384 2024-06-13T07:13:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:13:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:13:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.979+0000 7f7ed6800640 1 -- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7eb80a2c00 msgr2=0x7f7eb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.979+0000 7f7ed6800640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7eb80a2c00 0x7f7eb80a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55de9e133bb0 tx=0x7f7ec4005f90 comp rx=0 tx=0).stop 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.979+0000 7f7ed6800640 1 -- 172.21.15.26:0/3024322597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0104c50 msgr2=0x7f7ed019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.979+0000 7f7ed6800640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0104c50 0x7f7ed019a280 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00e9f0 tx=0x7f7ebc00edf0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.980+0000 7f7ed6800640 1 -- 172.21.15.26:0/3024322597 shutdown_connections 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.980+0000 7f7ed6800640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7eb80a2c00 0x7f7eb80a50c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.980+0000 7f7ed6800640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ed0105610 0x7f7ed01a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.980+0000 7f7ed6800640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ed0104c50 0x7f7ed019a280 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.980+0000 7f7ed6800640 1 --2- 172.21.15.26:0/3024322597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed0103a60 0x7f7ed0199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.980+0000 7f7ed6800640 1 -- 172.21.15.26:0/3024322597 >> 172.21.15.26:0/3024322597 conn(0x7f7ed00fefd0 msgr2=0x7f7ed0107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.980+0000 7f7ed6800640 1 -- 172.21.15.26:0/3024322597 shutdown_connections 2024-06-13T07:13:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:17.980+0000 7f7ed6800640 1 -- 172.21.15.26:0/3024322597 wait complete. 2024-06-13T07:13:18.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:17 smithi026 ceph-mon[32041]: pgmap v893: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-13T07:13:18.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:17 smithi135 ceph-mon[36276]: pgmap v893: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-13T07:13:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:17 smithi136 ceph-mon[37229]: pgmap v893: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-13T07:13:18.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 601046 4.75 MB/sec execute 572 sec latency 54.679 ms 2024-06-13T07:13:19.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 601950 4.75 MB/sec execute 573 sec latency 40.643 ms 2024-06-13T07:13:20.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:19 smithi026 ceph-mon[32041]: pgmap v894: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:13:20.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:19 smithi135 ceph-mon[36276]: pgmap v894: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:13:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:19 smithi136 ceph-mon[37229]: pgmap v894: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-13T07:13:20.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 603220 4.76 MB/sec execute 574 sec latency 27.376 ms 2024-06-13T07:13:21.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 604034 4.76 MB/sec execute 575 sec latency 22.278 ms 2024-06-13T07:13:22.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:22 smithi026 ceph-mon[32041]: pgmap v895: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-13T07:13:22.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:22 smithi135 ceph-mon[36276]: pgmap v895: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-13T07:13:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:22 smithi136 ceph-mon[37229]: pgmap v895: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-13T07:13:22.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 604396 4.75 MB/sec execute 576 sec latency 32.942 ms 2024-06-13T07:13:23.589 INFO:tasks.workunit.client.0.smithi026.stdout: 1 605462 4.75 MB/sec execute 577 sec latency 52.009 ms 2024-06-13T07:13:24.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:24 smithi135 ceph-mon[36276]: pgmap v896: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-06-13T07:13:24.378 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:24 smithi136 ceph-mon[37229]: pgmap v896: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-06-13T07:13:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:24 smithi026 ceph-mon[32041]: pgmap v896: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-06-13T07:13:24.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 606723 4.76 MB/sec execute 578 sec latency 35.583 ms 2024-06-13T07:13:25.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 607601 4.76 MB/sec execute 579 sec latency 26.127 ms 2024-06-13T07:13:26.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:26 smithi135 ceph-mon[36276]: pgmap v897: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:13:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:26 smithi136 ceph-mon[37229]: pgmap v897: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:13:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:26 smithi026 ceph-mon[32041]: pgmap v897: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-13T07:13:26.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 608077 4.75 MB/sec execute 580 sec latency 24.947 ms 2024-06-13T07:13:27.590 INFO:tasks.workunit.client.0.smithi026.stdout: 1 609012 4.75 MB/sec execute 581 sec latency 23.534 ms 2024-06-13T07:13:28.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:28 smithi135 ceph-mon[36276]: pgmap v898: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:13:28.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:28 smithi136 ceph-mon[37229]: pgmap v898: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:13:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:28 smithi026 ceph-mon[32041]: pgmap v898: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:13:28.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 610332 4.76 MB/sec execute 582 sec latency 47.893 ms 2024-06-13T07:13:29.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 611267 4.76 MB/sec execute 583 sec latency 22.889 ms 2024-06-13T07:13:30.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:30 smithi135 ceph-mon[36276]: pgmap v899: 129 pgs: 129 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:13:30.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:30 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:13:30.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:30 smithi136 ceph-mon[37229]: pgmap v899: 129 pgs: 129 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:13:30.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:30 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:13:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:30 smithi026 ceph-mon[32041]: pgmap v899: 129 pgs: 129 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-06-13T07:13:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:30 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:13:30.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 612081 4.75 MB/sec execute 584 sec latency 28.219 ms 2024-06-13T07:13:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 -- 172.21.15.26:0/1175508341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 msgr2=0x7f4b40109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 --2- 172.21.15.26:0/1175508341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 0x7f4b40109950 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009ca0 tx=0x7f4b3802a8c0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 -- 172.21.15.26:0/1175508341 shutdown_connections 2024-06-13T07:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 --2- 172.21.15.26:0/1175508341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 0x7f4b40109950 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 --2- 172.21.15.26:0/1175508341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40104bd0 0x7f4b40106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 --2- 172.21.15.26:0/1175508341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b401022a0 0x7f4b40104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 -- 172.21.15.26:0/1175508341 >> 172.21.15.26:0/1175508341 conn(0x7f4b400fbc90 msgr2=0x7f4b400fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 -- 172.21.15.26:0/1175508341 shutdown_connections 2024-06-13T07:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.135+0000 7f4b46800640 1 -- 172.21.15.26:0/1175508341 wait complete. 2024-06-13T07:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.137+0000 7f4b46800640 1 Processor -- start 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.137+0000 7f4b46800640 1 -- start start 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.137+0000 7f4b46800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b401022a0 0x7f4b40197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.137+0000 7f4b46800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40104bd0 0x7f4b40198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.137+0000 7f4b46800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 0x7f4b4019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.137+0000 7f4b46800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010d4c0 con 0x7f4b40104bd0 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.137+0000 7f4b46800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010d340 con 0x7f4b40107500 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.137+0000 7f4b46800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010d640 con 0x7f4b401022a0 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b401022a0 0x7f4b40197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b401022a0 0x7f4b40197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38024/0 (socket says 172.21.15.26:38024) 2024-06-13T07:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45400640 1 -- 172.21.15.26:0/3172774003 learned_addr learned my addr 172.21.15.26:0/3172774003 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45e00640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 0x7f4b4019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40104bd0 0x7f4b40198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45400640 1 -- 172.21.15.26:0/3172774003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 msgr2=0x7f4b4019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45400640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 0x7f4b4019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45400640 1 -- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40104bd0 msgr2=0x7f4b40198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45400640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40104bd0 0x7f4b40198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45400640 1 -- 172.21.15.26:0/3172774003 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b40100110 con 0x7f4b401022a0 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b45e00640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 0x7f4b4019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.138+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40104bd0 0x7f4b40198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.139+0000 7f4b45400640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b401022a0 0x7f4b40197b00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400e9f0 tx=0x7f4b3400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.139+0000 7f4b3e000640 1 -- 172.21.15.26:0/3172774003 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3400cd80 con 0x7f4b401022a0 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.139+0000 7f4b3e000640 1 -- 172.21.15.26:0/3172774003 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b34005500 con 0x7f4b401022a0 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.139+0000 7f4b46800640 1 -- 172.21.15.26:0/3172774003 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b40100400 con 0x7f4b401022a0 2024-06-13T07:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.139+0000 7f4b3e000640 1 -- 172.21.15.26:0/3172774003 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34010690 con 0x7f4b401022a0 2024-06-13T07:13:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.139+0000 7f4b46800640 1 -- 172.21.15.26:0/3172774003 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b40100900 con 0x7f4b401022a0 2024-06-13T07:13:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.143+0000 7f4b3e000640 1 -- 172.21.15.26:0/3172774003 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b340046c0 con 0x7f4b401022a0 2024-06-13T07:13:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.143+0000 7f4b4ac24740 1 Processor -- start 2024-06-13T07:13:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b4ac24740 1 -- start start 2024-06-13T07:13:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253bddc0 0x5606253be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253be7b0 0x5606253c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560625321420 con 0x5606253bddc0 2024-06-13T07:13:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606252c8c50 con 0x5606253c3920 2024-06-13T07:13:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56062531d3e0 con 0x5606253be7b0 2024-06-13T07:13:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b45400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253be7b0 0x5606253c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253bddc0 0x5606253be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53302/0 (socket says 172.21.15.26:53302) 2024-06-13T07:13:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253bddc0 0x5606253be1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35324/0 (socket says 172.21.15.26:35324) 2024-06-13T07:13:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b44a00640 1 -- 172.21.15.26:0/3699994624 learned_addr learned my addr 172.21.15.26:0/3699994624 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.144+0000 7f4b45400640 1 --2- 172.21.15.26:0/3699994624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253be7b0 0x5606253c32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38038/0 (socket says 172.21.15.26:38038) 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.145+0000 7f4b44a00640 1 -- 172.21.15.26:0/3699994624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253be7b0 msgr2=0x5606253c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.145+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3699994624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253be7b0 0x5606253c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.145+0000 7f4b44a00640 1 -- 172.21.15.26:0/3699994624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253bddc0 msgr2=0x5606253be1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.145+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3699994624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253bddc0 0x5606253be1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.145+0000 7f4b44a00640 1 -- 172.21.15.26:0/3699994624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560625035210 con 0x5606253c3920 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.145+0000 7f4b3e000640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b280a2b80 0x7f4b280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.145+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3699994624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253c5db0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009ca0 tx=0x7f4b2c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=438feeb4aec2a993 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.145+0000 7f4b3e000640 1 -- 172.21.15.26:0/3172774003 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4b340cf9f0 con 0x7f4b401022a0 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.146+0000 7f4b45e00640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b280a2b80 0x7f4b280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.146+0000 7f4b32600640 1 -- 172.21.15.26:0/3699994624 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c004030 con 0x5606253c3920 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.146+0000 7f4b32600640 1 -- 172.21.15.26:0/3699994624 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0041d0 con 0x5606253c3920 2024-06-13T07:13:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.146+0000 7f4b32600640 1 -- 172.21.15.26:0/3699994624 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c012780 con 0x5606253c3920 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.146+0000 7f4b45e00640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b280a2b80 0x7f4b280a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5606253bf010 tx=0x7f4b38031bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.146+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3699994624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 msgr2=0x5606253c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3699994624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253c5db0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009ca0 tx=0x7f4b2c010740 comp rx=0 tx=0).stop 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3699994624 shutdown_connections 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3699994624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253c5db0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3699994624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253be7b0 0x5606253c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3699994624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253bddc0 0x5606253be1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3699994624 >> 172.21.15.26:0/3699994624 conn(0x560625385fb0 msgr2=0x5606253b9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3699994624 shutdown_connections 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3699994624 wait complete. 2024-06-13T07:13:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.147+0000 7f4b4ac24740 1 -- start start 2024-06-13T07:13:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b4ac24740 1 Processor -- start 2024-06-13T07:13:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5606253c68d0 con 0x5606253be7b0 2024-06-13T07:13:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606252dc0b0 con 0x5606253c3920 2024-06-13T07:13:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5606253c6a50 con 0x5606253bddc0 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b4ac24740 10 client.0 _ll_register_callbacks cb 0x5606252c7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.148+0000 7f4b45400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b45400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38052/0 (socket says 172.21.15.26:38052) 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b45400640 1 -- 172.21.15.26:0/3886121776 learned_addr learned my addr 172.21.15.26:0/3886121776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b45e00640 1 -- 172.21.15.26:0/3886121776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 msgr2=0x5606253d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b45e00640 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b45e00640 1 -- 172.21.15.26:0/3886121776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 msgr2=0x5606253d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b45e00640 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.149+0000 7f4b45e00640 1 -- 172.21.15.26:0/3886121776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560625101620 con 0x5606253c3920 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b45e00640 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253d7e80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4b38031ae0 tx=0x7f4b3802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7da950b9f775550 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b45400640 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b26a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b26a00640 1 -- 172.21.15.26:0/3886121776 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38003e00 con 0x5606253c3920 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b26a00640 1 -- 172.21.15.26:0/3886121776 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4b38004510 con 0x5606253c3920 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3886121776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c0041d0 con 0x5606253c3920 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b26a00640 1 -- 172.21.15.26:0/3886121776 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4b38004710 con 0x5606253c3920 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b4ac24740 10 client.26492 fetch_fsmap learned FSMap version 36 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3886121776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5606253db260 con 0x5606253c3920 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.150+0000 7f4b26a00640 1 -- 172.21.15.26:0/3886121776 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0041d0 con 0x5606253c3920 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.151+0000 7f4b26a00640 1 -- 172.21.15.26:0/3886121776 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f4b380306c0 con 0x5606253c3920 2024-06-13T07:13:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.151+0000 7f4b4ac24740 10 client.26492 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:13:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.151+0000 7f4b4ac24740 10 client.26492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.151+0000 7f4b4ac24740 20 client.26492 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.151+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606253dc940 0x5606253dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.151+0000 7f4b4ac24740 4 client.26492 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:13:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.151+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3886121776 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56062532eb50 con 0x5606253dc940 2024-06-13T07:13:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.151+0000 7f4b45400640 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606253dc940 0x5606253dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.152+0000 7f4b45400640 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606253dc940 0x5606253dedd0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed72c9826087310 server_cookie=9e5b5c12868882d3 in_seq=0 out_seq=0 2024-06-13T07:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.152+0000 7f4b26a00640 10 client.26492 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.153+0000 7f4b26a00640 1 -- 172.21.15.26:0/3886121776 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56062532eb50 con 0x5606253dc940 2024-06-13T07:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.153+0000 7f4b26a00640 10 client.26492 handle_command_reply: tid=0 2024-06-13T07:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.153+0000 7f4b4ac24740 1 client.26492 shutdown 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3886121776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 msgr2=0x5606253d7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253d7e80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4b38031ae0 tx=0x7f4b3802a8c0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3886121776 shutdown_connections 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606253dc940 0x5606253dedd0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606253d7e80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3886121776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3886121776 >> 172.21.15.26:0/3886121776 conn(0x560625385fb0 msgr2=0x5606253b89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b26a00640 0 client.26492 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3886121776 shutdown_connections 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3886121776 wait complete. 2024-06-13T07:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.154+0000 7f4b4ac24740 20 client.26492 trim_cache size 0 max 16384 2024-06-13T07:13:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.174+0000 7f4b4ac24740 1 Processor -- start 2024-06-13T07:13:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.174+0000 7f4b4ac24740 1 -- start start 2024-06-13T07:13:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253be7b0 0x5606253d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253c3920 0x5606253bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401aac30 con 0x5606253c3920 2024-06-13T07:13:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401a8500 con 0x5606253be7b0 2024-06-13T07:13:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401a6350 con 0x5606253bddc0 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253be7b0 0x5606253d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253c3920 0x5606253bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b45400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253be7b0 0x5606253d7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53326/0 (socket says 172.21.15.26:53326) 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.175+0000 7f4b44a00640 1 -- 172.21.15.26:0/2860354167 learned_addr learned my addr 172.21.15.26:0/2860354167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b44a00640 1 -- 172.21.15.26:0/2860354167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 msgr2=0x5606253bc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b44a00640 1 --2- 172.21.15.26:0/2860354167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253bc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b44a00640 1 -- 172.21.15.26:0/2860354167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253c3920 msgr2=0x5606253bcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b44a00640 1 --2- 172.21.15.26:0/2860354167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253c3920 0x5606253bcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b44a00640 1 -- 172.21.15.26:0/2860354167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606253ba4f0 con 0x5606253be7b0 2024-06-13T07:13:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b44a00640 1 --2- 172.21.15.26:0/2860354167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253be7b0 0x5606253d7e40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c019000 tx=0x7f4b2c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e515cbc65f9a4bd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b31c00640 1 -- 172.21.15.26:0/2860354167 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c029070 con 0x5606253be7b0 2024-06-13T07:13:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b31c00640 1 -- 172.21.15.26:0/2860354167 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0055a0 con 0x5606253be7b0 2024-06-13T07:13:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.176+0000 7f4b31c00640 1 -- 172.21.15.26:0/2860354167 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0126b0 con 0x5606253be7b0 2024-06-13T07:13:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.177+0000 7f4b4ac24740 1 -- 172.21.15.26:0/2860354167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253be7b0 msgr2=0x5606253d7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.177+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/2860354167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253be7b0 0x5606253d7e40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c019000 tx=0x7f4b2c009ba0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.177+0000 7f4b4ac24740 1 -- 172.21.15.26:0/2860354167 shutdown_connections 2024-06-13T07:13:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.177+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/2860354167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253be7b0 0x5606253d7e40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.177+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/2860354167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253c3920 0x5606253bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.177+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/2860354167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253bc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.177+0000 7f4b4ac24740 1 -- 172.21.15.26:0/2860354167 >> 172.21.15.26:0/2860354167 conn(0x560625385fb0 msgr2=0x5606253bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.177+0000 7f4b4ac24740 1 -- 172.21.15.26:0/2860354167 shutdown_connections 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.178+0000 7f4b4ac24740 1 -- 172.21.15.26:0/2860354167 wait complete. 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.178+0000 7f4b4ac24740 1 -- start start 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.178+0000 7f4b4ac24740 1 Processor -- start 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.178+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.178+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b4ac24740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606254456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401a69c0 con 0x5606253be7b0 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401a7e30 con 0x5606253c3920 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b4ac24740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401a7090 con 0x5606253bddc0 2024-06-13T07:13:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b4ac24740 10 client.0 _ll_register_callbacks cb 0x7f4b34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606254456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38074/0 (socket says 172.21.15.26:38074) 2024-06-13T07:13:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45400640 1 -- 172.21.15.26:0/3513889561 learned_addr learned my addr 172.21.15.26:0/3513889561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45400640 1 -- 172.21.15.26:0/3513889561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 msgr2=0x5606254456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45400640 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606254456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45400640 1 -- 172.21.15.26:0/3513889561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 msgr2=0x5606253d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45400640 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45400640 1 -- 172.21.15.26:0/3513889561 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560625445be0 con 0x5606253bddc0 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b45e00640 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606254456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.179+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.180+0000 7f4b45400640 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d4bc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b340049d0 tx=0x7f4b34005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17ed500b5ac9d7e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.180+0000 7f4b26a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:13:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.180+0000 7f4b26a00640 1 -- 172.21.15.26:0/3513889561 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3409be00 con 0x5606253bddc0 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.180+0000 7f4b26a00640 1 -- 172.21.15.26:0/3513889561 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4b34016df0 con 0x5606253bddc0 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.180+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3513889561 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c0055a0 con 0x5606253bddc0 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.180+0000 7f4b26a00640 1 -- 172.21.15.26:0/3513889561 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4b340a01b0 con 0x5606253bddc0 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.180+0000 7f4b4ac24740 10 client.26527 fetch_fsmap learned FSMap version 36 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.180+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3513889561 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560625446e00 con 0x5606253bddc0 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.181+0000 7f4b26a00640 1 -- 172.21.15.26:0/3513889561 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0055a0 con 0x5606253bddc0 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.181+0000 7f4b26a00640 1 -- 172.21.15.26:0/3513889561 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f4b340a45e0 con 0x5606253bddc0 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.181+0000 7f4b4ac24740 10 client.26527 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.181+0000 7f4b4ac24740 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.181+0000 7f4b4ac24740 20 client.26527 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.181+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560625447790 0x560625449bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.181+0000 7f4b4ac24740 4 client.26527 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:13:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.181+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3513889561 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560625328ea0 con 0x560625447790 2024-06-13T07:13:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.182+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560625447790 0x560625449bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.182+0000 7f4b44a00640 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560625447790 0x560625449bb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579ad1910bff275f server_cookie=85cc8814d6c42836 in_seq=0 out_seq=0 2024-06-13T07:13:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.182+0000 7f4b26a00640 10 client.26527 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.269+0000 7f4b26a00640 1 -- 172.21.15.26:0/3513889561 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560625328ea0 con 0x560625447790 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.269+0000 7f4b26a00640 10 client.26527 handle_command_reply: tid=0 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.269+0000 7f4b4ac24740 1 client.26527 shutdown 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.269+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3513889561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 msgr2=0x5606253d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d4bc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b340049d0 tx=0x7f4b34005760 comp rx=0 tx=0).stop 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3513889561 shutdown_connections 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560625447790 0x560625449bb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606253c3920 0x5606254456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606253be7b0 0x5606253d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 --2- 172.21.15.26:0/3513889561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606253bddc0 0x5606253d4bc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3513889561 >> 172.21.15.26:0/3513889561 conn(0x560625385fb0 msgr2=0x5606253bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b26a00640 0 client.26527 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3513889561 shutdown_connections 2024-06-13T07:13:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 1 -- 172.21.15.26:0/3513889561 wait complete. 2024-06-13T07:13:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.270+0000 7f4b4ac24740 20 client.26527 trim_cache size 0 max 16384 2024-06-13T07:13:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:13:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:13:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:13:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.272+0000 7f4b46800640 1 -- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b280a2b80 msgr2=0x7f4b280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.272+0000 7f4b46800640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b280a2b80 0x7f4b280a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5606253bf010 tx=0x7f4b38031bd0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.272+0000 7f4b46800640 1 -- 172.21.15.26:0/3172774003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b401022a0 msgr2=0x7f4b40197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.272+0000 7f4b46800640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b401022a0 0x7f4b40197b00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400e9f0 tx=0x7f4b3400edf0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.273+0000 7f4b46800640 1 -- 172.21.15.26:0/3172774003 shutdown_connections 2024-06-13T07:13:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.273+0000 7f4b46800640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4b280a2b80 0x7f4b280a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.273+0000 7f4b46800640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40107500 0x7f4b4019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.273+0000 7f4b46800640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b40104bd0 0x7f4b40198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.273+0000 7f4b46800640 1 --2- 172.21.15.26:0/3172774003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b401022a0 0x7f4b40197b00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.273+0000 7f4b46800640 1 -- 172.21.15.26:0/3172774003 >> 172.21.15.26:0/3172774003 conn(0x7f4b400fbc90 msgr2=0x7f4b40102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.273+0000 7f4b46800640 1 -- 172.21.15.26:0/3172774003 shutdown_connections 2024-06-13T07:13:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:31.274+0000 7f4b46800640 1 -- 172.21.15.26:0/3172774003 wait complete. 2024-06-13T07:13:31.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:13:31.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:13:31.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:31 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:13:31.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:13:31.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:13:31.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:31 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:13:31.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:13:31.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:13:31.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:13:31.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:31 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:13:31.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 613792 4.76 MB/sec execute 585 sec latency 30.174 ms 2024-06-13T07:13:32.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:32 smithi135 ceph-mon[36276]: pgmap v900: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:13:32.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:32 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:13:32.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:32 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:13:32.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:32 smithi136 ceph-mon[37229]: pgmap v900: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:13:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:32 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:13:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:32 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:13:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:32 smithi026 ceph-mon[32041]: pgmap v900: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-06-13T07:13:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:32 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:13:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:32 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:13:32.591 INFO:tasks.workunit.client.0.smithi026.stdout: 1 614685 4.77 MB/sec execute 586 sec latency 19.662 ms 2024-06-13T07:13:33.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 615181 4.76 MB/sec execute 587 sec latency 35.320 ms 2024-06-13T07:13:34.305 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:34 smithi026 ceph-mon[32041]: pgmap v901: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-06-13T07:13:34.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:34 smithi136 ceph-mon[37229]: pgmap v901: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-06-13T07:13:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:34 smithi135 ceph-mon[36276]: pgmap v901: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-06-13T07:13:34.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 616298 4.76 MB/sec execute 588 sec latency 31.629 ms 2024-06-13T07:13:35.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 617603 4.77 MB/sec execute 589 sec latency 26.208 ms 2024-06-13T07:13:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:35 smithi026 ceph-mon[32041]: pgmap v902: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-06-13T07:13:35.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:35 smithi135 ceph-mon[36276]: pgmap v902: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-06-13T07:13:35.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:35 smithi136 ceph-mon[37229]: pgmap v902: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-06-13T07:13:36.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 618345 4.77 MB/sec execute 590 sec latency 25.674 ms 2024-06-13T07:13:37.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 618886 4.76 MB/sec execute 591 sec latency 31.379 ms 2024-06-13T07:13:38.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:37 smithi026 ceph-mon[32041]: pgmap v903: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-06-13T07:13:38.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:37 smithi135 ceph-mon[36276]: pgmap v903: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-06-13T07:13:38.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:37 smithi136 ceph-mon[37229]: pgmap v903: 129 pgs: 129 active+clean; 129 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-06-13T07:13:38.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 620284 4.76 MB/sec execute 592 sec latency 32.939 ms 2024-06-13T07:13:39.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 621347 4.77 MB/sec execute 593 sec latency 21.692 ms 2024-06-13T07:13:40.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:39 smithi026 ceph-mon[32041]: pgmap v904: 129 pgs: 129 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:13:40.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:39 smithi135 ceph-mon[36276]: pgmap v904: 129 pgs: 129 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:13:40.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:39 smithi136 ceph-mon[37229]: pgmap v904: 129 pgs: 129 active+clean; 130 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-06-13T07:13:40.592 INFO:tasks.workunit.client.0.smithi026.stdout: 1 621954 4.77 MB/sec execute 594 sec latency 21.710 ms 2024-06-13T07:13:41.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 622586 4.76 MB/sec execute 595 sec latency 28.744 ms 2024-06-13T07:13:42.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:41 smithi026 ceph-mon[32041]: pgmap v905: 129 pgs: 129 active+clean; 131 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-13T07:13:42.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:41 smithi135 ceph-mon[36276]: pgmap v905: 129 pgs: 129 active+clean; 131 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-13T07:13:42.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:41 smithi136 ceph-mon[37229]: pgmap v905: 129 pgs: 129 active+clean; 131 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-13T07:13:42.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 623968 4.76 MB/sec execute 596 sec latency 20.805 ms 2024-06-13T07:13:43.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 625079 4.77 MB/sec execute 597 sec latency 74.659 ms 2024-06-13T07:13:44.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:43 smithi026 ceph-mon[32041]: pgmap v906: 129 pgs: 129 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:13:44.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:43 smithi135 ceph-mon[36276]: pgmap v906: 129 pgs: 129 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:13:44.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:43 smithi136 ceph-mon[37229]: pgmap v906: 129 pgs: 129 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-13T07:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 -- 172.21.15.26:0/4021982242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14541089a0 msgr2=0x7f1454108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 --2- 172.21.15.26:0/4021982242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14541089a0 0x7f1454108dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f143c009900 tx=0x7f143c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 -- 172.21.15.26:0/4021982242 shutdown_connections 2024-06-13T07:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 --2- 172.21.15.26:0/4021982242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1454109300 0x7f145410fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 --2- 172.21.15.26:0/4021982242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1454110220 0x7f1454112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 --2- 172.21.15.26:0/4021982242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14541089a0 0x7f1454108dc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 -- 172.21.15.26:0/4021982242 >> 172.21.15.26:0/4021982242 conn(0x7f145406ebd0 msgr2=0x7f1454071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 -- 172.21.15.26:0/4021982242 shutdown_connections 2024-06-13T07:13:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.447+0000 7f145b400640 1 -- 172.21.15.26:0/4021982242 wait complete. 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.448+0000 7f145b400640 1 Processor -- start 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.448+0000 7f145b400640 1 -- start start 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.448+0000 7f145b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14541089a0 0x7f14541a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.448+0000 7f145b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1454109300 0x7f14541a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f145b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1454110220 0x7f14541a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f145b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1454115ea0 con 0x7f14541089a0 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f145b400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1454115d20 con 0x7f1454109300 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f145b400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1454116020 con 0x7f1454110220 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f1458a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14541089a0 0x7f14541a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f1458a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14541089a0 0x7f14541a2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40540/0 (socket says 172.21.15.26:40540) 2024-06-13T07:13:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f1458a00640 1 -- 172.21.15.26:0/3258706700 learned_addr learned my addr 172.21.15.26:0/3258706700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f1459400640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1454110220 0x7f14541a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f144be00640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1454109300 0x7f14541a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f144be00640 1 -- 172.21.15.26:0/3258706700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1454110220 msgr2=0x7f14541a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f144be00640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1454110220 0x7f14541a9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f144be00640 1 -- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14541089a0 msgr2=0x7f14541a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f144be00640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14541089a0 0x7f14541a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f144be00640 1 -- 172.21.15.26:0/3258706700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14541aa270 con 0x7f1454109300 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f1458a00640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14541089a0 0x7f14541a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.449+0000 7f1459400640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1454110220 0x7f14541a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.450+0000 7f144be00640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1454109300 0x7f14541a2b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f144400e9c0 tx=0x7f144400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.450+0000 7f1452000640 1 -- 172.21.15.26:0/3258706700 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144400ccc0 con 0x7f1454109300 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.450+0000 7f1452000640 1 -- 172.21.15.26:0/3258706700 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14440035f0 con 0x7f1454109300 2024-06-13T07:13:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.450+0000 7f145b400640 1 -- 172.21.15.26:0/3258706700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14541aa560 con 0x7f1454109300 2024-06-13T07:13:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.450+0000 7f1452000640 1 -- 172.21.15.26:0/3258706700 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144400ce60 con 0x7f1454109300 2024-06-13T07:13:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.450+0000 7f145b400640 1 -- 172.21.15.26:0/3258706700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14541aaab0 con 0x7f1454109300 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.454+0000 7f145e363740 1 Processor -- start 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.454+0000 7f145e363740 1 -- start start 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.454+0000 7f1452000640 1 -- 172.21.15.26:0/3258706700 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f14440037a0 con 0x7f1454109300 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f145e363740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef446020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f145e363740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef44b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f145e363740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef44dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f145e363740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5653ef3a9420 con 0x5653ef44b760 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f145e363740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653ef350c50 con 0x5653ef4465f0 2024-06-13T07:13:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f145e363740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5653ef3a53e0 con 0x5653ef445c00 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f1459400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef446020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f1459400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef446020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54384/0 (socket says 172.21.15.26:54384) 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f1459400640 1 -- 172.21.15.26:0/2346452989 learned_addr learned my addr 172.21.15.26:0/2346452989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f1458a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef44b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f144be00640 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef44dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f1459400640 1 -- 172.21.15.26:0/2346452989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 msgr2=0x5653ef44b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f1459400640 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef44b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.455+0000 7f1459400640 1 -- 172.21.15.26:0/2346452989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 msgr2=0x5653ef44dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.456+0000 7f1459400640 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef44dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.456+0000 7f1459400640 1 -- 172.21.15.26:0/2346452989 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ef0bd210 con 0x5653ef445c00 2024-06-13T07:13:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.456+0000 7f144be00640 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef44dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.456+0000 7f1459400640 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef446020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f144c0107e0 tx=0x7f144c010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b7b58450fd43761 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.456+0000 7f1449c00640 1 -- 172.21.15.26:0/2346452989 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c003b70 con 0x5653ef445c00 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.456+0000 7f1449c00640 1 -- 172.21.15.26:0/2346452989 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f144c00ee10 con 0x5653ef445c00 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.456+0000 7f1449c00640 1 -- 172.21.15.26:0/2346452989 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c011440 con 0x5653ef445c00 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f1452000640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14340a2c30 0x7f14340a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f1452000640 1 -- 172.21.15.26:0/3258706700 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f14440cf7c0 con 0x7f1454109300 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f1458a00640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14340a2c30 0x7f14340a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 -- 172.21.15.26:0/2346452989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 msgr2=0x5653ef446020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef446020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f144c0107e0 tx=0x7f144c010ca0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 -- 172.21.15.26:0/2346452989 shutdown_connections 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef44b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef44dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 --2- 172.21.15.26:0/2346452989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef446020 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 -- 172.21.15.26:0/2346452989 >> 172.21.15.26:0/2346452989 conn(0x5653ef40dfb0 msgr2=0x5653ef441680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 -- 172.21.15.26:0/2346452989 shutdown_connections 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.457+0000 7f145e363740 1 -- 172.21.15.26:0/2346452989 wait complete. 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.458+0000 7f1458a00640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14340a2c30 0x7f14340a50f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5653ef447060 tx=0x7f143c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.458+0000 7f145e363740 1 -- start start 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.458+0000 7f145e363740 1 Processor -- start 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.458+0000 7f145e363740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef445c00 0x5653ef450030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f145e363740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef4465f0 0x5653ef450570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f145e363740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef4574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f145e363740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5653ef44e710 con 0x5653ef44b760 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f145e363740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653ef3640b0 con 0x5653ef445c00 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f145e363740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5653ef44e890 con 0x5653ef4465f0 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f145e363740 10 client.0 _ll_register_callbacks cb 0x5653ef34ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1458a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef445c00 0x5653ef450030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1458a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef445c00 0x5653ef450030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:45942/0 (socket says 172.21.15.26:45942) 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1458a00640 1 -- 172.21.15.26:0/1029947133 learned_addr learned my addr 172.21.15.26:0/1029947133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f144be00640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef4465f0 0x5653ef450570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1459400640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef4574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1458a00640 1 -- 172.21.15.26:0/1029947133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef4465f0 msgr2=0x5653ef450570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1458a00640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef4465f0 0x5653ef450570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1458a00640 1 -- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 msgr2=0x5653ef4574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1458a00640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef4574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1458a00640 1 -- 172.21.15.26:0/1029947133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ef189620 con 0x5653ef445c00 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f144be00640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef4465f0 0x5653ef450570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.459+0000 7f1459400640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef4574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f1458a00640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef445c00 0x5653ef450030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f143c02a9e0 tx=0x7f143c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56d32eef2a117f64 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f1442000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f1442000640 1 -- 172.21.15.26:0/1029947133 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c035040 con 0x5653ef445c00 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f145e363740 1 -- 172.21.15.26:0/1029947133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f144c00ee10 con 0x5653ef445c00 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f1442000640 1 -- 172.21.15.26:0/1029947133 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f143c051470 con 0x5653ef445c00 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f1442000640 1 -- 172.21.15.26:0/1029947133 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f143c05a050 con 0x5653ef445c00 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f145e363740 10 client.26510 fetch_fsmap learned FSMap version 36 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f145e363740 1 -- 172.21.15.26:0/1029947133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5653ef45a870 con 0x5653ef445c00 2024-06-13T07:13:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.460+0000 7f1442000640 1 -- 172.21.15.26:0/1029947133 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f144c00ee10 con 0x5653ef445c00 2024-06-13T07:13:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.461+0000 7f1442000640 1 -- 172.21.15.26:0/1029947133 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f143c044030 con 0x5653ef445c00 2024-06-13T07:13:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.461+0000 7f145e363740 10 client.26510 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:13:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.461+0000 7f145e363740 10 client.26510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.461+0000 7f145e363740 20 client.26510 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.461+0000 7f145e363740 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5653ef45bf00 0x5653ef45e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.461+0000 7f145e363740 4 client.26510 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:13:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.461+0000 7f145e363740 1 -- 172.21.15.26:0/1029947133 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ef3b6b50 con 0x5653ef45bf00 2024-06-13T07:13:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.461+0000 7f144be00640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5653ef45bf00 0x5653ef45e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.462+0000 7f144be00640 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5653ef45bf00 0x5653ef45e390 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680d5bbce9d0305 server_cookie=8410e17ed0473d13 in_seq=0 out_seq=0 2024-06-13T07:13:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.462+0000 7f1442000640 10 client.26510 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.462+0000 7f1442000640 1 -- 172.21.15.26:0/1029947133 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5653ef3b6b50 con 0x5653ef45bf00 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.462+0000 7f1442000640 10 client.26510 handle_command_reply: tid=0 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 client.26510 shutdown 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 -- 172.21.15.26:0/1029947133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef445c00 msgr2=0x5653ef450030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef445c00 0x5653ef450030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f143c02a9e0 tx=0x7f143c009260 comp rx=0 tx=0).stop 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 -- 172.21.15.26:0/1029947133 shutdown_connections 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5653ef45bf00 0x5653ef45e390 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef4574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef4465f0 0x5653ef450570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 --2- 172.21.15.26:0/1029947133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef445c00 0x5653ef450030 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 -- 172.21.15.26:0/1029947133 >> 172.21.15.26:0/1029947133 conn(0x5653ef40dfb0 msgr2=0x5653ef440840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f1442000640 0 client.26510 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 -- 172.21.15.26:0/1029947133 shutdown_connections 2024-06-13T07:13:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 1 -- 172.21.15.26:0/1029947133 wait complete. 2024-06-13T07:13:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.463+0000 7f145e363740 20 client.26510 trim_cache size 0 max 16384 2024-06-13T07:13:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.483+0000 7f145e363740 1 Processor -- start 2024-06-13T07:13:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f145e363740 1 -- start start 2024-06-13T07:13:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f145e363740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef447090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f145e363740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef457490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f145e363740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef444d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f145e363740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14541b7970 con 0x5653ef44b760 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f145e363740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14541b5240 con 0x5653ef4465f0 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f145e363740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f14541b3090 con 0x5653ef445c00 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f1458a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef447090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f1458a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef447090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54404/0 (socket says 172.21.15.26:54404) 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f1459400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef444d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f1458a00640 1 -- 172.21.15.26:0/1494307670 learned_addr learned my addr 172.21.15.26:0/1494307670 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.484+0000 7f144be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef457490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1458a00640 1 -- 172.21.15.26:0/1494307670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 msgr2=0x5653ef457490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1458a00640 1 --2- 172.21.15.26:0/1494307670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef457490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1458a00640 1 -- 172.21.15.26:0/1494307670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 msgr2=0x5653ef444d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1458a00640 1 --2- 172.21.15.26:0/1494307670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef444d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1458a00640 1 -- 172.21.15.26:0/1494307670 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ef446e80 con 0x5653ef445c00 2024-06-13T07:13:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f144be00640 1 --2- 172.21.15.26:0/1494307670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef457490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1458a00640 1 --2- 172.21.15.26:0/1494307670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef447090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f143c042040 tx=0x7f143c005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09e7bc0e726ef45 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1449200640 1 -- 172.21.15.26:0/1494307670 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c003d50 con 0x5653ef445c00 2024-06-13T07:13:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1449200640 1 -- 172.21.15.26:0/1494307670 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c04c070 con 0x5653ef445c00 2024-06-13T07:13:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.485+0000 7f1449200640 1 -- 172.21.15.26:0/1494307670 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c048400 con 0x5653ef445c00 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 -- 172.21.15.26:0/1494307670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 msgr2=0x5653ef447090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 --2- 172.21.15.26:0/1494307670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef447090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f143c042040 tx=0x7f143c005890 comp rx=0 tx=0).stop 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 -- 172.21.15.26:0/1494307670 shutdown_connections 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 --2- 172.21.15.26:0/1494307670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef4465f0 0x5653ef457490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 --2- 172.21.15.26:0/1494307670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef44b760 0x5653ef444d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 --2- 172.21.15.26:0/1494307670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef447090 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 -- 172.21.15.26:0/1494307670 >> 172.21.15.26:0/1494307670 conn(0x5653ef40dfb0 msgr2=0x5653ef444630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 -- 172.21.15.26:0/1494307670 shutdown_connections 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.486+0000 7f145e363740 1 -- 172.21.15.26:0/1494307670 wait complete. 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.487+0000 7f145e363740 1 -- start start 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.487+0000 7f145e363740 1 Processor -- start 2024-06-13T07:13:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.487+0000 7f145e363740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef4546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.487+0000 7f145e363740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef4465f0 0x5653ef454c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.487+0000 7f145e363740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef44b760 0x5653ef455c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.487+0000 7f145e363740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14541b3700 con 0x5653ef4465f0 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.487+0000 7f145e363740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14541b4b70 con 0x5653ef44b760 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.487+0000 7f145e363740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f14541b3dd0 con 0x5653ef445c00 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f145e363740 10 client.0 _ll_register_callbacks cb 0x7f143c0318b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef4546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef4546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54408/0 (socket says 172.21.15.26:54408) 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 -- 172.21.15.26:0/294636581 learned_addr learned my addr 172.21.15.26:0/294636581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1459400640 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef44b760 0x5653ef455c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f144be00640 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef4465f0 0x5653ef454c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 -- 172.21.15.26:0/294636581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef44b760 msgr2=0x5653ef455c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef44b760 0x5653ef455c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 -- 172.21.15.26:0/294636581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef4465f0 msgr2=0x5653ef454c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef4465f0 0x5653ef454c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 -- 172.21.15.26:0/294636581 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ef456210 con 0x5653ef445c00 2024-06-13T07:13:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1458a00640 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef4546d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f143c03d6e0 tx=0x7f143c009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed8925bdd7f8e8b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.488+0000 7f1442000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:13:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.489+0000 7f1442000640 1 -- 172.21.15.26:0/294636581 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c03dad0 con 0x5653ef445c00 2024-06-13T07:13:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.489+0000 7f145e363740 1 -- 172.21.15.26:0/294636581 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143c04c070 con 0x5653ef445c00 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.489+0000 7f1442000640 1 -- 172.21.15.26:0/294636581 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f143c051470 con 0x5653ef445c00 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.489+0000 7f1442000640 1 -- 172.21.15.26:0/294636581 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f143c051670 con 0x5653ef445c00 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.489+0000 7f145e363740 10 client.26545 fetch_fsmap learned FSMap version 36 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.489+0000 7f145e363740 1 -- 172.21.15.26:0/294636581 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5653ef457520 con 0x5653ef445c00 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.489+0000 7f1442000640 1 -- 172.21.15.26:0/294636581 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c04c070 con 0x5653ef445c00 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.489+0000 7f1442000640 1 -- 172.21.15.26:0/294636581 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f143c044040 con 0x5653ef445c00 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.490+0000 7f145e363740 10 client.26545 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.490+0000 7f145e363740 10 client.26545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.490+0000 7f145e363740 20 client.26545 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.490+0000 7f145e363740 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5653ef4cf960 0x5653ef4d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.490+0000 7f145e363740 4 client.26545 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:13:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.490+0000 7f145e363740 1 -- 172.21.15.26:0/294636581 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653ef3b0ea0 con 0x5653ef4cf960 2024-06-13T07:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.490+0000 7f144be00640 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5653ef4cf960 0x5653ef4d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.491+0000 7f144be00640 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5653ef4cf960 0x5653ef4d1d50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18eddf21edf1410d server_cookie=a18aaac8277c68b4 in_seq=0 out_seq=0 2024-06-13T07:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.491+0000 7f1442000640 10 client.26545 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f1442000640 1 -- 172.21.15.26:0/294636581 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5653ef3b0ea0 con 0x5653ef4cf960 2024-06-13T07:13:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f1442000640 10 client.26545 handle_command_reply: tid=0 2024-06-13T07:13:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 client.26545 shutdown 2024-06-13T07:13:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 -- 172.21.15.26:0/294636581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 msgr2=0x5653ef4546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef4546d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f143c03d6e0 tx=0x7f143c009830 comp rx=0 tx=0).stop 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 -- 172.21.15.26:0/294636581 shutdown_connections 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5653ef4cf960 0x5653ef4d1d50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653ef44b760 0x5653ef455c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5653ef4465f0 0x5653ef454c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 --2- 172.21.15.26:0/294636581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ef445c00 0x5653ef4546d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f145e363740 1 -- 172.21.15.26:0/294636581 >> 172.21.15.26:0/294636581 conn(0x5653ef40dfb0 msgr2=0x5653ef44fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.581+0000 7f1442000640 0 client.26545 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.582+0000 7f145e363740 1 -- 172.21.15.26:0/294636581 shutdown_connections 2024-06-13T07:13:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.582+0000 7f145e363740 1 -- 172.21.15.26:0/294636581 wait complete. 2024-06-13T07:13:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.582+0000 7f145e363740 20 client.26545 trim_cache size 0 max 16384 2024-06-13T07:13:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:13:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:13:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 -- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14340a2c30 msgr2=0x7f14340a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14340a2c30 0x7f14340a50f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5653ef447060 tx=0x7f143c0315b0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 -- 172.21.15.26:0/3258706700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1454109300 msgr2=0x7f14541a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1454109300 0x7f14541a2b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f144400e9c0 tx=0x7f144400ee80 comp rx=0 tx=0).stop 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 -- 172.21.15.26:0/3258706700 shutdown_connections 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f14340a2c30 0x7f14340a50f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1454109300 0x7f14541a2b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1454110220 0x7f14541a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 --2- 172.21.15.26:0/3258706700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14541089a0 0x7f14541a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 -- 172.21.15.26:0/3258706700 >> 172.21.15.26:0/3258706700 conn(0x7f145406ebd0 msgr2=0x7f145406efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 -- 172.21.15.26:0/3258706700 shutdown_connections 2024-06-13T07:13:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:44.583+0000 7f145b400640 1 -- 172.21.15.26:0/3258706700 wait complete. 2024-06-13T07:13:44.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 625627 4.77 MB/sec execute 598 sec latency 32.100 ms 2024-06-13T07:13:45.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 626502 4.77 MB/sec execute 599 sec latency 23.107 ms 2024-06-13T07:13:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:45 smithi026 ceph-mon[32041]: pgmap v907: 129 pgs: 129 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:13:46.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:45 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:45 smithi135 ceph-mon[36276]: pgmap v907: 129 pgs: 129 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:13:46.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:45 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:46.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:45 smithi136 ceph-mon[37229]: pgmap v907: 129 pgs: 129 active+clean; 132 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-13T07:13:46.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:45 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:46.593 INFO:tasks.workunit.client.0.smithi026.stdout: 1 cleanup 600 sec 2024-06-13T07:13:47.030 INFO:tasks.workunit.client.0.smithi026.stdout: 0 cleanup 601 sec 2024-06-13T07:13:47.030 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-06-13T07:13:47.030 INFO:tasks.workunit.client.0.smithi026.stdout: Operation Count AvgLat MaxLat 2024-06-13T07:13:47.030 INFO:tasks.workunit.client.0.smithi026.stdout: ---------------------------------------- 2024-06-13T07:13:47.030 INFO:tasks.workunit.client.0.smithi026.stdout: NTCreateX 91356 2.781 1636.699 2024-06-13T07:13:47.030 INFO:tasks.workunit.client.0.smithi026.stdout: Close 67096 0.011 1.441 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Rename 3866 12.338 1758.370 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Unlink 18450 3.507 68.810 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Deltree 2 119.382 186.606 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Mkdir 1 0.017 0.017 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Qpathinfo 82822 1.790 2725.108 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Qfileinfo 14498 0.005 0.070 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Qfsinfo 15166 0.566 51.912 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Sfileinfo 7454 0.111 10.806 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Find 31997 0.464 66.105 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: WriteX 45470 0.073 31.360 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: ReadX 143028 0.038 59.414 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: LockX 296 0.949 2.068 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: UnlockX 296 0.820 7.219 2024-06-13T07:13:47.031 INFO:tasks.workunit.client.0.smithi026.stdout: Flush 6404 7.209 86.342 2024-06-13T07:13:47.032 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-06-13T07:13:47.032 INFO:tasks.workunit.client.0.smithi026.stdout:Throughput 4.76855 MB/sec 1 clients 1 procs max_latency=2725.123 ms 2024-06-13T07:13:47.032 INFO:tasks.workunit.client.0.smithi026.stderr:+ dbench 10 2024-06-13T07:13:47.035 INFO:tasks.workunit.client.0.smithi026.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-13T07:13:47.035 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-06-13T07:13:47.035 INFO:tasks.workunit.client.0.smithi026.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-13T07:13:47.039 INFO:tasks.workunit.client.0.smithi026.stdout:8 of 10 processes prepared for launch 0 sec 2024-06-13T07:13:47.139 INFO:tasks.workunit.client.0.smithi026.stdout:10 of 10 processes prepared for launch 0 sec 2024-06-13T07:13:47.139 INFO:tasks.workunit.client.0.smithi026.stdout:releasing clients 2024-06-13T07:13:48.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 242 92.44 MB/sec warmup 1 sec latency 99.881 ms 2024-06-13T07:13:48.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:47 smithi026 ceph-mon[32041]: pgmap v908: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:13:48.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:47 smithi135 ceph-mon[36276]: pgmap v908: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:13:48.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:47 smithi136 ceph-mon[37229]: pgmap v908: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-13T07:13:49.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 453 79.51 MB/sec warmup 2 sec latency 85.788 ms 2024-06-13T07:13:49.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:48 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:49.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:48 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:49.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:48 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:50.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 766 70.59 MB/sec warmup 3 sec latency 80.871 ms 2024-06-13T07:13:50.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:49 smithi026 ceph-mon[32041]: pgmap v909: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-13T07:13:50.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:49 smithi135 ceph-mon[36276]: pgmap v909: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-13T07:13:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:49 smithi136 ceph-mon[37229]: pgmap v909: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-13T07:13:51.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1139 57.22 MB/sec warmup 4 sec latency 66.987 ms 2024-06-13T07:13:51.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:50 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:51.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:50 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:51.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:50 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:13:52.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1476 51.52 MB/sec warmup 5 sec latency 80.564 ms 2024-06-13T07:13:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:51 smithi026 ceph-mon[32041]: pgmap v910: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-06-13T07:13:52.278 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:51 smithi136 ceph-mon[37229]: pgmap v910: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-06-13T07:13:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:51 smithi135 ceph-mon[36276]: pgmap v910: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-06-13T07:13:53.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 1998 44.18 MB/sec warmup 6 sec latency 80.597 ms 2024-06-13T07:13:54.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 2365 39.83 MB/sec warmup 7 sec latency 316.797 ms 2024-06-13T07:13:54.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:53 smithi026 ceph-mon[32041]: pgmap v911: 129 pgs: 129 active+clean; 122 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-13T07:13:54.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:53 smithi135 ceph-mon[36276]: pgmap v911: 129 pgs: 129 active+clean; 122 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-13T07:13:54.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:53 smithi136 ceph-mon[37229]: pgmap v911: 129 pgs: 129 active+clean; 122 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-13T07:13:55.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 2649 37.61 MB/sec warmup 8 sec latency 100.852 ms 2024-06-13T07:13:56.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3078 37.21 MB/sec warmup 9 sec latency 74.061 ms 2024-06-13T07:13:56.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:56 smithi026 ceph-mon[32041]: pgmap v912: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-06-13T07:13:56.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:56 smithi135 ceph-mon[36276]: pgmap v912: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-06-13T07:13:56.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:56 smithi136 ceph-mon[37229]: pgmap v912: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-06-13T07:13:57.140 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3490 37.94 MB/sec warmup 10 sec latency 95.801 ms 2024-06-13T07:13:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.746+0000 7f7fd7e00640 1 -- 172.21.15.26:0/2652259647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 msgr2=0x7f7fd8103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.746+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/2652259647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 0x7f7fd8103e80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009900 tx=0x7f7fc802a400 comp rx=0 tx=0).stop 2024-06-13T07:13:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.747+0000 7f7fd7e00640 1 -- 172.21.15.26:0/2652259647 shutdown_connections 2024-06-13T07:13:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.747+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/2652259647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd8105610 0x7f7fd810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.747+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/2652259647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd8104c50 0x7f7fd81050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.747+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/2652259647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 0x7f7fd8103e80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.747+0000 7f7fd7e00640 1 -- 172.21.15.26:0/2652259647 >> 172.21.15.26:0/2652259647 conn(0x7f7fd80fefd0 msgr2=0x7f7fd8101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.747+0000 7f7fd7e00640 1 -- 172.21.15.26:0/2652259647 shutdown_connections 2024-06-13T07:13:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.747+0000 7f7fd7e00640 1 -- 172.21.15.26:0/2652259647 wait complete. 2024-06-13T07:13:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.748+0000 7f7fd7e00640 1 Processor -- start 2024-06-13T07:13:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.748+0000 7f7fd7e00640 1 -- start start 2024-06-13T07:13:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.748+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 0x7f7fd8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.748+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd8104c50 0x7f7fd819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.748+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd8105610 0x7f7fd81a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.748+0000 7f7fd7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd810f6b0 con 0x7f7fd8105610 2024-06-13T07:13:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.748+0000 7f7fd7e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd810f530 con 0x7f7fd8104c50 2024-06-13T07:13:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.748+0000 7f7fd7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd810f830 con 0x7f7fd8103a60 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd8104c50 0x7f7fd819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 0x7f7fd8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd8104c50 0x7f7fd819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37878/0 (socket says 172.21.15.26:37878) 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6000640 1 -- 172.21.15.26:0/1251313303 learned_addr learned my addr 172.21.15.26:0/1251313303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd7400640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd8105610 0x7f7fd81a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6a00640 1 -- 172.21.15.26:0/1251313303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd8104c50 msgr2=0x7f7fd819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd8104c50 0x7f7fd819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6a00640 1 -- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd8105610 msgr2=0x7f7fd81a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd8105610 0x7f7fd81a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd6a00640 1 -- 172.21.15.26:0/1251313303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd81a19b0 con 0x7f7fd8103a60 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.749+0000 7f7fd7400640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd8105610 0x7f7fd81a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.750+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 0x7f7fd8199da0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009230 tx=0x7f7fc80043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.750+0000 7f7fcf400640 1 -- 172.21.15.26:0/1251313303 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc803f070 con 0x7f7fd8103a60 2024-06-13T07:13:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.750+0000 7f7fcf400640 1 -- 172.21.15.26:0/1251313303 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc803b460 con 0x7f7fd8103a60 2024-06-13T07:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.750+0000 7f7fd7e00640 1 -- 172.21.15.26:0/1251313303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd81a1c40 con 0x7f7fd8103a60 2024-06-13T07:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.750+0000 7f7fcf400640 1 -- 172.21.15.26:0/1251313303 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8035070 con 0x7f7fd8103a60 2024-06-13T07:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.750+0000 7f7fd7e00640 1 -- 172.21.15.26:0/1251313303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd81a2150 con 0x7f7fd8103a60 2024-06-13T07:13:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.755+0000 7f7fe066a740 1 Processor -- start 2024-06-13T07:13:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.755+0000 7f7fe066a740 1 -- start start 2024-06-13T07:13:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.755+0000 7f7fcf400640 1 -- 172.21.15.26:0/1251313303 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7fc803b910 con 0x7f7fd8103a60 2024-06-13T07:13:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.755+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec98dc0 0x55b29ec991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.755+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec997b0 0x55b29ec9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.755+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec9e920 0x55b29eca0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.755+0000 7f7fe066a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b29ebfc420 con 0x55b29ec997b0 2024-06-13T07:13:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.755+0000 7f7fe066a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b29eba3c50 con 0x55b29ec9e920 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fe066a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b29ebf83e0 con 0x55b29ec98dc0 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec9e920 0x55b29eca0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec9e920 0x55b29eca0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37892/0 (socket says 172.21.15.26:37892) 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6000640 1 -- 172.21.15.26:0/1955284417 learned_addr learned my addr 172.21.15.26:0/1955284417 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd7400640 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec98dc0 0x55b29ec991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec997b0 0x55b29ec9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6000640 1 -- 172.21.15.26:0/1955284417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec98dc0 msgr2=0x55b29ec991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6000640 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec98dc0 0x55b29ec991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6000640 1 -- 172.21.15.26:0/1955284417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec997b0 msgr2=0x55b29ec9e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6000640 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec997b0 0x55b29ec9e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.756+0000 7f7fd6000640 1 -- 172.21.15.26:0/1955284417 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29e910210 con 0x55b29ec9e920 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.757+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec997b0 0x55b29ec9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:13:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.757+0000 7f7fd6000640 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec9e920 0x55b29eca0db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400b890 tx=0x7f7fc400bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c21ee46beef331 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.757+0000 7f7fc3e00640 1 -- 172.21.15.26:0/1955284417 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4003dd0 con 0x55b29ec9e920 2024-06-13T07:13:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.757+0000 7f7fc3e00640 1 -- 172.21.15.26:0/1955284417 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc4015070 con 0x55b29ec9e920 2024-06-13T07:13:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.757+0000 7f7fc3e00640 1 -- 172.21.15.26:0/1955284417 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4011880 con 0x55b29ec9e920 2024-06-13T07:13:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.757+0000 7f7fcf400640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7fbc0a2c00 0x7f7fbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fcf400640 1 -- 172.21.15.26:0/1251313303 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7fc80efb20 con 0x7f7fd8103a60 2024-06-13T07:13:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fd7400640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7fbc0a2c00 0x7f7fbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fe066a740 1 -- 172.21.15.26:0/1955284417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec9e920 msgr2=0x55b29eca0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fe066a740 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec9e920 0x55b29eca0db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400b890 tx=0x7f7fc400bd50 comp rx=0 tx=0).stop 2024-06-13T07:13:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fe066a740 1 -- 172.21.15.26:0/1955284417 shutdown_connections 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fe066a740 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec9e920 0x55b29eca0db0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fe066a740 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec997b0 0x55b29ec9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fe066a740 1 --2- 172.21.15.26:0/1955284417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec98dc0 0x55b29ec991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.758+0000 7f7fe066a740 1 -- 172.21.15.26:0/1955284417 >> 172.21.15.26:0/1955284417 conn(0x55b29ec60fb0 msgr2=0x55b29ec94840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.759+0000 7f7fd7400640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7fbc0a2c00 0x7f7fbc0a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55b29ec9a010 tx=0x7f7fd0009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.759+0000 7f7fe066a740 1 -- 172.21.15.26:0/1955284417 shutdown_connections 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.759+0000 7f7fe066a740 1 -- 172.21.15.26:0/1955284417 wait complete. 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 1 -- start start 2024-06-13T07:13:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 1 Processor -- start 2024-06-13T07:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29eca3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29eca3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ecaa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b29eca18d0 con 0x55b29ec98dc0 2024-06-13T07:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b29ebb70b0 con 0x55b29ec997b0 2024-06-13T07:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b29eca1a50 con 0x55b29ec9e920 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.760+0000 7f7fe066a740 10 client.0 _ll_register_callbacks cb 0x55b29eba2f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ecaa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ecaa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37860/0 (socket says 172.21.15.26:37860) 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd7400640 1 -- 172.21.15.26:0/2961384383 learned_addr learned my addr 172.21.15.26:0/2961384383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd6000640 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29eca3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29eca3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd7400640 1 -- 172.21.15.26:0/2961384383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 msgr2=0x55b29eca3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd7400640 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29eca3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd7400640 1 -- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 msgr2=0x55b29eca3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd7400640 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29eca3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd7400640 1 -- 172.21.15.26:0/2961384383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29e9dc620 con 0x55b29ec9e920 2024-06-13T07:13:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.761+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29eca3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:13:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.762+0000 7f7fd7400640 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ecaa680 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00092b0 tx=0x7f7fd0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e636d7739c1e202a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.762+0000 7f7fc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.763+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2961384383 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0010ea0 con 0x55b29ec9e920 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.763+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2961384383 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7fd0028600 con 0x55b29ec9e920 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.763+0000 7f7fe066a740 1 -- 172.21.15.26:0/2961384383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc4015070 con 0x55b29ec9e920 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.763+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2961384383 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7fd0031050 con 0x55b29ec9e920 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.763+0000 7f7fe066a740 10 client.26557 fetch_fsmap learned FSMap version 39 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.763+0000 7f7fe066a740 1 -- 172.21.15.26:0/2961384383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b29ecb62c0 con 0x55b29ec9e920 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.763+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2961384383 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc4015070 con 0x55b29ec9e920 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.764+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2961384383 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7fd000f040 con 0x55b29ec9e920 2024-06-13T07:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.764+0000 7f7fe066a740 10 client.26557 fetch_fsmap finished waiting for FSMap version 39 2024-06-13T07:13:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.764+0000 7f7fe066a740 10 client.26557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.764+0000 7f7fe066a740 20 client.26557 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.764+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b29ecb7a00 0x55b29ecb9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.764+0000 7f7fe066a740 4 client.26557 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:13:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.764+0000 7f7fe066a740 1 -- 172.21.15.26:0/2961384383 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b29ec09b50 con 0x55b29ecb7a00 2024-06-13T07:13:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.764+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b29ecb7a00 0x55b29ecb9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.765+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b29ecb7a00 0x55b29ecb9e90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2043eb871528ce6b server_cookie=ab2fa564dd6b3afb in_seq=0 out_seq=0 2024-06-13T07:13:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.765+0000 7f7fc0c00640 10 client.26557 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2961384383 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b29ec09b50 con 0x55b29ecb7a00 2024-06-13T07:13:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fc0c00640 10 client.26557 handle_command_reply: tid=0 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 client.26557 shutdown 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 -- 172.21.15.26:0/2961384383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 msgr2=0x55b29ecaa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ecaa680 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00092b0 tx=0x7f7fd0004a40 comp rx=0 tx=0).stop 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 -- 172.21.15.26:0/2961384383 shutdown_connections 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b29ecb7a00 0x55b29ecb9e90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29eca3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ecaa680 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2961384383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29eca3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fc0c00640 0 client.26557 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 -- 172.21.15.26:0/2961384383 >> 172.21.15.26:0/2961384383 conn(0x55b29ec60fb0 msgr2=0x55b29ec939a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.769+0000 7f7fe066a740 1 -- 172.21.15.26:0/2961384383 shutdown_connections 2024-06-13T07:13:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.770+0000 7f7fe066a740 1 -- 172.21.15.26:0/2961384383 wait complete. 2024-06-13T07:13:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.770+0000 7f7fe066a740 20 client.26557 trim_cache size 0 max 16384 2024-06-13T07:13:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.791+0000 7f7fe066a740 1 Processor -- start 2024-06-13T07:13:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.791+0000 7f7fe066a740 1 -- start start 2024-06-13T07:13:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29ec94320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29ecaa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ec9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fe066a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81aac70 con 0x55b29ec98dc0 2024-06-13T07:13:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fe066a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81a84e0 con 0x55b29ec997b0 2024-06-13T07:13:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fe066a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81a6330 con 0x55b29ec9e920 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fd6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29ec94320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fd6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29ec94320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48404/0 (socket says 172.21.15.26:48404) 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fd6a00640 1 -- 172.21.15.26:0/1687947184 learned_addr learned my addr 172.21.15.26:0/1687947184 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fd6000640 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29ecaa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.792+0000 7f7fd7400640 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ec9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.793+0000 7f7fd6a00640 1 -- 172.21.15.26:0/1687947184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 msgr2=0x55b29ec9aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.793+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ec9aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.793+0000 7f7fd6a00640 1 -- 172.21.15.26:0/1687947184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 msgr2=0x55b29ecaa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.793+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29ecaa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.793+0000 7f7fd6a00640 1 -- 172.21.15.26:0/1687947184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29ec99b90 con 0x55b29ec98dc0 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.793+0000 7f7fd6000640 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29ecaa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.793+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29ec94320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005c40 tx=0x7f7fc8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3ad7308f23ba95 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.794+0000 7f7fc3400640 1 -- 172.21.15.26:0/1687947184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8048040 con 0x55b29ec98dc0 2024-06-13T07:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.794+0000 7f7fc3400640 1 -- 172.21.15.26:0/1687947184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc80bc690 con 0x55b29ec98dc0 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.794+0000 7f7fc3400640 1 -- 172.21.15.26:0/1687947184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80c4c00 con 0x55b29ec98dc0 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 -- 172.21.15.26:0/1687947184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 msgr2=0x55b29ec94320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29ec94320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005c40 tx=0x7f7fc8009260 comp rx=0 tx=0).stop 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 -- 172.21.15.26:0/1687947184 shutdown_connections 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec997b0 0x55b29ecaa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec9e920 0x55b29ec9aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 --2- 172.21.15.26:0/1687947184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec98dc0 0x55b29ec94320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 -- 172.21.15.26:0/1687947184 >> 172.21.15.26:0/1687947184 conn(0x55b29ec60fb0 msgr2=0x55b29eca3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 -- 172.21.15.26:0/1687947184 shutdown_connections 2024-06-13T07:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.795+0000 7f7fe066a740 1 -- 172.21.15.26:0/1687947184 wait complete. 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.796+0000 7f7fe066a740 1 -- start start 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.796+0000 7f7fe066a740 1 Processor -- start 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.796+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec98dc0 0x55b29eca7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec997b0 0x55b29eca7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fe066a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec9e920 0x55b29eca8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fe066a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81a69a0 con 0x55b29ec9e920 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fe066a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81a7e10 con 0x55b29ec98dc0 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fe066a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81a7070 con 0x55b29ec997b0 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fe066a740 10 client.0 _ll_register_callbacks cb 0x7f7fd0009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec997b0 0x55b29eca7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec98dc0 0x55b29eca7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec997b0 0x55b29eca7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37884/0 (socket says 172.21.15.26:37884) 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6000640 1 -- 172.21.15.26:0/2654385405 learned_addr learned my addr 172.21.15.26:0/2654385405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec98dc0 0x55b29eca7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37924/0 (socket says 172.21.15.26:37924) 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd7400640 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec9e920 0x55b29eca8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6000640 1 -- 172.21.15.26:0/2654385405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec98dc0 msgr2=0x55b29eca7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6000640 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec98dc0 0x55b29eca7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6000640 1 -- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec9e920 msgr2=0x55b29eca8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6000640 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec9e920 0x55b29eca8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6000640 1 -- 172.21.15.26:0/2654385405 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29eca9270 con 0x55b29ec997b0 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec98dc0 0x55b29eca7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:13:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.797+0000 7f7fd7400640 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec9e920 0x55b29eca8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.798+0000 7f7fd6000640 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec997b0 0x55b29eca7c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400b890 tx=0x7f7fc4003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c1173a5ab5124e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.798+0000 7f7fc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.798+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2654385405 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc402c060 con 0x55b29ec997b0 2024-06-13T07:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.798+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2654385405 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7fc401a880 con 0x55b29ec997b0 2024-06-13T07:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.798+0000 7f7fe066a740 1 -- 172.21.15.26:0/2654385405 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc80bc690 con 0x55b29ec997b0 2024-06-13T07:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2654385405 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7fc4033050 con 0x55b29ec997b0 2024-06-13T07:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fe066a740 10 client.26563 fetch_fsmap learned FSMap version 39 2024-06-13T07:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fe066a740 1 -- 172.21.15.26:0/2654385405 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b29ecaa5b0 con 0x55b29ec997b0 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2654385405 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc80bc690 con 0x55b29ec997b0 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2654385405 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7fc4011bc0 con 0x55b29ec997b0 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fe066a740 10 client.26563 fetch_fsmap finished waiting for FSMap version 39 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fe066a740 10 client.26563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fe066a740 20 client.26563 populate_metadata read hostname 'smithi026' 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b29ed22910 0x55b29ed24d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fe066a740 4 client.26563 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.799+0000 7f7fe066a740 1 -- 172.21.15.26:0/2654385405 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b29ec03ea0 con 0x55b29ed22910 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.800+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b29ed22910 0x55b29ed24d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.800+0000 7f7fd6a00640 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b29ed22910 0x55b29ed24d00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6fd2b52f26df66 server_cookie=bdee2353c2ed024b in_seq=0 out_seq=0 2024-06-13T07:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:57.800+0000 7f7fc0c00640 10 client.26563 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:58.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3659 35.12 MB/sec warmup 11 sec latency 214.720 ms 2024-06-13T07:13:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:13:58 smithi135 ceph-mon[36276]: pgmap v913: 129 pgs: 129 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 3.1 MiB/s wr, 208 op/s 2024-06-13T07:13:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:13:58 smithi136 ceph-mon[37229]: pgmap v913: 129 pgs: 129 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 3.1 MiB/s wr, 208 op/s 2024-06-13T07:13:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:13:58 smithi026 ceph-mon[32041]: pgmap v913: 129 pgs: 129 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 3.1 MiB/s wr, 208 op/s 2024-06-13T07:13:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.798+0000 7f7fc0c00640 1 -- 172.21.15.26:0/2654385405 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b29ec03ea0 con 0x55b29ed22910 2024-06-13T07:13:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.798+0000 7f7fc0c00640 10 client.26563 handle_command_reply: tid=0 2024-06-13T07:13:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.798+0000 7f7fe066a740 1 client.26563 shutdown 2024-06-13T07:13:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.799+0000 7f7fe066a740 1 -- 172.21.15.26:0/2654385405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec997b0 msgr2=0x55b29eca7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.799+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec997b0 0x55b29eca7c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400b890 tx=0x7f7fc4003e00 comp rx=0 tx=0).stop 2024-06-13T07:13:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.799+0000 7f7fe066a740 1 -- 172.21.15.26:0/2654385405 shutdown_connections 2024-06-13T07:13:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.799+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b29ed22910 0x55b29ed24d00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.799+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b29ec9e920 0x55b29eca8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.799+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b29ec997b0 0x55b29eca7c70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.799+0000 7f7fe066a740 1 --2- 172.21.15.26:0/2654385405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b29ec98dc0 0x55b29eca7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.799+0000 7f7fe066a740 1 -- 172.21.15.26:0/2654385405 >> 172.21.15.26:0/2654385405 conn(0x55b29ec60fb0 msgr2=0x55b29eca3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.800+0000 7f7fc0c00640 0 client.26563 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:13:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.800+0000 7f7fe066a740 1 -- 172.21.15.26:0/2654385405 shutdown_connections 2024-06-13T07:13:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.800+0000 7f7fe066a740 1 -- 172.21.15.26:0/2654385405 wait complete. 2024-06-13T07:13:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.801+0000 7f7fe066a740 20 client.26563 trim_cache size 0 max 16384 2024-06-13T07:13:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:13:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:13:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:13:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 -- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7fbc0a2c00 msgr2=0x7f7fbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7fbc0a2c00 0x7f7fbc0a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55b29ec9a010 tx=0x7f7fd0009780 comp rx=0 tx=0).stop 2024-06-13T07:13:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 -- 172.21.15.26:0/1251313303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 msgr2=0x7f7fd8199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 0x7f7fd8199da0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009230 tx=0x7f7fc80043b0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 -- 172.21.15.26:0/1251313303 shutdown_connections 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f7fbc0a2c00 0x7f7fbc0a50c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd8104c50 0x7f7fd819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd8105610 0x7f7fd81a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 --2- 172.21.15.26:0/1251313303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd8103a60 0x7f7fd8199da0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 -- 172.21.15.26:0/1251313303 >> 172.21.15.26:0/1251313303 conn(0x7f7fd80fefd0 msgr2=0x7f7fd8107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 -- 172.21.15.26:0/1251313303 shutdown_connections 2024-06-13T07:13:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:13:58.803+0000 7f7fd7e00640 1 -- 172.21.15.26:0/1251313303 wait complete. 2024-06-13T07:13:59.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3674 32.21 MB/sec warmup 12 sec latency 688.543 ms 2024-06-13T07:14:00.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3716 30.15 MB/sec warmup 13 sec latency 560.220 ms 2024-06-13T07:14:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:00 smithi135 ceph-mon[36276]: pgmap v914: 129 pgs: 129 active+clean; 144 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 5.0 MiB/s wr, 311 op/s 2024-06-13T07:14:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:00 smithi136 ceph-mon[37229]: pgmap v914: 129 pgs: 129 active+clean; 144 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 5.0 MiB/s wr, 311 op/s 2024-06-13T07:14:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:00 smithi026 ceph-mon[32041]: pgmap v914: 129 pgs: 129 active+clean; 144 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 5.0 MiB/s wr, 311 op/s 2024-06-13T07:14:01.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3795 28.09 MB/sec warmup 14 sec latency 355.284 ms 2024-06-13T07:14:02.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3880 26.54 MB/sec warmup 15 sec latency 513.392 ms 2024-06-13T07:14:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:02 smithi135 ceph-mon[36276]: pgmap v915: 129 pgs: 129 active+clean; 180 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 8.8 MiB/s wr, 408 op/s 2024-06-13T07:14:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:02 smithi136 ceph-mon[37229]: pgmap v915: 129 pgs: 129 active+clean; 180 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 8.8 MiB/s wr, 408 op/s 2024-06-13T07:14:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:02 smithi026 ceph-mon[32041]: pgmap v915: 129 pgs: 129 active+clean; 180 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 8.8 MiB/s wr, 408 op/s 2024-06-13T07:14:03.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 3991 25.02 MB/sec warmup 16 sec latency 88.335 ms 2024-06-13T07:14:04.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4090 23.56 MB/sec warmup 17 sec latency 240.240 ms 2024-06-13T07:14:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:04 smithi135 ceph-mon[36276]: pgmap v916: 129 pgs: 129 active+clean; 262 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 16 MiB/s wr, 553 op/s 2024-06-13T07:14:04.303 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:04 smithi026 ceph-mon[32041]: pgmap v916: 129 pgs: 129 active+clean; 262 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 16 MiB/s wr, 553 op/s 2024-06-13T07:14:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:04 smithi136 ceph-mon[37229]: pgmap v916: 129 pgs: 129 active+clean; 262 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 16 MiB/s wr, 553 op/s 2024-06-13T07:14:05.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4201 22.33 MB/sec warmup 18 sec latency 173.710 ms 2024-06-13T07:14:06.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 4567 21.80 MB/sec warmup 19 sec latency 207.284 ms 2024-06-13T07:14:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:06 smithi136 ceph-mon[37229]: pgmap v917: 129 pgs: 129 active+clean; 319 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 20 MiB/s wr, 619 op/s 2024-06-13T07:14:06.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:06 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:06 smithi026 ceph-mon[32041]: pgmap v917: 129 pgs: 129 active+clean; 319 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 20 MiB/s wr, 619 op/s 2024-06-13T07:14:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:06 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:06.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:06 smithi135 ceph-mon[36276]: pgmap v917: 129 pgs: 129 active+clean; 319 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 20 MiB/s wr, 619 op/s 2024-06-13T07:14:06.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:06 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:07.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 5230 22.64 MB/sec warmup 20 sec latency 62.825 ms 2024-06-13T07:14:08.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 6098 23.27 MB/sec warmup 21 sec latency 54.822 ms 2024-06-13T07:14:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:08 smithi136 ceph-mon[37229]: pgmap v918: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 20 MiB/s wr, 632 op/s 2024-06-13T07:14:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:08 smithi026 ceph-mon[32041]: pgmap v918: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 20 MiB/s wr, 632 op/s 2024-06-13T07:14:08.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:08 smithi135 ceph-mon[36276]: pgmap v918: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 20 MiB/s wr, 632 op/s 2024-06-13T07:14:09.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 6625 24.17 MB/sec warmup 22 sec latency 40.984 ms 2024-06-13T07:14:10.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 7171 24.57 MB/sec warmup 23 sec latency 86.360 ms 2024-06-13T07:14:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:10 smithi136 ceph-mon[37229]: pgmap v919: 129 pgs: 129 active+clean; 321 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 19 MiB/s wr, 607 op/s 2024-06-13T07:14:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:10 smithi026 ceph-mon[32041]: pgmap v919: 129 pgs: 129 active+clean; 321 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 19 MiB/s wr, 607 op/s 2024-06-13T07:14:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:10 smithi135 ceph-mon[36276]: pgmap v919: 129 pgs: 129 active+clean; 321 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 19 MiB/s wr, 607 op/s 2024-06-13T07:14:11.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 7728 25.07 MB/sec warmup 24 sec latency 168.986 ms 2024-06-13T07:14:11.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:11 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:11.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:11 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:11.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:11 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 -- 172.21.15.26:0/793281167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c0101910 msgr2=0x7f63c0101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 --2- 172.21.15.26:0/793281167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c0101910 0x7f63c0101d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009900 tx=0x7f63ac02a410 comp rx=0 tx=0).stop 2024-06-13T07:14:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 -- 172.21.15.26:0/793281167 shutdown_connections 2024-06-13T07:14:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 --2- 172.21.15.26:0/793281167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c0101910 0x7f63c0101d90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 --2- 172.21.15.26:0/793281167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c01022d0 0x7f63c0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 --2- 172.21.15.26:0/793281167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c0100720 0x7f63c0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 -- 172.21.15.26:0/793281167 >> 172.21.15.26:0/793281167 conn(0x7f63c00fbc90 msgr2=0x7f63c00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 -- 172.21.15.26:0/793281167 shutdown_connections 2024-06-13T07:14:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.980+0000 7f63c5e00640 1 -- 172.21.15.26:0/793281167 wait complete. 2024-06-13T07:14:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.982+0000 7f63c5e00640 1 Processor -- start 2024-06-13T07:14:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.982+0000 7f63c5e00640 1 -- start start 2024-06-13T07:14:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c0100720 0x7f63c0104660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0101910 0x7f63c0102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c01022d0 0x7f63c0103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010c3d0 con 0x7f63c0101910 2024-06-13T07:14:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010c250 con 0x7f63c0100720 2024-06-13T07:14:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010c550 con 0x7f63c01022d0 2024-06-13T07:14:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c01022d0 0x7f63c0103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c01022d0 0x7f63c0103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56472/0 (socket says 172.21.15.26:56472) 2024-06-13T07:14:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c5400640 1 -- 172.21.15.26:0/3254449179 learned_addr learned my addr 172.21.15.26:0/3254449179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.983+0000 7f63c4a00640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c0100720 0x7f63c0104660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.984+0000 7f63c5400640 1 -- 172.21.15.26:0/3254449179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c0100720 msgr2=0x7f63c0104660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.984+0000 7f63c5400640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c0100720 0x7f63c0104660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.984+0000 7f63c5400640 1 -- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0101910 msgr2=0x7f63c0102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.984+0000 7f63c5400640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0101910 0x7f63c0102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.984+0000 7f63c5400640 1 -- 172.21.15.26:0/3254449179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c0103af0 con 0x7f63c01022d0 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.984+0000 7f63c5400640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c01022d0 0x7f63c0103230 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f63b800eaa0 tx=0x7f63b800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.984+0000 7f63bd600640 1 -- 172.21.15.26:0/3254449179 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b800cdb0 con 0x7f63c01022d0 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.985+0000 7f63bd600640 1 -- 172.21.15.26:0/3254449179 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b8005020 con 0x7f63c01022d0 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.985+0000 7f63bd600640 1 -- 172.21.15.26:0/3254449179 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80116b0 con 0x7f63c01022d0 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.985+0000 7f63c5e00640 1 -- 172.21.15.26:0/3254449179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c01977e0 con 0x7f63c01022d0 2024-06-13T07:14:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.985+0000 7f63c5e00640 1 -- 172.21.15.26:0/3254449179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c0197ce0 con 0x7f63c01022d0 2024-06-13T07:14:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.989+0000 7f63ca24c740 1 Processor -- start 2024-06-13T07:14:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.992+0000 7f63bd600640 1 -- 172.21.15.26:0/3254449179 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f63b80051d0 con 0x7f63c01022d0 2024-06-13T07:14:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.992+0000 7f63ca24c740 1 -- start start 2024-06-13T07:14:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.993+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b75211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.993+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b75262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.993+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b7528db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.993+0000 7f63ca24c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d7b7484420 con 0x55d7b75217b0 2024-06-13T07:14:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.993+0000 7f63ca24c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d7b742bc50 con 0x55d7b7526920 2024-06-13T07:14:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.993+0000 7f63ca24c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d7b74803e0 con 0x55d7b7520dc0 2024-06-13T07:14:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.995+0000 7f63bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b75211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.995+0000 7f63bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b75211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56484/0 (socket says 172.21.15.26:56484) 2024-06-13T07:14:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.995+0000 7f63bfe00640 1 -- 172.21.15.26:0/525322838 learned_addr learned my addr 172.21.15.26:0/525322838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.996+0000 7f63c4a00640 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b75262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.998+0000 7f63bd600640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f63a80a2c00 0x7f63a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.998+0000 7f63c5400640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f63a80a2c00 0x7f63a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.998+0000 7f63bfe00640 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b7528db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.999+0000 7f63bfe00640 1 -- 172.21.15.26:0/525322838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 msgr2=0x55d7b7528db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.999+0000 7f63bfe00640 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b7528db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.999+0000 7f63bfe00640 1 -- 172.21.15.26:0/525322838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 msgr2=0x55d7b75262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.999+0000 7f63bfe00640 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b75262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.999+0000 7f63bfe00640 1 -- 172.21.15.26:0/525322838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b7198210 con 0x55d7b7520dc0 2024-06-13T07:14:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.999+0000 7f63bfe00640 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b75211e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0314e0 tx=0x7f63ac000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4bcdc3cc389da3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.999+0000 7f63c4a00640 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b75262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:14:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:11.999+0000 7f63bd600640 1 -- 172.21.15.26:0/3254449179 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f63b80cf6e0 con 0x7f63c01022d0 2024-06-13T07:14:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.000+0000 7f63c5400640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f63a80a2c00 0x7f63a80a50c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f63b809b7f0 tx=0x7f63b8005820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.001+0000 7f63b1c00640 1 -- 172.21.15.26:0/525322838 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac044070 con 0x55d7b7520dc0 2024-06-13T07:14:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.001+0000 7f63b1c00640 1 -- 172.21.15.26:0/525322838 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ac0046c0 con 0x55d7b7520dc0 2024-06-13T07:14:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.001+0000 7f63b1c00640 1 -- 172.21.15.26:0/525322838 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac035040 con 0x55d7b7520dc0 2024-06-13T07:14:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.001+0000 7f63ca24c740 1 -- 172.21.15.26:0/525322838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 msgr2=0x55d7b75211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.001+0000 7f63ca24c740 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b75211e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0314e0 tx=0x7f63ac000bc0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.002+0000 7f63ca24c740 1 -- 172.21.15.26:0/525322838 shutdown_connections 2024-06-13T07:14:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.002+0000 7f63ca24c740 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b7528db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.002+0000 7f63ca24c740 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b75262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.002+0000 7f63ca24c740 1 --2- 172.21.15.26:0/525322838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b75211e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.002+0000 7f63ca24c740 1 -- 172.21.15.26:0/525322838 >> 172.21.15.26:0/525322838 conn(0x55d7b74e8fb0 msgr2=0x55d7b751c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.002+0000 7f63ca24c740 1 -- 172.21.15.26:0/525322838 shutdown_connections 2024-06-13T07:14:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.002+0000 7f63ca24c740 1 -- 172.21.15.26:0/525322838 wait complete. 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.002+0000 7f63ca24c740 1 -- start start 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63ca24c740 1 Processor -- start 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b752b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b752b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b75326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63ca24c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d7b7529870 con 0x55d7b75217b0 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63ca24c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d7b743f0b0 con 0x55d7b7526920 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63ca24c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d7b75299f0 con 0x55d7b7520dc0 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63ca24c740 10 client.0 _ll_register_callbacks cb 0x55d7b742af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63c4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b752b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63bfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b75326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b752b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63c4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b752b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58210/0 (socket says 172.21.15.26:58210) 2024-06-13T07:14:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.003+0000 7f63c4a00640 1 -- 172.21.15.26:0/2049079691 learned_addr learned my addr 172.21.15.26:0/2049079691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.004+0000 7f63c4a00640 1 -- 172.21.15.26:0/2049079691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 msgr2=0x55d7b752b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.004+0000 7f63c4a00640 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b752b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.004+0000 7f63c4a00640 1 -- 172.21.15.26:0/2049079691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 msgr2=0x55d7b75326a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.004+0000 7f63c4a00640 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b75326a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.004+0000 7f63c4a00640 1 -- 172.21.15.26:0/2049079691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b7264620 con 0x55d7b75217b0 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.004+0000 7f63bfe00640 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b75326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.004+0000 7f63c4a00640 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b752b660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63b400f410 tx=0x7f63b400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a87fc9b2193a8da server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.005+0000 7f63a6000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.005+0000 7f63a6000640 1 -- 172.21.15.26:0/2049079691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4005220 con 0x55d7b75217b0 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.005+0000 7f63a6000640 1 -- 172.21.15.26:0/2049079691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f63b40053c0 con 0x55d7b75217b0 2024-06-13T07:14:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.005+0000 7f63ca24c740 1 -- 172.21.15.26:0/2049079691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ac0046c0 con 0x55d7b75217b0 2024-06-13T07:14:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.005+0000 7f63a6000640 1 -- 172.21.15.26:0/2049079691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f63b40055c0 con 0x55d7b75217b0 2024-06-13T07:14:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.005+0000 7f63ca24c740 10 client.16407 fetch_fsmap learned FSMap version 41 2024-06-13T07:14:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.005+0000 7f63ca24c740 1 -- 172.21.15.26:0/2049079691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d7b7535a50 con 0x55d7b75217b0 2024-06-13T07:14:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.006+0000 7f63a6000640 1 -- 172.21.15.26:0/2049079691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ac0046c0 con 0x55d7b75217b0 2024-06-13T07:14:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.006+0000 7f63a6000640 1 -- 172.21.15.26:0/2049079691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f63b4019430 con 0x55d7b75217b0 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.006+0000 7f63ca24c740 10 client.16407 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.006+0000 7f63ca24c740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.007+0000 7f63ca24c740 20 client.16407 populate_metadata read hostname 'smithi026' 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.007+0000 7f63ca24c740 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d7b7537140 0x55d7b7539580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.007+0000 7f63ca24c740 4 client.16407 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.007+0000 7f63ca24c740 1 -- 172.21.15.26:0/2049079691 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7b7491b50 con 0x55d7b7537140 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.007+0000 7f63bfe00640 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d7b7537140 0x55d7b7539580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.008+0000 7f63bfe00640 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d7b7537140 0x55d7b7539580 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd8fa4188cc041 server_cookie=92063b06e604065a in_seq=0 out_seq=0 2024-06-13T07:14:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.008+0000 7f63a6000640 10 client.16407 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.010+0000 7f63a6000640 1 -- 172.21.15.26:0/2049079691 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d7b7491b50 con 0x55d7b7537140 2024-06-13T07:14:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.010+0000 7f63a6000640 10 client.16407 handle_command_reply: tid=0 2024-06-13T07:14:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.010+0000 7f63ca24c740 1 client.16407 shutdown 2024-06-13T07:14:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63ca24c740 1 -- 172.21.15.26:0/2049079691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 msgr2=0x55d7b752b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63ca24c740 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b752b660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63b400f410 tx=0x7f63b400f8d0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63ca24c740 1 -- 172.21.15.26:0/2049079691 shutdown_connections 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63ca24c740 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d7b7537140 0x55d7b7539580 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63ca24c740 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7526920 0x55d7b75326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63ca24c740 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b75217b0 0x55d7b752b660 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63a6000640 0 client.16407 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63ca24c740 1 --2- 172.21.15.26:0/2049079691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7520dc0 0x55d7b752b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.011+0000 7f63ca24c740 1 -- 172.21.15.26:0/2049079691 >> 172.21.15.26:0/2049079691 conn(0x55d7b74e8fb0 msgr2=0x55d7b751b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.012+0000 7f63ca24c740 1 -- 172.21.15.26:0/2049079691 shutdown_connections 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.012+0000 7f63ca24c740 1 -- 172.21.15.26:0/2049079691 wait complete. 2024-06-13T07:14:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.012+0000 7f63ca24c740 20 client.16407 trim_cache size 0 max 16384 2024-06-13T07:14:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.036+0000 7f63ca24c740 1 Processor -- start 2024-06-13T07:14:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.036+0000 7f63ca24c740 1 -- start start 2024-06-13T07:14:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7520dc0 0x55d7b7522b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b75217b0 0x55d7b7532660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7526920 0x55d7b7523050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63ca24c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01b33d0 con 0x55d7b7526920 2024-06-13T07:14:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63ca24c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01b0c40 con 0x55d7b7520dc0 2024-06-13T07:14:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63ca24c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01aea90 con 0x55d7b75217b0 2024-06-13T07:14:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63c4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b75217b0 0x55d7b7532660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63bfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7520dc0 0x55d7b7522b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63c4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b75217b0 0x55d7b7532660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56498/0 (socket says 172.21.15.26:56498) 2024-06-13T07:14:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63c4a00640 1 -- 172.21.15.26:0/1310273165 learned_addr learned my addr 172.21.15.26:0/1310273165 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.037+0000 7f63bfe00640 1 --2- 172.21.15.26:0/1310273165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7526920 0x55d7b7523050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.038+0000 7f63c4a00640 1 -- 172.21.15.26:0/1310273165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7520dc0 msgr2=0x55d7b7522b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.038+0000 7f63c4a00640 1 --2- 172.21.15.26:0/1310273165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7520dc0 0x55d7b7522b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.038+0000 7f63c4a00640 1 -- 172.21.15.26:0/1310273165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7526920 msgr2=0x55d7b7523050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.038+0000 7f63c4a00640 1 --2- 172.21.15.26:0/1310273165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7526920 0x55d7b7523050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.038+0000 7f63c4a00640 1 -- 172.21.15.26:0/1310273165 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b7535840 con 0x55d7b75217b0 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.038+0000 7f63c4a00640 1 --2- 172.21.15.26:0/1310273165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b75217b0 0x55d7b7532660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f63b4005cf0 tx=0x7f63b400f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d7e39455e040a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.038+0000 7f63b1200640 1 -- 172.21.15.26:0/1310273165 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4019cc0 con 0x55d7b75217b0 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.039+0000 7f63b1200640 1 -- 172.21.15.26:0/1310273165 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b402a040 con 0x55d7b75217b0 2024-06-13T07:14:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.039+0000 7f63b1200640 1 -- 172.21.15.26:0/1310273165 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40236a0 con 0x55d7b75217b0 2024-06-13T07:14:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.039+0000 7f63ca24c740 1 -- 172.21.15.26:0/1310273165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b75217b0 msgr2=0x55d7b7532660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.039+0000 7f63ca24c740 1 --2- 172.21.15.26:0/1310273165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b75217b0 0x55d7b7532660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f63b4005cf0 tx=0x7f63b400f3a0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.040+0000 7f63ca24c740 1 -- 172.21.15.26:0/1310273165 shutdown_connections 2024-06-13T07:14:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.040+0000 7f63ca24c740 1 --2- 172.21.15.26:0/1310273165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7526920 0x55d7b7523050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.040+0000 7f63ca24c740 1 --2- 172.21.15.26:0/1310273165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b75217b0 0x55d7b7532660 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.040+0000 7f63ca24c740 1 --2- 172.21.15.26:0/1310273165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b7520dc0 0x55d7b7522b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.040+0000 7f63ca24c740 1 -- 172.21.15.26:0/1310273165 >> 172.21.15.26:0/1310273165 conn(0x55d7b74e8fb0 msgr2=0x55d7b752b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.040+0000 7f63ca24c740 1 -- 172.21.15.26:0/1310273165 shutdown_connections 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.040+0000 7f63ca24c740 1 -- 172.21.15.26:0/1310273165 wait complete. 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.040+0000 7f63ca24c740 1 -- start start 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63ca24c740 1 Processor -- start 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7520dc0 0x55d7b752fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b75217b0 0x55d7b7530150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63ca24c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7526920 0x55d7b7531160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63ca24c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01af100 con 0x55d7b7520dc0 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63ca24c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01b0570 con 0x55d7b75217b0 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63ca24c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01af7d0 con 0x55d7b7526920 2024-06-13T07:14:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63ca24c740 10 client.0 _ll_register_callbacks cb 0x7f63b400a370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:14:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63c4a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b75217b0 0x55d7b7530150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63bfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7520dc0 0x55d7b752fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63c4a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b75217b0 0x55d7b7530150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:44686/0 (socket says 172.21.15.26:44686) 2024-06-13T07:14:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.041+0000 7f63c4a00640 1 -- 172.21.15.26:0/3059855871 learned_addr learned my addr 172.21.15.26:0/3059855871 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63bfe00640 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7526920 0x55d7b7531160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63c4a00640 1 -- 172.21.15.26:0/3059855871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7526920 msgr2=0x55d7b7531160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63c4a00640 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7526920 0x55d7b7531160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63c4a00640 1 -- 172.21.15.26:0/3059855871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7520dc0 msgr2=0x55d7b752fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63c4a00640 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7520dc0 0x55d7b752fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63c4a00640 1 -- 172.21.15.26:0/3059855871 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b7531750 con 0x55d7b75217b0 2024-06-13T07:14:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63bfe00640 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7520dc0 0x55d7b752fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:14:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63c4a00640 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b75217b0 0x55d7b7530150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f63b4026040 tx=0x7f63b4019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f6f441006192ba server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.042+0000 7f63a6000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:14:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.043+0000 7f63a6000640 1 -- 172.21.15.26:0/3059855871 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4032070 con 0x55d7b75217b0 2024-06-13T07:14:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.043+0000 7f63a6000640 1 -- 172.21.15.26:0/3059855871 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f63b4011710 con 0x55d7b75217b0 2024-06-13T07:14:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.043+0000 7f63ca24c740 1 -- 172.21.15.26:0/3059855871 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b402a040 con 0x55d7b75217b0 2024-06-13T07:14:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.043+0000 7f63a6000640 1 -- 172.21.15.26:0/3059855871 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f63b403f050 con 0x55d7b75217b0 2024-06-13T07:14:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.043+0000 7f63ca24c740 10 client.26552 fetch_fsmap learned FSMap version 41 2024-06-13T07:14:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.043+0000 7f63ca24c740 1 -- 172.21.15.26:0/3059855871 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d7b75a1e60 con 0x55d7b75217b0 2024-06-13T07:14:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.043+0000 7f63a6000640 1 -- 172.21.15.26:0/3059855871 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b402a040 con 0x55d7b75217b0 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.044+0000 7f63a6000640 1 -- 172.21.15.26:0/3059855871 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f63b4003b40 con 0x55d7b75217b0 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.044+0000 7f63ca24c740 10 client.26552 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.044+0000 7f63ca24c740 10 client.26552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.044+0000 7f63ca24c740 20 client.26552 populate_metadata read hostname 'smithi026' 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.044+0000 7f63ca24c740 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d7b7537140 0x55d7b75a4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.044+0000 7f63ca24c740 4 client.26552 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.044+0000 7f63ca24c740 1 -- 172.21.15.26:0/3059855871 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7b748bea0 con 0x55d7b7537140 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.045+0000 7f63bfe00640 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d7b7537140 0x55d7b75a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.046+0000 7f63bfe00640 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d7b7537140 0x55d7b75a4710 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b73e45a9d7964e server_cookie=802d03ea9606704e in_seq=0 out_seq=0 2024-06-13T07:14:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.046+0000 7f63a6000640 10 client.26552 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:12.141 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8114 24.58 MB/sec warmup 25 sec latency 398.688 ms 2024-06-13T07:14:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:12 smithi136 ceph-mon[37229]: pgmap v920: 129 pgs: 129 active+clean; 328 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 565 op/s 2024-06-13T07:14:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:12 smithi026 ceph-mon[32041]: pgmap v920: 129 pgs: 129 active+clean; 328 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 565 op/s 2024-06-13T07:14:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.529+0000 7f63a6000640 1 -- 172.21.15.26:0/3059855871 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d7b748bea0 con 0x55d7b7537140 2024-06-13T07:14:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.529+0000 7f63a6000640 10 client.26552 handle_command_reply: tid=0 2024-06-13T07:14:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.530+0000 7f63ca24c740 1 client.26552 shutdown 2024-06-13T07:14:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.531+0000 7f63ca24c740 1 -- 172.21.15.26:0/3059855871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b75217b0 msgr2=0x55d7b7530150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.531+0000 7f63ca24c740 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b75217b0 0x55d7b7530150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f63b4026040 tx=0x7f63b4019660 comp rx=0 tx=0).stop 2024-06-13T07:14:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.532+0000 7f63ca24c740 1 -- 172.21.15.26:0/3059855871 shutdown_connections 2024-06-13T07:14:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.532+0000 7f63ca24c740 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d7b7537140 0x55d7b75a4710 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.532+0000 7f63ca24c740 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d7b75217b0 0x55d7b7530150 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.532+0000 7f63ca24c740 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7b7526920 0x55d7b7531160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.532+0000 7f63ca24c740 1 --2- 172.21.15.26:0/3059855871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7b7520dc0 0x55d7b752fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.532+0000 7f63ca24c740 1 -- 172.21.15.26:0/3059855871 >> 172.21.15.26:0/3059855871 conn(0x55d7b74e8fb0 msgr2=0x55d7b751b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.532+0000 7f63a6000640 0 client.26552 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.532+0000 7f63ca24c740 1 -- 172.21.15.26:0/3059855871 shutdown_connections 2024-06-13T07:14:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.533+0000 7f63ca24c740 1 -- 172.21.15.26:0/3059855871 wait complete. 2024-06-13T07:14:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.533+0000 7f63ca24c740 20 client.26552 trim_cache size 0 max 16384 2024-06-13T07:14:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:12 smithi135 ceph-mon[36276]: pgmap v920: 129 pgs: 129 active+clean; 328 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 565 op/s 2024-06-13T07:14:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:14:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:14:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:14:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 -- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f63a80a2c00 msgr2=0x7f63a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f63a80a2c00 0x7f63a80a50c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f63b809b7f0 tx=0x7f63b8005820 comp rx=0 tx=0).stop 2024-06-13T07:14:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 -- 172.21.15.26:0/3254449179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c01022d0 msgr2=0x7f63c0103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c01022d0 0x7f63c0103230 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f63b800eaa0 tx=0x7f63b800ef60 comp rx=0 tx=0).stop 2024-06-13T07:14:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 -- 172.21.15.26:0/3254449179 shutdown_connections 2024-06-13T07:14:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f63a80a2c00 0x7f63a80a50c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63c01022d0 0x7f63c0103230 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0101910 0x7f63c0102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 --2- 172.21.15.26:0/3254449179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c0100720 0x7f63c0104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.534+0000 7f63c5e00640 1 -- 172.21.15.26:0/3254449179 >> 172.21.15.26:0/3254449179 conn(0x7f63c00fbc90 msgr2=0x7f63c00fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.537+0000 7f63c5e00640 1 -- 172.21.15.26:0/3254449179 shutdown_connections 2024-06-13T07:14:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:12.537+0000 7f63c5e00640 1 -- 172.21.15.26:0/3254449179 wait complete. 2024-06-13T07:14:13.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 8571 24.46 MB/sec warmup 26 sec latency 189.211 ms 2024-06-13T07:14:14.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 9466 25.27 MB/sec warmup 27 sec latency 34.776 ms 2024-06-13T07:14:14.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:14 smithi136 ceph-mon[37229]: pgmap v921: 129 pgs: 129 active+clean; 333 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 14 MiB/s wr, 526 op/s 2024-06-13T07:14:14.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:14 smithi026 ceph-mon[32041]: pgmap v921: 129 pgs: 129 active+clean; 333 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 14 MiB/s wr, 526 op/s 2024-06-13T07:14:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:14 smithi135 ceph-mon[36276]: pgmap v921: 129 pgs: 129 active+clean; 333 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 14 MiB/s wr, 526 op/s 2024-06-13T07:14:15.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 10555 26.52 MB/sec warmup 28 sec latency 32.676 ms 2024-06-13T07:14:16.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11428 27.58 MB/sec warmup 29 sec latency 32.041 ms 2024-06-13T07:14:16.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:16 smithi135 ceph-mon[36276]: pgmap v922: 129 pgs: 129 active+clean; 330 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 449 op/s 2024-06-13T07:14:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:16 smithi136 ceph-mon[37229]: pgmap v922: 129 pgs: 129 active+clean; 330 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 449 op/s 2024-06-13T07:14:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:16 smithi026 ceph-mon[32041]: pgmap v922: 129 pgs: 129 active+clean; 330 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 449 op/s 2024-06-13T07:14:17.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 11985 27.53 MB/sec warmup 30 sec latency 46.301 ms 2024-06-13T07:14:18.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 12451 27.22 MB/sec warmup 31 sec latency 114.463 ms 2024-06-13T07:14:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:18 smithi136 ceph-mon[37229]: pgmap v923: 129 pgs: 129 active+clean; 336 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 524 op/s 2024-06-13T07:14:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:18 smithi026 ceph-mon[32041]: pgmap v923: 129 pgs: 129 active+clean; 336 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 524 op/s 2024-06-13T07:14:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:18 smithi135 ceph-mon[36276]: pgmap v923: 129 pgs: 129 active+clean; 336 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 524 op/s 2024-06-13T07:14:19.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 13321 27.73 MB/sec warmup 32 sec latency 45.209 ms 2024-06-13T07:14:20.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 14285 28.33 MB/sec warmup 33 sec latency 150.599 ms 2024-06-13T07:14:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:20 smithi136 ceph-mon[37229]: pgmap v924: 129 pgs: 129 active+clean; 340 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.7 MiB/s wr, 550 op/s 2024-06-13T07:14:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:20 smithi026 ceph-mon[32041]: pgmap v924: 129 pgs: 129 active+clean; 340 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.7 MiB/s wr, 550 op/s 2024-06-13T07:14:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:20 smithi135 ceph-mon[36276]: pgmap v924: 129 pgs: 129 active+clean; 340 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.7 MiB/s wr, 550 op/s 2024-06-13T07:14:21.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15170 29.63 MB/sec warmup 34 sec latency 42.053 ms 2024-06-13T07:14:22.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 15565 29.13 MB/sec warmup 35 sec latency 61.469 ms 2024-06-13T07:14:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:22 smithi136 ceph-mon[37229]: pgmap v925: 129 pgs: 129 active+clean; 346 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 582 op/s 2024-06-13T07:14:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:22 smithi026 ceph-mon[32041]: pgmap v925: 129 pgs: 129 active+clean; 346 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 582 op/s 2024-06-13T07:14:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:22 smithi135 ceph-mon[36276]: pgmap v925: 129 pgs: 129 active+clean; 346 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 582 op/s 2024-06-13T07:14:23.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 16203 29.10 MB/sec warmup 36 sec latency 31.494 ms 2024-06-13T07:14:24.142 INFO:tasks.workunit.client.0.smithi026.stdout: 10 16900 28.99 MB/sec warmup 37 sec latency 124.236 ms 2024-06-13T07:14:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:24 smithi136 ceph-mon[37229]: pgmap v926: 129 pgs: 129 active+clean; 358 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.3 MiB/s wr, 613 op/s 2024-06-13T07:14:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:24 smithi026 ceph-mon[32041]: pgmap v926: 129 pgs: 129 active+clean; 358 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.3 MiB/s wr, 613 op/s 2024-06-13T07:14:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:24 smithi135 ceph-mon[36276]: pgmap v926: 129 pgs: 129 active+clean; 358 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.3 MiB/s wr, 613 op/s 2024-06-13T07:14:25.143 INFO:tasks.workunit.client.0.smithi026.stdout: 10 17916 29.96 MB/sec warmup 38 sec latency 37.168 ms 2024-06-13T07:14:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.731+0000 7fdbfb200640 1 -- 172.21.15.26:0/173560219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 msgr2=0x7fdbf4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.731+0000 7fdbfb200640 1 --2- 172.21.15.26:0/173560219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 0x7fdbf4100e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009900 tx=0x7fdbe002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.732+0000 7fdbfb200640 1 -- 172.21.15.26:0/173560219 shutdown_connections 2024-06-13T07:14:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.732+0000 7fdbfb200640 1 --2- 172.21.15.26:0/173560219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf4101370 0x7fdbf4101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.732+0000 7fdbfb200640 1 --2- 172.21.15.26:0/173560219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 0x7fdbf4100e30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.732+0000 7fdbfb200640 1 --2- 172.21.15.26:0/173560219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf40ff8a0 0x7fdbf40ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.732+0000 7fdbfb200640 1 -- 172.21.15.26:0/173560219 >> 172.21.15.26:0/173560219 conn(0x7fdbf40fb1c0 msgr2=0x7fdbf40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.732+0000 7fdbfb200640 1 -- 172.21.15.26:0/173560219 shutdown_connections 2024-06-13T07:14:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.732+0000 7fdbfb200640 1 -- 172.21.15.26:0/173560219 wait complete. 2024-06-13T07:14:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.734+0000 7fdbfb200640 1 Processor -- start 2024-06-13T07:14:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.734+0000 7fdbfb200640 1 -- start start 2024-06-13T07:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.735+0000 7fdbfb200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf40ff8a0 0x7fdbf419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.735+0000 7fdbfb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 0x7fdbf419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.735+0000 7fdbfb200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf4101370 0x7fdbf41a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.735+0000 7fdbfb200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410f8a0 con 0x7fdbf4101370 2024-06-13T07:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.735+0000 7fdbfb200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410f720 con 0x7fdbf40ff8a0 2024-06-13T07:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.735+0000 7fdbfb200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410fa20 con 0x7fdbf41009d0 2024-06-13T07:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf40ff8a0 0x7fdbf419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 0x7fdbf419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf40ff8a0 0x7fdbf419e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39162/0 (socket says 172.21.15.26:39162) 2024-06-13T07:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3e00640 1 -- 172.21.15.26:0/853168690 learned_addr learned my addr 172.21.15.26:0/853168690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf9200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf4101370 0x7fdbf41a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3400640 1 -- 172.21.15.26:0/853168690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf40ff8a0 msgr2=0x7fdbf419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3400640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf40ff8a0 0x7fdbf419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3400640 1 -- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf4101370 msgr2=0x7fdbf41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3400640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf4101370 0x7fdbf41a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf3400640 1 -- 172.21.15.26:0/853168690 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf41a5e50 con 0x7fdbf41009d0 2024-06-13T07:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.736+0000 7fdbf9200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf4101370 0x7fdbf41a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.737+0000 7fdbf3400640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 0x7fdbf419e770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdbe002a910 tx=0x7fdbe0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.737+0000 7fdbf0c00640 1 -- 172.21.15.26:0/853168690 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe003f030 con 0x7fdbf41009d0 2024-06-13T07:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.737+0000 7fdbfb200640 1 -- 172.21.15.26:0/853168690 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf41a60e0 con 0x7fdbf41009d0 2024-06-13T07:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.738+0000 7fdbf0c00640 1 -- 172.21.15.26:0/853168690 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe0003ce0 con 0x7fdbf41009d0 2024-06-13T07:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.738+0000 7fdbf0c00640 1 -- 172.21.15.26:0/853168690 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0035070 con 0x7fdbf41009d0 2024-06-13T07:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.738+0000 7fdbfb200640 1 -- 172.21.15.26:0/853168690 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf41a65f0 con 0x7fdbf41009d0 2024-06-13T07:14:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.743+0000 7fdbfdfec740 1 Processor -- start 2024-06-13T07:14:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.744+0000 7fdbfdfec740 1 -- start start 2024-06-13T07:14:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.744+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.744+0000 7fdbf0c00640 1 -- 172.21.15.26:0/853168690 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdbe0042030 con 0x7fdbf41009d0 2024-06-13T07:14:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfcb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfcdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5564fdf29420 con 0x5564fdfc5c00 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5564fded0c50 con 0x5564fdfc65f0 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5564fdf253e0 con 0x5564fdfcb760 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbf9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbf9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44918/0 (socket says 172.21.15.26:44918) 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbf9200640 1 -- 172.21.15.26:0/3392124809 learned_addr learned my addr 172.21.15.26:0/3392124809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbf3400640 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfcdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.745+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfcb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.746+0000 7fdbf3e00640 1 -- 172.21.15.26:0/3392124809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 msgr2=0x5564fdfcdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.746+0000 7fdbf0c00640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbdc0a2cb0 0x7fdbdc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.746+0000 7fdbf0c00640 1 -- 172.21.15.26:0/853168690 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdbe00ef980 con 0x7fdbf41009d0 2024-06-13T07:14:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.746+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfcdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.746+0000 7fdbf3e00640 1 -- 172.21.15.26:0/3392124809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 msgr2=0x5564fdfc6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.746+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.746+0000 7fdbf3e00640 1 -- 172.21.15.26:0/3392124809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fdc3d210 con 0x5564fdfc65f0 2024-06-13T07:14:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.746+0000 7fdbf9200640 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:14:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.747+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfcb0f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009c10 tx=0x7fdbe80227c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=831dbca61e23cc05 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.747+0000 7fdbf9200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbdc0a2cb0 0x7fdbdc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.747+0000 7fdbe5600640 1 -- 172.21.15.26:0/3392124809 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8009dc0 con 0x5564fdfc65f0 2024-06-13T07:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.748+0000 7fdbe5600640 1 -- 172.21.15.26:0/3392124809 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8005500 con 0x5564fdfc65f0 2024-06-13T07:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.748+0000 7fdbe5600640 1 -- 172.21.15.26:0/3392124809 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8009dc0 con 0x5564fdfc65f0 2024-06-13T07:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3392124809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 msgr2=0x5564fdfcb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfcb0f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009c10 tx=0x7fdbe80227c0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbf9200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbdc0a2cb0 0x7fdbdc0a5170 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5564fdfc6e50 tx=0x7fdbec004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3392124809 shutdown_connections 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfcb0f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfcdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3392124809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3392124809 >> 172.21.15.26:0/3392124809 conn(0x5564fdf8dfb0 msgr2=0x5564fdfc1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3392124809 shutdown_connections 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.749+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3392124809 wait complete. 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.750+0000 7fdbfdfec740 1 -- start start 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.750+0000 7fdbfdfec740 1 Processor -- start 2024-06-13T07:14:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfcfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfd0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfd7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5564fdfce760 con 0x5564fdfc5c00 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5564fdee40b0 con 0x5564fdfc65f0 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5564fdfce8e0 con 0x5564fdfcb760 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbfdfec740 10 client.0 _ll_register_callbacks cb 0x5564fdecff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfd0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfd0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39176/0 (socket says 172.21.15.26:39176) 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.751+0000 7fdbf3400640 1 -- 172.21.15.26:0/2545558340 learned_addr learned my addr 172.21.15.26:0/2545558340 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf3400640 1 -- 172.21.15.26:0/2545558340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 msgr2=0x5564fdfd7460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfcfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf9200640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfd7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf3400640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfd7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf3400640 1 -- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 msgr2=0x5564fdfcfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf3400640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfcfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf3400640 1 -- 172.21.15.26:0/2545558340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fdd09620 con 0x5564fdfc65f0 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf9200640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfd7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.752+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfcfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.753+0000 7fdbf3400640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfd0420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00c4530 tx=0x7fdbe0003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d8b086f2f9dd26 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.753+0000 7fdbda000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.753+0000 7fdbda000640 1 -- 172.21.15.26:0/2545558340 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0049070 con 0x5564fdfc65f0 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.753+0000 7fdbda000640 1 -- 172.21.15.26:0/2545558340 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdbe00b8460 con 0x5564fdfc65f0 2024-06-13T07:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.753+0000 7fdbfdfec740 1 -- 172.21.15.26:0/2545558340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe8005500 con 0x5564fdfc65f0 2024-06-13T07:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.753+0000 7fdbda000640 1 -- 172.21.15.26:0/2545558340 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbe00bbbc0 con 0x5564fdfc65f0 2024-06-13T07:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.753+0000 7fdbfdfec740 10 client.26564 fetch_fsmap learned FSMap version 41 2024-06-13T07:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.753+0000 7fdbfdfec740 1 -- 172.21.15.26:0/2545558340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5564fdfda7d0 con 0x5564fdfc65f0 2024-06-13T07:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.754+0000 7fdbda000640 1 -- 172.21.15.26:0/2545558340 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8005500 con 0x5564fdfc65f0 2024-06-13T07:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.754+0000 7fdbda000640 1 -- 172.21.15.26:0/2545558340 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fdbe003b460 con 0x5564fdfc65f0 2024-06-13T07:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.754+0000 7fdbfdfec740 10 client.26564 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.754+0000 7fdbfdfec740 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.754+0000 7fdbfdfec740 20 client.26564 populate_metadata read hostname 'smithi026' 2024-06-13T07:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.754+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5564fdfdc1c0 0x5564fdfde660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.754+0000 7fdbfdfec740 4 client.26564 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.754+0000 7fdbfdfec740 1 -- 172.21.15.26:0/2545558340 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564fdf36b50 con 0x5564fdfdc1c0 2024-06-13T07:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.755+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5564fdfdc1c0 0x5564fdfde660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.756+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5564fdfdc1c0 0x5564fdfde660 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f33c0be2097a4a3 server_cookie=54c7ed1fd02348d0 in_seq=0 out_seq=0 2024-06-13T07:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.756+0000 7fdbda000640 10 client.26564 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.759+0000 7fdbda000640 1 -- 172.21.15.26:0/2545558340 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5564fdf36b50 con 0x5564fdfdc1c0 2024-06-13T07:14:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.759+0000 7fdbda000640 10 client.26564 handle_command_reply: tid=0 2024-06-13T07:14:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.759+0000 7fdbfdfec740 1 client.26564 shutdown 2024-06-13T07:14:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbfdfec740 1 -- 172.21.15.26:0/2545558340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 msgr2=0x5564fdfd0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfd0420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00c4530 tx=0x7fdbe0003f50 comp rx=0 tx=0).stop 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbfdfec740 1 -- 172.21.15.26:0/2545558340 shutdown_connections 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5564fdfdc1c0 0x5564fdfde660 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfd0420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfd7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/2545558340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfcfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbfdfec740 1 -- 172.21.15.26:0/2545558340 >> 172.21.15.26:0/2545558340 conn(0x5564fdf8dfb0 msgr2=0x5564fdfc0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.760+0000 7fdbda000640 0 client.26564 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.761+0000 7fdbfdfec740 1 -- 172.21.15.26:0/2545558340 shutdown_connections 2024-06-13T07:14:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.761+0000 7fdbfdfec740 1 -- 172.21.15.26:0/2545558340 wait complete. 2024-06-13T07:14:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.761+0000 7fdbfdfec740 20 client.26564 trim_cache size 0 max 16384 2024-06-13T07:14:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.788+0000 7fdbfdfec740 1 Processor -- start 2024-06-13T07:14:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.788+0000 7fdbfdfec740 1 -- start start 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfc1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfc4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41aaee0 con 0x5564fdfc5c00 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41a8750 con 0x5564fdfc65f0 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbfdfec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf419d400 con 0x5564fdfcb760 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfc1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfc1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39186/0 (socket says 172.21.15.26:39186) 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbf3400640 1 -- 172.21.15.26:0/752253644 learned_addr learned my addr 172.21.15.26:0/752253644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.789+0000 7fdbf9200640 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfc4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbf3400640 1 -- 172.21.15.26:0/752253644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 msgr2=0x5564fdfc4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbf3400640 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfc4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbf3400640 1 -- 172.21.15.26:0/752253644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 msgr2=0x5564fdfc1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbf3400640 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbf3400640 1 -- 172.21.15.26:0/752253644 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fdfc2330 con 0x5564fdfc65f0 2024-06-13T07:14:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbf3400640 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfc1640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00ed400 tx=0x7fdbe00ba300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea5b89ca9611c11 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbe4c00640 1 -- 172.21.15.26:0/752253644 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0049070 con 0x5564fdfc65f0 2024-06-13T07:14:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbe4c00640 1 -- 172.21.15.26:0/752253644 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe00bc4f0 con 0x5564fdfc65f0 2024-06-13T07:14:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.790+0000 7fdbe4c00640 1 -- 172.21.15.26:0/752253644 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0005d50 con 0x5564fdfc65f0 2024-06-13T07:14:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 -- 172.21.15.26:0/752253644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 msgr2=0x5564fdfc1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfc1640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00ed400 tx=0x7fdbe00ba300 comp rx=0 tx=0).stop 2024-06-13T07:14:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 -- 172.21.15.26:0/752253644 shutdown_connections 2024-06-13T07:14:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc65f0 0x5564fdfc1640 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fdfc4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/752253644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc5c00 0x5564fdfc1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 -- 172.21.15.26:0/752253644 >> 172.21.15.26:0/752253644 conn(0x5564fdf8dfb0 msgr2=0x5564fdfc7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 -- 172.21.15.26:0/752253644 shutdown_connections 2024-06-13T07:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 -- 172.21.15.26:0/752253644 wait complete. 2024-06-13T07:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.792+0000 7fdbfdfec740 1 -- start start 2024-06-13T07:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.793+0000 7fdbfdfec740 1 Processor -- start 2024-06-13T07:14:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.793+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc5c00 0x5564fdfe3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.793+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc65f0 0x5564fdfe37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.793+0000 7fdbfdfec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fe044dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.793+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc5c00 0x5564fdfe3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc5c00 0x5564fdfe3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39192/0 (socket says 172.21.15.26:39192) 2024-06-13T07:14:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3e00640 1 -- 172.21.15.26:0/3153651289 learned_addr learned my addr 172.21.15.26:0/3153651289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf9200640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fe044dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41a6c10 con 0x5564fdfc65f0 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41a8080 con 0x5564fdfc5c00 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41a72e0 con 0x5564fdfcb760 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbfdfec740 10 client.0 _ll_register_callbacks cb 0x7fdbe0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3400640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc65f0 0x5564fdfe37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3e00640 1 -- 172.21.15.26:0/3153651289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 msgr2=0x5564fe044dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fe044dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3e00640 1 -- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc65f0 msgr2=0x5564fdfe37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc65f0 0x5564fdfe37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3e00640 1 -- 172.21.15.26:0/3153651289 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fe045310 con 0x5564fdfc5c00 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf3400640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc65f0 0x5564fdfe37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:14:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.794+0000 7fdbf9200640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fe044dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.795+0000 7fdbf3e00640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc5c00 0x5564fdfe3260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009b60 tx=0x7fdbe8005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c063047c47f2b36 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.795+0000 7fdbda000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.795+0000 7fdbda000640 1 -- 172.21.15.26:0/3153651289 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8043060 con 0x5564fdfc5c00 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.795+0000 7fdbda000640 1 -- 172.21.15.26:0/3153651289 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdbe8027040 con 0x5564fdfc5c00 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.795+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe00bc4f0 con 0x5564fdfc5c00 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.795+0000 7fdbda000640 1 -- 172.21.15.26:0/3153651289 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbe804a050 con 0x5564fdfc5c00 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.795+0000 7fdbfdfec740 10 client.26570 fetch_fsmap learned FSMap version 41 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.795+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5564fe046480 con 0x5564fdfc5c00 2024-06-13T07:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.796+0000 7fdbda000640 1 -- 172.21.15.26:0/3153651289 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe00bc4f0 con 0x5564fdfc5c00 2024-06-13T07:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.796+0000 7fdbda000640 1 -- 172.21.15.26:0/3153651289 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fdbe803d070 con 0x5564fdfc5c00 2024-06-13T07:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.796+0000 7fdbfdfec740 10 client.26570 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.796+0000 7fdbfdfec740 10 client.26570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.796+0000 7fdbfdfec740 20 client.26570 populate_metadata read hostname 'smithi026' 2024-06-13T07:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.796+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5564fe046d20 0x5564fe049140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.796+0000 7fdbfdfec740 4 client.26570 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.796+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564fdf30ea0 con 0x5564fe046d20 2024-06-13T07:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.797+0000 7fdbf3400640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5564fe046d20 0x5564fe049140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.798+0000 7fdbda000640 10 client.26570 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:25.798+0000 7fdbf3400640 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5564fe046d20 0x5564fe049140 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6678a8855f75c4 server_cookie=342873fbc70c33a5 in_seq=0 out_seq=0 2024-06-13T07:14:26.144 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18505 30.08 MB/sec warmup 39 sec latency 197.387 ms 2024-06-13T07:14:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:26 smithi136 ceph-mon[37229]: pgmap v927: 129 pgs: 129 active+clean; 360 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-06-13T07:14:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:26 smithi026 ceph-mon[32041]: pgmap v927: 129 pgs: 129 active+clean; 360 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-06-13T07:14:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:26 smithi135 ceph-mon[36276]: pgmap v927: 129 pgs: 129 active+clean; 360 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-06-13T07:14:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.717+0000 7fdbda000640 1 -- 172.21.15.26:0/3153651289 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5564fdf30ea0 con 0x5564fe046d20 2024-06-13T07:14:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.717+0000 7fdbda000640 10 client.26570 handle_command_reply: tid=0 2024-06-13T07:14:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.717+0000 7fdbfdfec740 1 client.26570 shutdown 2024-06-13T07:14:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.718+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc5c00 msgr2=0x5564fdfe3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.718+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc5c00 0x5564fdfe3260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009b60 tx=0x7fdbe8005f10 comp rx=0 tx=0).stop 2024-06-13T07:14:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 shutdown_connections 2024-06-13T07:14:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5564fe046d20 0x5564fe049140 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564fdfcb760 0x5564fe044dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5564fdfc65f0 0x5564fdfe37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbda000640 0 client.26570 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 1 --2- 172.21.15.26:0/3153651289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564fdfc5c00 0x5564fdfe3260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 >> 172.21.15.26:0/3153651289 conn(0x5564fdf8dfb0 msgr2=0x5564fdfcfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 shutdown_connections 2024-06-13T07:14:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 1 -- 172.21.15.26:0/3153651289 wait complete. 2024-06-13T07:14:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.719+0000 7fdbfdfec740 20 client.26570 trim_cache size 0 max 16384 2024-06-13T07:14:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:14:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:14:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.723+0000 7fdbfb200640 1 -- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbdc0a2cb0 msgr2=0x7fdbdc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.723+0000 7fdbfb200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbdc0a2cb0 0x7fdbdc0a5170 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5564fdfc6e50 tx=0x7fdbec004ab0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.723+0000 7fdbfb200640 1 -- 172.21.15.26:0/853168690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 msgr2=0x7fdbf419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.723+0000 7fdbfb200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 0x7fdbf419e770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdbe002a910 tx=0x7fdbe0032ce0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.724+0000 7fdbfb200640 1 -- 172.21.15.26:0/853168690 shutdown_connections 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.724+0000 7fdbfb200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdbdc0a2cb0 0x7fdbdc0a5170 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.724+0000 7fdbfb200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf4101370 0x7fdbf41a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.724+0000 7fdbfb200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf41009d0 0x7fdbf419e770 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.724+0000 7fdbfb200640 1 --2- 172.21.15.26:0/853168690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf40ff8a0 0x7fdbf419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.724+0000 7fdbfb200640 1 -- 172.21.15.26:0/853168690 >> 172.21.15.26:0/853168690 conn(0x7fdbf40fb1c0 msgr2=0x7fdbf40fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.725+0000 7fdbfb200640 1 -- 172.21.15.26:0/853168690 shutdown_connections 2024-06-13T07:14:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:26.726+0000 7fdbfb200640 1 -- 172.21.15.26:0/853168690 wait complete. 2024-06-13T07:14:27.143 INFO:tasks.workunit.client.0.smithi026.stdout: 10 18689 29.65 MB/sec warmup 40 sec latency 561.736 ms 2024-06-13T07:14:28.143 INFO:tasks.workunit.client.0.smithi026.stdout: 10 19324 29.72 MB/sec warmup 41 sec latency 42.538 ms 2024-06-13T07:14:28.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:28 smithi136 ceph-mon[37229]: pgmap v928: 129 pgs: 129 active+clean; 371 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 548 op/s 2024-06-13T07:14:28.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:28 smithi026 ceph-mon[32041]: pgmap v928: 129 pgs: 129 active+clean; 371 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 548 op/s 2024-06-13T07:14:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:28 smithi135 ceph-mon[36276]: pgmap v928: 129 pgs: 129 active+clean; 371 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 548 op/s 2024-06-13T07:14:29.143 INFO:tasks.workunit.client.0.smithi026.stdout: 10 20017 29.84 MB/sec warmup 42 sec latency 65.865 ms 2024-06-13T07:14:30.143 INFO:tasks.workunit.client.0.smithi026.stdout: 10 20766 29.88 MB/sec warmup 43 sec latency 82.198 ms 2024-06-13T07:14:30.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:30 smithi136 ceph-mon[37229]: pgmap v929: 129 pgs: 129 active+clean; 375 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-06-13T07:14:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:30 smithi026 ceph-mon[32041]: pgmap v929: 129 pgs: 129 active+clean; 375 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-06-13T07:14:30.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:30 smithi135 ceph-mon[36276]: pgmap v929: 129 pgs: 129 active+clean; 375 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-06-13T07:14:31.143 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21512 30.34 MB/sec warmup 44 sec latency 178.826 ms 2024-06-13T07:14:32.143 INFO:tasks.workunit.client.0.smithi026.stdout: 10 21734 30.06 MB/sec warmup 45 sec latency 561.132 ms 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: pgmap v930: 129 pgs: 129 active+clean; 377 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.5 MiB/s wr, 579 op/s 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:32 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: pgmap v930: 129 pgs: 129 active+clean; 377 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.5 MiB/s wr, 579 op/s 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:32 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: pgmap v930: 129 pgs: 129 active+clean; 377 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.5 MiB/s wr, 579 op/s 2024-06-13T07:14:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:14:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:32.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:32 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:14:33.144 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22082 29.69 MB/sec warmup 46 sec latency 125.243 ms 2024-06-13T07:14:33.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:33 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:14:33.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:33 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:14:33.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:33 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:14:33.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:33 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:14:33.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:33 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:14:33.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:33 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:14:33.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:33 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:14:33.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:33 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:14:33.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:33 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:14:33.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:33 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:14:33.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:33 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:14:33.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:33 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:14:33.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:33 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:14:33.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:33 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:14:33.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:33 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:14:33.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:33 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:14:33.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:33 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:14:33.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:33 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:14:34.144 INFO:tasks.workunit.client.0.smithi026.stdout: 10 22708 29.76 MB/sec warmup 47 sec latency 205.946 ms 2024-06-13T07:14:34.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:34 smithi136 ceph-mon[37229]: pgmap v931: 129 pgs: 129 active+clean; 380 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 7.2 MiB/s wr, 611 op/s 2024-06-13T07:14:34.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:34 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:14:34.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:34 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:34.402 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:34 smithi026 ceph-mon[32041]: pgmap v931: 129 pgs: 129 active+clean; 380 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 7.2 MiB/s wr, 611 op/s 2024-06-13T07:14:34.402 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:34 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:14:34.402 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:34 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:34 smithi135 ceph-mon[36276]: pgmap v931: 129 pgs: 129 active+clean; 380 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 7.2 MiB/s wr, 611 op/s 2024-06-13T07:14:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:34 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:14:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:34 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:14:35.144 INFO:tasks.workunit.client.0.smithi026.stdout: 10 23474 29.90 MB/sec warmup 48 sec latency 55.977 ms 2024-06-13T07:14:36.144 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24333 30.36 MB/sec warmup 49 sec latency 71.737 ms 2024-06-13T07:14:36.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:36 smithi136 ceph-mon[37229]: pgmap v932: 129 pgs: 129 active+clean; 386 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 606 op/s 2024-06-13T07:14:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:36 smithi026 ceph-mon[32041]: pgmap v932: 129 pgs: 129 active+clean; 386 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 606 op/s 2024-06-13T07:14:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:36 smithi135 ceph-mon[36276]: pgmap v932: 129 pgs: 129 active+clean; 386 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 606 op/s 2024-06-13T07:14:37.144 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24529 30.00 MB/sec warmup 50 sec latency 442.364 ms 2024-06-13T07:14:38.144 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24583 29.42 MB/sec warmup 51 sec latency 1414.460 ms 2024-06-13T07:14:38.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:38 smithi026 ceph-mon[32041]: pgmap v933: 129 pgs: 129 active+clean; 383 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 700 op/s 2024-06-13T07:14:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:38 smithi135 ceph-mon[36276]: pgmap v933: 129 pgs: 129 active+clean; 383 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 700 op/s 2024-06-13T07:14:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:38 smithi136 ceph-mon[37229]: pgmap v933: 129 pgs: 129 active+clean; 383 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 700 op/s 2024-06-13T07:14:39.144 INFO:tasks.workunit.client.0.smithi026.stdout: 10 24853 29.04 MB/sec warmup 52 sec latency 1724.054 ms 2024-06-13T07:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.951+0000 7f79aa200640 1 -- 172.21.15.26:0/367735952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a4077f40 msgr2=0x7f79a40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.951+0000 7f79aa200640 1 --2- 172.21.15.26:0/367735952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a4077f40 0x7f79a40787e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f799c009be0 tx=0x7f799c02a6c0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.952+0000 7f79aa200640 1 -- 172.21.15.26:0/367735952 shutdown_connections 2024-06-13T07:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.952+0000 7f79aa200640 1 --2- 172.21.15.26:0/367735952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a4077f40 0x7f79a40787e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.952+0000 7f79aa200640 1 --2- 172.21.15.26:0/367735952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79a40775a0 0x7f79a4077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.952+0000 7f79aa200640 1 --2- 172.21.15.26:0/367735952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79a4076360 0x7f79a4076780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.952+0000 7f79aa200640 1 -- 172.21.15.26:0/367735952 >> 172.21.15.26:0/367735952 conn(0x7f79a40ff190 msgr2=0x7f79a41015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.952+0000 7f79aa200640 1 -- 172.21.15.26:0/367735952 shutdown_connections 2024-06-13T07:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.952+0000 7f79aa200640 1 -- 172.21.15.26:0/367735952 wait complete. 2024-06-13T07:14:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.953+0000 7f79aa200640 1 Processor -- start 2024-06-13T07:14:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.954+0000 7f79aa200640 1 -- start start 2024-06-13T07:14:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.954+0000 7f79aa200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a4076360 0x7f79a4071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.954+0000 7f79aa200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79a40775a0 0x7f79a4071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.954+0000 7f79aa200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79a4077f40 0x7f79a4072b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.954+0000 7f79aa200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79a406bdd0 con 0x7f79a4076360 2024-06-13T07:14:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.954+0000 7f79aa200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f79a406bc50 con 0x7f79a4077f40 2024-06-13T07:14:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.954+0000 7f79aa200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79a406bf50 con 0x7f79a40775a0 2024-06-13T07:14:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.955+0000 7f79a3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79a4077f40 0x7f79a4072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.955+0000 7f79a3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79a4077f40 0x7f79a4072b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47476/0 (socket says 172.21.15.26:47476) 2024-06-13T07:14:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.955+0000 7f79a3e00640 1 -- 172.21.15.26:0/4160973026 learned_addr learned my addr 172.21.15.26:0/4160973026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.955+0000 7f79a3e00640 1 -- 172.21.15.26:0/4160973026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79a40775a0 msgr2=0x7f79a4071ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:14:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.955+0000 7f79a3e00640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79a40775a0 0x7f79a4071ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.955+0000 7f79a3e00640 1 -- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a4076360 msgr2=0x7f79a4071660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:14:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.955+0000 7f79a3400640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a4076360 0x7f79a4071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.956+0000 7f79a3e00640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a4076360 0x7f79a4071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.956+0000 7f79a3e00640 1 -- 172.21.15.26:0/4160973026 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a4073200 con 0x7f79a4077f40 2024-06-13T07:14:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.956+0000 7f79a3e00640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79a4077f40 0x7f79a4072b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f799c034040 tx=0x7f799c0305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.957+0000 7f79a0c00640 1 -- 172.21.15.26:0/4160973026 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c041020 con 0x7f79a4077f40 2024-06-13T07:14:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.957+0000 7f79a0c00640 1 -- 172.21.15.26:0/4160973026 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c003e50 con 0x7f79a4077f40 2024-06-13T07:14:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.957+0000 7f79aa200640 1 -- 172.21.15.26:0/4160973026 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a4073490 con 0x7f79a4077f40 2024-06-13T07:14:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.957+0000 7f79a0c00640 1 -- 172.21.15.26:0/4160973026 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c036040 con 0x7f79a4077f40 2024-06-13T07:14:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.957+0000 7f79aa200640 1 -- 172.21.15.26:0/4160973026 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a4073980 con 0x7f79a4077f40 2024-06-13T07:14:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.964+0000 7f79a0c00640 1 -- 172.21.15.26:0/4160973026 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f799c044020 con 0x7f79a4077f40 2024-06-13T07:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.967+0000 7f79ad166740 1 Processor -- start 2024-06-13T07:14:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.968+0000 7f79ad166740 1 -- start start 2024-06-13T07:14:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.968+0000 7f79a0c00640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79880a2d00 0x7f79880a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.968+0000 7f79ad166740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be57b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.968+0000 7f79ad166740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 0x5594be5800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.968+0000 7f79ad166740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be582bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.968+0000 7f79ad166740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5594be4de420 con 0x5594be57b5f0 2024-06-13T07:14:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.968+0000 7f79ad166740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5594be485c50 con 0x5594be57ac00 2024-06-13T07:14:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.968+0000 7f79ad166740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5594be4da3e0 con 0x5594be580760 2024-06-13T07:14:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a0c00640 1 -- 172.21.15.26:0/4160973026 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f799c0efcf0 con 0x7f79a4077f40 2024-06-13T07:14:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be582bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be582bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52892/0 (socket says 172.21.15.26:52892) 2024-06-13T07:14:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a3e00640 1 -- 172.21.15.26:0/1330667510 learned_addr learned my addr 172.21.15.26:0/1330667510 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a3e00640 1 -- 172.21.15.26:0/1330667510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 msgr2=0x5594be57b020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:14:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a3e00640 1 --2- 172.21.15.26:0/1330667510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be57b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a3e00640 1 -- 172.21.15.26:0/1330667510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 msgr2=0x5594be5800f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:14:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a3e00640 1 --2- 172.21.15.26:0/1330667510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 0x5594be5800f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.969+0000 7f79a3e00640 1 -- 172.21.15.26:0/1330667510 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594be1f2210 con 0x5594be580760 2024-06-13T07:14:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.970+0000 7f799be00640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79880a2d00 0x7f79880a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.970+0000 7f79a3e00640 1 --2- 172.21.15.26:0/1330667510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be582bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f799c0c48a0 tx=0x7f799c02a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d234614410ed51c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.970+0000 7f7993e00640 1 -- 172.21.15.26:0/1330667510 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0bc160 con 0x5594be580760 2024-06-13T07:14:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.970+0000 7f7993e00640 1 -- 172.21.15.26:0/1330667510 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c0c0510 con 0x5594be580760 2024-06-13T07:14:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.970+0000 7f7993e00640 1 -- 172.21.15.26:0/1330667510 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0b87d0 con 0x5594be580760 2024-06-13T07:14:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.971+0000 7f79ad166740 1 -- 172.21.15.26:0/1330667510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 msgr2=0x5594be582bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.971+0000 7f79ad166740 1 --2- 172.21.15.26:0/1330667510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be582bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f799c0c48a0 tx=0x7f799c02a6c0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.971+0000 7f79ad166740 1 -- 172.21.15.26:0/1330667510 shutdown_connections 2024-06-13T07:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.971+0000 7f79ad166740 1 --2- 172.21.15.26:0/1330667510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be582bf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.971+0000 7f79ad166740 1 --2- 172.21.15.26:0/1330667510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 0x5594be5800f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.971+0000 7f79ad166740 1 --2- 172.21.15.26:0/1330667510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be57b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.972+0000 7f79ad166740 1 -- 172.21.15.26:0/1330667510 >> 172.21.15.26:0/1330667510 conn(0x5594be542fb0 msgr2=0x5594be576680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.972+0000 7f79ad166740 1 -- 172.21.15.26:0/1330667510 shutdown_connections 2024-06-13T07:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.972+0000 7f79ad166740 1 -- 172.21.15.26:0/1330667510 wait complete. 2024-06-13T07:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.973+0000 7f79ad166740 1 -- start start 2024-06-13T07:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.973+0000 7f799be00640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79880a2d00 0x7f79880a51c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5594be57b9f0 tx=0x7f798c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.973+0000 7f79ad166740 1 Processor -- start 2024-06-13T07:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.973+0000 7f79ad166740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be584b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.973+0000 7f79ad166740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be585070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.973+0000 7f79ad166740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be580760 0x5594be58c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.974+0000 7f799be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be585070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.974+0000 7f799be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be585070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52898/0 (socket says 172.21.15.26:52898) 2024-06-13T07:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.974+0000 7f79a3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be584b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.974+0000 7f79a3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be584b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47492/0 (socket says 172.21.15.26:47492) 2024-06-13T07:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.974+0000 7f79ad166740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5594be4da3e0 con 0x5594be580760 2024-06-13T07:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.974+0000 7f79ad166740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5594be4990b0 con 0x5594be57ac00 2024-06-13T07:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.974+0000 7f79ad166740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5594be5834d0 con 0x5594be57b5f0 2024-06-13T07:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.974+0000 7f799be00640 1 -- 172.21.15.26:0/250830289 learned_addr learned my addr 172.21.15.26:0/250830289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.975+0000 7f79a3e00640 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be580760 0x5594be58c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.975+0000 7f799be00640 1 -- 172.21.15.26:0/250830289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 msgr2=0x5594be584b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.975+0000 7f799be00640 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be584b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.975+0000 7f799be00640 1 -- 172.21.15.26:0/250830289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be580760 msgr2=0x5594be58c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.975+0000 7f799be00640 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be580760 0x5594be58c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.975+0000 7f799be00640 1 -- 172.21.15.26:0/250830289 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594be2be620 con 0x5594be57b5f0 2024-06-13T07:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.976+0000 7f79a3400640 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be584b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.976+0000 7f79ad166740 10 client.0 _ll_register_callbacks cb 0x5594be484f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.976+0000 7f79ad166740 1 -- 172.21.15.26:0/250830289 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c0c0510 con 0x5594be57b5f0 2024-06-13T07:14:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.976+0000 7f799be00640 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be585070 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f798c0106e0 tx=0x7f798c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f34bfa1e350a2ac server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.976+0000 7f7990c00640 10 client.26602 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.977+0000 7f7990c00640 1 -- 172.21.15.26:0/250830289 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c020070 con 0x5594be57b5f0 2024-06-13T07:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.977+0000 7f7990c00640 1 -- 172.21.15.26:0/250830289 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f798c005220 con 0x5594be57b5f0 2024-06-13T07:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.978+0000 7f7990c00640 1 -- 172.21.15.26:0/250830289 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f798c005460 con 0x5594be57b5f0 2024-06-13T07:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.978+0000 7f79ad166740 10 client.26602 fetch_fsmap learned FSMap version 42 2024-06-13T07:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.978+0000 7f79ad166740 1 -- 172.21.15.26:0/250830289 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=42}) -- 0x5594be58f0b0 con 0x5594be57b5f0 2024-06-13T07:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.979+0000 7f7990c00640 1 -- 172.21.15.26:0/250830289 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f798c0047a0 con 0x5594be57b5f0 2024-06-13T07:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.980+0000 7f79ad166740 10 client.26602 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.980+0000 7f79ad166740 10 client.26602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.980+0000 7f79ad166740 20 client.26602 populate_metadata read hostname 'smithi026' 2024-06-13T07:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.980+0000 7f79ad166740 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5594be590820 0x5594be592ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.980+0000 7f79ad166740 4 client.26602 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.980+0000 7f79ad166740 1 -- 172.21.15.26:0/250830289 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594be4ebb50 con 0x5594be590820 2024-06-13T07:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.980+0000 7f79a3400640 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5594be590820 0x5594be592ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.982+0000 7f79a3400640 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5594be590820 0x5594be592ce0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d517c524c2ddfba server_cookie=557c7cb4e9ce099b in_seq=0 out_seq=0 2024-06-13T07:14:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.982+0000 7f7990c00640 10 client.26602 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f7990c00640 1 -- 172.21.15.26:0/250830289 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5594be4ebb50 con 0x5594be590820 2024-06-13T07:14:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f7990c00640 10 client.26602 handle_command_reply: tid=0 2024-06-13T07:14:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 client.26602 shutdown 2024-06-13T07:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 -- 172.21.15.26:0/250830289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 msgr2=0x5594be585070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be585070 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f798c0106e0 tx=0x7f798c0046c0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 -- 172.21.15.26:0/250830289 shutdown_connections 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5594be590820 0x5594be592ce0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be580760 0x5594be58c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be585070 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 --2- 172.21.15.26:0/250830289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be584b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.998+0000 7f79ad166740 1 -- 172.21.15.26:0/250830289 >> 172.21.15.26:0/250830289 conn(0x5594be542fb0 msgr2=0x5594be575840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.999+0000 7f7990c00640 0 client.26602 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.999+0000 7f79ad166740 1 -- 172.21.15.26:0/250830289 shutdown_connections 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.999+0000 7f79ad166740 1 -- 172.21.15.26:0/250830289 wait complete. 2024-06-13T07:14:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:39.999+0000 7f79ad166740 20 client.26602 trim_cache size 0 max 16384 2024-06-13T07:14:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.042+0000 7f79ad166740 1 Processor -- start 2024-06-13T07:14:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.042+0000 7f79ad166740 1 -- start start 2024-06-13T07:14:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.042+0000 7f79ad166740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be579ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.042+0000 7f79ad166740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 0x5594be58c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.042+0000 7f79ad166740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be584150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.043+0000 7f79ad166740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41bbe70 con 0x5594be57b5f0 2024-06-13T07:14:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.043+0000 7f79ad166740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b96b0 con 0x5594be57ac00 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.043+0000 7f79ad166740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b7500 con 0x5594be580760 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.043+0000 7f79a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be584150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.043+0000 7f79a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be584150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52910/0 (socket says 172.21.15.26:52910) 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.043+0000 7f79a3e00640 1 -- 172.21.15.26:0/3918959356 learned_addr learned my addr 172.21.15.26:0/3918959356 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.043+0000 7f79a3400640 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be579ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.043+0000 7f799be00640 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 0x5594be58c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.044+0000 7f79a3400640 1 -- 172.21.15.26:0/3918959356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 msgr2=0x5594be584150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.044+0000 7f79a3400640 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be584150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.044+0000 7f79a3400640 1 -- 172.21.15.26:0/3918959356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 msgr2=0x5594be58c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.044+0000 7f79a3400640 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 0x5594be58c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.044+0000 7f79a3400640 1 -- 172.21.15.26:0/3918959356 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594be57fc00 con 0x5594be57ac00 2024-06-13T07:14:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.044+0000 7f79a3e00640 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be584150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.045+0000 7f79a3400640 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be579ca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5594be5934f0 tx=0x7f7994009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997732415b2e7465 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.045+0000 7f7993400640 1 -- 172.21.15.26:0/3918959356 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994004220 con 0x5594be57ac00 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.046+0000 7f7993400640 1 -- 172.21.15.26:0/3918959356 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799401f030 con 0x5594be57ac00 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.046+0000 7f7993400640 1 -- 172.21.15.26:0/3918959356 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799401b650 con 0x5594be57ac00 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.047+0000 7f79ad166740 1 -- 172.21.15.26:0/3918959356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 msgr2=0x5594be579ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.047+0000 7f79ad166740 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be579ca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5594be5934f0 tx=0x7f7994009260 comp rx=0 tx=0).stop 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.048+0000 7f79ad166740 1 -- 172.21.15.26:0/3918959356 shutdown_connections 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.048+0000 7f79ad166740 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be580760 0x5594be584150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.048+0000 7f79ad166740 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57b5f0 0x5594be58c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.048+0000 7f79ad166740 1 --2- 172.21.15.26:0/3918959356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be57ac00 0x5594be579ca0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.048+0000 7f79ad166740 1 -- 172.21.15.26:0/3918959356 >> 172.21.15.26:0/3918959356 conn(0x5594be542fb0 msgr2=0x5594be579590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.048+0000 7f79ad166740 1 -- 172.21.15.26:0/3918959356 shutdown_connections 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.048+0000 7f79ad166740 1 -- 172.21.15.26:0/3918959356 wait complete. 2024-06-13T07:14:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.049+0000 7f79ad166740 1 -- start start 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.049+0000 7f79ad166740 1 Processor -- start 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.051+0000 7f79ad166740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57ac00 0x5594be58c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.051+0000 7f79ad166740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be588b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.051+0000 7f79ad166740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be580760 0x5594be5890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.052+0000 7f799be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be588b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.052+0000 7f799be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be588b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52922/0 (socket says 172.21.15.26:52922) 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.052+0000 7f79ad166740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b7b70 con 0x5594be57ac00 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.052+0000 7f799be00640 1 -- 172.21.15.26:0/3324221753 learned_addr learned my addr 172.21.15.26:0/3324221753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.052+0000 7f79ad166740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b8fe0 con 0x5594be580760 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.052+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b8240 con 0x5594be57b5f0 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.053+0000 7f79a3e00640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be580760 0x5594be5890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.053+0000 7f79ad166740 10 client.0 _ll_register_callbacks cb 0x7f798c014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:14:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.053+0000 7f79a3400640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57ac00 0x5594be58c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.054+0000 7f799be00640 1 -- 172.21.15.26:0/3324221753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be580760 msgr2=0x5594be5890c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.054+0000 7f799be00640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be580760 0x5594be5890c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.055+0000 7f799be00640 1 -- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57ac00 msgr2=0x5594be58c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.055+0000 7f799be00640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57ac00 0x5594be58c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.055+0000 7f799be00640 1 -- 172.21.15.26:0/3324221753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594be5f9570 con 0x5594be57b5f0 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.055+0000 7f79a3e00640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be580760 0x5594be5890c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.055+0000 7f79a3400640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57ac00 0x5594be58c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.056+0000 7f799be00640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be588b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f798c009fd0 tx=0x7f798c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9727e4331c880a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.056+0000 7f7990c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:14:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.056+0000 7f7990c00640 1 -- 172.21.15.26:0/3324221753 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c020070 con 0x5594be57b5f0 2024-06-13T07:14:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.057+0000 7f7990c00640 1 -- 172.21.15.26:0/3324221753 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f798c009040 con 0x5594be57b5f0 2024-06-13T07:14:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.057+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799401f030 con 0x5594be57b5f0 2024-06-13T07:14:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.057+0000 7f7990c00640 1 -- 172.21.15.26:0/3324221753 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f798c004ca0 con 0x5594be57b5f0 2024-06-13T07:14:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.058+0000 7f79ad166740 10 client.26608 fetch_fsmap learned FSMap version 42 2024-06-13T07:14:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.058+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5594be5fa8f0 con 0x5594be57b5f0 2024-06-13T07:14:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.059+0000 7f7990c00640 1 -- 172.21.15.26:0/3324221753 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799401f030 con 0x5594be57b5f0 2024-06-13T07:14:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.059+0000 7f7990c00640 1 -- 172.21.15.26:0/3324221753 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f798c005020 con 0x5594be57b5f0 2024-06-13T07:14:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.059+0000 7f79ad166740 10 client.26608 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:14:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.059+0000 7f79ad166740 10 client.26608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:14:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.060+0000 7f79ad166740 20 client.26608 populate_metadata read hostname 'smithi026' 2024-06-13T07:14:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.060+0000 7f79ad166740 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5594be5fb210 0x5594be5fd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.061+0000 7f79ad166740 4 client.26608 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:14:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.061+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594be4e5ea0 con 0x5594be5fb210 2024-06-13T07:14:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.065+0000 7f79a3400640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5594be5fb210 0x5594be5fd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.068+0000 7f7990c00640 10 client.26608 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.068+0000 7f79a3400640 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5594be5fb210 0x5594be5fd630 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1895d2b1c8c724 server_cookie=e1b9b40c46d3e924 in_seq=0 out_seq=0 2024-06-13T07:14:40.145 INFO:tasks.workunit.client.0.smithi026.stdout: 10 25162 28.82 MB/sec warmup 53 sec latency 195.956 ms 2024-06-13T07:14:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.357+0000 7f7990c00640 1 -- 172.21.15.26:0/3324221753 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5594be4e5ea0 con 0x5594be5fb210 2024-06-13T07:14:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.357+0000 7f7990c00640 10 client.26608 handle_command_reply: tid=0 2024-06-13T07:14:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.357+0000 7f79ad166740 1 client.26608 shutdown 2024-06-13T07:14:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 msgr2=0x5594be588b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be588b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f798c009fd0 tx=0x7f798c016000 comp rx=0 tx=0).stop 2024-06-13T07:14:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 shutdown_connections 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5594be5fb210 0x5594be5fd630 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5594be580760 0x5594be5890c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594be57b5f0 0x5594be588b40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 --2- 172.21.15.26:0/3324221753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5594be57ac00 0x5594be58c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 >> 172.21.15.26:0/3324221753 conn(0x5594be542fb0 msgr2=0x5594be574fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f7990c00640 0 client.26608 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 shutdown_connections 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.358+0000 7f79ad166740 1 -- 172.21.15.26:0/3324221753 wait complete. 2024-06-13T07:14:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.359+0000 7f79ad166740 20 client.26608 trim_cache size 0 max 16384 2024-06-13T07:14:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:14:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:14:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:14:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.359+0000 7f79aa200640 1 -- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79880a2d00 msgr2=0x7f79880a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.359+0000 7f79aa200640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79880a2d00 0x7f79880a51c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5594be57b9f0 tx=0x7f798c005f90 comp rx=0 tx=0).stop 2024-06-13T07:14:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.359+0000 7f79aa200640 1 -- 172.21.15.26:0/4160973026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79a4077f40 msgr2=0x7f79a4072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.359+0000 7f79aa200640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79a4077f40 0x7f79a4072b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f799c034040 tx=0x7f799c0305d0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.360+0000 7f79aa200640 1 -- 172.21.15.26:0/4160973026 shutdown_connections 2024-06-13T07:14:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.360+0000 7f79aa200640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f79880a2d00 0x7f79880a51c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.360+0000 7f79aa200640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79a4077f40 0x7f79a4072b90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.360+0000 7f79aa200640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79a40775a0 0x7f79a4071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.360+0000 7f79aa200640 1 --2- 172.21.15.26:0/4160973026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a4076360 0x7f79a4071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.360+0000 7f79aa200640 1 -- 172.21.15.26:0/4160973026 >> 172.21.15.26:0/4160973026 conn(0x7f79a40ff190 msgr2=0x7f79a4100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.360+0000 7f79aa200640 1 -- 172.21.15.26:0/4160973026 shutdown_connections 2024-06-13T07:14:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:40.360+0000 7f79aa200640 1 -- 172.21.15.26:0/4160973026 wait complete. 2024-06-13T07:14:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:40 smithi026 ceph-mon[32041]: pgmap v934: 129 pgs: 129 active+clean; 431 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-06-13T07:14:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:40 smithi135 ceph-mon[36276]: pgmap v934: 129 pgs: 129 active+clean; 431 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-06-13T07:14:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:40 smithi136 ceph-mon[37229]: pgmap v934: 129 pgs: 129 active+clean; 431 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-06-13T07:14:41.146 INFO:tasks.workunit.client.0.smithi026.stdout: 10 25580 28.77 MB/sec warmup 54 sec latency 197.202 ms 2024-06-13T07:14:42.146 INFO:tasks.workunit.client.0.smithi026.stdout: 10 26352 28.83 MB/sec warmup 55 sec latency 53.704 ms 2024-06-13T07:14:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:42 smithi026 ceph-mon[32041]: pgmap v935: 129 pgs: 129 active+clean; 450 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 713 op/s 2024-06-13T07:14:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:42 smithi135 ceph-mon[36276]: pgmap v935: 129 pgs: 129 active+clean; 450 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 713 op/s 2024-06-13T07:14:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:42 smithi136 ceph-mon[37229]: pgmap v935: 129 pgs: 129 active+clean; 450 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 713 op/s 2024-06-13T07:14:43.146 INFO:tasks.workunit.client.0.smithi026.stdout: 10 27136 29.24 MB/sec warmup 56 sec latency 60.177 ms 2024-06-13T07:14:44.146 INFO:tasks.workunit.client.0.smithi026.stdout: 10 27820 29.37 MB/sec warmup 57 sec latency 42.079 ms 2024-06-13T07:14:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:44 smithi026 ceph-mon[32041]: pgmap v936: 129 pgs: 129 active+clean; 423 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 784 op/s 2024-06-13T07:14:44.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:44 smithi135 ceph-mon[36276]: pgmap v936: 129 pgs: 129 active+clean; 423 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 784 op/s 2024-06-13T07:14:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:44 smithi136 ceph-mon[37229]: pgmap v936: 129 pgs: 129 active+clean; 423 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 784 op/s 2024-06-13T07:14:45.146 INFO:tasks.workunit.client.0.smithi026.stdout: 10 28460 29.48 MB/sec warmup 58 sec latency 60.065 ms 2024-06-13T07:14:46.146 INFO:tasks.workunit.client.0.smithi026.stdout: 10 29050 29.46 MB/sec warmup 59 sec latency 71.696 ms 2024-06-13T07:14:46.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:46 smithi026 ceph-mon[32041]: pgmap v937: 129 pgs: 129 active+clean; 383 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 806 op/s 2024-06-13T07:14:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:46 smithi135 ceph-mon[36276]: pgmap v937: 129 pgs: 129 active+clean; 383 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 806 op/s 2024-06-13T07:14:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:46 smithi136 ceph-mon[37229]: pgmap v937: 129 pgs: 129 active+clean; 383 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 806 op/s 2024-06-13T07:14:47.146 INFO:tasks.workunit.client.0.smithi026.stdout: 10 29758 29.58 MB/sec warmup 60 sec latency 75.895 ms 2024-06-13T07:14:48.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 30473 29.68 MB/sec warmup 61 sec latency 65.930 ms 2024-06-13T07:14:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:48 smithi026 ceph-mon[32041]: pgmap v938: 129 pgs: 129 active+clean; 377 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 798 op/s 2024-06-13T07:14:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:48 smithi135 ceph-mon[36276]: pgmap v938: 129 pgs: 129 active+clean; 377 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 798 op/s 2024-06-13T07:14:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:48 smithi136 ceph-mon[37229]: pgmap v938: 129 pgs: 129 active+clean; 377 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 798 op/s 2024-06-13T07:14:49.146 INFO:tasks.workunit.client.0.smithi026.stdout: 10 31251 29.96 MB/sec warmup 62 sec latency 67.111 ms 2024-06-13T07:14:50.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 32025 30.24 MB/sec warmup 63 sec latency 66.020 ms 2024-06-13T07:14:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:50 smithi026 ceph-mon[32041]: pgmap v939: 129 pgs: 129 active+clean; 384 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 772 op/s 2024-06-13T07:14:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:50 smithi135 ceph-mon[36276]: pgmap v939: 129 pgs: 129 active+clean; 384 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 772 op/s 2024-06-13T07:14:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:50 smithi136 ceph-mon[37229]: pgmap v939: 129 pgs: 129 active+clean; 384 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 772 op/s 2024-06-13T07:14:51.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 32597 30.25 MB/sec warmup 64 sec latency 36.730 ms 2024-06-13T07:14:52.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 33132 30.18 MB/sec warmup 65 sec latency 62.946 ms 2024-06-13T07:14:52.428 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:52 smithi136 ceph-mon[37229]: pgmap v940: 129 pgs: 129 active+clean; 388 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 691 op/s 2024-06-13T07:14:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:52 smithi026 ceph-mon[32041]: pgmap v940: 129 pgs: 129 active+clean; 388 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 691 op/s 2024-06-13T07:14:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:52 smithi135 ceph-mon[36276]: pgmap v940: 129 pgs: 129 active+clean; 388 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 691 op/s 2024-06-13T07:14:53.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 33848 30.13 MB/sec warmup 66 sec latency 50.933 ms 2024-06-13T07:14:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.553+0000 7faf5d200640 1 -- 172.21.15.26:0/948018980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 msgr2=0x7faf58105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.553+0000 7faf5d200640 1 --2- 172.21.15.26:0/948018980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 0x7faf58105c30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faf50009ca0 tx=0x7faf5002a8c0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.554+0000 7faf5d200640 1 -- 172.21.15.26:0/948018980 shutdown_connections 2024-06-13T07:14:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.554+0000 7faf5d200640 1 --2- 172.21.15.26:0/948018980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 0x7faf58105c30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.554+0000 7faf5d200640 1 --2- 172.21.15.26:0/948018980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf581009d0 0x7faf58100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.554+0000 7faf5d200640 1 --2- 172.21.15.26:0/948018980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf580ff8a0 0x7faf580ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.554+0000 7faf5d200640 1 -- 172.21.15.26:0/948018980 >> 172.21.15.26:0/948018980 conn(0x7faf580fb1c0 msgr2=0x7faf580fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.554+0000 7faf5d200640 1 -- 172.21.15.26:0/948018980 shutdown_connections 2024-06-13T07:14:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.554+0000 7faf5d200640 1 -- 172.21.15.26:0/948018980 wait complete. 2024-06-13T07:14:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.555+0000 7faf5d200640 1 Processor -- start 2024-06-13T07:14:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.555+0000 7faf5d200640 1 -- start start 2024-06-13T07:14:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.556+0000 7faf5d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf580ff8a0 0x7faf5819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.556+0000 7faf5d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf581009d0 0x7faf5819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.556+0000 7faf5d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 0x7faf581a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.556+0000 7faf5d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf58113c80 con 0x7faf580ff8a0 2024-06-13T07:14:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.556+0000 7faf5d200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf58113b00 con 0x7faf58101370 2024-06-13T07:14:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.556+0000 7faf5d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf58113e00 con 0x7faf581009d0 2024-06-13T07:14:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf55400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf581009d0 0x7faf5819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf55400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf581009d0 0x7faf5819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55956/0 (socket says 172.21.15.26:55956) 2024-06-13T07:14:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf55400640 1 -- 172.21.15.26:0/3848932976 learned_addr learned my addr 172.21.15.26:0/3848932976 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf55e00640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf580ff8a0 0x7faf5819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf55400640 1 -- 172.21.15.26:0/3848932976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 msgr2=0x7faf581a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf56800640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 0x7faf581a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf55400640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 0x7faf581a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf55400640 1 -- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf580ff8a0 msgr2=0x7faf5819e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.557+0000 7faf55400640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf580ff8a0 0x7faf5819e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.558+0000 7faf55400640 1 -- 172.21.15.26:0/3848932976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf581a5de0 con 0x7faf581009d0 2024-06-13T07:14:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.558+0000 7faf55e00640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf580ff8a0 0x7faf5819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:14:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.558+0000 7faf56800640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 0x7faf581a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:14:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.558+0000 7faf55400640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf581009d0 0x7faf5819e750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faf4800d950 tx=0x7faf4800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.558+0000 7faf4ea00640 1 -- 172.21.15.26:0/3848932976 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48017070 con 0x7faf581009d0 2024-06-13T07:14:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.558+0000 7faf4ea00640 1 -- 172.21.15.26:0/3848932976 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf48009c30 con 0x7faf581009d0 2024-06-13T07:14:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.558+0000 7faf4ea00640 1 -- 172.21.15.26:0/3848932976 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4800f6b0 con 0x7faf581009d0 2024-06-13T07:14:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.558+0000 7faf5d200640 1 -- 172.21.15.26:0/3848932976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf581a60d0 con 0x7faf581009d0 2024-06-13T07:14:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.559+0000 7faf5d200640 1 -- 172.21.15.26:0/3848932976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf581a6510 con 0x7faf581009d0 2024-06-13T07:14:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.564+0000 7faf6001c740 1 Processor -- start 2024-06-13T07:14:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.564+0000 7faf6001c740 1 -- start start 2024-06-13T07:14:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.564+0000 7faf6001c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e607c00 0x55ee0e608020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.565+0000 7faf56800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e607c00 0x55ee0e608020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.565+0000 7faf56800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e607c00 0x55ee0e608020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47928/0 (socket says 172.21.15.26:47928) 2024-06-13T07:14:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.565+0000 7faf6001c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e6085f0 0x55ee0e60d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.565+0000 7faf56800640 1 -- 172.21.15.26:0/303063428 learned_addr learned my addr 172.21.15.26:0/303063428 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.565+0000 7faf6001c740 1 --2- 172.21.15.26:0/303063428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 0x55ee0e60fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.565+0000 7faf6001c740 1 -- 172.21.15.26:0/303063428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee0e56b420 con 0x55ee0e607c00 2024-06-13T07:14:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.565+0000 7faf6001c740 1 -- 172.21.15.26:0/303063428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ee0e512c50 con 0x55ee0e6085f0 2024-06-13T07:14:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.565+0000 7faf6001c740 1 -- 172.21.15.26:0/303063428 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ee0e5673e0 con 0x55ee0e60d760 2024-06-13T07:14:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.566+0000 7faf56800640 1 -- 172.21.15.26:0/303063428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 msgr2=0x55ee0e60fbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:14:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.566+0000 7faf56800640 1 --2- 172.21.15.26:0/303063428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 0x55ee0e60fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.566+0000 7faf56800640 1 -- 172.21.15.26:0/303063428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e6085f0 msgr2=0x55ee0e60d0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:14:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.566+0000 7faf56800640 1 --2- 172.21.15.26:0/303063428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e6085f0 0x55ee0e60d0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.566+0000 7faf56800640 1 -- 172.21.15.26:0/303063428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0e27f210 con 0x55ee0e607c00 2024-06-13T07:14:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.567+0000 7faf4ea00640 1 -- 172.21.15.26:0/3848932976 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf480109d0 con 0x7faf581009d0 2024-06-13T07:14:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.568+0000 7faf56800640 1 --2- 172.21.15.26:0/303063428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e607c00 0x55ee0e608020 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faf5002ae30 tx=0x7faf50009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73307bf41b736eee server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.568+0000 7faf47400640 1 -- 172.21.15.26:0/303063428 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50031650 con 0x55ee0e607c00 2024-06-13T07:14:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.569+0000 7faf47400640 1 -- 172.21.15.26:0/303063428 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf50036070 con 0x55ee0e607c00 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.569+0000 7faf47400640 1 -- 172.21.15.26:0/303063428 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50004740 con 0x55ee0e607c00 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.569+0000 7faf4ea00640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf3c0a2d00 0x7faf3c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.569+0000 7faf4ea00640 1 -- 172.21.15.26:0/3848932976 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faf480cf140 con 0x7faf581009d0 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.570+0000 7faf6001c740 1 -- 172.21.15.26:0/303063428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e607c00 msgr2=0x55ee0e608020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.570+0000 7faf6001c740 1 --2- 172.21.15.26:0/303063428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e607c00 0x55ee0e608020 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faf5002ae30 tx=0x7faf50009800 comp rx=0 tx=0).stop 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.570+0000 7faf6001c740 1 -- 172.21.15.26:0/303063428 shutdown_connections 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.570+0000 7faf6001c740 1 --2- 172.21.15.26:0/303063428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e6085f0 0x55ee0e60d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.570+0000 7faf6001c740 1 --2- 172.21.15.26:0/303063428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 0x55ee0e60fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.570+0000 7faf6001c740 1 --2- 172.21.15.26:0/303063428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e607c00 0x55ee0e608020 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.570+0000 7faf6001c740 1 -- 172.21.15.26:0/303063428 >> 172.21.15.26:0/303063428 conn(0x55ee0e5cffb0 msgr2=0x55ee0e603680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.572+0000 7faf55e00640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf3c0a2d00 0x7faf3c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.572+0000 7faf6001c740 1 -- 172.21.15.26:0/303063428 shutdown_connections 2024-06-13T07:14:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.572+0000 7faf6001c740 1 -- 172.21.15.26:0/303063428 wait complete. 2024-06-13T07:14:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.573+0000 7faf6001c740 1 -- start start 2024-06-13T07:14:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.573+0000 7faf6001c740 1 Processor -- start 2024-06-13T07:14:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf6001c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e611f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf6001c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e6085f0 0x55ee0e6124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf6001c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e60d760 0x55ee0e6194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf6001c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee0e6106b0 con 0x55ee0e60d760 2024-06-13T07:14:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf6001c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ee0e5260b0 con 0x55ee0e607c00 2024-06-13T07:14:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf6001c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ee0e610830 con 0x55ee0e6085f0 2024-06-13T07:14:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf6001c740 10 client.0 _ll_register_callbacks cb 0x55ee0e511f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:14:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf55400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e6085f0 0x55ee0e6124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.575+0000 7faf55e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e611f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf55400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e6085f0 0x55ee0e6124b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55970/0 (socket says 172.21.15.26:55970) 2024-06-13T07:14:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf55400640 1 -- 172.21.15.26:0/3443314016 learned_addr learned my addr 172.21.15.26:0/3443314016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf56800640 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e60d760 0x55ee0e6194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf55e00640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf3c0a2d00 0x7faf3c0a51c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55ee0e604530 tx=0x7faf40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf55400640 1 -- 172.21.15.26:0/3443314016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 msgr2=0x55ee0e611f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf55400640 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e611f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf55400640 1 -- 172.21.15.26:0/3443314016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e60d760 msgr2=0x55ee0e6194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf55400640 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e60d760 0x55ee0e6194f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.576+0000 7faf55400640 1 -- 172.21.15.26:0/3443314016 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0e34b620 con 0x55ee0e6085f0 2024-06-13T07:14:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.578+0000 7faf3be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:14:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.578+0000 7faf55400640 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e6085f0 0x55ee0e6124b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7faf480979b0 tx=0x7faf4809afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53cdf93ea0166420 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.579+0000 7faf3be00640 1 -- 172.21.15.26:0/3443314016 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48020040 con 0x55ee0e6085f0 2024-06-13T07:14:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.579+0000 7faf6001c740 1 -- 172.21.15.26:0/3443314016 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf50036070 con 0x55ee0e6085f0 2024-06-13T07:14:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.579+0000 7faf3be00640 1 -- 172.21.15.26:0/3443314016 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faf48097bc0 con 0x55ee0e6085f0 2024-06-13T07:14:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.580+0000 7faf3be00640 1 -- 172.21.15.26:0/3443314016 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7faf48020040 con 0x55ee0e6085f0 2024-06-13T07:14:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.580+0000 7faf6001c740 10 client.26617 fetch_fsmap learned FSMap version 42 2024-06-13T07:14:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.580+0000 7faf6001c740 1 -- 172.21.15.26:0/3443314016 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ee0e61c8d0 con 0x55ee0e6085f0 2024-06-13T07:14:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.581+0000 7faf3be00640 1 -- 172.21.15.26:0/3443314016 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf50036070 con 0x55ee0e6085f0 2024-06-13T07:14:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.582+0000 7faf3be00640 1 -- 172.21.15.26:0/3443314016 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7faf4809a9a0 con 0x55ee0e6085f0 2024-06-13T07:14:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.582+0000 7faf6001c740 10 client.26617 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:14:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.582+0000 7faf6001c740 10 client.26617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:14:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.582+0000 7faf6001c740 20 client.26617 populate_metadata read hostname 'smithi026' 2024-06-13T07:14:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.582+0000 7faf6001c740 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ee0e61df00 0x55ee0e6202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.582+0000 7faf6001c740 4 client.26617 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:14:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.582+0000 7faf6001c740 1 -- 172.21.15.26:0/3443314016 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee0e578b50 con 0x55ee0e61df00 2024-06-13T07:14:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.582+0000 7faf55e00640 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ee0e61df00 0x55ee0e6202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.586+0000 7faf55e00640 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ee0e61df00 0x55ee0e6202f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c240d1ed522c4d40 server_cookie=4e7714311dc2403f in_seq=0 out_seq=0 2024-06-13T07:14:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.586+0000 7faf3be00640 10 client.26617 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.589+0000 7faf3be00640 1 -- 172.21.15.26:0/3443314016 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ee0e578b50 con 0x55ee0e61df00 2024-06-13T07:14:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.589+0000 7faf3be00640 10 client.26617 handle_command_reply: tid=0 2024-06-13T07:14:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.590+0000 7faf6001c740 1 client.26617 shutdown 2024-06-13T07:14:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.590+0000 7faf6001c740 1 -- 172.21.15.26:0/3443314016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e6085f0 msgr2=0x55ee0e6124b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.591+0000 7faf6001c740 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e6085f0 0x55ee0e6124b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7faf480979b0 tx=0x7faf4809afb0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.591+0000 7faf6001c740 1 -- 172.21.15.26:0/3443314016 shutdown_connections 2024-06-13T07:14:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.591+0000 7faf6001c740 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ee0e61df00 0x55ee0e6202f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.591+0000 7faf6001c740 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e60d760 0x55ee0e6194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.591+0000 7faf6001c740 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e6085f0 0x55ee0e6124b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.591+0000 7faf6001c740 1 --2- 172.21.15.26:0/3443314016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e611f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.591+0000 7faf6001c740 1 -- 172.21.15.26:0/3443314016 >> 172.21.15.26:0/3443314016 conn(0x55ee0e5cffb0 msgr2=0x55ee0e602840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.591+0000 7faf3be00640 0 client.26617 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.592+0000 7faf6001c740 1 -- 172.21.15.26:0/3443314016 shutdown_connections 2024-06-13T07:14:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.592+0000 7faf6001c740 1 -- 172.21.15.26:0/3443314016 wait complete. 2024-06-13T07:14:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.592+0000 7faf6001c740 20 client.26617 trim_cache size 0 max 16384 2024-06-13T07:14:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.635+0000 7faf6001c740 1 Processor -- start 2024-06-13T07:14:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.635+0000 7faf6001c740 1 -- start start 2024-06-13T07:14:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf6001c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e607c00 0x55ee0e602fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf6001c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 0x55ee0e6194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf6001c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e60d760 0x55ee0e606c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf6001c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b3700 con 0x55ee0e6085f0 2024-06-13T07:14:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf6001c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b0f40 con 0x55ee0e60d760 2024-06-13T07:14:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf6001c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf581aed30 con 0x55ee0e607c00 2024-06-13T07:14:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e607c00 0x55ee0e602fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf56800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e60d760 0x55ee0e606c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e607c00 0x55ee0e602fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55982/0 (socket says 172.21.15.26:55982) 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf55e00640 1 -- 172.21.15.26:0/882808909 learned_addr learned my addr 172.21.15.26:0/882808909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.636+0000 7faf55400640 1 --2- 172.21.15.26:0/882808909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 0x55ee0e6194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.637+0000 7faf55e00640 1 -- 172.21.15.26:0/882808909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e60d760 msgr2=0x55ee0e606c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.637+0000 7faf55e00640 1 --2- 172.21.15.26:0/882808909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e60d760 0x55ee0e606c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.637+0000 7faf55e00640 1 -- 172.21.15.26:0/882808909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 msgr2=0x55ee0e6194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.637+0000 7faf55e00640 1 --2- 172.21.15.26:0/882808909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 0x55ee0e6194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.637+0000 7faf55e00640 1 -- 172.21.15.26:0/882808909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0e608e80 con 0x55ee0e607c00 2024-06-13T07:14:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.637+0000 7faf55e00640 1 --2- 172.21.15.26:0/882808909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e607c00 0x55ee0e602fc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55ee0e609060 tx=0x7faf40019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a0ae08c582fa84 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.638+0000 7faf46a00640 1 -- 172.21.15.26:0/882808909 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4002d070 con 0x55ee0e607c00 2024-06-13T07:14:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.638+0000 7faf46a00640 1 -- 172.21.15.26:0/882808909 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ee0e578b50 con 0x55ee0e607c00 2024-06-13T07:14:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.638+0000 7faf46a00640 1 -- 172.21.15.26:0/882808909 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40028650 con 0x55ee0e607c00 2024-06-13T07:14:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.639+0000 7faf6001c740 1 -- 172.21.15.26:0/882808909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e607c00 msgr2=0x55ee0e602fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.639+0000 7faf6001c740 1 --2- 172.21.15.26:0/882808909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e607c00 0x55ee0e602fc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55ee0e609060 tx=0x7faf40019000 comp rx=0 tx=0).stop 2024-06-13T07:14:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.641+0000 7faf6001c740 1 -- 172.21.15.26:0/882808909 shutdown_connections 2024-06-13T07:14:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.641+0000 7faf6001c740 1 --2- 172.21.15.26:0/882808909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e60d760 0x55ee0e606c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.641+0000 7faf6001c740 1 --2- 172.21.15.26:0/882808909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 0x55ee0e6194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.641+0000 7faf6001c740 1 --2- 172.21.15.26:0/882808909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e607c00 0x55ee0e602fc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.641+0000 7faf6001c740 1 -- 172.21.15.26:0/882808909 >> 172.21.15.26:0/882808909 conn(0x55ee0e5cffb0 msgr2=0x55ee0e606570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.642+0000 7faf6001c740 1 -- 172.21.15.26:0/882808909 shutdown_connections 2024-06-13T07:14:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.643+0000 7faf6001c740 1 -- 172.21.15.26:0/882808909 wait complete. 2024-06-13T07:14:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.643+0000 7faf6001c740 1 -- start start 2024-06-13T07:14:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.644+0000 7faf6001c740 1 Processor -- start 2024-06-13T07:14:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.644+0000 7faf6001c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e614f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf6001c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 0x55ee0e6154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf6001c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 0x55ee0e618400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf55e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e614f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf55e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e614f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:41762/0 (socket says 172.21.15.26:41762) 2024-06-13T07:14:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf581af400 con 0x55ee0e6085f0 2024-06-13T07:14:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf55e00640 1 -- 172.21.15.26:0/3410115647 learned_addr learned my addr 172.21.15.26:0/3410115647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b0870 con 0x55ee0e607c00 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf581afad0 con 0x55ee0e60d760 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf6001c740 10 client.0 _ll_register_callbacks cb 0x7faf4800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.645+0000 7faf56800640 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 0x55ee0e618400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf55e00640 1 -- 172.21.15.26:0/3410115647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 msgr2=0x55ee0e618400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf55e00640 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 0x55ee0e618400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf55e00640 1 -- 172.21.15.26:0/3410115647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 msgr2=0x55ee0e6154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf55e00640 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 0x55ee0e6154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf55e00640 1 -- 172.21.15.26:0/3410115647 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0e6189f0 con 0x55ee0e607c00 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf55e00640 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e614f60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faf4002b040 tx=0x7faf4001eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a29891cd153122 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf3be00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf3be00640 1 -- 172.21.15.26:0/3410115647 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4002d070 con 0x55ee0e607c00 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.646+0000 7faf3be00640 1 -- 172.21.15.26:0/3410115647 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faf4001e640 con 0x55ee0e607c00 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee0e578b50 con 0x55ee0e607c00 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf3be00640 1 -- 172.21.15.26:0/3410115647 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7faf4001e840 con 0x55ee0e607c00 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf6001c740 10 client.26597 fetch_fsmap learned FSMap version 42 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ee0e699ea0 con 0x55ee0e607c00 2024-06-13T07:14:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf3be00640 1 -- 172.21.15.26:0/3410115647 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ee0e578b50 con 0x55ee0e607c00 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf3be00640 1 -- 172.21.15.26:0/3410115647 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7faf40004c60 con 0x55ee0e607c00 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf6001c740 10 client.26597 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf6001c740 10 client.26597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf6001c740 20 client.26597 populate_metadata read hostname 'smithi026' 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.647+0000 7faf6001c740 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ee0e69a850 0x55ee0e69cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.648+0000 7faf6001c740 4 client.26597 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.648+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee0e572ea0 con 0x55ee0e69a850 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.648+0000 7faf55400640 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ee0e69a850 0x55ee0e69cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.648+0000 7faf55400640 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ee0e69a850 0x55ee0e69cc70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb76afbfcc6b1a9 server_cookie=cb7973ae77511d25 in_seq=0 out_seq=0 2024-06-13T07:14:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:53.649+0000 7faf3be00640 10 client.26597 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf3be00640 1 -- 172.21.15.26:0/3410115647 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ee0e572ea0 con 0x55ee0e69a850 2024-06-13T07:14:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf3be00640 10 client.26597 handle_command_reply: tid=0 2024-06-13T07:14:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 client.26597 shutdown 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 msgr2=0x55ee0e614f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e614f60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faf4002b040 tx=0x7faf4001eba0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 shutdown_connections 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55ee0e69a850 0x55ee0e69cc70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee0e60d760 0x55ee0e618400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0e6085f0 0x55ee0e6154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 --2- 172.21.15.26:0/3410115647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee0e607c00 0x55ee0e614f60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 >> 172.21.15.26:0/3410115647 conn(0x55ee0e5cffb0 msgr2=0x55ee0e601fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.070+0000 7faf3be00640 0 client.26597 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.071+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 shutdown_connections 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.071+0000 7faf6001c740 1 -- 172.21.15.26:0/3410115647 wait complete. 2024-06-13T07:14:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.071+0000 7faf6001c740 20 client.26597 trim_cache size 0 max 16384 2024-06-13T07:14:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:14:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:14:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:14:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 -- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf3c0a2d00 msgr2=0x7faf3c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf3c0a2d00 0x7faf3c0a51c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55ee0e604530 tx=0x7faf40005f90 comp rx=0 tx=0).stop 2024-06-13T07:14:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 -- 172.21.15.26:0/3848932976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf581009d0 msgr2=0x7faf5819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:14:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf581009d0 0x7faf5819e750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faf4800d950 tx=0x7faf4800de10 comp rx=0 tx=0).stop 2024-06-13T07:14:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 -- 172.21.15.26:0/3848932976 shutdown_connections 2024-06-13T07:14:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faf3c0a2d00 0x7faf3c0a51c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf58101370 0x7faf581a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf581009d0 0x7faf5819e750 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 --2- 172.21.15.26:0/3848932976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf580ff8a0 0x7faf5819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:14:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.072+0000 7faf5d200640 1 -- 172.21.15.26:0/3848932976 >> 172.21.15.26:0/3848932976 conn(0x7faf580fb1c0 msgr2=0x7faf58101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:14:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.073+0000 7faf5d200640 1 -- 172.21.15.26:0/3848932976 shutdown_connections 2024-06-13T07:14:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:14:54.073+0000 7faf5d200640 1 -- 172.21.15.26:0/3848932976 wait complete. 2024-06-13T07:14:54.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 34524 30.21 MB/sec warmup 67 sec latency 258.004 ms 2024-06-13T07:14:54.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:54 smithi026 ceph-mon[32041]: pgmap v941: 129 pgs: 129 active+clean; 396 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-06-13T07:14:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:54 smithi135 ceph-mon[36276]: pgmap v941: 129 pgs: 129 active+clean; 396 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-06-13T07:14:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:54 smithi136 ceph-mon[37229]: pgmap v941: 129 pgs: 129 active+clean; 396 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-06-13T07:14:55.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 35449 30.64 MB/sec warmup 68 sec latency 45.320 ms 2024-06-13T07:14:56.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 36092 30.82 MB/sec warmup 69 sec latency 54.340 ms 2024-06-13T07:14:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:56 smithi026 ceph-mon[32041]: pgmap v942: 129 pgs: 129 active+clean; 398 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 551 op/s 2024-06-13T07:14:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:56 smithi135 ceph-mon[36276]: pgmap v942: 129 pgs: 129 active+clean; 398 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 551 op/s 2024-06-13T07:14:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:56 smithi136 ceph-mon[37229]: pgmap v942: 129 pgs: 129 active+clean; 398 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 551 op/s 2024-06-13T07:14:57.147 INFO:tasks.workunit.client.0.smithi026.stdout: 10 36616 30.73 MB/sec warmup 70 sec latency 92.435 ms 2024-06-13T07:14:58.148 INFO:tasks.workunit.client.0.smithi026.stdout: 10 36939 30.46 MB/sec warmup 71 sec latency 291.247 ms 2024-06-13T07:14:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:14:58 smithi026 ceph-mon[32041]: pgmap v943: 129 pgs: 129 active+clean; 402 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-06-13T07:14:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:14:58 smithi135 ceph-mon[36276]: pgmap v943: 129 pgs: 129 active+clean; 402 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-06-13T07:14:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:14:58 smithi136 ceph-mon[37229]: pgmap v943: 129 pgs: 129 active+clean; 402 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-06-13T07:14:59.148 INFO:tasks.workunit.client.0.smithi026.stdout: 10 37361 30.26 MB/sec warmup 72 sec latency 175.477 ms 2024-06-13T07:15:00.148 INFO:tasks.workunit.client.0.smithi026.stdout: 10 38180 30.50 MB/sec warmup 73 sec latency 70.004 ms 2024-06-13T07:15:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:00 smithi026 ceph-mon[32041]: pgmap v944: 129 pgs: 129 active+clean; 409 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 600 op/s 2024-06-13T07:15:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:00 smithi135 ceph-mon[36276]: pgmap v944: 129 pgs: 129 active+clean; 409 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 600 op/s 2024-06-13T07:15:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:00 smithi136 ceph-mon[37229]: pgmap v944: 129 pgs: 129 active+clean; 409 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 600 op/s 2024-06-13T07:15:01.148 INFO:tasks.workunit.client.0.smithi026.stdout: 10 39021 30.69 MB/sec warmup 74 sec latency 62.376 ms 2024-06-13T07:15:02.148 INFO:tasks.workunit.client.0.smithi026.stdout: 10 39463 30.72 MB/sec warmup 75 sec latency 346.846 ms 2024-06-13T07:15:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:02 smithi026 ceph-mon[32041]: pgmap v945: 129 pgs: 129 active+clean; 417 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-06-13T07:15:02.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:02 smithi135 ceph-mon[36276]: pgmap v945: 129 pgs: 129 active+clean; 417 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-06-13T07:15:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:02 smithi136 ceph-mon[37229]: pgmap v945: 129 pgs: 129 active+clean; 417 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-06-13T07:15:03.149 INFO:tasks.workunit.client.0.smithi026.stdout: 10 39760 30.54 MB/sec warmup 76 sec latency 469.519 ms 2024-06-13T07:15:04.149 INFO:tasks.workunit.client.0.smithi026.stdout: 10 39874 30.23 MB/sec warmup 77 sec latency 1449.291 ms 2024-06-13T07:15:04.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:04 smithi026 ceph-mon[32041]: pgmap v946: 129 pgs: 129 active+clean; 421 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 633 op/s 2024-06-13T07:15:04.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:04 smithi135 ceph-mon[36276]: pgmap v946: 129 pgs: 129 active+clean; 421 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 633 op/s 2024-06-13T07:15:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:04 smithi136 ceph-mon[37229]: pgmap v946: 129 pgs: 129 active+clean; 421 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 633 op/s 2024-06-13T07:15:05.149 INFO:tasks.workunit.client.0.smithi026.stdout: 10 40211 30.05 MB/sec warmup 78 sec latency 1468.386 ms 2024-06-13T07:15:06.149 INFO:tasks.workunit.client.0.smithi026.stdout: 10 40497 29.91 MB/sec warmup 79 sec latency 251.576 ms 2024-06-13T07:15:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:06 smithi026 ceph-mon[32041]: pgmap v947: 129 pgs: 129 active+clean; 458 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.9 MiB/s wr, 668 op/s 2024-06-13T07:15:06.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:06 smithi135 ceph-mon[36276]: pgmap v947: 129 pgs: 129 active+clean; 458 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.9 MiB/s wr, 668 op/s 2024-06-13T07:15:06.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:06 smithi136 ceph-mon[37229]: pgmap v947: 129 pgs: 129 active+clean; 458 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.9 MiB/s wr, 668 op/s 2024-06-13T07:15:07.149 INFO:tasks.workunit.client.0.smithi026.stdout: 10 40933 29.76 MB/sec warmup 80 sec latency 99.923 ms 2024-06-13T07:15:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.294+0000 7f2efd600640 1 -- 172.21.15.26:0/2857231379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 msgr2=0x7f2ef8076780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.294+0000 7f2efd600640 1 --2- 172.21.15.26:0/2857231379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 0x7f2ef8076780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009900 tx=0x7f2ee802a410 comp rx=0 tx=0).stop 2024-06-13T07:15:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.295+0000 7f2efd600640 1 -- 172.21.15.26:0/2857231379 shutdown_connections 2024-06-13T07:15:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.295+0000 7f2efd600640 1 --2- 172.21.15.26:0/2857231379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ef80775a0 0x7f2ef8077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.295+0000 7f2efd600640 1 --2- 172.21.15.26:0/2857231379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ef8077f40 0x7f2ef80787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.295+0000 7f2efd600640 1 --2- 172.21.15.26:0/2857231379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 0x7f2ef8076780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.295+0000 7f2efd600640 1 -- 172.21.15.26:0/2857231379 >> 172.21.15.26:0/2857231379 conn(0x7f2ef80ff1c0 msgr2=0x7f2ef81015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.295+0000 7f2efd600640 1 -- 172.21.15.26:0/2857231379 shutdown_connections 2024-06-13T07:15:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.295+0000 7f2efd600640 1 -- 172.21.15.26:0/2857231379 wait complete. 2024-06-13T07:15:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.296+0000 7f2efd600640 1 Processor -- start 2024-06-13T07:15:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.297+0000 7f2efd600640 1 -- start start 2024-06-13T07:15:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.297+0000 7f2efd600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 0x7f2ef819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.297+0000 7f2efd600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ef80775a0 0x7f2ef819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.297+0000 7f2efd600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ef8077f40 0x7f2ef81a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.297+0000 7f2efd600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8113cc0 con 0x7f2ef80775a0 2024-06-13T07:15:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.297+0000 7f2efd600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8113b40 con 0x7f2ef8077f40 2024-06-13T07:15:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.297+0000 7f2efd600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8113e40 con 0x7f2ef8076360 2024-06-13T07:15:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.297+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 0x7f2ef819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.298+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 0x7f2ef819e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53072/0 (socket says 172.21.15.26:53072) 2024-06-13T07:15:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.298+0000 7f2ef6000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ef80775a0 0x7f2ef819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.298+0000 7f2ef7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ef8077f40 0x7f2ef81a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.298+0000 7f2ef7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ef8077f40 0x7f2ef81a57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35572/0 (socket says 172.21.15.26:35572) 2024-06-13T07:15:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.298+0000 7f2ef6000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ef80775a0 0x7f2ef819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49688/0 (socket says 172.21.15.26:49688) 2024-06-13T07:15:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.298+0000 7f2ef6a00640 1 -- 172.21.15.26:0/341451784 learned_addr learned my addr 172.21.15.26:0/341451784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.299+0000 7f2ef6a00640 1 -- 172.21.15.26:0/341451784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ef8077f40 msgr2=0x7f2ef81a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.299+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ef8077f40 0x7f2ef81a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.299+0000 7f2ef6a00640 1 -- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ef80775a0 msgr2=0x7f2ef819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.299+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ef80775a0 0x7f2ef819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.299+0000 7f2ef6a00640 1 -- 172.21.15.26:0/341451784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef81a5e40 con 0x7f2ef8076360 2024-06-13T07:15:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.299+0000 7f2ef7400640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ef8077f40 0x7f2ef81a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:15:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.299+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 0x7f2ef819e220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8000c00 tx=0x7f2ee80050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.300+0000 7f2eef400640 1 -- 172.21.15.26:0/341451784 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8004030 con 0x7f2ef8076360 2024-06-13T07:15:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.300+0000 7f2efd600640 1 -- 172.21.15.26:0/341451784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef81a60d0 con 0x7f2ef8076360 2024-06-13T07:15:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.300+0000 7f2eef400640 1 -- 172.21.15.26:0/341451784 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee80318e0 con 0x7f2ef8076360 2024-06-13T07:15:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.300+0000 7f2eef400640 1 -- 172.21.15.26:0/341451784 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8004b10 con 0x7f2ef8076360 2024-06-13T07:15:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.300+0000 7f2efd600640 1 -- 172.21.15.26:0/341451784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef81a6610 con 0x7f2ef8076360 2024-06-13T07:15:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.307+0000 7f2eef400640 1 -- 172.21.15.26:0/341451784 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ee8004cf0 con 0x7f2ef8076360 2024-06-13T07:15:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.308+0000 7f2f00453740 1 Processor -- start 2024-06-13T07:15:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.309+0000 7f2f00453740 1 -- start start 2024-06-13T07:15:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.310+0000 7f2eef400640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2edc0a2cb0 0x7f2edc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.310+0000 7f2eef400640 1 -- 172.21.15.26:0/341451784 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2ee80eff00 con 0x7f2ef8076360 2024-06-13T07:15:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.310+0000 7f2ef7400640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2edc0a2cb0 0x7f2edc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2f00453740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635d6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2f00453740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2f00453740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e635ddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2f00453740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558e63539420 con 0x558e635db760 2024-06-13T07:15:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2f00453740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558e634e0c50 con 0x558e635d5c00 2024-06-13T07:15:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2f00453740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558e635353e0 con 0x558e635d65f0 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2ef6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2ef7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e635ddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2ef6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635db0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53076/0 (socket says 172.21.15.26:53076) 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2ef6000640 1 -- 172.21.15.26:0/208838144 learned_addr learned my addr 172.21.15.26:0/208838144 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.311+0000 7f2ef7400640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2edc0a2cb0 0x7f2edc0a5170 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0005fd0 tx=0x7f2ef0011bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ef7400640 1 -- 172.21.15.26:0/208838144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 msgr2=0x558e635db0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ef7400640 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635db0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ef7400640 1 -- 172.21.15.26:0/208838144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 msgr2=0x558e635d6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ef7400640 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635d6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ef7400640 1 -- 172.21.15.26:0/208838144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6324d210 con 0x558e635db760 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635d6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:15:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ef7400640 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e635ddbf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000fc00 tx=0x7f2ef001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d288e7e8ea64867 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ee3400640 1 -- 172.21.15.26:0/208838144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0017040 con 0x558e635db760 2024-06-13T07:15:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ee3400640 1 -- 172.21.15.26:0/208838144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef0019050 con 0x558e635db760 2024-06-13T07:15:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.312+0000 7f2ee3400640 1 -- 172.21.15.26:0/208838144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef001daa0 con 0x558e635db760 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.313+0000 7f2f00453740 1 -- 172.21.15.26:0/208838144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 msgr2=0x558e635ddbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.313+0000 7f2f00453740 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e635ddbf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000fc00 tx=0x7f2ef001c000 comp rx=0 tx=0).stop 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.313+0000 7f2f00453740 1 -- 172.21.15.26:0/208838144 shutdown_connections 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.313+0000 7f2f00453740 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e635ddbf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.313+0000 7f2f00453740 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635db0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.313+0000 7f2f00453740 1 --2- 172.21.15.26:0/208838144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635d6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.313+0000 7f2f00453740 1 -- 172.21.15.26:0/208838144 >> 172.21.15.26:0/208838144 conn(0x558e6359dfb0 msgr2=0x558e635d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.313+0000 7f2f00453740 1 -- 172.21.15.26:0/208838144 shutdown_connections 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.314+0000 7f2f00453740 1 -- 172.21.15.26:0/208838144 wait complete. 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.314+0000 7f2f00453740 1 -- start start 2024-06-13T07:15:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.314+0000 7f2f00453740 1 Processor -- start 2024-06-13T07:15:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2f00453740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d5c00 0x558e635dff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2f00453740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d65f0 0x558e635e04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2f00453740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635db760 0x558e635e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2f00453740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558e635de710 con 0x558e635d65f0 2024-06-13T07:15:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2f00453740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558e634f40b0 con 0x558e635db760 2024-06-13T07:15:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2f00453740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558e635de890 con 0x558e635d5c00 2024-06-13T07:15:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2f00453740 10 client.0 _ll_register_callbacks cb 0x558e634dff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:15:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635db760 0x558e635e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635db760 0x558e635e74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35588/0 (socket says 172.21.15.26:35588) 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef7400640 1 -- 172.21.15.26:0/537428507 learned_addr learned my addr 172.21.15.26:0/537428507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d5c00 0x558e635dff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef7400640 1 -- 172.21.15.26:0/537428507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d5c00 msgr2=0x558e635dff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef7400640 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d5c00 0x558e635dff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef7400640 1 -- 172.21.15.26:0/537428507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d65f0 msgr2=0x558e635e04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef7400640 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d65f0 0x558e635e04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.315+0000 7f2ef7400640 1 -- 172.21.15.26:0/537428507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e63319620 con 0x558e635db760 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.316+0000 7f2ed7e00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.316+0000 7f2ef7400640 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635db760 0x558e635e74e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2ef001dda0 tx=0x7f2ef001ddd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3efe77e76ceae355 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.316+0000 7f2ed7e00640 1 -- 172.21.15.26:0/537428507 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0017050 con 0x558e635db760 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.316+0000 7f2f00453740 1 -- 172.21.15.26:0/537428507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef0019050 con 0x558e635db760 2024-06-13T07:15:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.316+0000 7f2ed7e00640 1 -- 172.21.15.26:0/537428507 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2ef00041b0 con 0x558e635db760 2024-06-13T07:15:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.317+0000 7f2ed7e00640 1 -- 172.21.15.26:0/537428507 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2ef0039050 con 0x558e635db760 2024-06-13T07:15:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.317+0000 7f2f00453740 10 client.26609 fetch_fsmap learned FSMap version 42 2024-06-13T07:15:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.317+0000 7f2f00453740 1 -- 172.21.15.26:0/537428507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558e635ea8a0 con 0x558e635db760 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.317+0000 7f2ed7e00640 1 -- 172.21.15.26:0/537428507 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef0019050 con 0x558e635db760 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.317+0000 7f2ed7e00640 1 -- 172.21.15.26:0/537428507 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f2ef0025070 con 0x558e635db760 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.318+0000 7f2f00453740 10 client.26609 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.318+0000 7f2f00453740 10 client.26609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.318+0000 7f2f00453740 20 client.26609 populate_metadata read hostname 'smithi026' 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.318+0000 7f2f00453740 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558e635ebf80 0x558e635ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.318+0000 7f2f00453740 4 client.26609 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.318+0000 7f2f00453740 1 -- 172.21.15.26:0/537428507 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e63546b50 con 0x558e635ebf80 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.318+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558e635ebf80 0x558e635ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.319+0000 7f2ed7e00640 10 client.26609 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.319+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558e635ebf80 0x558e635ee3b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217f011eac4993f9 server_cookie=b2bf3e51882e7c99 in_seq=0 out_seq=0 2024-06-13T07:15:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.321+0000 7f2ed7e00640 1 -- 172.21.15.26:0/537428507 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558e63546b50 con 0x558e635ebf80 2024-06-13T07:15:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.321+0000 7f2ed7e00640 10 client.26609 handle_command_reply: tid=0 2024-06-13T07:15:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.321+0000 7f2f00453740 1 client.26609 shutdown 2024-06-13T07:15:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.321+0000 7f2f00453740 1 -- 172.21.15.26:0/537428507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635db760 msgr2=0x558e635e74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.321+0000 7f2f00453740 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635db760 0x558e635e74e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2ef001dda0 tx=0x7f2ef001ddd0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 1 -- 172.21.15.26:0/537428507 shutdown_connections 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558e635ebf80 0x558e635ee3b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635db760 0x558e635e74e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d65f0 0x558e635e04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 1 --2- 172.21.15.26:0/537428507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d5c00 0x558e635dff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 1 -- 172.21.15.26:0/537428507 >> 172.21.15.26:0/537428507 conn(0x558e6359dfb0 msgr2=0x558e635d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2ed7e00640 0 client.26609 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 1 -- 172.21.15.26:0/537428507 shutdown_connections 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 1 -- 172.21.15.26:0/537428507 wait complete. 2024-06-13T07:15:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.322+0000 7f2f00453740 20 client.26609 trim_cache size 0 max 16384 2024-06-13T07:15:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.358+0000 7f2f00453740 1 Processor -- start 2024-06-13T07:15:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.358+0000 7f2f00453740 1 -- start start 2024-06-13T07:15:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.358+0000 7f2f00453740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d5c00 0x558e635d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.358+0000 7f2f00453740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d65f0 0x558e635e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2f00453740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635db760 0x558e635d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2f00453740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81b3510 con 0x558e635d5c00 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2f00453740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81b0d50 con 0x558e635d65f0 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2f00453740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81aeba0 con 0x558e635db760 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635db760 0x558e635d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635db760 0x558e635d7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53098/0 (socket says 172.21.15.26:53098) 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef7400640 1 -- 172.21.15.26:0/399499642 learned_addr learned my addr 172.21.15.26:0/399499642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef6000640 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d65f0 0x558e635e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d5c00 0x558e635d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef7400640 1 -- 172.21.15.26:0/399499642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d65f0 msgr2=0x558e635e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef7400640 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d65f0 0x558e635e74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef7400640 1 -- 172.21.15.26:0/399499642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d5c00 msgr2=0x558e635d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef7400640 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d5c00 0x558e635d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef7400640 1 -- 172.21.15.26:0/399499642 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e635d2330 con 0x558e635db760 2024-06-13T07:15:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.359+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d5c00 0x558e635d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:15:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.360+0000 7f2ef7400640 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635db760 0x558e635d7910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000f830 tx=0x7f2ef0011480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7055e72a7118e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.360+0000 7f2ee2a00640 1 -- 172.21.15.26:0/399499642 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0017040 con 0x558e635db760 2024-06-13T07:15:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.360+0000 7f2ee2a00640 1 -- 172.21.15.26:0/399499642 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef001d640 con 0x558e635db760 2024-06-13T07:15:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.360+0000 7f2ee2a00640 1 -- 172.21.15.26:0/399499642 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00058f0 con 0x558e635db760 2024-06-13T07:15:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.361+0000 7f2f00453740 1 -- 172.21.15.26:0/399499642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635db760 msgr2=0x558e635d7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.361+0000 7f2f00453740 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635db760 0x558e635d7910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000f830 tx=0x7f2ef0011480 comp rx=0 tx=0).stop 2024-06-13T07:15:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.361+0000 7f2f00453740 1 -- 172.21.15.26:0/399499642 shutdown_connections 2024-06-13T07:15:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.361+0000 7f2f00453740 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d65f0 0x558e635e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.361+0000 7f2f00453740 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635db760 0x558e635d7910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.361+0000 7f2f00453740 1 --2- 172.21.15.26:0/399499642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635d5c00 0x558e635d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.361+0000 7f2f00453740 1 -- 172.21.15.26:0/399499642 >> 172.21.15.26:0/399499642 conn(0x558e6359dfb0 msgr2=0x558e635dff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.362+0000 7f2f00453740 1 -- 172.21.15.26:0/399499642 shutdown_connections 2024-06-13T07:15:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.362+0000 7f2f00453740 1 -- 172.21.15.26:0/399499642 wait complete. 2024-06-13T07:15:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.362+0000 7f2f00453740 1 -- start start 2024-06-13T07:15:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.363+0000 7f2f00453740 1 Processor -- start 2024-06-13T07:15:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2f00453740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635e4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2f00453740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2f00453740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e63654c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2f00453740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81af210 con 0x558e635db760 2024-06-13T07:15:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2f00453740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81b0680 con 0x558e635d5c00 2024-06-13T07:15:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2f00453740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81af8e0 con 0x558e635d65f0 2024-06-13T07:15:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2f00453740 10 client.0 _ll_register_callbacks cb 0x7f2ee4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:15:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2ef6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2ef6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635e46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53100/0 (socket says 172.21.15.26:53100) 2024-06-13T07:15:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.364+0000 7f2ef6000640 1 -- 172.21.15.26:0/2430384382 learned_addr learned my addr 172.21.15.26:0/2430384382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.365+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.365+0000 7f2ef7400640 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e63654c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.365+0000 7f2ef6000640 1 -- 172.21.15.26:0/2430384382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 msgr2=0x558e635e4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.365+0000 7f2ef6000640 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635e4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.365+0000 7f2ef6000640 1 -- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 msgr2=0x558e63654c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.365+0000 7f2ef6000640 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e63654c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.365+0000 7f2ef6000640 1 -- 172.21.15.26:0/2430384382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e635e4ce0 con 0x558e635d65f0 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.365+0000 7f2ef7400640 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e63654c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.366+0000 7f2ef6000640 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635e46a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x558e635d7060 tx=0x7f2ee4010d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95b984d336d1e59 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.366+0000 7f2ed7e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:15:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.366+0000 7f2ed7e00640 1 -- 172.21.15.26:0/2430384382 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4021060 con 0x558e635d65f0 2024-06-13T07:15:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.366+0000 7f2f00453740 1 -- 172.21.15.26:0/2430384382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef001d640 con 0x558e635d65f0 2024-06-13T07:15:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.366+0000 7f2ed7e00640 1 -- 172.21.15.26:0/2430384382 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2ee400fc80 con 0x558e635d65f0 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.367+0000 7f2ed7e00640 1 -- 172.21.15.26:0/2430384382 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2ee4028050 con 0x558e635d65f0 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.367+0000 7f2f00453740 10 client.26641 fetch_fsmap learned FSMap version 42 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.367+0000 7f2f00453740 1 -- 172.21.15.26:0/2430384382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558e63656320 con 0x558e635d65f0 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.367+0000 7f2ed7e00640 1 -- 172.21.15.26:0/2430384382 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef001d640 con 0x558e635d65f0 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.368+0000 7f2ed7e00640 1 -- 172.21.15.26:0/2430384382 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f2ee401b070 con 0x558e635d65f0 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.368+0000 7f2f00453740 10 client.26641 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.368+0000 7f2f00453740 10 client.26641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.368+0000 7f2f00453740 20 client.26641 populate_metadata read hostname 'smithi026' 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.368+0000 7f2f00453740 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558e63656d40 0x558e63659160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.368+0000 7f2f00453740 4 client.26641 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:15:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.368+0000 7f2f00453740 1 -- 172.21.15.26:0/2430384382 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e63540ea0 con 0x558e63656d40 2024-06-13T07:15:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.369+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558e63656d40 0x558e63659160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.370+0000 7f2ef6a00640 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558e63656d40 0x558e63659160 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f32c11b457d0473f server_cookie=a82bb6007ac79756 in_seq=0 out_seq=0 2024-06-13T07:15:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.370+0000 7f2ed7e00640 10 client.26641 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.670+0000 7f2ed7e00640 1 -- 172.21.15.26:0/2430384382 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558e63540ea0 con 0x558e63656d40 2024-06-13T07:15:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.670+0000 7f2ed7e00640 10 client.26641 handle_command_reply: tid=0 2024-06-13T07:15:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.670+0000 7f2f00453740 1 client.26641 shutdown 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.670+0000 7f2f00453740 1 -- 172.21.15.26:0/2430384382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 msgr2=0x558e635e46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.670+0000 7f2f00453740 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635e46a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x558e635d7060 tx=0x7f2ee4010d50 comp rx=0 tx=0).stop 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2f00453740 1 -- 172.21.15.26:0/2430384382 shutdown_connections 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2f00453740 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558e63656d40 0x558e63659160 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2f00453740 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e635db760 0x558e63654c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2f00453740 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e635d65f0 0x558e635e46a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2ed7e00640 0 client.26641 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2f00453740 1 --2- 172.21.15.26:0/2430384382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e635d5c00 0x558e635e4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2f00453740 1 -- 172.21.15.26:0/2430384382 >> 172.21.15.26:0/2430384382 conn(0x558e6359dfb0 msgr2=0x558e635d7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2f00453740 1 -- 172.21.15.26:0/2430384382 shutdown_connections 2024-06-13T07:15:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.671+0000 7f2f00453740 1 -- 172.21.15.26:0/2430384382 wait complete. 2024-06-13T07:15:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.672+0000 7f2f00453740 20 client.26641 trim_cache size 0 max 16384 2024-06-13T07:15:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:15:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:15:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:15:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 -- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2edc0a2cb0 msgr2=0x7f2edc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2edc0a2cb0 0x7f2edc0a5170 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0005fd0 tx=0x7f2ef0011bd0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 -- 172.21.15.26:0/341451784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 msgr2=0x7f2ef819e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 0x7f2ef819e220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8000c00 tx=0x7f2ee80050c0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 -- 172.21.15.26:0/341451784 shutdown_connections 2024-06-13T07:15:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2edc0a2cb0 0x7f2edc0a5170 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ef8077f40 0x7f2ef81a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ef80775a0 0x7f2ef819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 --2- 172.21.15.26:0/341451784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ef8076360 0x7f2ef819e220 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 -- 172.21.15.26:0/341451784 >> 172.21.15.26:0/341451784 conn(0x7f2ef80ff1c0 msgr2=0x7f2ef8100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 -- 172.21.15.26:0/341451784 shutdown_connections 2024-06-13T07:15:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:07.674+0000 7f2efd600640 1 -- 172.21.15.26:0/341451784 wait complete. 2024-06-13T07:15:08.149 INFO:tasks.workunit.client.0.smithi026.stdout: 10 41433 29.72 MB/sec warmup 81 sec latency 158.542 ms 2024-06-13T07:15:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:08 smithi026 ceph-mon[32041]: pgmap v948: 129 pgs: 129 active+clean; 473 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 736 op/s 2024-06-13T07:15:08.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:08 smithi135 ceph-mon[36276]: pgmap v948: 129 pgs: 129 active+clean; 473 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 736 op/s 2024-06-13T07:15:08.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:08 smithi136 ceph-mon[37229]: pgmap v948: 129 pgs: 129 active+clean; 473 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 736 op/s 2024-06-13T07:15:09.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 42131 29.79 MB/sec warmup 82 sec latency 67.049 ms 2024-06-13T07:15:10.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 42817 29.81 MB/sec warmup 83 sec latency 57.395 ms 2024-06-13T07:15:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:10 smithi026 ceph-mon[32041]: pgmap v949: 129 pgs: 129 active+clean; 431 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 805 op/s 2024-06-13T07:15:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:10 smithi135 ceph-mon[36276]: pgmap v949: 129 pgs: 129 active+clean; 431 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 805 op/s 2024-06-13T07:15:10.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:10 smithi136 ceph-mon[37229]: pgmap v949: 129 pgs: 129 active+clean; 431 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 805 op/s 2024-06-13T07:15:11.149 INFO:tasks.workunit.client.0.smithi026.stdout: 10 43662 30.08 MB/sec warmup 84 sec latency 99.592 ms 2024-06-13T07:15:12.149 INFO:tasks.workunit.client.0.smithi026.stdout: 10 44233 30.07 MB/sec warmup 85 sec latency 366.571 ms 2024-06-13T07:15:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:12 smithi026 ceph-mon[32041]: pgmap v950: 129 pgs: 129 active+clean; 411 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 825 op/s 2024-06-13T07:15:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:12 smithi135 ceph-mon[36276]: pgmap v950: 129 pgs: 129 active+clean; 411 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 825 op/s 2024-06-13T07:15:12.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:12 smithi136 ceph-mon[37229]: pgmap v950: 129 pgs: 129 active+clean; 411 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 825 op/s 2024-06-13T07:15:13.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 44954 30.21 MB/sec warmup 86 sec latency 82.360 ms 2024-06-13T07:15:14.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 45488 30.21 MB/sec warmup 87 sec latency 87.598 ms 2024-06-13T07:15:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:14 smithi026 ceph-mon[32041]: pgmap v951: 129 pgs: 129 active+clean; 412 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 762 op/s 2024-06-13T07:15:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:14 smithi135 ceph-mon[36276]: pgmap v951: 129 pgs: 129 active+clean; 412 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 762 op/s 2024-06-13T07:15:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:14 smithi136 ceph-mon[37229]: pgmap v951: 129 pgs: 129 active+clean; 412 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 762 op/s 2024-06-13T07:15:15.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 46116 30.20 MB/sec warmup 88 sec latency 57.434 ms 2024-06-13T07:15:16.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 46723 30.22 MB/sec warmup 89 sec latency 80.677 ms 2024-06-13T07:15:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:16 smithi026 ceph-mon[32041]: pgmap v952: 129 pgs: 129 active+clean; 419 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 801 op/s 2024-06-13T07:15:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:16 smithi135 ceph-mon[36276]: pgmap v952: 129 pgs: 129 active+clean; 419 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 801 op/s 2024-06-13T07:15:16.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:16 smithi136 ceph-mon[37229]: pgmap v952: 129 pgs: 129 active+clean; 419 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 801 op/s 2024-06-13T07:15:17.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 47601 30.34 MB/sec warmup 90 sec latency 69.639 ms 2024-06-13T07:15:18.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 48336 30.49 MB/sec warmup 91 sec latency 64.761 ms 2024-06-13T07:15:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:18 smithi026 ceph-mon[32041]: pgmap v953: 129 pgs: 129 active+clean; 427 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 754 op/s 2024-06-13T07:15:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:18 smithi135 ceph-mon[36276]: pgmap v953: 129 pgs: 129 active+clean; 427 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 754 op/s 2024-06-13T07:15:18.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:18 smithi136 ceph-mon[37229]: pgmap v953: 129 pgs: 129 active+clean; 427 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 754 op/s 2024-06-13T07:15:19.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 48756 30.49 MB/sec warmup 92 sec latency 209.592 ms 2024-06-13T07:15:20.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 49167 30.43 MB/sec warmup 93 sec latency 98.643 ms 2024-06-13T07:15:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:20 smithi026 ceph-mon[32041]: pgmap v954: 129 pgs: 129 active+clean; 432 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.8 MiB/s wr, 677 op/s 2024-06-13T07:15:20.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:20 smithi135 ceph-mon[36276]: pgmap v954: 129 pgs: 129 active+clean; 432 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.8 MiB/s wr, 677 op/s 2024-06-13T07:15:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:20 smithi136 ceph-mon[37229]: pgmap v954: 129 pgs: 129 active+clean; 432 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.8 MiB/s wr, 677 op/s 2024-06-13T07:15:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.867+0000 7fb3e7400640 1 -- 172.21.15.26:0/1483553771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e01076e0 msgr2=0x7fb3e0109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.867+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1483553771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e01076e0 0x7fb3e0109b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009900 tx=0x7fb3d802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.868+0000 7fb3e7400640 1 -- 172.21.15.26:0/1483553771 shutdown_connections 2024-06-13T07:15:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.868+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1483553771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e01076e0 0x7fb3e0109b30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.868+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1483553771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e0104db0 0x7fb3e01071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.868+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1483553771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e0102480 0x7fb3e0104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.868+0000 7fb3e7400640 1 -- 172.21.15.26:0/1483553771 >> 172.21.15.26:0/1483553771 conn(0x7fb3e00fbe70 msgr2=0x7fb3e00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.868+0000 7fb3e7400640 1 -- 172.21.15.26:0/1483553771 shutdown_connections 2024-06-13T07:15:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.869+0000 7fb3e7400640 1 -- 172.21.15.26:0/1483553771 wait complete. 2024-06-13T07:15:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.870+0000 7fb3e7400640 1 Processor -- start 2024-06-13T07:15:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.870+0000 7fb3e7400640 1 -- start start 2024-06-13T07:15:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.870+0000 7fb3e7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e0102480 0x7fb3e019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e0104db0 0x7fb3e019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e01076e0 0x7fb3e01a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e0104db0 0x7fb3e019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e0102480 0x7fb3e019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e0104db0 0x7fb3e019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49890/0 (socket says 172.21.15.26:49890) 2024-06-13T07:15:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e01076e0 0x7fb3e01a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e0102480 0x7fb3e019e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42408/0 (socket says 172.21.15.26:42408) 2024-06-13T07:15:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e4a00640 1 -- 172.21.15.26:0/1280930208 learned_addr learned my addr 172.21.15.26:0/1280930208 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0101ab0 con 0x7fb3e0102480 2024-06-13T07:15:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0101930 con 0x7fb3e0104db0 2024-06-13T07:15:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.871+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0101c30 con 0x7fb3e01076e0 2024-06-13T07:15:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.872+0000 7fb3e4a00640 1 -- 172.21.15.26:0/1280930208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e01076e0 msgr2=0x7fb3e01a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.872+0000 7fb3e4a00640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e01076e0 0x7fb3e01a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.872+0000 7fb3e4a00640 1 -- 172.21.15.26:0/1280930208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e0104db0 msgr2=0x7fb3e019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.872+0000 7fb3e4a00640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e0104db0 0x7fb3e019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.872+0000 7fb3e4a00640 1 -- 172.21.15.26:0/1280930208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e01a5e70 con 0x7fb3e0102480 2024-06-13T07:15:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.872+0000 7fb3e4a00640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e0102480 0x7fb3e019e2c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400ebc0 tx=0x7fb3d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.873+0000 7fb3dd600640 1 -- 172.21.15.26:0/1280930208 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4010630 con 0x7fb3e0102480 2024-06-13T07:15:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.873+0000 7fb3dd600640 1 -- 172.21.15.26:0/1280930208 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d4010cb0 con 0x7fb3e0102480 2024-06-13T07:15:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.873+0000 7fb3dd600640 1 -- 172.21.15.26:0/1280930208 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4018c70 con 0x7fb3e0102480 2024-06-13T07:15:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.873+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e01a6160 con 0x7fb3e0102480 2024-06-13T07:15:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.873+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e01a6660 con 0x7fb3e0102480 2024-06-13T07:15:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.879+0000 7fb3ea184740 1 Processor -- start 2024-06-13T07:15:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.879+0000 7fb3dd600640 1 -- 172.21.15.26:0/1280930208 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb3d40107d0 con 0x7fb3e0102480 2024-06-13T07:15:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.880+0000 7fb3ea184740 1 -- start start 2024-06-13T07:15:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.881+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026633c00 0x55a026634020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.881+0000 7fb3dd600640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3c80a2cb0 0x7fb3c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.882+0000 7fb3dd600640 1 -- 172.21.15.26:0/1280930208 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb3d40cfc60 con 0x7fb3e0102480 2024-06-13T07:15:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.882+0000 7fb3e4a00640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3c80a2cb0 0x7fb3c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.883+0000 7fb3e5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026633c00 0x55a026634020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.883+0000 7fb3e5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026633c00 0x55a026634020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:47134/0 (socket says 172.21.15.26:47134) 2024-06-13T07:15:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.883+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0266345f0 0x55a0266390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.883+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026639760 0x55a02663bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.883+0000 7fb3ea184740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a026597420 con 0x55a026639760 2024-06-13T07:15:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.883+0000 7fb3ea184740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a02653ec50 con 0x55a0266345f0 2024-06-13T07:15:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.883+0000 7fb3ea184740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0265933e0 con 0x55a026633c00 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.883+0000 7fb3e5400640 1 -- 172.21.15.26:0/3642891821 learned_addr learned my addr 172.21.15.26:0/3642891821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.884+0000 7fb3e5400640 1 --2- 172.21.15.26:0/3642891821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026639760 0x55a02663bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.884+0000 7fb3e4a00640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3c80a2cb0 0x7fb3c80a5170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400ef00 tx=0x7fb3d4018710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.884+0000 7fb3e5400640 1 -- 172.21.15.26:0/3642891821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0266345f0 msgr2=0x55a0266390f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.884+0000 7fb3e5400640 1 --2- 172.21.15.26:0/3642891821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0266345f0 0x55a0266390f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.884+0000 7fb3e5400640 1 -- 172.21.15.26:0/3642891821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026639760 msgr2=0x55a02663bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.884+0000 7fb3e5400640 1 --2- 172.21.15.26:0/3642891821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026639760 0x55a02663bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.884+0000 7fb3e5400640 1 -- 172.21.15.26:0/3642891821 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0262ab210 con 0x55a026633c00 2024-06-13T07:15:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.885+0000 7fb3e5400640 1 --2- 172.21.15.26:0/3642891821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026633c00 0x55a026634020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009900 tx=0x7fb3d8031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669da599d5511c5f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.885+0000 7fb3d1c00640 1 -- 172.21.15.26:0/3642891821 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8035040 con 0x55a026633c00 2024-06-13T07:15:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.885+0000 7fb3d1c00640 1 -- 172.21.15.26:0/3642891821 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d8005500 con 0x55a026633c00 2024-06-13T07:15:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.885+0000 7fb3d1c00640 1 -- 172.21.15.26:0/3642891821 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8031910 con 0x55a026633c00 2024-06-13T07:15:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.886+0000 7fb3ea184740 1 -- 172.21.15.26:0/3642891821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026633c00 msgr2=0x55a026634020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.886+0000 7fb3ea184740 1 --2- 172.21.15.26:0/3642891821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026633c00 0x55a026634020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009900 tx=0x7fb3d8031400 comp rx=0 tx=0).stop 2024-06-13T07:15:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.886+0000 7fb3ea184740 1 -- 172.21.15.26:0/3642891821 shutdown_connections 2024-06-13T07:15:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.886+0000 7fb3ea184740 1 --2- 172.21.15.26:0/3642891821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0266345f0 0x55a0266390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.886+0000 7fb3ea184740 1 --2- 172.21.15.26:0/3642891821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026639760 0x55a02663bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.886+0000 7fb3ea184740 1 --2- 172.21.15.26:0/3642891821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026633c00 0x55a026634020 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.886+0000 7fb3ea184740 1 -- 172.21.15.26:0/3642891821 >> 172.21.15.26:0/3642891821 conn(0x55a0265fbfb0 msgr2=0x55a02662f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.887+0000 7fb3ea184740 1 -- 172.21.15.26:0/3642891821 shutdown_connections 2024-06-13T07:15:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.887+0000 7fb3ea184740 1 -- 172.21.15.26:0/3642891821 wait complete. 2024-06-13T07:15:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.888+0000 7fb3ea184740 1 -- start start 2024-06-13T07:15:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.888+0000 7fb3ea184740 1 Processor -- start 2024-06-13T07:15:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.888+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026633c00 0x55a02663df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.888+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0266345f0 0x55a02663e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026639760 0x55a0266454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3ea184740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a02663c6b0 con 0x55a026633c00 2024-06-13T07:15:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3ea184740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0265520b0 con 0x55a026639760 2024-06-13T07:15:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3ea184740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a02663c830 con 0x55a0266345f0 2024-06-13T07:15:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3ea184740 10 client.0 _ll_register_callbacks cb 0x55a02653df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:15:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0266345f0 0x55a02663e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0266345f0 0x55a02663e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:47136/0 (socket says 172.21.15.26:47136) 2024-06-13T07:15:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3dfe00640 1 -- 172.21.15.26:0/2374977383 learned_addr learned my addr 172.21.15.26:0/2374977383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3e5400640 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026633c00 0x55a02663df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3dfe00640 1 -- 172.21.15.26:0/2374977383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026639760 msgr2=0x55a0266454c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:15:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026639760 0x55a0266454c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3dfe00640 1 -- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026633c00 msgr2=0x55a02663df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026633c00 0x55a02663df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3dfe00640 1 -- 172.21.15.26:0/2374977383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a026377620 con 0x55a0266345f0 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.889+0000 7fb3e5400640 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026633c00 0x55a02663df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.890+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0266345f0 0x55a02663e480 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0069bb0 tx=0x7fb3cc00fa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8852d90fcd08045b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.890+0000 7fb3c6000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.890+0000 7fb3c6000640 1 -- 172.21.15.26:0/2374977383 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc004220 con 0x55a0266345f0 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.890+0000 7fb3c6000640 1 -- 172.21.15.26:0/2374977383 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb3cc0043c0 con 0x55a0266345f0 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.890+0000 7fb3ea184740 1 -- 172.21.15.26:0/2374977383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d8005500 con 0x55a0266345f0 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3c6000640 1 -- 172.21.15.26:0/2374977383 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb3cc022050 con 0x55a0266345f0 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3ea184740 10 client.26650 fetch_fsmap learned FSMap version 42 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3ea184740 1 -- 172.21.15.26:0/2374977383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a026648900 con 0x55a0266345f0 2024-06-13T07:15:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3c6000640 1 -- 172.21.15.26:0/2374977383 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d8005500 con 0x55a0266345f0 2024-06-13T07:15:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3c6000640 1 -- 172.21.15.26:0/2374977383 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7fb3cc00d770 con 0x55a0266345f0 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3ea184740 10 client.26650 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3ea184740 10 client.26650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3ea184740 20 client.26650 populate_metadata read hostname 'smithi026' 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3ea184740 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a02664a150 0x55a02664c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3ea184740 4 client.26650 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.891+0000 7fb3ea184740 1 -- 172.21.15.26:0/2374977383 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0265a4b50 con 0x55a02664a150 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.892+0000 7fb3e5400640 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a02664a150 0x55a02664c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.892+0000 7fb3c6000640 10 client.26650 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.893+0000 7fb3e5400640 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a02664a150 0x55a02664c5f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56f3ad63542b690d server_cookie=c7c4b04a65147270 in_seq=0 out_seq=0 2024-06-13T07:15:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.896+0000 7fb3c6000640 1 -- 172.21.15.26:0/2374977383 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a0265a4b50 con 0x55a02664a150 2024-06-13T07:15:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.896+0000 7fb3c6000640 10 client.26650 handle_command_reply: tid=0 2024-06-13T07:15:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.897+0000 7fb3ea184740 1 client.26650 shutdown 2024-06-13T07:15:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 -- 172.21.15.26:0/2374977383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0266345f0 msgr2=0x55a02663e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0266345f0 0x55a02663e480 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0069bb0 tx=0x7fb3cc00fa80 comp rx=0 tx=0).stop 2024-06-13T07:15:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 -- 172.21.15.26:0/2374977383 shutdown_connections 2024-06-13T07:15:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a02664a150 0x55a02664c5f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026639760 0x55a0266454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0266345f0 0x55a02663e480 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 --2- 172.21.15.26:0/2374977383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a026633c00 0x55a02663df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 -- 172.21.15.26:0/2374977383 >> 172.21.15.26:0/2374977383 conn(0x55a0265fbfb0 msgr2=0x55a02662e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3c6000640 0 client.26650 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 -- 172.21.15.26:0/2374977383 shutdown_connections 2024-06-13T07:15:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.898+0000 7fb3ea184740 1 -- 172.21.15.26:0/2374977383 wait complete. 2024-06-13T07:15:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.901+0000 7fb3ea184740 20 client.26650 trim_cache size 0 max 16384 2024-06-13T07:15:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.925+0000 7fb3ea184740 1 Processor -- start 2024-06-13T07:15:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.925+0000 7fb3ea184740 1 -- start start 2024-06-13T07:15:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.925+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a02662f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.926+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026645480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.926+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 0x55a026632650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.926+0000 7fb3ea184740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e01b36c0 con 0x55a0266345f0 2024-06-13T07:15:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.926+0000 7fb3ea184740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e01b0f00 con 0x55a026633c00 2024-06-13T07:15:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.926+0000 7fb3ea184740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e01aed50 con 0x55a026639760 2024-06-13T07:15:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.926+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026645480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026645480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42424/0 (socket says 172.21.15.26:42424) 2024-06-13T07:15:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3dfe00640 1 -- 172.21.15.26:0/282562922 learned_addr learned my addr 172.21.15.26:0/282562922 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3e5400640 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a02662f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3e5400640 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 0x55a026632650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3dfe00640 1 -- 172.21.15.26:0/282562922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 msgr2=0x55a026632650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 0x55a026632650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3dfe00640 1 -- 172.21.15.26:0/282562922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 msgr2=0x55a02662f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a02662f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.927+0000 7fb3dfe00640 1 -- 172.21.15.26:0/282562922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a026633fe0 con 0x55a0266345f0 2024-06-13T07:15:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.928+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026645480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00a500 tx=0x7fb3cc00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55df313608b43a4a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.930+0000 7fb3d1200640 1 -- 172.21.15.26:0/282562922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00dea0 con 0x55a0266345f0 2024-06-13T07:15:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.930+0000 7fb3d1200640 1 -- 172.21.15.26:0/282562922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3cc00d970 con 0x55a0266345f0 2024-06-13T07:15:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.930+0000 7fb3d1200640 1 -- 172.21.15.26:0/282562922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc004c90 con 0x55a0266345f0 2024-06-13T07:15:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 -- 172.21.15.26:0/282562922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 msgr2=0x55a026645480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026645480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00a500 tx=0x7fb3cc00aed0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 -- 172.21.15.26:0/282562922 shutdown_connections 2024-06-13T07:15:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 0x55a026632650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026645480 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 --2- 172.21.15.26:0/282562922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a02662f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 -- 172.21.15.26:0/282562922 >> 172.21.15.26:0/282562922 conn(0x55a0265fbfb0 msgr2=0x55a02662f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 -- 172.21.15.26:0/282562922 shutdown_connections 2024-06-13T07:15:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.931+0000 7fb3ea184740 1 -- 172.21.15.26:0/282562922 wait complete. 2024-06-13T07:15:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 1 -- start start 2024-06-13T07:15:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 1 Processor -- start 2024-06-13T07:15:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a0266424f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026642a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 0x55a026643a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e01af3c0 con 0x55a0266345f0 2024-06-13T07:15:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e01b0830 con 0x55a026633c00 2024-06-13T07:15:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e01afa90 con 0x55a026639760 2024-06-13T07:15:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.932+0000 7fb3ea184740 10 client.0 _ll_register_callbacks cb 0x7fb3cc00a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:15:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.933+0000 7fb3e5400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a0266424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.933+0000 7fb3e5400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a0266424f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49936/0 (socket says 172.21.15.26:49936) 2024-06-13T07:15:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.933+0000 7fb3e5400640 1 -- 172.21.15.26:0/1911923878 learned_addr learned my addr 172.21.15.26:0/1911923878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.933+0000 7fb3e5400640 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 0x55a026643a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026642a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3e5400640 1 -- 172.21.15.26:0/1911923878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 msgr2=0x55a026643a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3e5400640 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 0x55a026643a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3e5400640 1 -- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 msgr2=0x55a026642a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3e5400640 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026642a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3e5400640 1 -- 172.21.15.26:0/1911923878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a026644030 con 0x55a026633c00 2024-06-13T07:15:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026642a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:15:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3e5400640 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a0266424f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8031610 tx=0x7fb3d8050e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e06727b5a75ae1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3c6000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3c6000640 1 -- 172.21.15.26:0/1911923878 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8035040 con 0x55a026633c00 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3ea184740 1 -- 172.21.15.26:0/1911923878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3cc00d970 con 0x55a026633c00 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.934+0000 7fb3c6000640 1 -- 172.21.15.26:0/1911923878 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb3d8032b40 con 0x55a026633c00 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3c6000640 1 -- 172.21.15.26:0/1911923878 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb3d8032d40 con 0x55a026633c00 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3ea184740 10 client.26627 fetch_fsmap learned FSMap version 42 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3ea184740 1 -- 172.21.15.26:0/1911923878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a0266453e0 con 0x55a026633c00 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3c6000640 1 -- 172.21.15.26:0/1911923878 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3cc00d970 con 0x55a026633c00 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3c6000640 1 -- 172.21.15.26:0/1911923878 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7fb3d80494c0 con 0x55a026633c00 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3ea184740 10 client.26627 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3ea184740 10 client.26627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3ea184740 20 client.26627 populate_metadata read hostname 'smithi026' 2024-06-13T07:15:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3ea184740 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0266b4ed0 0x55a0266b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3ea184740 4 client.26627 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:15:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.935+0000 7fb3ea184740 1 -- 172.21.15.26:0/1911923878 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a02659eea0 con 0x55a0266b4ed0 2024-06-13T07:15:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.936+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0266b4ed0 0x55a0266b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.937+0000 7fb3dfe00640 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0266b4ed0 0x55a0266b72c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f2cf66d251abcca server_cookie=6bf58a93dae581c4 in_seq=0 out_seq=0 2024-06-13T07:15:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:20.937+0000 7fb3c6000640 10 client.26627 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:21.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 49659 30.31 MB/sec warmup 94 sec latency 204.891 ms 2024-06-13T07:15:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.406+0000 7fb3c6000640 1 -- 172.21.15.26:0/1911923878 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a02659eea0 con 0x55a0266b4ed0 2024-06-13T07:15:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.406+0000 7fb3c6000640 10 client.26627 handle_command_reply: tid=0 2024-06-13T07:15:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.406+0000 7fb3ea184740 1 client.26627 shutdown 2024-06-13T07:15:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 -- 172.21.15.26:0/1911923878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 msgr2=0x55a0266424f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a0266424f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8031610 tx=0x7fb3d8050e80 comp rx=0 tx=0).stop 2024-06-13T07:15:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 -- 172.21.15.26:0/1911923878 shutdown_connections 2024-06-13T07:15:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a0266b4ed0 0x55a0266b72c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a026639760 0x55a026643a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0266345f0 0x55a026642a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 --2- 172.21.15.26:0/1911923878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a026633c00 0x55a0266424f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3c6000640 0 client.26627 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 -- 172.21.15.26:0/1911923878 >> 172.21.15.26:0/1911923878 conn(0x55a0265fbfb0 msgr2=0x55a026632650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.407+0000 7fb3ea184740 1 -- 172.21.15.26:0/1911923878 shutdown_connections 2024-06-13T07:15:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.408+0000 7fb3ea184740 1 -- 172.21.15.26:0/1911923878 wait complete. 2024-06-13T07:15:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.408+0000 7fb3ea184740 20 client.26627 trim_cache size 0 max 16384 2024-06-13T07:15:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:15:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:15:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:15:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3c80a2cb0 msgr2=0x7fb3c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3c80a2cb0 0x7fb3c80a5170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400ef00 tx=0x7fb3d4018710 comp rx=0 tx=0).stop 2024-06-13T07:15:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e0102480 msgr2=0x7fb3e019e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e0102480 0x7fb3e019e2c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400ebc0 tx=0x7fb3d400c6a0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 shutdown_connections 2024-06-13T07:15:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb3c80a2cb0 0x7fb3c80a5170 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e0104db0 0x7fb3e019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e01076e0 0x7fb3e01a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.410+0000 7fb3e7400640 1 --2- 172.21.15.26:0/1280930208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e0102480 0x7fb3e019e2c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.411+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 >> 172.21.15.26:0/1280930208 conn(0x7fb3e00fbe70 msgr2=0x7fb3e0102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.411+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 shutdown_connections 2024-06-13T07:15:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:21.412+0000 7fb3e7400640 1 -- 172.21.15.26:0/1280930208 wait complete. 2024-06-13T07:15:22.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 50184 30.32 MB/sec warmup 95 sec latency 341.966 ms 2024-06-13T07:15:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:22 smithi026 ceph-mon[32041]: pgmap v955: 129 pgs: 129 active+clean; 434 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 505 op/s 2024-06-13T07:15:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:22 smithi135 ceph-mon[36276]: pgmap v955: 129 pgs: 129 active+clean; 434 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 505 op/s 2024-06-13T07:15:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:22 smithi136 ceph-mon[37229]: pgmap v955: 129 pgs: 129 active+clean; 434 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 505 op/s 2024-06-13T07:15:23.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 50791 30.33 MB/sec warmup 96 sec latency 87.612 ms 2024-06-13T07:15:24.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 51174 30.17 MB/sec warmup 97 sec latency 428.843 ms 2024-06-13T07:15:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:24 smithi026 ceph-mon[32041]: pgmap v956: 129 pgs: 129 active+clean; 438 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-06-13T07:15:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:24 smithi135 ceph-mon[36276]: pgmap v956: 129 pgs: 129 active+clean; 438 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-06-13T07:15:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:24 smithi136 ceph-mon[37229]: pgmap v956: 129 pgs: 129 active+clean; 438 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-06-13T07:15:25.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 51310 29.93 MB/sec warmup 98 sec latency 907.571 ms 2024-06-13T07:15:26.150 INFO:tasks.workunit.client.0.smithi026.stdout: 10 51713 29.87 MB/sec warmup 99 sec latency 101.959 ms 2024-06-13T07:15:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:26 smithi135 ceph-mon[36276]: pgmap v957: 129 pgs: 129 active+clean; 440 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.1 MiB/s wr, 596 op/s 2024-06-13T07:15:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:26 smithi136 ceph-mon[37229]: pgmap v957: 129 pgs: 129 active+clean; 440 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.1 MiB/s wr, 596 op/s 2024-06-13T07:15:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:26 smithi026 ceph-mon[32041]: pgmap v957: 129 pgs: 129 active+clean; 440 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.1 MiB/s wr, 596 op/s 2024-06-13T07:15:27.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 52119 29.80 MB/sec warmup 100 sec latency 85.789 ms 2024-06-13T07:15:28.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 52810 29.91 MB/sec warmup 101 sec latency 64.533 ms 2024-06-13T07:15:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:28 smithi135 ceph-mon[36276]: pgmap v958: 129 pgs: 129 active+clean; 455 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.6 MiB/s wr, 700 op/s 2024-06-13T07:15:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:28 smithi136 ceph-mon[37229]: pgmap v958: 129 pgs: 129 active+clean; 455 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.6 MiB/s wr, 700 op/s 2024-06-13T07:15:28.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:28 smithi026 ceph-mon[32041]: pgmap v958: 129 pgs: 129 active+clean; 455 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.6 MiB/s wr, 700 op/s 2024-06-13T07:15:29.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 53512 30.04 MB/sec warmup 102 sec latency 47.906 ms 2024-06-13T07:15:30.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 54020 30.01 MB/sec warmup 103 sec latency 137.853 ms 2024-06-13T07:15:30.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:30 smithi135 ceph-mon[36276]: pgmap v959: 129 pgs: 129 active+clean; 451 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 795 op/s 2024-06-13T07:15:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:30 smithi136 ceph-mon[37229]: pgmap v959: 129 pgs: 129 active+clean; 451 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 795 op/s 2024-06-13T07:15:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:30 smithi026 ceph-mon[32041]: pgmap v959: 129 pgs: 129 active+clean; 451 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 795 op/s 2024-06-13T07:15:31.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 54644 29.95 MB/sec warmup 104 sec latency 52.084 ms 2024-06-13T07:15:32.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 55441 30.08 MB/sec warmup 105 sec latency 50.487 ms 2024-06-13T07:15:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:32 smithi135 ceph-mon[36276]: pgmap v960: 129 pgs: 129 active+clean; 441 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-06-13T07:15:32.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:32 smithi136 ceph-mon[37229]: pgmap v960: 129 pgs: 129 active+clean; 441 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-06-13T07:15:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:32 smithi026 ceph-mon[32041]: pgmap v960: 129 pgs: 129 active+clean; 441 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-06-13T07:15:33.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 56150 30.16 MB/sec warmup 106 sec latency 57.183 ms 2024-06-13T07:15:34.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 56923 30.31 MB/sec warmup 107 sec latency 45.952 ms 2024-06-13T07:15:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:34 smithi135 ceph-mon[36276]: pgmap v961: 129 pgs: 129 active+clean; 440 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 827 op/s 2024-06-13T07:15:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:34 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:15:34.554 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:34 smithi026 ceph-mon[32041]: pgmap v961: 129 pgs: 129 active+clean; 440 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 827 op/s 2024-06-13T07:15:34.554 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:34 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:15:34.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:34 smithi136 ceph-mon[37229]: pgmap v961: 129 pgs: 129 active+clean; 440 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 827 op/s 2024-06-13T07:15:34.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:34 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:15:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.639+0000 7f5ad4000640 1 -- 172.21.15.26:0/3320761549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc108020 msgr2=0x7f5acc108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.639+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3320761549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc108020 0x7f5acc108440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009900 tx=0x7f5ac402a400 comp rx=0 tx=0).stop 2024-06-13T07:15:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.641+0000 7f5ad4000640 1 -- 172.21.15.26:0/3320761549 shutdown_connections 2024-06-13T07:15:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.641+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3320761549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5acc109210 0x7f5acc109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.641+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3320761549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5acc109bd0 0x7f5acc110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.641+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3320761549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc108020 0x7f5acc108440 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.641+0000 7f5ad4000640 1 -- 172.21.15.26:0/3320761549 >> 172.21.15.26:0/3320761549 conn(0x7f5acc076fc0 msgr2=0x7f5acc079400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.642+0000 7f5ad4000640 1 -- 172.21.15.26:0/3320761549 shutdown_connections 2024-06-13T07:15:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.642+0000 7f5ad4000640 1 -- 172.21.15.26:0/3320761549 wait complete. 2024-06-13T07:15:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.644+0000 7f5ad4000640 1 Processor -- start 2024-06-13T07:15:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.644+0000 7f5ad4000640 1 -- start start 2024-06-13T07:15:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.645+0000 7f5ad4000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5acc108020 0x7f5acc19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.645+0000 7f5ad4000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5acc109210 0x7f5acc19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.645+0000 7f5ad4000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc109bd0 0x7f5acc1a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.645+0000 7f5ad4000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc113d00 con 0x7f5acc109bd0 2024-06-13T07:15:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.645+0000 7f5ad4000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc113b80 con 0x7f5acc109210 2024-06-13T07:15:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.645+0000 7f5ad4000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc113e80 con 0x7f5acc108020 2024-06-13T07:15:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.646+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc109bd0 0x7f5acc1a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.646+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc109bd0 0x7f5acc1a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36904/0 (socket says 172.21.15.26:36904) 2024-06-13T07:15:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.646+0000 7f5ad2000640 1 -- 172.21.15.26:0/3455235484 learned_addr learned my addr 172.21.15.26:0/3455235484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.647+0000 7f5ad1600640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5acc108020 0x7f5acc19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.647+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5acc109210 0x7f5acc19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.648+0000 7f5ad2000640 1 -- 172.21.15.26:0/3455235484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5acc108020 msgr2=0x7f5acc19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.648+0000 7f5ad2000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5acc108020 0x7f5acc19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.648+0000 7f5ad2000640 1 -- 172.21.15.26:0/3455235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5acc109210 msgr2=0x7f5acc19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.648+0000 7f5ad2000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5acc109210 0x7f5acc19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.648+0000 7f5ad2000640 1 -- 172.21.15.26:0/3455235484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc06b4f0 con 0x7f5acc109bd0 2024-06-13T07:15:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.648+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5acc109210 0x7f5acc19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:15:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.649+0000 7f5ad2000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc109bd0 0x7f5acc1a5730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00cce0 tx=0x7f5abc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.652+0000 7f5aca000640 1 -- 172.21.15.26:0/3455235484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00f450 con 0x7f5acc109bd0 2024-06-13T07:15:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.652+0000 7f5aca000640 1 -- 172.21.15.26:0/3455235484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc00fad0 con 0x7f5acc109bd0 2024-06-13T07:15:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.652+0000 7f5aca000640 1 -- 172.21.15.26:0/3455235484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc018b60 con 0x7f5acc109bd0 2024-06-13T07:15:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.653+0000 7f5ad4000640 1 -- 172.21.15.26:0/3455235484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc06b7e0 con 0x7f5acc109bd0 2024-06-13T07:15:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.653+0000 7f5ad4000640 1 -- 172.21.15.26:0/3455235484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc06bcb0 con 0x7f5acc109bd0 2024-06-13T07:15:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.658+0000 7f5ad6e26740 1 Processor -- start 2024-06-13T07:15:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.660+0000 7f5aca000640 1 -- 172.21.15.26:0/3455235484 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5abc018d00 con 0x7f5acc109bd0 2024-06-13T07:15:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.663+0000 7f5aca000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5ab00a2da0 0x7f5ab00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.664+0000 7f5aca000640 1 -- 172.21.15.26:0/3455235484 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5abc09a240 con 0x7f5acc109bd0 2024-06-13T07:15:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.664+0000 7f5ad6e26740 1 -- start start 2024-06-13T07:15:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.664+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda828020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.664+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda82d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.664+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda82fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.664+0000 7f5ad6e26740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560bda78b420 con 0x560bda82d760 2024-06-13T07:15:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.665+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda82fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.665+0000 7f5ad1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda828020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.665+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda82fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36916/0 (socket says 172.21.15.26:36916) 2024-06-13T07:15:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.665+0000 7f5ad2000640 1 -- 172.21.15.26:0/519518085 learned_addr learned my addr 172.21.15.26:0/519518085 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.664+0000 7f5ad6e26740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560bda732c50 con 0x560bda8285f0 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.665+0000 7f5ad6e26740 1 -- 172.21.15.26:0/519518085 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560bda7873e0 con 0x560bda827c00 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.666+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda82d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.666+0000 7f5ad2000640 1 -- 172.21.15.26:0/519518085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 msgr2=0x560bda828020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.667+0000 7f5ad2000640 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda828020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.667+0000 7f5ad2000640 1 -- 172.21.15.26:0/519518085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 msgr2=0x560bda82d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.667+0000 7f5ad2000640 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda82d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.667+0000 7f5ad2000640 1 -- 172.21.15.26:0/519518085 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bda49f210 con 0x560bda82d760 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.667+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda82d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.667+0000 7f5ad1600640 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda828020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:15:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.667+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5ab00a2da0 0x7f5ab00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.668+0000 7f5ad2000640 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda82fbf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5abc099420 tx=0x7f5abc0a3860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f0c2196aee8366 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.668+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5ab00a2da0 0x7f5ab00a5260 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x560bda8289f0 tx=0x7f5ab8012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.669+0000 7f5ac2600640 1 -- 172.21.15.26:0/519518085 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc098bd0 con 0x560bda82d760 2024-06-13T07:15:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.669+0000 7f5ac2600640 1 -- 172.21.15.26:0/519518085 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc00de50 con 0x560bda82d760 2024-06-13T07:15:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.670+0000 7f5ac2600640 1 -- 172.21.15.26:0/519518085 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc098d70 con 0x560bda82d760 2024-06-13T07:15:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.671+0000 7f5ad6e26740 1 -- 172.21.15.26:0/519518085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 msgr2=0x560bda82fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.671+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda82fbf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5abc099420 tx=0x7f5abc0a3860 comp rx=0 tx=0).stop 2024-06-13T07:15:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.673+0000 7f5ad6e26740 1 -- 172.21.15.26:0/519518085 shutdown_connections 2024-06-13T07:15:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.673+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda82d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.673+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda82fbf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.673+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/519518085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda828020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.673+0000 7f5ad6e26740 1 -- 172.21.15.26:0/519518085 >> 172.21.15.26:0/519518085 conn(0x560bda7effb0 msgr2=0x560bda823680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.673+0000 7f5ad6e26740 1 -- 172.21.15.26:0/519518085 shutdown_connections 2024-06-13T07:15:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.674+0000 7f5ad6e26740 1 -- 172.21.15.26:0/519518085 wait complete. 2024-06-13T07:15:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.674+0000 7f5ad6e26740 1 -- start start 2024-06-13T07:15:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.675+0000 7f5ad6e26740 1 Processor -- start 2024-06-13T07:15:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.676+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda831e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.676+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 0x560bda832380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.676+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda8393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda831e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda831e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:48718/0 (socket says 172.21.15.26:48718) 2024-06-13T07:15:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad6e26740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560bda830610 con 0x560bda8285f0 2024-06-13T07:15:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad6e26740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560bda7460b0 con 0x560bda827c00 2024-06-13T07:15:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad6e26740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560bda830790 con 0x560bda82d760 2024-06-13T07:15:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda8393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda8393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36524/0 (socket says 172.21.15.26:36524) 2024-06-13T07:15:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad2000640 1 -- 172.21.15.26:0/2439982459 learned_addr learned my addr 172.21.15.26:0/2439982459 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:34.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.677+0000 7f5ad6e26740 10 client.0 _ll_register_callbacks cb 0x560bda731f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:15:34.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.678+0000 7f5ad2000640 1 -- 172.21.15.26:0/2439982459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 msgr2=0x560bda831e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:34.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.678+0000 7f5ad2000640 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda831e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.678+0000 7f5ad2000640 1 -- 172.21.15.26:0/2439982459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 msgr2=0x560bda832380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:34.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.678+0000 7f5ad2000640 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 0x560bda832380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.678+0000 7f5ad2000640 1 -- 172.21.15.26:0/2439982459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bda56b620 con 0x560bda82d760 2024-06-13T07:15:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.679+0000 7f5ad2000640 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda8393c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0986a0 tx=0x7f5abc09bb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3773e4e5d5a298aa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.679+0000 7f5ab6a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:15:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.679+0000 7f5ab6a00640 1 -- 172.21.15.26:0/2439982459 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0b4040 con 0x560bda82d760 2024-06-13T07:15:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.679+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2439982459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5abc00de50 con 0x560bda82d760 2024-06-13T07:15:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.679+0000 7f5ab6a00640 1 -- 172.21.15.26:0/2439982459 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5abc00fc80 con 0x560bda82d760 2024-06-13T07:15:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.679+0000 7f5ab6a00640 1 -- 172.21.15.26:0/2439982459 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5abc0ad050 con 0x560bda82d760 2024-06-13T07:15:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.680+0000 7f5ad6e26740 10 client.26665 fetch_fsmap learned FSMap version 42 2024-06-13T07:15:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.680+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2439982459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560bda830660 con 0x560bda82d760 2024-06-13T07:15:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.681+0000 7f5ab6a00640 1 -- 172.21.15.26:0/2439982459 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc00de50 con 0x560bda82d760 2024-06-13T07:15:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.681+0000 7f5ab6a00640 1 -- 172.21.15.26:0/2439982459 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f5abc0a1ce0 con 0x560bda82d760 2024-06-13T07:15:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.681+0000 7f5ad6e26740 10 client.26665 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:15:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.681+0000 7f5ad6e26740 10 client.26665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:15:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.681+0000 7f5ad6e26740 20 client.26665 populate_metadata read hostname 'smithi026' 2024-06-13T07:15:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.681+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560bda83da20 0x560bda83fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.681+0000 7f5ad6e26740 4 client.26665 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:15:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.681+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2439982459 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bda798b50 con 0x560bda83da20 2024-06-13T07:15:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.682+0000 7f5ad1600640 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560bda83da20 0x560bda83fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.683+0000 7f5ab6a00640 10 client.26665 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.683+0000 7f5ad1600640 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560bda83da20 0x560bda83fe10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48de0656adcfa2 server_cookie=4f682a56c1c7d88d in_seq=0 out_seq=0 2024-06-13T07:15:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.702+0000 7f5ab6a00640 1 -- 172.21.15.26:0/2439982459 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560bda798b50 con 0x560bda83da20 2024-06-13T07:15:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.702+0000 7f5ab6a00640 10 client.26665 handle_command_reply: tid=0 2024-06-13T07:15:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.702+0000 7f5ad6e26740 1 client.26665 shutdown 2024-06-13T07:15:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.703+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2439982459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 msgr2=0x560bda8393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.703+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda8393c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0986a0 tx=0x7f5abc09bb80 comp rx=0 tx=0).stop 2024-06-13T07:15:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.704+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2439982459 shutdown_connections 2024-06-13T07:15:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.704+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560bda83da20 0x560bda83fe10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.704+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda8393c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.704+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 0x560bda832380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.704+0000 7f5ab6a00640 0 client.26665 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.704+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2439982459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda831e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.704+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2439982459 >> 172.21.15.26:0/2439982459 conn(0x560bda7effb0 msgr2=0x560bda822840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.704+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2439982459 shutdown_connections 2024-06-13T07:15:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.705+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2439982459 wait complete. 2024-06-13T07:15:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.706+0000 7f5ad6e26740 20 client.26665 trim_cache size 0 max 16384 2024-06-13T07:15:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.750+0000 7f5ad6e26740 1 Processor -- start 2024-06-13T07:15:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.751+0000 7f5ad6e26740 1 -- start start 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda827fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda839380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda829910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda827fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad6e26740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1bbcb0 con 0x560bda82d760 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda827fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:36538/0 (socket says 172.21.15.26:36538) 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad6e26740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1b94f0 con 0x560bda8285f0 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2040210348 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc073980 con 0x560bda827c00 2024-06-13T07:15:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.752+0000 7f5ad1600640 1 -- 172.21.15.26:0/2040210348 learned_addr learned my addr 172.21.15.26:0/2040210348 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad2000640 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda829910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda839380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad1600640 1 -- 172.21.15.26:0/2040210348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 msgr2=0x560bda839380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad1600640 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda839380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad1600640 1 -- 172.21.15.26:0/2040210348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 msgr2=0x560bda829910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad1600640 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda829910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad1600640 1 -- 172.21.15.26:0/2040210348 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bda82cbf0 con 0x560bda827c00 2024-06-13T07:15:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda839380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:15:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.753+0000 7f5ad2000640 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda829910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:15:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.754+0000 7f5ad1600640 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda827fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4031400 tx=0x7f5ac4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb47a6ffad377fe server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.754+0000 7f5ac1c00640 1 -- 172.21.15.26:0/2040210348 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac404a020 con 0x560bda827c00 2024-06-13T07:15:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.755+0000 7f5ac1c00640 1 -- 172.21.15.26:0/2040210348 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560bda798b50 con 0x560bda827c00 2024-06-13T07:15:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.755+0000 7f5ac1c00640 1 -- 172.21.15.26:0/2040210348 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4035040 con 0x560bda827c00 2024-06-13T07:15:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.756+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2040210348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 msgr2=0x560bda827fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.756+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda827fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4031400 tx=0x7f5ac4005850 comp rx=0 tx=0).stop 2024-06-13T07:15:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.758+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2040210348 shutdown_connections 2024-06-13T07:15:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.758+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda8285f0 0x560bda839380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.758+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda82d760 0x560bda829910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.758+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/2040210348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda827c00 0x560bda827fe0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.758+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2040210348 >> 172.21.15.26:0/2040210348 conn(0x560bda7effb0 msgr2=0x560bda831e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.758+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2040210348 shutdown_connections 2024-06-13T07:15:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.758+0000 7f5ad6e26740 1 -- 172.21.15.26:0/2040210348 wait complete. 2024-06-13T07:15:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.758+0000 7f5ad6e26740 1 -- start start 2024-06-13T07:15:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.759+0000 7f5ad6e26740 1 Processor -- start 2024-06-13T07:15:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.761+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda836620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.761+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 0x560bda836b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.761+0000 7f5ad6e26740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda837b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.761+0000 7f5ad1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda836620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.761+0000 7f5ad1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda836620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:48736/0 (socket says 172.21.15.26:48736) 2024-06-13T07:15:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.761+0000 7f5ad1600640 1 -- 172.21.15.26:0/1793084908 learned_addr learned my addr 172.21.15.26:0/1793084908 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1b79b0 con 0x560bda8285f0 2024-06-13T07:15:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1b8e20 con 0x560bda827c00 2024-06-13T07:15:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1b8080 con 0x560bda82d760 2024-06-13T07:15:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad6e26740 10 client.0 _ll_register_callbacks cb 0x7f5abc0674e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:15:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad2000640 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda837b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 0x560bda836b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad1600640 1 -- 172.21.15.26:0/1793084908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 msgr2=0x560bda837b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad1600640 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda837b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad1600640 1 -- 172.21.15.26:0/1793084908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 msgr2=0x560bda836b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad1600640 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 0x560bda836b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.762+0000 7f5ad1600640 1 -- 172.21.15.26:0/1793084908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bda838160 con 0x560bda827c00 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.763+0000 7f5ad1600640 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda836620 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009230 tx=0x7f5ac40315d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5a4b8ac4c217578 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.763+0000 7f5ab6a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.764+0000 7f5ab6a00640 1 -- 172.21.15.26:0/1793084908 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac404a030 con 0x560bda827c00 2024-06-13T07:15:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.764+0000 7f5ab6a00640 1 -- 172.21.15.26:0/1793084908 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5ac40036b0 con 0x560bda827c00 2024-06-13T07:15:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.764+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bda798b50 con 0x560bda827c00 2024-06-13T07:15:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.765+0000 7f5ab6a00640 1 -- 172.21.15.26:0/1793084908 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5ac405d050 con 0x560bda827c00 2024-06-13T07:15:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.765+0000 7f5ad6e26740 10 client.26639 fetch_fsmap learned FSMap version 42 2024-06-13T07:15:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.765+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560bda839480 con 0x560bda827c00 2024-06-13T07:15:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.765+0000 7f5ab6a00640 1 -- 172.21.15.26:0/1793084908 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560bda798b50 con 0x560bda827c00 2024-06-13T07:15:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.766+0000 7f5ab6a00640 1 -- 172.21.15.26:0/1793084908 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f5ac40467b0 con 0x560bda827c00 2024-06-13T07:15:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.766+0000 7f5ad6e26740 10 client.26639 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:15:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.766+0000 7f5ad6e26740 10 client.26639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:15:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.766+0000 7f5ad6e26740 20 client.26639 populate_metadata read hostname 'smithi026' 2024-06-13T07:15:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.766+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560bda8a8d20 0x560bda8ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.766+0000 7f5ad6e26740 4 client.26639 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:15:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.766+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bda792ea0 con 0x560bda8a8d20 2024-06-13T07:15:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.767+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560bda8a8d20 0x560bda8ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.769+0000 7f5ad0c00640 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560bda8a8d20 0x560bda8ab140 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c85ea3745ab669 server_cookie=7d02eceaa2168568 in_seq=0 out_seq=0 2024-06-13T07:15:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:34.769+0000 7f5ab6a00640 10 client.26639 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:35.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 57347 30.31 MB/sec warmup 108 sec latency 242.033 ms 2024-06-13T07:15:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.336+0000 7f5ab6a00640 1 -- 172.21.15.26:0/1793084908 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560bda792ea0 con 0x560bda8a8d20 2024-06-13T07:15:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.336+0000 7f5ab6a00640 10 client.26639 handle_command_reply: tid=0 2024-06-13T07:15:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.336+0000 7f5ad6e26740 1 client.26639 shutdown 2024-06-13T07:15:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.336+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 msgr2=0x560bda836620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.336+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda836620 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009230 tx=0x7f5ac40315d0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.336+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 shutdown_connections 2024-06-13T07:15:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.336+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560bda8a8d20 0x560bda8ab140 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.337+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bda82d760 0x560bda837b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.337+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bda8285f0 0x560bda836b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.337+0000 7f5ad6e26740 1 --2- 172.21.15.26:0/1793084908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560bda827c00 0x560bda836620 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.337+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 >> 172.21.15.26:0/1793084908 conn(0x560bda7effb0 msgr2=0x560bda829910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.337+0000 7f5ab6a00640 0 client.26639 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.337+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 shutdown_connections 2024-06-13T07:15:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.338+0000 7f5ad6e26740 1 -- 172.21.15.26:0/1793084908 wait complete. 2024-06-13T07:15:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.338+0000 7f5ad6e26740 20 client.26639 trim_cache size 0 max 16384 2024-06-13T07:15:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:15:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:15:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:15:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 -- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5ab00a2da0 msgr2=0x7f5ab00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5ab00a2da0 0x7f5ab00a5260 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x560bda8289f0 tx=0x7f5ab8012040 comp rx=0 tx=0).stop 2024-06-13T07:15:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 -- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc109bd0 msgr2=0x7f5acc1a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc109bd0 0x7f5acc1a5730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00cce0 tx=0x7f5abc00a620 comp rx=0 tx=0).stop 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 -- 172.21.15.26:0/3455235484 shutdown_connections 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5ab00a2da0 0x7f5ab00a5260 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5acc109210 0x7f5acc19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc109bd0 0x7f5acc1a5730 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 --2- 172.21.15.26:0/3455235484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5acc108020 0x7f5acc19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 -- 172.21.15.26:0/3455235484 >> 172.21.15.26:0/3455235484 conn(0x7f5acc076fc0 msgr2=0x7f5acc10c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 -- 172.21.15.26:0/3455235484 shutdown_connections 2024-06-13T07:15:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:35.339+0000 7f5ad4000640 1 -- 172.21.15.26:0/3455235484 wait complete. 2024-06-13T07:15:35.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:15:35.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:15:35.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:35 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:15:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:15:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:15:35.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:35 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:15:35.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:15:35.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:15:35.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:35 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:15:36.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 57961 30.25 MB/sec warmup 109 sec latency 277.856 ms 2024-06-13T07:15:36.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:36 smithi136 ceph-mon[37229]: pgmap v962: 129 pgs: 129 active+clean; 447 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-06-13T07:15:36.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:36 smithi026 ceph-mon[32041]: pgmap v962: 129 pgs: 129 active+clean; 447 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-06-13T07:15:36.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:36 smithi135 ceph-mon[36276]: pgmap v962: 129 pgs: 129 active+clean; 447 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-06-13T07:15:37.151 INFO:tasks.workunit.client.0.smithi026.stdout: 10 58575 30.27 MB/sec warmup 110 sec latency 56.434 ms 2024-06-13T07:15:38.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 59235 30.32 MB/sec warmup 111 sec latency 71.821 ms 2024-06-13T07:15:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:38 smithi136 ceph-mon[37229]: pgmap v963: 129 pgs: 129 active+clean; 454 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 736 op/s 2024-06-13T07:15:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:38 smithi026 ceph-mon[32041]: pgmap v963: 129 pgs: 129 active+clean; 454 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 736 op/s 2024-06-13T07:15:38.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:38 smithi135 ceph-mon[36276]: pgmap v963: 129 pgs: 129 active+clean; 454 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 736 op/s 2024-06-13T07:15:39.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 60027 30.42 MB/sec warmup 112 sec latency 70.498 ms 2024-06-13T07:15:40.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 60841 30.54 MB/sec warmup 113 sec latency 50.157 ms 2024-06-13T07:15:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:40 smithi136 ceph-mon[37229]: pgmap v964: 129 pgs: 129 active+clean; 459 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 727 op/s 2024-06-13T07:15:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:40 smithi026 ceph-mon[32041]: pgmap v964: 129 pgs: 129 active+clean; 459 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 727 op/s 2024-06-13T07:15:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:40 smithi135 ceph-mon[36276]: pgmap v964: 129 pgs: 129 active+clean; 459 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 727 op/s 2024-06-13T07:15:41.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 61674 30.70 MB/sec warmup 114 sec latency 42.858 ms 2024-06-13T07:15:42.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 62118 30.65 MB/sec warmup 115 sec latency 372.976 ms 2024-06-13T07:15:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:42 smithi136 ceph-mon[37229]: pgmap v965: 129 pgs: 129 active+clean; 456 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 577 op/s 2024-06-13T07:15:42.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:42 smithi026 ceph-mon[32041]: pgmap v965: 129 pgs: 129 active+clean; 456 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 577 op/s 2024-06-13T07:15:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:42 smithi135 ceph-mon[36276]: pgmap v965: 129 pgs: 129 active+clean; 456 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 577 op/s 2024-06-13T07:15:43.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 62610 30.64 MB/sec warmup 116 sec latency 68.665 ms 2024-06-13T07:15:44.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 63071 30.53 MB/sec warmup 117 sec latency 106.613 ms 2024-06-13T07:15:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:44 smithi136 ceph-mon[37229]: pgmap v966: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 553 op/s 2024-06-13T07:15:44.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:44 smithi026 ceph-mon[32041]: pgmap v966: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 553 op/s 2024-06-13T07:15:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:44 smithi135 ceph-mon[36276]: pgmap v966: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 553 op/s 2024-06-13T07:15:45.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 63432 30.39 MB/sec warmup 118 sec latency 397.342 ms 2024-06-13T07:15:46.153 INFO:tasks.workunit.client.0.smithi026.stdout: 10 63655 30.22 MB/sec warmup 119 sec latency 573.335 ms 2024-06-13T07:15:46.584 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:46 smithi135 ceph-mon[36276]: pgmap v967: 129 pgs: 129 active+clean; 471 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.1 MiB/s wr, 591 op/s 2024-06-13T07:15:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:46 smithi136 ceph-mon[37229]: pgmap v967: 129 pgs: 129 active+clean; 471 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.1 MiB/s wr, 591 op/s 2024-06-13T07:15:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:46 smithi026 ceph-mon[32041]: pgmap v967: 129 pgs: 129 active+clean; 471 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.1 MiB/s wr, 591 op/s 2024-06-13T07:15:48.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 64888 45.68 MB/sec execute 1 sec latency 318.047 ms 2024-06-13T07:15:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 -- 172.21.15.26:0/3714370591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 msgr2=0x7fa7f4073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/3714370591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 0x7fa7f4073dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009ca0 tx=0x7fa7ec02a960 comp rx=0 tx=0).stop 2024-06-13T07:15:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 -- 172.21.15.26:0/3714370591 shutdown_connections 2024-06-13T07:15:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/3714370591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f41096f0 0x7fa7f410bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/3714370591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7f40743a0 0x7fa7f4074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/3714370591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 0x7fa7f4073dd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 -- 172.21.15.26:0/3714370591 >> 172.21.15.26:0/3714370591 conn(0x7fa7f40fcdd0 msgr2=0x7fa7f40ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 -- 172.21.15.26:0/3714370591 shutdown_connections 2024-06-13T07:15:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.528+0000 7fa7f8e00640 1 -- 172.21.15.26:0/3714370591 wait complete. 2024-06-13T07:15:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.530+0000 7fa7f8e00640 1 Processor -- start 2024-06-13T07:15:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.530+0000 7fa7f8e00640 1 -- start start 2024-06-13T07:15:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.531+0000 7fa7f8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f40743a0 0x7fa7f4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.531+0000 7fa7f8e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 0x7fa7f419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.531+0000 7fa7f3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f40743a0 0x7fa7f4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.531+0000 7fa7f3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f40743a0 0x7fa7f4199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41364/0 (socket says 172.21.15.26:41364) 2024-06-13T07:15:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f3400640 1 -- 172.21.15.26:0/1356429471 learned_addr learned my addr 172.21.15.26:0/1356429471 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7f41096f0 0x7fa7f41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f410f7a0 con 0x7fa7f40743a0 2024-06-13T07:15:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f410f620 con 0x7fa7f4075980 2024-06-13T07:15:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f410f920 con 0x7fa7f41096f0 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 0x7fa7f419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f3e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7f41096f0 0x7fa7f41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f2a00640 1 -- 172.21.15.26:0/1356429471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7f41096f0 msgr2=0x7fa7f41a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7f41096f0 0x7fa7f41a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f2a00640 1 -- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f40743a0 msgr2=0x7fa7f4199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f40743a0 0x7fa7f4199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f2a00640 1 -- 172.21.15.26:0/1356429471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f41a19c0 con 0x7fa7f4075980 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.533+0000 7fa7f3400640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f40743a0 0x7fa7f4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:15:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.534+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 0x7fa7f419a2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400e9f0 tx=0x7fa7e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.534+0000 7fa7ebe00640 1 -- 172.21.15.26:0/1356429471 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400ccc0 con 0x7fa7f4075980 2024-06-13T07:15:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.534+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f41a1cb0 con 0x7fa7f4075980 2024-06-13T07:15:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.534+0000 7fa7ebe00640 1 -- 172.21.15.26:0/1356429471 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e4010910 con 0x7fa7f4075980 2024-06-13T07:15:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.534+0000 7fa7ebe00640 1 -- 172.21.15.26:0/1356429471 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4018960 con 0x7fa7f4075980 2024-06-13T07:15:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.534+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f41a2220 con 0x7fa7f4075980 2024-06-13T07:15:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.538+0000 7fa7fd181740 1 Processor -- start 2024-06-13T07:15:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.538+0000 7fa7fd181740 1 -- start start 2024-06-13T07:15:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.538+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.538+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a4d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.538+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a4d920 0x55c3a1a4fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.538+0000 7fa7fd181740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c3a19ab420 con 0x55c3a1a487b0 2024-06-13T07:15:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.538+0000 7fa7fd181740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c3a1952c50 con 0x55c3a1a47dc0 2024-06-13T07:15:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.538+0000 7fa7fd181740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c3a19a73e0 con 0x55c3a1a4d920 2024-06-13T07:15:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:42734/0 (socket says 172.21.15.26:42734) 2024-06-13T07:15:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 -- 172.21.15.26:0/722321178 learned_addr learned my addr 172.21.15.26:0/722321178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3400640 1 --2- 172.21.15.26:0/722321178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a4d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 -- 172.21.15.26:0/722321178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a4d920 msgr2=0x55c3a1a4fdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 --2- 172.21.15.26:0/722321178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a4d920 0x55c3a1a4fdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 -- 172.21.15.26:0/722321178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 msgr2=0x55c3a1a4d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 --2- 172.21.15.26:0/722321178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a4d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 -- 172.21.15.26:0/722321178 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a16bf210 con 0x55c3a1a47dc0 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.539+0000 7fa7f3e00640 1 --2- 172.21.15.26:0/722321178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a481e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00099c0 tx=0x7fa7e0009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4079a67ce73490 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7e8c00640 1 -- 172.21.15.26:0/722321178 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0024070 con 0x55c3a1a47dc0 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7e8c00640 1 -- 172.21.15.26:0/722321178 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e0017d80 con 0x55c3a1a47dc0 2024-06-13T07:15:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7e8c00640 1 -- 172.21.15.26:0/722321178 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e001fc50 con 0x55c3a1a47dc0 2024-06-13T07:15:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7ebe00640 1 -- 172.21.15.26:0/1356429471 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7e4010ac0 con 0x7fa7f4075980 2024-06-13T07:15:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7fd181740 1 -- 172.21.15.26:0/722321178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 msgr2=0x55c3a1a481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7fd181740 1 --2- 172.21.15.26:0/722321178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a481e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00099c0 tx=0x7fa7e0009e80 comp rx=0 tx=0).stop 2024-06-13T07:15:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7fd181740 1 -- 172.21.15.26:0/722321178 shutdown_connections 2024-06-13T07:15:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7fd181740 1 --2- 172.21.15.26:0/722321178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a4d920 0x55c3a1a4fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7fd181740 1 --2- 172.21.15.26:0/722321178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a4d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7fd181740 1 --2- 172.21.15.26:0/722321178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a481e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.540+0000 7fa7fd181740 1 -- 172.21.15.26:0/722321178 >> 172.21.15.26:0/722321178 conn(0x55c3a1a0ffb0 msgr2=0x55c3a1a43840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.541+0000 7fa7fd181740 1 -- 172.21.15.26:0/722321178 shutdown_connections 2024-06-13T07:15:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.541+0000 7fa7fd181740 1 -- 172.21.15.26:0/722321178 wait complete. 2024-06-13T07:15:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.541+0000 7fa7fd181740 1 -- start start 2024-06-13T07:15:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.541+0000 7fa7fd181740 1 Processor -- start 2024-06-13T07:15:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.541+0000 7fa7ebe00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7d80a2a90 0x7fa7d80a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7ebe00640 1 -- 172.21.15.26:0/1356429471 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa7e40cf830 con 0x7fa7f4075980 2024-06-13T07:15:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7f3e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7d80a2a90 0x7fa7d80a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a63160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a487b0 0x55c3a1a636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a4d920 0x55c3a1a6a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7fd181740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c3a1a618e0 con 0x55c3a1a4d920 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7fd181740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c3a19660b0 con 0x55c3a1a487b0 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7fd181740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c3a1a61a60 con 0x55c3a1a47dc0 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7fd181740 10 client.0 _ll_register_callbacks cb 0x55c3a1951f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7f2a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a487b0 0x55c3a1a636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7f2a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a487b0 0x55c3a1a636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:42736/0 (socket says 172.21.15.26:42736) 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7f2a00640 1 -- 172.21.15.26:0/3453404446 learned_addr learned my addr 172.21.15.26:0/3453404446 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7f2a00640 1 -- 172.21.15.26:0/3453404446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 msgr2=0x55c3a1a63160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a63160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.542+0000 7fa7f2a00640 1 -- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a4d920 msgr2=0x55c3a1a6a6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7f3e00640 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a4d920 0x55c3a1a6a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a4d920 0x55c3a1a6a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7f2a00640 1 -- 172.21.15.26:0/3453404446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a178b620 con 0x55c3a1a487b0 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7f3e00640 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a4d920 0x55c3a1a6a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a487b0 0x55c3a1a636a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400e9c0 tx=0x7fa7e4099e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30b27f33eab11648 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7dd600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7dd600640 1 -- 172.21.15.26:0/3453404446 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40a4910 con 0x55c3a1a487b0 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7fd181740 1 -- 172.21.15.26:0/3453404446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e0017d80 con 0x55c3a1a487b0 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7dd600640 1 -- 172.21.15.26:0/3453404446 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa7e4010430 con 0x55c3a1a487b0 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7dd600640 1 -- 172.21.15.26:0/3453404446 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa7e40b9050 con 0x55c3a1a487b0 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7fd181740 10 client.26651 fetch_fsmap learned FSMap version 42 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.543+0000 7fa7fd181740 1 -- 172.21.15.26:0/3453404446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c3a1a6daf0 con 0x55c3a1a487b0 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7f3e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7d80a2a90 0x7fa7d80a4f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000a010 tx=0x7fa7e00095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7dd600640 1 -- 172.21.15.26:0/3453404446 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e0017d80 con 0x55c3a1a487b0 2024-06-13T07:15:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7dd600640 1 -- 172.21.15.26:0/3453404446 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7fa7e40b3030 con 0x55c3a1a487b0 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7fd181740 10 client.26651 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7fd181740 10 client.26651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7fd181740 20 client.26651 populate_metadata read hostname 'smithi026' 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7fd181740 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c3a1a6f1b0 0x55c3a1a715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7fd181740 4 client.26651 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7fd181740 1 -- 172.21.15.26:0/3453404446 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3a19b8b50 con 0x55c3a1a6f1b0 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.544+0000 7fa7f3400640 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c3a1a6f1b0 0x55c3a1a715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.545+0000 7fa7f3400640 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c3a1a6f1b0 0x55c3a1a715f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48c8b429be60f1c server_cookie=7ae67bff78155caa in_seq=0 out_seq=0 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.545+0000 7fa7dd600640 10 client.26651 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.545+0000 7fa7dd600640 1 -- 172.21.15.26:0/3453404446 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c3a19b8b50 con 0x55c3a1a6f1b0 2024-06-13T07:15:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.545+0000 7fa7dd600640 10 client.26651 handle_command_reply: tid=0 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 client.26651 shutdown 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 -- 172.21.15.26:0/3453404446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a487b0 msgr2=0x55c3a1a636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a487b0 0x55c3a1a636a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400e9c0 tx=0x7fa7e4099e90 comp rx=0 tx=0).stop 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 -- 172.21.15.26:0/3453404446 shutdown_connections 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c3a1a6f1b0 0x55c3a1a715f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a487b0 0x55c3a1a636a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7dd600640 0 client.26651 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a4d920 0x55c3a1a6a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 --2- 172.21.15.26:0/3453404446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a63160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 -- 172.21.15.26:0/3453404446 >> 172.21.15.26:0/3453404446 conn(0x55c3a1a0ffb0 msgr2=0x55c3a1a429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 -- 172.21.15.26:0/3453404446 shutdown_connections 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 1 -- 172.21.15.26:0/3453404446 wait complete. 2024-06-13T07:15:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.546+0000 7fa7fd181740 20 client.26651 trim_cache size 0 max 16384 2024-06-13T07:15:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.562+0000 7fa7fd181740 1 Processor -- start 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7fd181740 1 -- start start 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a3e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a46c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7fd181740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41a2600 con 0x55c3a1a487b0 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7fd181740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4071500 con 0x55c3a1a47dc0 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7fd181740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f406f350 con 0x55c3a1a0ffb0 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7f3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a46c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7f3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a46c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41382/0 (socket says 172.21.15.26:41382) 2024-06-13T07:15:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7f3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a3e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7f3e00640 1 -- 172.21.15.26:0/1193911352 learned_addr learned my addr 172.21.15.26:0/1193911352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.563+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/1193911352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7f3400640 1 -- 172.21.15.26:0/1193911352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 msgr2=0x55c3a1a6a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7f3400640 1 --2- 172.21.15.26:0/1193911352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a6a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7f3400640 1 -- 172.21.15.26:0/1193911352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 msgr2=0x55c3a1a46c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7f3400640 1 --2- 172.21.15.26:0/1193911352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a46c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7f3400640 1 -- 172.21.15.26:0/1193911352 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a1a3f000 con 0x55c3a1a0ffb0 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7f3400640 1 --2- 172.21.15.26:0/1193911352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a3e830 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa7f419b280 tx=0x7fa7ec02f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7df01cda4666de1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7dfe00640 1 -- 172.21.15.26:0/1193911352 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec049070 con 0x55c3a1a0ffb0 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7dfe00640 1 -- 172.21.15.26:0/1193911352 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec003e50 con 0x55c3a1a0ffb0 2024-06-13T07:15:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.564+0000 7fa7dfe00640 1 -- 172.21.15.26:0/1193911352 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec034040 con 0x55c3a1a0ffb0 2024-06-13T07:15:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 -- 172.21.15.26:0/1193911352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a0ffb0 msgr2=0x55c3a1a3e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 --2- 172.21.15.26:0/1193911352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a3e830 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa7f419b280 tx=0x7fa7ec02f9e0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 -- 172.21.15.26:0/1193911352 shutdown_connections 2024-06-13T07:15:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 --2- 172.21.15.26:0/1193911352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a6a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 --2- 172.21.15.26:0/1193911352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a46c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 --2- 172.21.15.26:0/1193911352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a3e830 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 -- 172.21.15.26:0/1193911352 >> 172.21.15.26:0/1193911352 conn(0x55c3a1a62d20 msgr2=0x55c3a1a46500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 -- 172.21.15.26:0/1193911352 shutdown_connections 2024-06-13T07:15:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 -- 172.21.15.26:0/1193911352 wait complete. 2024-06-13T07:15:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.565+0000 7fa7fd181740 1 -- start start 2024-06-13T07:15:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7fd181740 1 Processor -- start 2024-06-13T07:15:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a67f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a68470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7fd181740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a69410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7fd181740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f406f9c0 con 0x55c3a1a487b0 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7fd181740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4070e30 con 0x55c3a1a0ffb0 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7fd181740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4070090 con 0x55c3a1a47dc0 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7fd181740 10 client.0 _ll_register_callbacks cb 0x7fa7e400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7f3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a69410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7f2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a68470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7f3400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a67f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7f2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a68470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50488/0 (socket says 172.21.15.26:50488) 2024-06-13T07:15:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.566+0000 7fa7f2a00640 1 -- 172.21.15.26:0/524446672 learned_addr learned my addr 172.21.15.26:0/524446672 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:15:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7f2a00640 1 -- 172.21.15.26:0/524446672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a0ffb0 msgr2=0x55c3a1a67f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a67f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7f2a00640 1 -- 172.21.15.26:0/524446672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 msgr2=0x55c3a1a69410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:15:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a69410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7f2a00640 1 -- 172.21.15.26:0/524446672 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a1a69a00 con 0x55c3a1a47dc0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7f2a00640 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a68470 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400c6a0 tx=0x7fa7e409b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf47850d63c52fcc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7dd600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7dd600640 1 -- 172.21.15.26:0/524446672 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e409b890 con 0x55c3a1a47dc0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7dd600640 1 -- 172.21.15.26:0/524446672 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa7e4010640 con 0x55c3a1a47dc0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7fd181740 1 -- 172.21.15.26:0/524446672 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ec003e50 con 0x55c3a1a47dc0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7dd600640 1 -- 172.21.15.26:0/524446672 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa7e409bb20 con 0x55c3a1a47dc0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7fd181740 10 client.26683 fetch_fsmap learned FSMap version 42 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.567+0000 7fa7fd181740 1 -- 172.21.15.26:0/524446672 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c3a1ada040 con 0x55c3a1a47dc0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7dd600640 1 -- 172.21.15.26:0/524446672 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec003e50 con 0x55c3a1a47dc0 2024-06-13T07:15:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7dd600640 1 -- 172.21.15.26:0/524446672 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7fa7e40b3040 con 0x55c3a1a47dc0 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7fd181740 10 client.26683 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7fd181740 10 client.26683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7fd181740 20 client.26683 populate_metadata read hostname 'smithi026' 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7fd181740 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c3a1a4d920 0x55c3a1adcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7fd181740 4 client.26683 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7fd181740 1 -- 172.21.15.26:0/524446672 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3a19b2ea0 con 0x55c3a1a4d920 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.568+0000 7fa7f3400640 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c3a1a4d920 0x55c3a1adcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.569+0000 7fa7f3400640 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c3a1a4d920 0x55c3a1adcad0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7e4e03bc6efb1d server_cookie=afbae2ef6e2b381 in_seq=0 out_seq=0 2024-06-13T07:15:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:48.569+0000 7fa7dd600640 10 client.26683 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:48 smithi136 ceph-mon[37229]: pgmap v968: 129 pgs: 129 active+clean; 488 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.0 MiB/s wr, 628 op/s 2024-06-13T07:15:48.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:48 smithi026 ceph-mon[32041]: pgmap v968: 129 pgs: 129 active+clean; 488 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.0 MiB/s wr, 628 op/s 2024-06-13T07:15:48.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:48 smithi135 ceph-mon[36276]: pgmap v968: 129 pgs: 129 active+clean; 488 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.0 MiB/s wr, 628 op/s 2024-06-13T07:15:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.007+0000 7fa7dd600640 1 -- 172.21.15.26:0/524446672 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3a19b2ea0 con 0x55c3a1a4d920 2024-06-13T07:15:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.007+0000 7fa7dd600640 10 client.26683 handle_command_reply: tid=0 2024-06-13T07:15:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.007+0000 7fa7fd181740 1 client.26683 shutdown 2024-06-13T07:15:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7fd181740 1 -- 172.21.15.26:0/524446672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 msgr2=0x55c3a1a68470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7fd181740 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a68470 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400c6a0 tx=0x7fa7e409b5e0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7fd181740 1 -- 172.21.15.26:0/524446672 shutdown_connections 2024-06-13T07:15:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7fd181740 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c3a1a4d920 0x55c3a1adcad0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7fd181740 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c3a1a487b0 0x55c3a1a69410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7fd181740 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3a1a47dc0 0x55c3a1a68470 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7fd181740 1 --2- 172.21.15.26:0/524446672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3a1a0ffb0 0x55c3a1a67f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7fd181740 1 -- 172.21.15.26:0/524446672 >> 172.21.15.26:0/524446672 conn(0x55c3a1a62d20 msgr2=0x55c3a1a46d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.008+0000 7fa7dd600640 0 client.26683 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:15:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.009+0000 7fa7fd181740 1 -- 172.21.15.26:0/524446672 shutdown_connections 2024-06-13T07:15:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.009+0000 7fa7fd181740 1 -- 172.21.15.26:0/524446672 wait complete. 2024-06-13T07:15:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.009+0000 7fa7fd181740 20 client.26683 trim_cache size 0 max 16384 2024-06-13T07:15:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:15:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:15:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:15:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7d80a2a90 msgr2=0x7fa7d80a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7d80a2a90 0x7fa7d80a4f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000a010 tx=0x7fa7e00095b0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 msgr2=0x7fa7f419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:15:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 0x7fa7f419a2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400e9f0 tx=0x7fa7e400eeb0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 shutdown_connections 2024-06-13T07:15:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa7d80a2a90 0x7fa7d80a4f50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f4075980 0x7fa7f419a2f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7f41096f0 0x7fa7f41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 --2- 172.21.15.26:0/1356429471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f40743a0 0x7fa7f4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:15:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.011+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 >> 172.21.15.26:0/1356429471 conn(0x7fa7f40fcdd0 msgr2=0x7fa7f4105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:15:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.013+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 shutdown_connections 2024-06-13T07:15:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:15:49.014+0000 7fa7f8e00640 1 -- 172.21.15.26:0/1356429471 wait complete. 2024-06-13T07:15:49.152 INFO:tasks.workunit.client.0.smithi026.stdout: 10 65395 39.95 MB/sec execute 2 sec latency 319.494 ms 2024-06-13T07:15:50.153 INFO:tasks.workunit.client.0.smithi026.stdout: 10 66024 41.56 MB/sec execute 3 sec latency 62.244 ms 2024-06-13T07:15:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:50 smithi136 ceph-mon[37229]: pgmap v969: 129 pgs: 129 active+clean; 489 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 673 op/s 2024-06-13T07:15:50.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:50 smithi026 ceph-mon[32041]: pgmap v969: 129 pgs: 129 active+clean; 489 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 673 op/s 2024-06-13T07:15:50.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:50 smithi135 ceph-mon[36276]: pgmap v969: 129 pgs: 129 active+clean; 489 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 673 op/s 2024-06-13T07:15:51.153 INFO:tasks.workunit.client.0.smithi026.stdout: 10 66126 32.19 MB/sec execute 4 sec latency 666.091 ms 2024-06-13T07:15:52.153 INFO:tasks.workunit.client.0.smithi026.stdout: 10 66322 27.36 MB/sec execute 5 sec latency 546.464 ms 2024-06-13T07:15:52.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:52 smithi136 ceph-mon[37229]: pgmap v970: 129 pgs: 129 active+clean; 480 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.7 MiB/s wr, 590 op/s 2024-06-13T07:15:52.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:52 smithi026 ceph-mon[32041]: pgmap v970: 129 pgs: 129 active+clean; 480 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.7 MiB/s wr, 590 op/s 2024-06-13T07:15:52.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:52 smithi135 ceph-mon[36276]: pgmap v970: 129 pgs: 129 active+clean; 480 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.7 MiB/s wr, 590 op/s 2024-06-13T07:15:53.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 66611 24.83 MB/sec execute 6 sec latency 151.992 ms 2024-06-13T07:15:54.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 67182 24.81 MB/sec execute 7 sec latency 126.070 ms 2024-06-13T07:15:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:54 smithi136 ceph-mon[37229]: pgmap v971: 129 pgs: 129 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 667 op/s 2024-06-13T07:15:54.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:54 smithi026 ceph-mon[32041]: pgmap v971: 129 pgs: 129 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 667 op/s 2024-06-13T07:15:54.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:54 smithi135 ceph-mon[36276]: pgmap v971: 129 pgs: 129 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 667 op/s 2024-06-13T07:15:55.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 67907 26.11 MB/sec execute 8 sec latency 57.494 ms 2024-06-13T07:15:55.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:55 smithi026 ceph-mon[32041]: pgmap v972: 129 pgs: 129 active+clean; 487 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-06-13T07:15:55.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:55 smithi135 ceph-mon[36276]: pgmap v972: 129 pgs: 129 active+clean; 487 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-06-13T07:15:55.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:55 smithi136 ceph-mon[37229]: pgmap v972: 129 pgs: 129 active+clean; 487 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-06-13T07:15:56.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 68719 28.48 MB/sec execute 9 sec latency 58.936 ms 2024-06-13T07:15:57.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 69266 28.87 MB/sec execute 10 sec latency 301.310 ms 2024-06-13T07:15:58.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 69914 29.59 MB/sec execute 11 sec latency 66.580 ms 2024-06-13T07:15:58.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:57 smithi026 ceph-mon[32041]: pgmap v973: 129 pgs: 129 active+clean; 470 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 848 op/s 2024-06-13T07:15:58.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:15:57 smithi135 ceph-mon[36276]: pgmap v973: 129 pgs: 129 active+clean; 470 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 848 op/s 2024-06-13T07:15:58.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:15:57 smithi136 ceph-mon[37229]: pgmap v973: 129 pgs: 129 active+clean; 470 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 848 op/s 2024-06-13T07:15:59.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 70648 30.41 MB/sec execute 12 sec latency 71.685 ms 2024-06-13T07:16:00.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 71213 29.93 MB/sec execute 13 sec latency 57.037 ms 2024-06-13T07:16:00.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:15:59 smithi026 ceph-mon[32041]: pgmap v974: 129 pgs: 129 active+clean; 476 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-06-13T07:16:00.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:00 smithi135 ceph-mon[36276]: pgmap v974: 129 pgs: 129 active+clean; 476 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-06-13T07:16:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:00 smithi136 ceph-mon[37229]: pgmap v974: 129 pgs: 129 active+clean; 476 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-06-13T07:16:01.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 72033 31.03 MB/sec execute 14 sec latency 46.542 ms 2024-06-13T07:16:02.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 72751 31.90 MB/sec execute 15 sec latency 34.600 ms 2024-06-13T07:16:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.195+0000 7f32b0200640 1 -- 172.21.15.26:0/1873305135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a80ff770 msgr2=0x7f32a80ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.195+0000 7f32b0200640 1 --2- 172.21.15.26:0/1873305135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a80ff770 0x7f32a80ffb90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ca0 tx=0x7f329c02a8b0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.196+0000 7f32b0200640 1 -- 172.21.15.26:0/1873305135 shutdown_connections 2024-06-13T07:16:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.196+0000 7f32b0200640 1 --2- 172.21.15.26:0/1873305135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32a8107660 0x7f32a8109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.196+0000 7f32b0200640 1 --2- 172.21.15.26:0/1873305135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32a81000d0 0x7f32a81070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.196+0000 7f32b0200640 1 --2- 172.21.15.26:0/1873305135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a80ff770 0x7f32a80ffb90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.196+0000 7f32b0200640 1 -- 172.21.15.26:0/1873305135 >> 172.21.15.26:0/1873305135 conn(0x7f32a80fb1c0 msgr2=0x7f32a80fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.196+0000 7f32b0200640 1 -- 172.21.15.26:0/1873305135 shutdown_connections 2024-06-13T07:16:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.197+0000 7f32b0200640 1 -- 172.21.15.26:0/1873305135 wait complete. 2024-06-13T07:16:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.199+0000 7f32b0200640 1 Processor -- start 2024-06-13T07:16:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.199+0000 7f32b0200640 1 -- start start 2024-06-13T07:16:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.199+0000 7f32b0200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32a80ff770 0x7f32a8199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.199+0000 7f32b0200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32a81000d0 0x7f32a819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.199+0000 7f32b0200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a8107660 0x7f32a81a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32b0200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810d750 con 0x7f32a80ff770 2024-06-13T07:16:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32b0200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810d5d0 con 0x7f32a8107660 2024-06-13T07:16:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32b0200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810d8d0 con 0x7f32a81000d0 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ace00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32a81000d0 0x7f32a819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ae200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a8107660 0x7f32a81a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ace00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32a81000d0 0x7f32a819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45448/0 (socket says 172.21.15.26:45448) 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ace00640 1 -- 172.21.15.26:0/1743859723 learned_addr learned my addr 172.21.15.26:0/1743859723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ad800640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32a80ff770 0x7f32a8199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ae200640 1 -- 172.21.15.26:0/1743859723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32a81000d0 msgr2=0x7f32a819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ae200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32a81000d0 0x7f32a819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ae200640 1 -- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32a80ff770 msgr2=0x7f32a8199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ae200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32a80ff770 0x7f32a8199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.200+0000 7f32ae200640 1 -- 172.21.15.26:0/1743859723 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a81a1b10 con 0x7f32a8107660 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.201+0000 7f32ae200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a8107660 0x7f32a81a14a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f32a000e970 tx=0x7f32a000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.201+0000 7f32a6000640 1 -- 172.21.15.26:0/1743859723 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a000ccc0 con 0x7f32a8107660 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.201+0000 7f32a6000640 1 -- 172.21.15.26:0/1743859723 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a0005500 con 0x7f32a8107660 2024-06-13T07:16:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.201+0000 7f32a6000640 1 -- 172.21.15.26:0/1743859723 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0010430 con 0x7f32a8107660 2024-06-13T07:16:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.201+0000 7f32b0200640 1 -- 172.21.15.26:0/1743859723 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a81a1e00 con 0x7f32a8107660 2024-06-13T07:16:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.201+0000 7f32b0200640 1 -- 172.21.15.26:0/1743859723 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81a2390 con 0x7f32a8107660 2024-06-13T07:16:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.205+0000 7f32b2f9e740 1 Processor -- start 2024-06-13T07:16:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32b2f9e740 1 -- start start 2024-06-13T07:16:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc7c00 0x564158dc8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc85f0 0x564158dcd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 0x564158dcfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564158d2b420 con 0x564158dc85f0 2024-06-13T07:16:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564158cd2c50 con 0x564158dcd760 2024-06-13T07:16:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564158d273e0 con 0x564158dc7c00 2024-06-13T07:16:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32ace00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc7c00 0x564158dc8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32ace00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc7c00 0x564158dc8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45462/0 (socket says 172.21.15.26:45462) 2024-06-13T07:16:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.206+0000 7f32ace00640 1 -- 172.21.15.26:0/2091022644 learned_addr learned my addr 172.21.15.26:0/2091022644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.207+0000 7f32ace00640 1 --2- 172.21.15.26:0/2091022644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 0x564158dcfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.207+0000 7f32ace00640 1 -- 172.21.15.26:0/2091022644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 msgr2=0x564158dcfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.207+0000 7f32ace00640 1 --2- 172.21.15.26:0/2091022644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 0x564158dcfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.207+0000 7f32ace00640 1 -- 172.21.15.26:0/2091022644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc85f0 msgr2=0x564158dcd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.207+0000 7f32ace00640 1 --2- 172.21.15.26:0/2091022644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc85f0 0x564158dcd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.207+0000 7f32ace00640 1 -- 172.21.15.26:0/2091022644 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564158a3f210 con 0x564158dc7c00 2024-06-13T07:16:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.207+0000 7f32ace00640 1 --2- 172.21.15.26:0/2091022644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc7c00 0x564158dc8020 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3294006aa0 tx=0x7f3294009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326347905e525841 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.208+0000 7f329aa00640 1 -- 172.21.15.26:0/2091022644 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940157c0 con 0x564158dc7c00 2024-06-13T07:16:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.208+0000 7f329aa00640 1 -- 172.21.15.26:0/2091022644 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329401d070 con 0x564158dc7c00 2024-06-13T07:16:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.208+0000 7f329aa00640 1 -- 172.21.15.26:0/2091022644 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294018be0 con 0x564158dc7c00 2024-06-13T07:16:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.209+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2091022644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc7c00 msgr2=0x564158dc8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.209+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2091022644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc7c00 0x564158dc8020 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3294006aa0 tx=0x7f3294009590 comp rx=0 tx=0).stop 2024-06-13T07:16:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.210+0000 7f32a6000640 1 -- 172.21.15.26:0/1743859723 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32a0005000 con 0x7f32a8107660 2024-06-13T07:16:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.210+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2091022644 shutdown_connections 2024-06-13T07:16:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.210+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2091022644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 0x564158dcfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.210+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2091022644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc85f0 0x564158dcd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.210+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2091022644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc7c00 0x564158dc8020 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.210+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2091022644 >> 172.21.15.26:0/2091022644 conn(0x564158d8ffb0 msgr2=0x564158dc3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.210+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2091022644 shutdown_connections 2024-06-13T07:16:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.210+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2091022644 wait complete. 2024-06-13T07:16:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.211+0000 7f32b2f9e740 1 -- start start 2024-06-13T07:16:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.211+0000 7f32b2f9e740 1 Processor -- start 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158de2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 0x564158dea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564158de16c0 con 0x564158dc7c00 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564158ce60b0 con 0x564158dcd760 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564158de1840 con 0x564158dc85f0 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32b2f9e740 10 client.0 _ll_register_callbacks cb 0x564158cd1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32ad800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32ad800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45470/0 (socket says 172.21.15.26:45470) 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32ad800640 1 -- 172.21.15.26:0/125103284 learned_addr learned my addr 172.21.15.26:0/125103284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.212+0000 7f32ace00640 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158de2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f32ad800640 1 -- 172.21.15.26:0/125103284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 msgr2=0x564158dea430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:16:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f32ad800640 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 0x564158dea430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f32ad800640 1 -- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 msgr2=0x564158de2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f32ad800640 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158de2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f32ad800640 1 -- 172.21.15.26:0/125103284 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564158b0b620 con 0x564158dc85f0 2024-06-13T07:16:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f32ace00640 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158de2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:16:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f32ad800640 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de33f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f329c030860 tx=0x7f329c030c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f2554cdbb62611 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f32a6000640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32900a2bc0 0x7f32900a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.213+0000 7f328f400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:16:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.214+0000 7f32a6000640 1 -- 172.21.15.26:0/1743859723 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f32a00cf660 con 0x7f32a8107660 2024-06-13T07:16:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.214+0000 7f32ace00640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32900a2bc0 0x7f32900a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.214+0000 7f328f400640 1 -- 172.21.15.26:0/125103284 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c003b70 con 0x564158dc85f0 2024-06-13T07:16:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.215+0000 7f32ace00640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32900a2bc0 0x7f32900a5080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3294006aa0 tx=0x7f3294008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.216+0000 7f328f400640 1 -- 172.21.15.26:0/125103284 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f329c003d10 con 0x564158dc85f0 2024-06-13T07:16:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.216+0000 7f32b2f9e740 1 -- 172.21.15.26:0/125103284 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329401d070 con 0x564158dc85f0 2024-06-13T07:16:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f328f400640 1 -- 172.21.15.26:0/125103284 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f329c004040 con 0x564158dc85f0 2024-06-13T07:16:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f32b2f9e740 10 client.26695 fetch_fsmap learned FSMap version 42 2024-06-13T07:16:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f32b2f9e740 1 -- 172.21.15.26:0/125103284 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564158df6020 con 0x564158dc85f0 2024-06-13T07:16:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f328f400640 1 -- 172.21.15.26:0/125103284 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329401d070 con 0x564158dc85f0 2024-06-13T07:16:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f328f400640 1 -- 172.21.15.26:0/125103284 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f329c043070 con 0x564158dc85f0 2024-06-13T07:16:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f32b2f9e740 10 client.26695 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:16:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f32b2f9e740 10 client.26695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:16:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f32b2f9e740 20 client.26695 populate_metadata read hostname 'smithi026' 2024-06-13T07:16:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564158df7710 0x564158df9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f32b2f9e740 4 client.26695 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:16:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.217+0000 7f32b2f9e740 1 -- 172.21.15.26:0/125103284 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564158d38b50 con 0x564158df7710 2024-06-13T07:16:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.218+0000 7f32ace00640 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564158df7710 0x564158df9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.219+0000 7f32ace00640 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564158df7710 0x564158df9b30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8952aff981f5e306 server_cookie=c49f95887fa96d22 in_seq=0 out_seq=0 2024-06-13T07:16:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.219+0000 7f328f400640 10 client.26695 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.221+0000 7f328f400640 1 -- 172.21.15.26:0/125103284 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564158d38b50 con 0x564158df7710 2024-06-13T07:16:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.221+0000 7f328f400640 10 client.26695 handle_command_reply: tid=0 2024-06-13T07:16:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.221+0000 7f32b2f9e740 1 client.26695 shutdown 2024-06-13T07:16:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f32b2f9e740 1 -- 172.21.15.26:0/125103284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 msgr2=0x564158de33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de33f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f329c030860 tx=0x7f329c030c60 comp rx=0 tx=0).stop 2024-06-13T07:16:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f32b2f9e740 1 -- 172.21.15.26:0/125103284 shutdown_connections 2024-06-13T07:16:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564158df7710 0x564158df9b30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f328f400640 0 client.26695 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dcd760 0x564158dea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de33f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/125103284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158de2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.222+0000 7f32b2f9e740 1 -- 172.21.15.26:0/125103284 >> 172.21.15.26:0/125103284 conn(0x564158d8ffb0 msgr2=0x564158dc2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.223+0000 7f32b2f9e740 1 -- 172.21.15.26:0/125103284 shutdown_connections 2024-06-13T07:16:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.223+0000 7f32b2f9e740 1 -- 172.21.15.26:0/125103284 wait complete. 2024-06-13T07:16:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.223+0000 7f32b2f9e740 20 client.26695 trim_cache size 0 max 16384 2024-06-13T07:16:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.248+0000 7f32b2f9e740 1 Processor -- start 2024-06-13T07:16:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.248+0000 7f32b2f9e740 1 -- start start 2024-06-13T07:16:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.248+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158dc3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.248+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc85f0 0x564158dea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.248+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dcd760 0x564158dc6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.248+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81a2840 con 0x564158dc7c00 2024-06-13T07:16:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.248+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8071600 con 0x564158dc85f0 2024-06-13T07:16:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.248+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32a806f450 con 0x564158dcd760 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ace00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc85f0 0x564158dea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ace00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc85f0 0x564158dea3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:34300/0 (socket says 172.21.15.26:34300) 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ace00640 1 -- 172.21.15.26:0/3561437411 learned_addr learned my addr 172.21.15.26:0/3561437411 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ae200640 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dcd760 0x564158dc6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ad800640 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158dc3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ae200640 1 -- 172.21.15.26:0/3561437411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc85f0 msgr2=0x564158dea3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ae200640 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc85f0 0x564158dea3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ae200640 1 -- 172.21.15.26:0/3561437411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 msgr2=0x564158dc3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ae200640 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158dc3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ae200640 1 -- 172.21.15.26:0/3561437411 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564158df9ea0 con 0x564158dcd760 2024-06-13T07:16:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ace00640 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc85f0 0x564158dea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:16:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ad800640 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158dc3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:16:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.249+0000 7f32ae200640 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dcd760 0x564158dc6630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f32a000cfd0 tx=0x7f32a009b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2cb7654ec60ca39 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.250+0000 7f329a000640 1 -- 172.21.15.26:0/3561437411 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a009bad0 con 0x564158dcd760 2024-06-13T07:16:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.250+0000 7f329a000640 1 -- 172.21.15.26:0/3561437411 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a009fe80 con 0x564158dcd760 2024-06-13T07:16:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.250+0000 7f329a000640 1 -- 172.21.15.26:0/3561437411 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0098140 con 0x564158dcd760 2024-06-13T07:16:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.250+0000 7f32b2f9e740 1 -- 172.21.15.26:0/3561437411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dcd760 msgr2=0x564158dc6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.250+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dcd760 0x564158dc6630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f32a000cfd0 tx=0x7f32a009b610 comp rx=0 tx=0).stop 2024-06-13T07:16:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 -- 172.21.15.26:0/3561437411 shutdown_connections 2024-06-13T07:16:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc85f0 0x564158dea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dcd760 0x564158dc6630 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/3561437411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dc7c00 0x564158dc3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 -- 172.21.15.26:0/3561437411 >> 172.21.15.26:0/3561437411 conn(0x564158d8ffb0 msgr2=0x564158dc3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 -- 172.21.15.26:0/3561437411 shutdown_connections 2024-06-13T07:16:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 -- 172.21.15.26:0/3561437411 wait complete. 2024-06-13T07:16:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 -- start start 2024-06-13T07:16:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.251+0000 7f32b2f9e740 1 Processor -- start 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc7c00 0x564158de79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32b2f9e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dcd760 0x564158de8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32a806fac0 con 0x564158dcd760 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8070f30 con 0x564158dc7c00 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32b2f9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8070190 con 0x564158dc85f0 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32b2f9e740 10 client.0 _ll_register_callbacks cb 0x7f329c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32ad800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc7c00 0x564158de79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32ad800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc7c00 0x564158de79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:34308/0 (socket says 172.21.15.26:34308) 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32ad800640 1 -- 172.21.15.26:0/2323620561 learned_addr learned my addr 172.21.15.26:0/2323620561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32ace00640 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.252+0000 7f32ae200640 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dcd760 0x564158de8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f32ad800640 1 -- 172.21.15.26:0/2323620561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 msgr2=0x564158de7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f32ad800640 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f32ad800640 1 -- 172.21.15.26:0/2323620561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dcd760 msgr2=0x564158de8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f32ad800640 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dcd760 0x564158de8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f32ad800640 1 -- 172.21.15.26:0/2323620561 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564158de9440 con 0x564158dc7c00 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f32ace00640 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f32ad800640 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc7c00 0x564158de79c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f329c000f80 tx=0x7f329c045670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4dffc2a46cc1f0b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f328f400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f328f400640 1 -- 172.21.15.26:0/2323620561 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0098b0 con 0x564158dc7c00 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.253+0000 7f328f400640 1 -- 172.21.15.26:0/2323620561 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f329c003970 con 0x564158dc7c00 2024-06-13T07:16:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.254+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2323620561 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a009fe80 con 0x564158dc7c00 2024-06-13T07:16:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.254+0000 7f328f400640 1 -- 172.21.15.26:0/2323620561 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f329c003b70 con 0x564158dc7c00 2024-06-13T07:16:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.254+0000 7f32b2f9e740 10 client.26672 fetch_fsmap learned FSMap version 42 2024-06-13T07:16:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.254+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2323620561 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564158e62410 con 0x564158dc7c00 2024-06-13T07:16:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.254+0000 7f328f400640 1 -- 172.21.15.26:0/2323620561 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a009fe80 con 0x564158dc7c00 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.255+0000 7f328f400640 1 -- 172.21.15.26:0/2323620561 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f329c043040 con 0x564158dc7c00 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.255+0000 7f32b2f9e740 10 client.26672 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.255+0000 7f32b2f9e740 10 client.26672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.255+0000 7f32b2f9e740 20 client.26672 populate_metadata read hostname 'smithi026' 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.255+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564158e62ca0 0x564158e650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.255+0000 7f32b2f9e740 4 client.26672 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.255+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2323620561 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564158d32ea0 con 0x564158e62ca0 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.256+0000 7f32ace00640 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564158e62ca0 0x564158e650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.257+0000 7f32ace00640 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564158e62ca0 0x564158e650c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c762f61dc68f9a09 server_cookie=8bc8bec5a1a89084 in_seq=0 out_seq=0 2024-06-13T07:16:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.257+0000 7f328f400640 10 client.26672 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:02.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:02 smithi135 ceph-mon[36276]: pgmap v975: 129 pgs: 129 active+clean; 481 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.8 MiB/s wr, 718 op/s 2024-06-13T07:16:02.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:02 smithi136 ceph-mon[37229]: pgmap v975: 129 pgs: 129 active+clean; 481 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.8 MiB/s wr, 718 op/s 2024-06-13T07:16:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:02 smithi026 ceph-mon[32041]: pgmap v975: 129 pgs: 129 active+clean; 481 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.8 MiB/s wr, 718 op/s 2024-06-13T07:16:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.715+0000 7f328f400640 1 -- 172.21.15.26:0/2323620561 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564158d32ea0 con 0x564158e62ca0 2024-06-13T07:16:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.715+0000 7f328f400640 10 client.26672 handle_command_reply: tid=0 2024-06-13T07:16:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.715+0000 7f32b2f9e740 1 client.26672 shutdown 2024-06-13T07:16:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2323620561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc7c00 msgr2=0x564158de79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc7c00 0x564158de79c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f329c000f80 tx=0x7f329c045670 comp rx=0 tx=0).stop 2024-06-13T07:16:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2323620561 shutdown_connections 2024-06-13T07:16:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564158e62ca0 0x564158e650c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564158dcd760 0x564158de8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564158dc85f0 0x564158de7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f328f400640 0 client.26672 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 --2- 172.21.15.26:0/2323620561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564158dc7c00 0x564158de79c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2323620561 >> 172.21.15.26:0/2323620561 conn(0x564158d8ffb0 msgr2=0x564158dc6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2323620561 shutdown_connections 2024-06-13T07:16:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.716+0000 7f32b2f9e740 1 -- 172.21.15.26:0/2323620561 wait complete. 2024-06-13T07:16:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.717+0000 7f32b2f9e740 20 client.26672 trim_cache size 0 max 16384 2024-06-13T07:16:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:16:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:16:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:16:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.717+0000 7f32b0200640 1 -- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32900a2bc0 msgr2=0x7f32900a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32900a2bc0 0x7f32900a5080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3294006aa0 tx=0x7f3294008040 comp rx=0 tx=0).stop 2024-06-13T07:16:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 -- 172.21.15.26:0/1743859723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a8107660 msgr2=0x7f32a81a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a8107660 0x7f32a81a14a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f32a000e970 tx=0x7f32a000ee30 comp rx=0 tx=0).stop 2024-06-13T07:16:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 -- 172.21.15.26:0/1743859723 shutdown_connections 2024-06-13T07:16:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f32900a2bc0 0x7f32900a5080 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32a8107660 0x7f32a81a14a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32a81000d0 0x7f32a819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 --2- 172.21.15.26:0/1743859723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32a80ff770 0x7f32a8199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 -- 172.21.15.26:0/1743859723 >> 172.21.15.26:0/1743859723 conn(0x7f32a80fb1c0 msgr2=0x7f32a80fb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 -- 172.21.15.26:0/1743859723 shutdown_connections 2024-06-13T07:16:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:02.718+0000 7f32b0200640 1 -- 172.21.15.26:0/1743859723 wait complete. 2024-06-13T07:16:03.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 73166 31.02 MB/sec execute 16 sec latency 268.535 ms 2024-06-13T07:16:04.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 73786 31.03 MB/sec execute 17 sec latency 207.192 ms 2024-06-13T07:16:04.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:04 smithi135 ceph-mon[36276]: pgmap v976: 129 pgs: 129 active+clean; 488 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 716 op/s 2024-06-13T07:16:04.304 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:04 smithi026 ceph-mon[32041]: pgmap v976: 129 pgs: 129 active+clean; 488 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 716 op/s 2024-06-13T07:16:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:04 smithi136 ceph-mon[37229]: pgmap v976: 129 pgs: 129 active+clean; 488 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 716 op/s 2024-06-13T07:16:05.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 74428 31.38 MB/sec execute 18 sec latency 79.700 ms 2024-06-13T07:16:06.156 INFO:tasks.workunit.client.0.smithi026.stdout: 10 75144 32.28 MB/sec execute 19 sec latency 39.905 ms 2024-06-13T07:16:06.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:06 smithi135 ceph-mon[36276]: pgmap v977: 129 pgs: 129 active+clean; 486 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 739 op/s 2024-06-13T07:16:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:06 smithi136 ceph-mon[37229]: pgmap v977: 129 pgs: 129 active+clean; 486 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 739 op/s 2024-06-13T07:16:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:06 smithi026 ceph-mon[32041]: pgmap v977: 129 pgs: 129 active+clean; 486 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 739 op/s 2024-06-13T07:16:07.154 INFO:tasks.workunit.client.0.smithi026.stdout: 10 75346 31.18 MB/sec execute 20 sec latency 337.846 ms 2024-06-13T07:16:08.155 INFO:tasks.workunit.client.0.smithi026.stdout: 10 75856 30.40 MB/sec execute 21 sec latency 90.189 ms 2024-06-13T07:16:08.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:08 smithi135 ceph-mon[36276]: pgmap v978: 129 pgs: 129 active+clean; 497 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.4 MiB/s wr, 655 op/s 2024-06-13T07:16:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:08 smithi136 ceph-mon[37229]: pgmap v978: 129 pgs: 129 active+clean; 497 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.4 MiB/s wr, 655 op/s 2024-06-13T07:16:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:08 smithi026 ceph-mon[32041]: pgmap v978: 129 pgs: 129 active+clean; 497 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.4 MiB/s wr, 655 op/s 2024-06-13T07:16:09.155 INFO:tasks.workunit.client.0.smithi026.stdout: 10 76633 31.34 MB/sec execute 22 sec latency 89.281 ms 2024-06-13T07:16:10.155 INFO:tasks.workunit.client.0.smithi026.stdout: 10 77428 31.55 MB/sec execute 23 sec latency 68.496 ms 2024-06-13T07:16:10.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:10 smithi135 ceph-mon[36276]: pgmap v979: 129 pgs: 129 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-06-13T07:16:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:10 smithi136 ceph-mon[37229]: pgmap v979: 129 pgs: 129 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-06-13T07:16:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:10 smithi026 ceph-mon[32041]: pgmap v979: 129 pgs: 129 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-06-13T07:16:11.155 INFO:tasks.workunit.client.0.smithi026.stdout: 10 78184 32.29 MB/sec execute 24 sec latency 63.531 ms 2024-06-13T07:16:12.155 INFO:tasks.workunit.client.0.smithi026.stdout: 10 78327 31.21 MB/sec execute 25 sec latency 659.913 ms 2024-06-13T07:16:12.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:12 smithi135 ceph-mon[36276]: pgmap v980: 129 pgs: 129 active+clean; 503 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.1 MiB/s wr, 592 op/s 2024-06-13T07:16:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:12 smithi136 ceph-mon[37229]: pgmap v980: 129 pgs: 129 active+clean; 503 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.1 MiB/s wr, 592 op/s 2024-06-13T07:16:12.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:12 smithi026 ceph-mon[32041]: pgmap v980: 129 pgs: 129 active+clean; 503 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.1 MiB/s wr, 592 op/s 2024-06-13T07:16:13.155 INFO:tasks.workunit.client.0.smithi026.stdout: 10 78582 30.67 MB/sec execute 26 sec latency 750.792 ms 2024-06-13T07:16:14.156 INFO:tasks.workunit.client.0.smithi026.stdout: 10 78890 30.03 MB/sec execute 27 sec latency 87.971 ms 2024-06-13T07:16:14.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:14 smithi135 ceph-mon[36276]: pgmap v981: 129 pgs: 129 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.7 MiB/s wr, 657 op/s 2024-06-13T07:16:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:14 smithi136 ceph-mon[37229]: pgmap v981: 129 pgs: 129 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.7 MiB/s wr, 657 op/s 2024-06-13T07:16:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:14 smithi026 ceph-mon[32041]: pgmap v981: 129 pgs: 129 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.7 MiB/s wr, 657 op/s 2024-06-13T07:16:15.156 INFO:tasks.workunit.client.0.smithi026.stdout: 10 79356 29.68 MB/sec execute 28 sec latency 60.364 ms 2024-06-13T07:16:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.929+0000 7fdfee000640 1 -- 172.21.15.26:0/3255377535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 msgr2=0x7fdfe8073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.929+0000 7fdfee000640 1 --2- 172.21.15.26:0/3255377535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 0x7fdfe8073dd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009900 tx=0x7fdfd002a4a0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.929+0000 7fdfee000640 1 -- 172.21.15.26:0/3255377535 shutdown_connections 2024-06-13T07:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.929+0000 7fdfee000640 1 --2- 172.21.15.26:0/3255377535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe81098b0 0x7fdfe810bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.929+0000 7fdfee000640 1 --2- 172.21.15.26:0/3255377535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe80743a0 0x7fdfe8074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.929+0000 7fdfee000640 1 --2- 172.21.15.26:0/3255377535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 0x7fdfe8073dd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.929+0000 7fdfee000640 1 -- 172.21.15.26:0/3255377535 >> 172.21.15.26:0/3255377535 conn(0x7fdfe80fcf90 msgr2=0x7fdfe80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.930+0000 7fdfee000640 1 -- 172.21.15.26:0/3255377535 shutdown_connections 2024-06-13T07:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.930+0000 7fdfee000640 1 -- 172.21.15.26:0/3255377535 wait complete. 2024-06-13T07:16:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.931+0000 7fdfee000640 1 Processor -- start 2024-06-13T07:16:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.931+0000 7fdfee000640 1 -- start start 2024-06-13T07:16:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfee000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe80743a0 0x7fdfe819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfee000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 0x7fdfe819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfee000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe81098b0 0x7fdfe81a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfee000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810f8e0 con 0x7fdfe80743a0 2024-06-13T07:16:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfee000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810f760 con 0x7fdfe8075980 2024-06-13T07:16:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfee000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810fa60 con 0x7fdfe81098b0 2024-06-13T07:16:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfe7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe81098b0 0x7fdfe81a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfe7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe81098b0 0x7fdfe81a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59558/0 (socket says 172.21.15.26:59558) 2024-06-13T07:16:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.932+0000 7fdfe7e00640 1 -- 172.21.15.26:0/285366634 learned_addr learned my addr 172.21.15.26:0/285366634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.933+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 0x7fdfe819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.933+0000 7fdfe7e00640 1 -- 172.21.15.26:0/285366634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 msgr2=0x7fdfe819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.933+0000 7fdfe7e00640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 0x7fdfe819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.933+0000 7fdfe7e00640 1 -- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe80743a0 msgr2=0x7fdfe819e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.933+0000 7fdfe7e00640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe80743a0 0x7fdfe819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.933+0000 7fdfe7e00640 1 -- 172.21.15.26:0/285366634 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81a5ed0 con 0x7fdfe81098b0 2024-06-13T07:16:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.933+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 0x7fdfe819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:16:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.933+0000 7fdfe7e00640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe81098b0 0x7fdfe81a5860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000eaa0 tx=0x7fdfe000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.934+0000 7fdfdfe00640 1 -- 172.21.15.26:0/285366634 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000cdb0 con 0x7fdfe81098b0 2024-06-13T07:16:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.934+0000 7fdfee000640 1 -- 172.21.15.26:0/285366634 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe81a61c0 con 0x7fdfe81098b0 2024-06-13T07:16:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.934+0000 7fdfdfe00640 1 -- 172.21.15.26:0/285366634 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe0005020 con 0x7fdfe81098b0 2024-06-13T07:16:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.934+0000 7fdfdfe00640 1 -- 172.21.15.26:0/285366634 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000cdb0 con 0x7fdfe81098b0 2024-06-13T07:16:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.934+0000 7fdfee000640 1 -- 172.21.15.26:0/285366634 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe81a6710 con 0x7fdfe81098b0 2024-06-13T07:16:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.942+0000 7fdfdfe00640 1 -- 172.21.15.26:0/285366634 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfe00051d0 con 0x7fdfe81098b0 2024-06-13T07:16:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.944+0000 7fdfdfe00640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdfcc0a2f40 0x7fdfcc0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.945+0000 7fdff0d7d740 1 Processor -- start 2024-06-13T07:16:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.945+0000 7fdfdfe00640 1 -- 172.21.15.26:0/285366634 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdfe00cf330 con 0x7fdfe81098b0 2024-06-13T07:16:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.945+0000 7fdfe7400640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdfcc0a2f40 0x7fdfcc0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.946+0000 7fdff0d7d740 1 -- start start 2024-06-13T07:16:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.946+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a556a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.946+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a556f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.946+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a5571bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.946+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d0a54cd420 con 0x55d0a5569c00 2024-06-13T07:16:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.946+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0a5474c50 con 0x55d0a556a5f0 2024-06-13T07:16:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.946+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d0a54c93e0 con 0x55d0a556f760 2024-06-13T07:16:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe6a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a556f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a5571bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe6a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a556f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:57186/0 (socket says 172.21.15.26:57186) 2024-06-13T07:16:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe6a00640 1 -- 172.21.15.26:0/983034446 learned_addr learned my addr 172.21.15.26:0/983034446 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe7400640 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a556a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe6a00640 1 -- 172.21.15.26:0/983034446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 msgr2=0x55d0a5571bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a5571bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe6a00640 1 -- 172.21.15.26:0/983034446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 msgr2=0x55d0a556a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a556a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.947+0000 7fdfe6a00640 1 -- 172.21.15.26:0/983034446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a51e1210 con 0x55d0a556a5f0 2024-06-13T07:16:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.948+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a556f0f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009c00 tx=0x7fdfd8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6086326e929acab2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.948+0000 7fdfe7400640 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a556a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:16:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.949+0000 7fdfd7e00640 1 -- 172.21.15.26:0/983034446 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8004030 con 0x55d0a556a5f0 2024-06-13T07:16:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.949+0000 7fdfd7e00640 1 -- 172.21.15.26:0/983034446 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd80041d0 con 0x55d0a556a5f0 2024-06-13T07:16:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.949+0000 7fdfd7e00640 1 -- 172.21.15.26:0/983034446 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8004030 con 0x55d0a556a5f0 2024-06-13T07:16:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdff0d7d740 1 -- 172.21.15.26:0/983034446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 msgr2=0x55d0a556f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a556f0f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009c00 tx=0x7fdfd8010740 comp rx=0 tx=0).stop 2024-06-13T07:16:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdff0d7d740 1 -- 172.21.15.26:0/983034446 shutdown_connections 2024-06-13T07:16:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a556f0f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a5571bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/983034446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a556a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdff0d7d740 1 -- 172.21.15.26:0/983034446 >> 172.21.15.26:0/983034446 conn(0x55d0a5531fb0 msgr2=0x55d0a5565680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdfe7400640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdfcc0a2f40 0x7fdfcc0a5400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009230 tx=0x7fdfd0005bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.951+0000 7fdff0d7d740 1 -- 172.21.15.26:0/983034446 shutdown_connections 2024-06-13T07:16:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.952+0000 7fdff0d7d740 1 -- 172.21.15.26:0/983034446 wait complete. 2024-06-13T07:16:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.953+0000 7fdff0d7d740 1 -- start start 2024-06-13T07:16:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.954+0000 7fdff0d7d740 1 Processor -- start 2024-06-13T07:16:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.954+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5584f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.954+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a5585460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.954+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a558c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.955+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d0a5572710 con 0x55d0a5569c00 2024-06-13T07:16:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.955+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0a54880b0 con 0x55d0a556a5f0 2024-06-13T07:16:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.955+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d0a5572890 con 0x55d0a556f760 2024-06-13T07:16:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.955+0000 7fdff0d7d740 10 client.0 _ll_register_callbacks cb 0x55d0a5473f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.955+0000 7fdfe7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a558c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.955+0000 7fdfe7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a558c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59574/0 (socket says 172.21.15.26:59574) 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.955+0000 7fdfe7e00640 1 -- 172.21.15.26:0/3838488188 learned_addr learned my addr 172.21.15.26:0/3838488188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.955+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a5585460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.956+0000 7fdfe7400640 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5584f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.956+0000 7fdfe7e00640 1 -- 172.21.15.26:0/3838488188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 msgr2=0x55d0a5585460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.956+0000 7fdfe7e00640 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a5585460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.956+0000 7fdfe7e00640 1 -- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 msgr2=0x55d0a5584f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.956+0000 7fdfe7e00640 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5584f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.957+0000 7fdfe7e00640 1 -- 172.21.15.26:0/3838488188 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a52ad620 con 0x55d0a556f760 2024-06-13T07:16:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.957+0000 7fdfe7400640 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5584f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:16:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.957+0000 7fdfe7e00640 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a558c4a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0097b50 tx=0x7fdfe009b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34b27b3a90d0913 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.957+0000 7fdfd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:16:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.957+0000 7fdfd4c00640 1 -- 172.21.15.26:0/3838488188 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe001f040 con 0x55d0a556f760 2024-06-13T07:16:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.958+0000 7fdfd4c00640 1 -- 172.21.15.26:0/3838488188 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdfe00ccaf0 con 0x55d0a556f760 2024-06-13T07:16:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.958+0000 7fdff0d7d740 1 -- 172.21.15.26:0/3838488188 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd80041d0 con 0x55d0a556f760 2024-06-13T07:16:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.958+0000 7fdfd4c00640 1 -- 172.21.15.26:0/3838488188 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdfe0005500 con 0x55d0a556f760 2024-06-13T07:16:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.959+0000 7fdff0d7d740 10 client.26713 fetch_fsmap learned FSMap version 43 2024-06-13T07:16:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.959+0000 7fdff0d7d740 1 -- 172.21.15.26:0/3838488188 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d0a558f880 con 0x55d0a556f760 2024-06-13T07:16:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.959+0000 7fdfd4c00640 1 -- 172.21.15.26:0/3838488188 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd80041d0 con 0x55d0a556f760 2024-06-13T07:16:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.959+0000 7fdfd4c00640 1 -- 172.21.15.26:0/3838488188 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fdfe000f040 con 0x55d0a556f760 2024-06-13T07:16:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.959+0000 7fdff0d7d740 10 client.26713 fetch_fsmap finished waiting for FSMap version 43 2024-06-13T07:16:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.959+0000 7fdff0d7d740 10 client.26713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:16:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.960+0000 7fdff0d7d740 20 client.26713 populate_metadata read hostname 'smithi026' 2024-06-13T07:16:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.960+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d0a5591010 0x55d0a5593440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.960+0000 7fdff0d7d740 4 client.26713 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:16:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.960+0000 7fdff0d7d740 1 -- 172.21.15.26:0/3838488188 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0a54dab50 con 0x55d0a5591010 2024-06-13T07:16:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.961+0000 7fdfe7400640 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d0a5591010 0x55d0a5593440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.965+0000 7fdfe7400640 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d0a5591010 0x55d0a5593440 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd322f73fecd7680 server_cookie=c8a6e93569992846 in_seq=0 out_seq=0 2024-06-13T07:16:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.965+0000 7fdfd4c00640 10 client.26713 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.972+0000 7fdfd4c00640 1 -- 172.21.15.26:0/3838488188 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d0a54dab50 con 0x55d0a5591010 2024-06-13T07:16:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.973+0000 7fdfd4c00640 10 client.26713 handle_command_reply: tid=0 2024-06-13T07:16:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.973+0000 7fdff0d7d740 1 client.26713 shutdown 2024-06-13T07:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdff0d7d740 1 -- 172.21.15.26:0/3838488188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 msgr2=0x55d0a558c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a558c4a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0097b50 tx=0x7fdfe009b800 comp rx=0 tx=0).stop 2024-06-13T07:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdff0d7d740 1 -- 172.21.15.26:0/3838488188 shutdown_connections 2024-06-13T07:16:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d0a5591010 0x55d0a5593440 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a5585460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a558c4a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdfd4c00640 0 client.26713 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/3838488188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5584f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.974+0000 7fdff0d7d740 1 -- 172.21.15.26:0/3838488188 >> 172.21.15.26:0/3838488188 conn(0x55d0a5531fb0 msgr2=0x55d0a5564840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.976+0000 7fdff0d7d740 1 -- 172.21.15.26:0/3838488188 shutdown_connections 2024-06-13T07:16:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.977+0000 7fdff0d7d740 1 -- 172.21.15.26:0/3838488188 wait complete. 2024-06-13T07:16:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:15.978+0000 7fdff0d7d740 20 client.26713 trim_cache size 0 max 16384 2024-06-13T07:16:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.007+0000 7fdff0d7d740 1 Processor -- start 2024-06-13T07:16:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.007+0000 7fdff0d7d740 1 -- start start 2024-06-13T07:16:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5565810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a558c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a5568570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81aaee0 con 0x55d0a5569c00 2024-06-13T07:16:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a8750 con 0x55d0a556a5f0 2024-06-13T07:16:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8073790 con 0x55d0a556f760 2024-06-13T07:16:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdfe6a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a558c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdfe7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5565810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdfe6a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a558c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:57210/0 (socket says 172.21.15.26:57210) 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdfe6a00640 1 -- 172.21.15.26:0/2127569388 learned_addr learned my addr 172.21.15.26:0/2127569388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.008+0000 7fdfe7400640 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5565810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40934/0 (socket says 172.21.15.26:40934) 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.009+0000 7fdfe7400640 1 -- 172.21.15.26:0/2127569388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 msgr2=0x55d0a5568570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.009+0000 7fdfe7400640 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a5568570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.009+0000 7fdfe7400640 1 -- 172.21.15.26:0/2127569388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 msgr2=0x55d0a558c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.009+0000 7fdfe7400640 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a558c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.009+0000 7fdfe7400640 1 -- 172.21.15.26:0/2127569388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a55606d0 con 0x55d0a5569c00 2024-06-13T07:16:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.009+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a558c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:16:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.009+0000 7fdfe7400640 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5565810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0005c20 tx=0x7fdfd003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4003ec1c82f6e30 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.010+0000 7fdfd7400640 1 -- 172.21.15.26:0/2127569388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0035040 con 0x55d0a5569c00 2024-06-13T07:16:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.010+0000 7fdfd7400640 1 -- 172.21.15.26:0/2127569388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd00035f0 con 0x55d0a5569c00 2024-06-13T07:16:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.010+0000 7fdfd7400640 1 -- 172.21.15.26:0/2127569388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd00566c0 con 0x55d0a5569c00 2024-06-13T07:16:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.011+0000 7fdff0d7d740 1 -- 172.21.15.26:0/2127569388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 msgr2=0x55d0a5565810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.011+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5565810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0005c20 tx=0x7fdfd003c000 comp rx=0 tx=0).stop 2024-06-13T07:16:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.011+0000 7fdff0d7d740 1 -- 172.21.15.26:0/2127569388 shutdown_connections 2024-06-13T07:16:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.011+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556a5f0 0x55d0a558c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.011+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556f760 0x55d0a5568570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.011+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/2127569388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a5565810 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.011+0000 7fdff0d7d740 1 -- 172.21.15.26:0/2127569388 >> 172.21.15.26:0/2127569388 conn(0x55d0a5531fb0 msgr2=0x55d0a5565100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.011+0000 7fdff0d7d740 1 -- 172.21.15.26:0/2127569388 shutdown_connections 2024-06-13T07:16:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.012+0000 7fdff0d7d740 1 -- 172.21.15.26:0/2127569388 wait complete. 2024-06-13T07:16:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.012+0000 7fdff0d7d740 1 -- start start 2024-06-13T07:16:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.012+0000 7fdff0d7d740 1 Processor -- start 2024-06-13T07:16:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.014+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a55899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.014+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556a5f0 0x55d0a5589ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.014+0000 7fdff0d7d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556f760 0x55d0a558aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.014+0000 7fdfe6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556a5f0 0x55d0a5589ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.014+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a6c10 con 0x55d0a5569c00 2024-06-13T07:16:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.014+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a8080 con 0x55d0a556f760 2024-06-13T07:16:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.014+0000 7fdff0d7d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a72e0 con 0x55d0a556a5f0 2024-06-13T07:16:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.014+0000 7fdff0d7d740 10 client.0 _ll_register_callbacks cb 0x7fdfe009b0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:16:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.015+0000 7fdfe6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556a5f0 0x55d0a5589ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59580/0 (socket says 172.21.15.26:59580) 2024-06-13T07:16:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.015+0000 7fdfe7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556f760 0x55d0a558aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.015+0000 7fdfe6a00640 1 -- 172.21.15.26:0/1112587473 learned_addr learned my addr 172.21.15.26:0/1112587473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.015+0000 7fdfe7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a55899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.015+0000 7fdfe6a00640 1 -- 172.21.15.26:0/1112587473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556f760 msgr2=0x55d0a558aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.015+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556f760 0x55d0a558aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.015+0000 7fdfe6a00640 1 -- 172.21.15.26:0/1112587473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 msgr2=0x55d0a55899a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.015+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a55899a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.016+0000 7fdfe6a00640 1 -- 172.21.15.26:0/1112587473 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a558b4e0 con 0x55d0a556a5f0 2024-06-13T07:16:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.016+0000 7fdfe7e00640 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556f760 0x55d0a558aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:16:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.016+0000 7fdfe6a00640 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556a5f0 0x55d0a5589ee0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009c00 tx=0x7fdfd8009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f367edb28acc8af server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.016+0000 7fdfd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:16:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.016+0000 7fdfd4c00640 1 -- 172.21.15.26:0/1112587473 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd802c060 con 0x55d0a556a5f0 2024-06-13T07:16:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.017+0000 7fdff0d7d740 1 -- 172.21.15.26:0/1112587473 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd00035f0 con 0x55d0a556a5f0 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.017+0000 7fdfd4c00640 1 -- 172.21.15.26:0/1112587473 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdfd80050b0 con 0x55d0a556a5f0 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.017+0000 7fdfd4c00640 1 -- 172.21.15.26:0/1112587473 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdfd8033050 con 0x55d0a556a5f0 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.017+0000 7fdff0d7d740 10 client.26716 fetch_fsmap learned FSMap version 43 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.017+0000 7fdff0d7d740 1 -- 172.21.15.26:0/1112587473 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d0a55fbd20 con 0x55d0a556a5f0 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.017+0000 7fdfd4c00640 1 -- 172.21.15.26:0/1112587473 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd00035f0 con 0x55d0a556a5f0 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.018+0000 7fdfd4c00640 1 -- 172.21.15.26:0/1112587473 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fdfd800f040 con 0x55d0a556a5f0 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.018+0000 7fdff0d7d740 10 client.26716 fetch_fsmap finished waiting for FSMap version 43 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.018+0000 7fdff0d7d740 10 client.26716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.018+0000 7fdff0d7d740 20 client.26716 populate_metadata read hostname 'smithi026' 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.018+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d0a55fc320 0x55d0a55fe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.018+0000 7fdff0d7d740 4 client.26716 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:16:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.018+0000 7fdff0d7d740 1 -- 172.21.15.26:0/1112587473 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0a54d4ea0 con 0x55d0a55fc320 2024-06-13T07:16:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.019+0000 7fdfe7400640 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d0a55fc320 0x55d0a55fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.020+0000 7fdfe7400640 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d0a55fc320 0x55d0a55fe740 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee13773d47a874f9 server_cookie=fa9d9d1198f37069 in_seq=0 out_seq=0 2024-06-13T07:16:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.020+0000 7fdfd4c00640 10 client.26716 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:16.156 INFO:tasks.workunit.client.0.smithi026.stdout: 10 80091 30.08 MB/sec execute 29 sec latency 74.713 ms 2024-06-13T07:16:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.356+0000 7fdfd4c00640 1 -- 172.21.15.26:0/1112587473 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d0a54d4ea0 con 0x55d0a55fc320 2024-06-13T07:16:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.356+0000 7fdfd4c00640 10 client.26716 handle_command_reply: tid=0 2024-06-13T07:16:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.356+0000 7fdff0d7d740 1 client.26716 shutdown 2024-06-13T07:16:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 -- 172.21.15.26:0/1112587473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556a5f0 msgr2=0x55d0a5589ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556a5f0 0x55d0a5589ee0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009c00 tx=0x7fdfd8009f20 comp rx=0 tx=0).stop 2024-06-13T07:16:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 -- 172.21.15.26:0/1112587473 shutdown_connections 2024-06-13T07:16:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55d0a55fc320 0x55d0a55fe740 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0a556f760 0x55d0a558aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d0a556a5f0 0x55d0a5589ee0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 --2- 172.21.15.26:0/1112587473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d0a5569c00 0x55d0a55899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdfd4c00640 0 client.26716 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 -- 172.21.15.26:0/1112587473 >> 172.21.15.26:0/1112587473 conn(0x55d0a5531fb0 msgr2=0x55d0a5568570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 -- 172.21.15.26:0/1112587473 shutdown_connections 2024-06-13T07:16:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.357+0000 7fdff0d7d740 1 -- 172.21.15.26:0/1112587473 wait complete. 2024-06-13T07:16:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.358+0000 7fdff0d7d740 20 client.26716 trim_cache size 0 max 16384 2024-06-13T07:16:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:16:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:16:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:16:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 -- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdfcc0a2f40 msgr2=0x7fdfcc0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdfcc0a2f40 0x7fdfcc0a5400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009230 tx=0x7fdfd0005bb0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 -- 172.21.15.26:0/285366634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe81098b0 msgr2=0x7fdfe81a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe81098b0 0x7fdfe81a5860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000eaa0 tx=0x7fdfe000ef60 comp rx=0 tx=0).stop 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 -- 172.21.15.26:0/285366634 shutdown_connections 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdfcc0a2f40 0x7fdfcc0a5400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe8075980 0x7fdfe819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe81098b0 0x7fdfe81a5860 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 --2- 172.21.15.26:0/285366634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe80743a0 0x7fdfe819e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.359+0000 7fdfee000640 1 -- 172.21.15.26:0/285366634 >> 172.21.15.26:0/285366634 conn(0x7fdfe80fcf90 msgr2=0x7fdfe8101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.360+0000 7fdfee000640 1 -- 172.21.15.26:0/285366634 shutdown_connections 2024-06-13T07:16:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:16.360+0000 7fdfee000640 1 -- 172.21.15.26:0/285366634 wait complete. 2024-06-13T07:16:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:16 smithi136 ceph-mon[37229]: pgmap v982: 129 pgs: 129 active+clean; 536 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 677 op/s 2024-06-13T07:16:16.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:16 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:16 smithi026 ceph-mon[32041]: pgmap v982: 129 pgs: 129 active+clean; 536 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 677 op/s 2024-06-13T07:16:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:16 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:16 smithi135 ceph-mon[36276]: pgmap v982: 129 pgs: 129 active+clean; 536 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 677 op/s 2024-06-13T07:16:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:16 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:17.157 INFO:tasks.workunit.client.0.smithi026.stdout: 10 80412 29.56 MB/sec execute 30 sec latency 373.552 ms 2024-06-13T07:16:18.157 INFO:tasks.workunit.client.0.smithi026.stdout: 10 81162 30.04 MB/sec execute 31 sec latency 63.591 ms 2024-06-13T07:16:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:18 smithi136 ceph-mon[37229]: pgmap v983: 129 pgs: 129 active+clean; 529 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 699 op/s 2024-06-13T07:16:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:18 smithi026 ceph-mon[32041]: pgmap v983: 129 pgs: 129 active+clean; 529 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 699 op/s 2024-06-13T07:16:18.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:18 smithi135 ceph-mon[36276]: pgmap v983: 129 pgs: 129 active+clean; 529 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 699 op/s 2024-06-13T07:16:19.157 INFO:tasks.workunit.client.0.smithi026.stdout: 10 81817 30.35 MB/sec execute 32 sec latency 88.552 ms 2024-06-13T07:16:20.157 INFO:tasks.workunit.client.0.smithi026.stdout: 10 82395 30.01 MB/sec execute 33 sec latency 79.522 ms 2024-06-13T07:16:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:20 smithi136 ceph-mon[37229]: pgmap v984: 129 pgs: 129 active+clean; 502 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-06-13T07:16:20.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:20 smithi026 ceph-mon[32041]: pgmap v984: 129 pgs: 129 active+clean; 502 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-06-13T07:16:20.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:20 smithi135 ceph-mon[36276]: pgmap v984: 129 pgs: 129 active+clean; 502 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-06-13T07:16:21.157 INFO:tasks.workunit.client.0.smithi026.stdout: 10 83153 30.46 MB/sec execute 34 sec latency 62.796 ms 2024-06-13T07:16:22.158 INFO:tasks.workunit.client.0.smithi026.stdout: 10 83792 30.60 MB/sec execute 35 sec latency 98.062 ms 2024-06-13T07:16:22.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:22 smithi136 ceph-mon[37229]: pgmap v985: 129 pgs: 129 active+clean; 505 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 753 op/s 2024-06-13T07:16:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:22 smithi026 ceph-mon[32041]: pgmap v985: 129 pgs: 129 active+clean; 505 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 753 op/s 2024-06-13T07:16:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:22 smithi135 ceph-mon[36276]: pgmap v985: 129 pgs: 129 active+clean; 505 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 753 op/s 2024-06-13T07:16:23.158 INFO:tasks.workunit.client.0.smithi026.stdout: 10 84507 30.93 MB/sec execute 36 sec latency 62.375 ms 2024-06-13T07:16:24.158 INFO:tasks.workunit.client.0.smithi026.stdout: 10 85152 31.23 MB/sec execute 37 sec latency 75.795 ms 2024-06-13T07:16:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:24 smithi136 ceph-mon[37229]: pgmap v986: 129 pgs: 129 active+clean; 511 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.6 MiB/s wr, 721 op/s 2024-06-13T07:16:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:24 smithi026 ceph-mon[32041]: pgmap v986: 129 pgs: 129 active+clean; 511 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.6 MiB/s wr, 721 op/s 2024-06-13T07:16:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:24 smithi135 ceph-mon[36276]: pgmap v986: 129 pgs: 129 active+clean; 511 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.6 MiB/s wr, 721 op/s 2024-06-13T07:16:25.158 INFO:tasks.workunit.client.0.smithi026.stdout: 10 85672 30.78 MB/sec execute 38 sec latency 85.396 ms 2024-06-13T07:16:26.158 INFO:tasks.workunit.client.0.smithi026.stdout: 10 86477 30.99 MB/sec execute 39 sec latency 54.237 ms 2024-06-13T07:16:26.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:26 smithi136 ceph-mon[37229]: pgmap v987: 129 pgs: 129 active+clean; 515 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.0 MiB/s wr, 659 op/s 2024-06-13T07:16:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:26 smithi026 ceph-mon[32041]: pgmap v987: 129 pgs: 129 active+clean; 515 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.0 MiB/s wr, 659 op/s 2024-06-13T07:16:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:26 smithi135 ceph-mon[36276]: pgmap v987: 129 pgs: 129 active+clean; 515 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.0 MiB/s wr, 659 op/s 2024-06-13T07:16:27.158 INFO:tasks.workunit.client.0.smithi026.stdout: 10 87246 31.40 MB/sec execute 40 sec latency 91.487 ms 2024-06-13T07:16:28.158 INFO:tasks.workunit.client.0.smithi026.stdout: 10 88010 31.82 MB/sec execute 41 sec latency 42.913 ms 2024-06-13T07:16:28.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:28 smithi136 ceph-mon[37229]: pgmap v988: 129 pgs: 129 active+clean; 523 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.9 MiB/s wr, 651 op/s 2024-06-13T07:16:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:28 smithi026 ceph-mon[32041]: pgmap v988: 129 pgs: 129 active+clean; 523 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.9 MiB/s wr, 651 op/s 2024-06-13T07:16:28.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:28 smithi135 ceph-mon[36276]: pgmap v988: 129 pgs: 129 active+clean; 523 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.9 MiB/s wr, 651 op/s 2024-06-13T07:16:29.158 INFO:tasks.workunit.client.0.smithi026.stdout: 10 88367 31.51 MB/sec execute 42 sec latency 209.638 ms 2024-06-13T07:16:30.159 INFO:tasks.workunit.client.0.smithi026.stdout: 10 88640 31.07 MB/sec execute 43 sec latency 599.324 ms 2024-06-13T07:16:30.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:30 smithi136 ceph-mon[37229]: pgmap v989: 129 pgs: 129 active+clean; 528 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 554 op/s 2024-06-13T07:16:30.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:30 smithi135 ceph-mon[36276]: pgmap v989: 129 pgs: 129 active+clean; 528 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 554 op/s 2024-06-13T07:16:30.650 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:30 smithi026 ceph-mon[32041]: pgmap v989: 129 pgs: 129 active+clean; 528 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 554 op/s 2024-06-13T07:16:31.159 INFO:tasks.workunit.client.0.smithi026.stdout: 10 88704 30.45 MB/sec execute 44 sec latency 1451.896 ms 2024-06-13T07:16:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 -- 172.21.15.26:0/3046357787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff314105590 msgr2=0x7ff314105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 --2- 172.21.15.26:0/3046357787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff314105590 0x7ff314105a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009ca0 tx=0x7ff30c02a8d0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 -- 172.21.15.26:0/3046357787 shutdown_connections 2024-06-13T07:16:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 --2- 172.21.15.26:0/3046357787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff314102be0 0x7ff314105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 --2- 172.21.15.26:0/3046357787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff314105590 0x7ff314105a10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 --2- 172.21.15.26:0/3046357787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff314100270 0x7ff3141026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 -- 172.21.15.26:0/3046357787 >> 172.21.15.26:0/3046357787 conn(0x7ff3140fbe70 msgr2=0x7ff3140fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 -- 172.21.15.26:0/3046357787 shutdown_connections 2024-06-13T07:16:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.466+0000 7ff31ac00640 1 -- 172.21.15.26:0/3046357787 wait complete. 2024-06-13T07:16:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.468+0000 7ff31ac00640 1 Processor -- start 2024-06-13T07:16:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.468+0000 7ff31ac00640 1 -- start start 2024-06-13T07:16:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.469+0000 7ff31ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff314100270 0x7ff31410b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.469+0000 7ff31ac00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff314102be0 0x7ff31410bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.469+0000 7ff31ac00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff314105590 0x7ff3141ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.469+0000 7ff31ac00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff314113cf0 con 0x7ff314105590 2024-06-13T07:16:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.469+0000 7ff31ac00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff314113b70 con 0x7ff314102be0 2024-06-13T07:16:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.469+0000 7ff31ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff314113e70 con 0x7ff314100270 2024-06-13T07:16:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.470+0000 7ff313400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff314102be0 0x7ff31410bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.470+0000 7ff313400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff314102be0 0x7ff31410bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50314/0 (socket says 172.21.15.26:50314) 2024-06-13T07:16:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.470+0000 7ff313400640 1 -- 172.21.15.26:0/1264886972 learned_addr learned my addr 172.21.15.26:0/1264886972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.471+0000 7ff313400640 1 -- 172.21.15.26:0/1264886972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff314100270 msgr2=0x7ff31410b710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:16:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.471+0000 7ff313400640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff314100270 0x7ff31410b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.471+0000 7ff313400640 1 -- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff314105590 msgr2=0x7ff3141ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.471+0000 7ff313400640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff314105590 0x7ff3141ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.471+0000 7ff313400640 1 -- 172.21.15.26:0/1264886972 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31410edd0 con 0x7ff314102be0 2024-06-13T07:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.471+0000 7ff313400640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff314102be0 0x7ff31410bc50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff30400d950 tx=0x7ff30400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.472+0000 7ff310c00640 1 -- 172.21.15.26:0/1264886972 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304017070 con 0x7ff314102be0 2024-06-13T07:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.472+0000 7ff310c00640 1 -- 172.21.15.26:0/1264886972 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff304009c30 con 0x7ff314102be0 2024-06-13T07:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.472+0000 7ff31ac00640 1 -- 172.21.15.26:0/1264886972 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3141ae940 con 0x7ff314102be0 2024-06-13T07:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.472+0000 7ff31ac00640 1 -- 172.21.15.26:0/1264886972 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3141aeeb0 con 0x7ff314102be0 2024-06-13T07:16:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.474+0000 7ff310c00640 1 -- 172.21.15.26:0/1264886972 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400f6b0 con 0x7ff314102be0 2024-06-13T07:16:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.479+0000 7ff310c00640 1 -- 172.21.15.26:0/1264886972 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff30400f8d0 con 0x7ff314102be0 2024-06-13T07:16:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.481+0000 7ff310c00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2f80a2cb0 0x7ff2f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.481+0000 7ff310c00640 1 -- 172.21.15.26:0/1264886972 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff3040d03d0 con 0x7ff314102be0 2024-06-13T07:16:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.482+0000 7ff313e00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2f80a2cb0 0x7ff2f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.515+0000 7ff313e00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2f80a2cb0 0x7ff2f80a5170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00b990 tx=0x7ff2fc00b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.515+0000 7ff31db85740 1 Processor -- start 2024-06-13T07:16:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.516+0000 7ff31db85740 1 -- start start 2024-06-13T07:16:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.516+0000 7ff31db85740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.516+0000 7ff31db85740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91ca5f0 0x5624c91cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.516+0000 7ff31db85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91cf760 0x5624c91d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.516+0000 7ff313400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91ca5f0 0x5624c91cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff313400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91ca5f0 0x5624c91cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34574/0 (socket says 172.21.15.26:34574) 2024-06-13T07:16:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff313e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff313400640 1 -- 172.21.15.26:0/3409810939 learned_addr learned my addr 172.21.15.26:0/3409810939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff31db85740 1 -- 172.21.15.26:0/3409810939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5624c912d420 con 0x5624c91c9c00 2024-06-13T07:16:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff31db85740 1 -- 172.21.15.26:0/3409810939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5624c90d4c50 con 0x5624c91cf760 2024-06-13T07:16:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff31db85740 1 -- 172.21.15.26:0/3409810939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5624c91293e0 con 0x5624c91ca5f0 2024-06-13T07:16:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff318c00640 1 --2- 172.21.15.26:0/3409810939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91cf760 0x5624c91d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff313400640 1 -- 172.21.15.26:0/3409810939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91cf760 msgr2=0x5624c91d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff313400640 1 --2- 172.21.15.26:0/3409810939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91cf760 0x5624c91d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff313400640 1 -- 172.21.15.26:0/3409810939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 msgr2=0x5624c91ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff313400640 1 --2- 172.21.15.26:0/3409810939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.517+0000 7ff313400640 1 -- 172.21.15.26:0/3409810939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c8e41210 con 0x5624c91ca5f0 2024-06-13T07:16:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.518+0000 7ff313400640 1 --2- 172.21.15.26:0/3409810939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91ca5f0 0x5624c91cf0f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a5020 tx=0x7ff30409d0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b3c9669c085413 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.518+0000 7ff309200640 1 -- 172.21.15.26:0/3409810939 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304020070 con 0x5624c91ca5f0 2024-06-13T07:16:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.518+0000 7ff309200640 1 -- 172.21.15.26:0/3409810939 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3040a0c90 con 0x5624c91ca5f0 2024-06-13T07:16:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.518+0000 7ff309200640 1 -- 172.21.15.26:0/3409810939 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040870b0 con 0x5624c91ca5f0 2024-06-13T07:16:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.520+0000 7ff31db85740 1 -- 172.21.15.26:0/3409810939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91ca5f0 msgr2=0x5624c91cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.520+0000 7ff31db85740 1 --2- 172.21.15.26:0/3409810939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91ca5f0 0x5624c91cf0f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a5020 tx=0x7ff30409d0a0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.520+0000 7ff31db85740 1 -- 172.21.15.26:0/3409810939 shutdown_connections 2024-06-13T07:16:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.520+0000 7ff31db85740 1 --2- 172.21.15.26:0/3409810939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91cf760 0x5624c91d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.520+0000 7ff31db85740 1 --2- 172.21.15.26:0/3409810939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91ca5f0 0x5624c91cf0f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.520+0000 7ff31db85740 1 --2- 172.21.15.26:0/3409810939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.520+0000 7ff31db85740 1 -- 172.21.15.26:0/3409810939 >> 172.21.15.26:0/3409810939 conn(0x5624c9191fb0 msgr2=0x5624c91c5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.521+0000 7ff31db85740 1 -- 172.21.15.26:0/3409810939 shutdown_connections 2024-06-13T07:16:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.523+0000 7ff31db85740 1 -- 172.21.15.26:0/3409810939 wait complete. 2024-06-13T07:16:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.523+0000 7ff31db85740 1 -- start start 2024-06-13T07:16:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.523+0000 7ff31db85740 1 Processor -- start 2024-06-13T07:16:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.524+0000 7ff31db85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91c9c00 0x5624c91d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.524+0000 7ff31db85740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91ca5f0 0x5624c91d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.524+0000 7ff31db85740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.524+0000 7ff31db85740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5624c91d26b0 con 0x5624c91ca5f0 2024-06-13T07:16:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.524+0000 7ff31db85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5624c90e80b0 con 0x5624c91c9c00 2024-06-13T07:16:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.525+0000 7ff31db85740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5624c91d2830 con 0x5624c91cf760 2024-06-13T07:16:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.526+0000 7ff31db85740 10 client.0 _ll_register_callbacks cb 0x5624c90d3f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:16:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.526+0000 7ff313e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91c9c00 0x5624c91d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.526+0000 7ff313e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91c9c00 0x5624c91d3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50336/0 (socket says 172.21.15.26:50336) 2024-06-13T07:16:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.526+0000 7ff313e00640 1 -- 172.21.15.26:0/4027788646 learned_addr learned my addr 172.21.15.26:0/4027788646 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.526+0000 7ff318c00640 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.527+0000 7ff313e00640 1 -- 172.21.15.26:0/4027788646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 msgr2=0x5624c91db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.528+0000 7ff313e00640 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91db4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.528+0000 7ff313e00640 1 -- 172.21.15.26:0/4027788646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91ca5f0 msgr2=0x5624c91d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.528+0000 7ff313e00640 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91ca5f0 0x5624c91d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.528+0000 7ff313e00640 1 -- 172.21.15.26:0/4027788646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c8f0d620 con 0x5624c91c9c00 2024-06-13T07:16:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.528+0000 7ff318c00640 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:16:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.528+0000 7ff31db85740 1 -- 172.21.15.26:0/4027788646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3040a0c90 con 0x5624c91c9c00 2024-06-13T07:16:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.529+0000 7ff313e00640 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91c9c00 0x5624c91d3f70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00dfc0 tx=0x7ff2fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4703c52f74ff9dac server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.529+0000 7ff301600640 10 client.26699 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:16:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.530+0000 7ff301600640 1 -- 172.21.15.26:0/4027788646 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc026600 con 0x5624c91c9c00 2024-06-13T07:16:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.530+0000 7ff301600640 1 -- 172.21.15.26:0/4027788646 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff2fc0267a0 con 0x5624c91c9c00 2024-06-13T07:16:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.530+0000 7ff301600640 1 -- 172.21.15.26:0/4027788646 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff2fc0269e0 con 0x5624c91c9c00 2024-06-13T07:16:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.531+0000 7ff31db85740 10 client.26699 fetch_fsmap learned FSMap version 43 2024-06-13T07:16:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.531+0000 7ff31db85740 1 -- 172.21.15.26:0/4027788646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=43}) -- 0x5624c91ef850 con 0x5624c91c9c00 2024-06-13T07:16:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.532+0000 7ff301600640 1 -- 172.21.15.26:0/4027788646 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff2fc026b80 con 0x5624c91c9c00 2024-06-13T07:16:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.532+0000 7ff31db85740 10 client.26699 fetch_fsmap finished waiting for FSMap version 43 2024-06-13T07:16:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.532+0000 7ff31db85740 10 client.26699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:16:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.532+0000 7ff31db85740 20 client.26699 populate_metadata read hostname 'smithi026' 2024-06-13T07:16:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.533+0000 7ff31db85740 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5624c91f0eb0 0x5624c91f3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.533+0000 7ff31db85740 4 client.26699 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:16:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.533+0000 7ff31db85740 1 -- 172.21.15.26:0/4027788646 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624c913ab50 con 0x5624c91f0eb0 2024-06-13T07:16:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.533+0000 7ff313400640 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5624c91f0eb0 0x5624c91f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.539+0000 7ff313400640 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5624c91f0eb0 0x5624c91f3370 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6f445e88287a69 server_cookie=6e50487e784f6977 in_seq=0 out_seq=0 2024-06-13T07:16:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.539+0000 7ff301600640 10 client.26699 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.547+0000 7ff301600640 1 -- 172.21.15.26:0/4027788646 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5624c913ab50 con 0x5624c91f0eb0 2024-06-13T07:16:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff301600640 10 client.26699 handle_command_reply: tid=0 2024-06-13T07:16:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 client.26699 shutdown 2024-06-13T07:16:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 -- 172.21.15.26:0/4027788646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91c9c00 msgr2=0x5624c91d3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91c9c00 0x5624c91d3f70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00dfc0 tx=0x7ff2fc004730 comp rx=0 tx=0).stop 2024-06-13T07:16:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 -- 172.21.15.26:0/4027788646 shutdown_connections 2024-06-13T07:16:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5624c91f0eb0 0x5624c91f3370 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91ca5f0 0x5624c91d44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 --2- 172.21.15.26:0/4027788646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91c9c00 0x5624c91d3f70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff31db85740 1 -- 172.21.15.26:0/4027788646 >> 172.21.15.26:0/4027788646 conn(0x5624c9191fb0 msgr2=0x5624c91c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.548+0000 7ff301600640 0 client.26699 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.549+0000 7ff31db85740 1 -- 172.21.15.26:0/4027788646 shutdown_connections 2024-06-13T07:16:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.549+0000 7ff31db85740 1 -- 172.21.15.26:0/4027788646 wait complete. 2024-06-13T07:16:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.550+0000 7ff31db85740 20 client.26699 trim_cache size 0 max 16384 2024-06-13T07:16:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.589+0000 7ff31db85740 1 Processor -- start 2024-06-13T07:16:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.589+0000 7ff31db85740 1 -- start start 2024-06-13T07:16:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff31db85740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91c9c00 0x5624c91cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff31db85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91ca5f0 0x5624c91db480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff31db85740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91cf760 0x5624c91cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff31db85740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0012720 con 0x5624c91cf760 2024-06-13T07:16:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff31db85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f000ff90 con 0x5624c91ca5f0 2024-06-13T07:16:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff31db85740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f000dde0 con 0x5624c91c9c00 2024-06-13T07:16:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff313400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91ca5f0 0x5624c91db480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff313400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91ca5f0 0x5624c91db480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50340/0 (socket says 172.21.15.26:50340) 2024-06-13T07:16:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.590+0000 7ff313400640 1 -- 172.21.15.26:0/453591280 learned_addr learned my addr 172.21.15.26:0/453591280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.591+0000 7ff313e00640 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91c9c00 0x5624c91cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.591+0000 7ff318c00640 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91cf760 0x5624c91cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.591+0000 7ff313400640 1 -- 172.21.15.26:0/453591280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91c9c00 msgr2=0x5624c91cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.591+0000 7ff313400640 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91c9c00 0x5624c91cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.591+0000 7ff313400640 1 -- 172.21.15.26:0/453591280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91cf760 msgr2=0x5624c91cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.591+0000 7ff313400640 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91cf760 0x5624c91cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.591+0000 7ff313400640 1 -- 172.21.15.26:0/453591280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c91cb400 con 0x5624c91ca5f0 2024-06-13T07:16:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.593+0000 7ff313400640 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91ca5f0 0x5624c91db480 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff3040058c0 tx=0x7ff30409abe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2834bcc49bd7cad7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.594+0000 7ff303e00640 1 -- 172.21.15.26:0/453591280 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304020070 con 0x5624c91ca5f0 2024-06-13T07:16:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.594+0000 7ff303e00640 1 -- 172.21.15.26:0/453591280 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3040b5070 con 0x5624c91ca5f0 2024-06-13T07:16:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.594+0000 7ff303e00640 1 -- 172.21.15.26:0/453591280 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040a50c0 con 0x5624c91ca5f0 2024-06-13T07:16:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.595+0000 7ff31db85740 1 -- 172.21.15.26:0/453591280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91ca5f0 msgr2=0x5624c91db480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.595+0000 7ff31db85740 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91ca5f0 0x5624c91db480 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff3040058c0 tx=0x7ff30409abe0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.596+0000 7ff31db85740 1 -- 172.21.15.26:0/453591280 shutdown_connections 2024-06-13T07:16:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.596+0000 7ff31db85740 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c91ca5f0 0x5624c91db480 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff3040058c0 tx=0x7ff30409abe0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.596+0000 7ff31db85740 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91cf760 0x5624c91cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.596+0000 7ff31db85740 1 --2- 172.21.15.26:0/453591280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91c9c00 0x5624c91cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.596+0000 7ff31db85740 1 -- 172.21.15.26:0/453591280 >> 172.21.15.26:0/453591280 conn(0x5624c9191fb0 msgr2=0x5624c91c5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.597+0000 7ff31db85740 1 -- 172.21.15.26:0/453591280 shutdown_connections 2024-06-13T07:16:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.597+0000 7ff31db85740 1 -- 172.21.15.26:0/453591280 wait complete. 2024-06-13T07:16:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.597+0000 7ff31db85740 1 -- start start 2024-06-13T07:16:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.598+0000 7ff31db85740 1 Processor -- start 2024-06-13T07:16:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff31db85740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff31db85740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91d8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff31db85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c9257bf0 0x5624c9259fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff31db85740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f000e450 con 0x5624c91c9c00 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff31db85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f000f8c0 con 0x5624c9257bf0 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff31db85740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f000eb20 con 0x5624c91cf760 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff31db85740 10 client.0 _ll_register_callbacks cb 0x7ff2fc00dc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff313400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91d8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff313400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91d8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34610/0 (socket says 172.21.15.26:34610) 2024-06-13T07:16:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.600+0000 7ff313400640 1 -- 172.21.15.26:0/2068555683 learned_addr learned my addr 172.21.15.26:0/2068555683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.601+0000 7ff313e00640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91d83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff318c00640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c9257bf0 0x5624c9259fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff313400640 1 -- 172.21.15.26:0/2068555683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c9257bf0 msgr2=0x5624c9259fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff313400640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c9257bf0 0x5624c9259fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff313400640 1 -- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 msgr2=0x5624c91d83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff313400640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91d83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff313400640 1 -- 172.21.15.26:0/2068555683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c925a520 con 0x5624c91cf760 2024-06-13T07:16:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff313e00640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:16:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff318c00640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c9257bf0 0x5624c9259fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:16:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff313400640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91d8910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff304010e50 tx=0x7ff30409c2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6a69d054e99dc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.602+0000 7ff301600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:16:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.603+0000 7ff301600640 1 -- 172.21.15.26:0/2068555683 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304020050 con 0x5624c91cf760 2024-06-13T07:16:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.603+0000 7ff31db85740 1 -- 172.21.15.26:0/2068555683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3040b5070 con 0x5624c91cf760 2024-06-13T07:16:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.603+0000 7ff301600640 1 -- 172.21.15.26:0/2068555683 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff3040b0e00 con 0x5624c91cf760 2024-06-13T07:16:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.603+0000 7ff301600640 1 -- 172.21.15.26:0/2068555683 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff304083050 con 0x5624c91cf760 2024-06-13T07:16:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.603+0000 7ff31db85740 10 client.26728 fetch_fsmap learned FSMap version 43 2024-06-13T07:16:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.603+0000 7ff31db85740 1 -- 172.21.15.26:0/2068555683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5624c925b800 con 0x5624c91cf760 2024-06-13T07:16:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.606+0000 7ff301600640 1 -- 172.21.15.26:0/2068555683 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3040b5070 con 0x5624c91cf760 2024-06-13T07:16:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.606+0000 7ff301600640 1 -- 172.21.15.26:0/2068555683 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff304087040 con 0x5624c91cf760 2024-06-13T07:16:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.606+0000 7ff31db85740 10 client.26728 fetch_fsmap finished waiting for FSMap version 43 2024-06-13T07:16:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.606+0000 7ff31db85740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:16:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.606+0000 7ff31db85740 20 client.26728 populate_metadata read hostname 'smithi026' 2024-06-13T07:16:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.606+0000 7ff31db85740 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5624c925bc60 0x5624c925e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.606+0000 7ff31db85740 4 client.26728 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:16:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.606+0000 7ff31db85740 1 -- 172.21.15.26:0/2068555683 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624c9134ea0 con 0x5624c925bc60 2024-06-13T07:16:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.607+0000 7ff313e00640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5624c925bc60 0x5624c925e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.609+0000 7ff313e00640 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5624c925bc60 0x5624c925e080 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6baa41cd3f2169a server_cookie=3a55a3772f68262e in_seq=0 out_seq=0 2024-06-13T07:16:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:31.609+0000 7ff301600640 10 client.26728 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:32.159 INFO:tasks.workunit.client.0.smithi026.stdout: 10 88813 29.81 MB/sec execute 45 sec latency 1831.683 ms 2024-06-13T07:16:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.363+0000 7ff301600640 1 -- 172.21.15.26:0/2068555683 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5624c9134ea0 con 0x5624c925bc60 2024-06-13T07:16:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.363+0000 7ff301600640 10 client.26728 handle_command_reply: tid=0 2024-06-13T07:16:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 client.26728 shutdown 2024-06-13T07:16:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 -- 172.21.15.26:0/2068555683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 msgr2=0x5624c91d8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91d8910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff304010e50 tx=0x7ff30409c2f0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 -- 172.21.15.26:0/2068555683 shutdown_connections 2024-06-13T07:16:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5624c925bc60 0x5624c925e080 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff301600640 0 client.26728 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624c9257bf0 0x5624c9259fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624c91cf760 0x5624c91d8910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 --2- 172.21.15.26:0/2068555683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5624c91c9c00 0x5624c91d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 -- 172.21.15.26:0/2068555683 >> 172.21.15.26:0/2068555683 conn(0x5624c9191fb0 msgr2=0x5624c91c85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 -- 172.21.15.26:0/2068555683 shutdown_connections 2024-06-13T07:16:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.364+0000 7ff31db85740 1 -- 172.21.15.26:0/2068555683 wait complete. 2024-06-13T07:16:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.365+0000 7ff31db85740 20 client.26728 trim_cache size 0 max 16384 2024-06-13T07:16:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:16:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:16:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:16:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 -- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2f80a2cb0 msgr2=0x7ff2f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2f80a2cb0 0x7ff2f80a5170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00b990 tx=0x7ff2fc00b370 comp rx=0 tx=0).stop 2024-06-13T07:16:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 -- 172.21.15.26:0/1264886972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff314102be0 msgr2=0x7ff31410bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff314102be0 0x7ff31410bc50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff30400d950 tx=0x7ff30400de10 comp rx=0 tx=0).stop 2024-06-13T07:16:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 -- 172.21.15.26:0/1264886972 shutdown_connections 2024-06-13T07:16:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff2f80a2cb0 0x7ff2f80a5170 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff314102be0 0x7ff31410bc50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff314105590 0x7ff3141ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 --2- 172.21.15.26:0/1264886972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff314100270 0x7ff31410b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 -- 172.21.15.26:0/1264886972 >> 172.21.15.26:0/1264886972 conn(0x7ff3140fbe70 msgr2=0x7ff314100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 -- 172.21.15.26:0/1264886972 shutdown_connections 2024-06-13T07:16:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:32.367+0000 7ff31ac00640 1 -- 172.21.15.26:0/1264886972 wait complete. 2024-06-13T07:16:32.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:32 smithi136 ceph-mon[37229]: pgmap v990: 129 pgs: 129 active+clean; 545 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.7 MiB/s wr, 478 op/s 2024-06-13T07:16:32.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:32 smithi026 ceph-mon[32041]: pgmap v990: 129 pgs: 129 active+clean; 545 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.7 MiB/s wr, 478 op/s 2024-06-13T07:16:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:32 smithi135 ceph-mon[36276]: pgmap v990: 129 pgs: 129 active+clean; 545 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.7 MiB/s wr, 478 op/s 2024-06-13T07:16:33.159 INFO:tasks.workunit.client.0.smithi026.stdout: 10 89106 29.48 MB/sec execute 46 sec latency 424.724 ms 2024-06-13T07:16:34.159 INFO:tasks.workunit.client.0.smithi026.stdout: 10 89405 29.26 MB/sec execute 47 sec latency 176.296 ms 2024-06-13T07:16:34.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:34 smithi136 ceph-mon[37229]: pgmap v991: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.8 MiB/s wr, 491 op/s 2024-06-13T07:16:34.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:34 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:34 smithi026 ceph-mon[32041]: pgmap v991: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.8 MiB/s wr, 491 op/s 2024-06-13T07:16:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:34 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:34 smithi135 ceph-mon[36276]: pgmap v991: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.8 MiB/s wr, 491 op/s 2024-06-13T07:16:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:34 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:35.159 INFO:tasks.workunit.client.0.smithi026.stdout: 10 89768 28.99 MB/sec execute 48 sec latency 159.342 ms 2024-06-13T07:16:36.159 INFO:tasks.workunit.client.0.smithi026.stdout: 10 90574 29.31 MB/sec execute 49 sec latency 54.051 ms 2024-06-13T07:16:36.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:36 smithi136 ceph-mon[37229]: pgmap v992: 129 pgs: 129 active+clean; 587 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 648 op/s 2024-06-13T07:16:36.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:36 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:16:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:36 smithi026 ceph-mon[32041]: pgmap v992: 129 pgs: 129 active+clean; 587 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 648 op/s 2024-06-13T07:16:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:36 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:16:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:36 smithi135 ceph-mon[36276]: pgmap v992: 129 pgs: 129 active+clean; 587 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 648 op/s 2024-06-13T07:16:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:36 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:16:37.159 INFO:tasks.workunit.client.0.smithi026.stdout: 10 91021 29.15 MB/sec execute 50 sec latency 331.218 ms 2024-06-13T07:16:38.160 INFO:tasks.workunit.client.0.smithi026.stdout: 10 91553 29.27 MB/sec execute 51 sec latency 181.627 ms 2024-06-13T07:16:38.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:38 smithi136 ceph-mon[37229]: pgmap v993: 129 pgs: 129 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 13 MiB/s wr, 691 op/s 2024-06-13T07:16:38.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:38 smithi026 ceph-mon[32041]: pgmap v993: 129 pgs: 129 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 13 MiB/s wr, 691 op/s 2024-06-13T07:16:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:38 smithi135 ceph-mon[36276]: pgmap v993: 129 pgs: 129 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 13 MiB/s wr, 691 op/s 2024-06-13T07:16:39.160 INFO:tasks.workunit.client.0.smithi026.stdout: 10 91971 29.08 MB/sec execute 52 sec latency 109.075 ms 2024-06-13T07:16:40.160 INFO:tasks.workunit.client.0.smithi026.stdout: 10 92593 29.13 MB/sec execute 53 sec latency 80.736 ms 2024-06-13T07:16:40.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:40 smithi136 ceph-mon[37229]: pgmap v994: 129 pgs: 129 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 804 op/s 2024-06-13T07:16:40.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:40.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:40.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:16:40.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:16:40.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:40 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:40 smithi026 ceph-mon[32041]: pgmap v994: 129 pgs: 129 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 804 op/s 2024-06-13T07:16:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:40.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:40.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:16:40.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:16:40.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:40 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:40 smithi135 ceph-mon[36276]: pgmap v994: 129 pgs: 129 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 804 op/s 2024-06-13T07:16:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:40 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:40 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:40 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:16:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:40 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:16:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:40 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:16:41.160 INFO:tasks.workunit.client.0.smithi026.stdout: 10 93339 29.36 MB/sec execute 54 sec latency 108.009 ms 2024-06-13T07:16:42.160 INFO:tasks.workunit.client.0.smithi026.stdout: 10 93759 29.31 MB/sec execute 55 sec latency 290.119 ms 2024-06-13T07:16:42.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:42 smithi136 ceph-mon[37229]: pgmap v995: 129 pgs: 129 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 764 op/s 2024-06-13T07:16:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:42 smithi026 ceph-mon[32041]: pgmap v995: 129 pgs: 129 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 764 op/s 2024-06-13T07:16:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:42 smithi135 ceph-mon[36276]: pgmap v995: 129 pgs: 129 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 764 op/s 2024-06-13T07:16:43.160 INFO:tasks.workunit.client.0.smithi026.stdout: 10 94226 29.16 MB/sec execute 56 sec latency 294.834 ms 2024-06-13T07:16:44.161 INFO:tasks.workunit.client.0.smithi026.stdout: 10 94722 29.18 MB/sec execute 57 sec latency 95.757 ms 2024-06-13T07:16:44.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:44 smithi136 ceph-mon[37229]: pgmap v996: 129 pgs: 129 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 840 op/s 2024-06-13T07:16:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:44 smithi026 ceph-mon[32041]: pgmap v996: 129 pgs: 129 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 840 op/s 2024-06-13T07:16:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:44 smithi135 ceph-mon[36276]: pgmap v996: 129 pgs: 129 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 840 op/s 2024-06-13T07:16:45.161 INFO:tasks.workunit.client.0.smithi026.stdout: 10 95462 29.23 MB/sec execute 58 sec latency 75.909 ms 2024-06-13T07:16:46.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 96118 29.37 MB/sec execute 59 sec latency 96.216 ms 2024-06-13T07:16:46.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:46 smithi136 ceph-mon[37229]: pgmap v997: 129 pgs: 129 active+clean; 545 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-06-13T07:16:46.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:46 smithi026 ceph-mon[32041]: pgmap v997: 129 pgs: 129 active+clean; 545 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-06-13T07:16:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:46 smithi135 ceph-mon[36276]: pgmap v997: 129 pgs: 129 active+clean; 545 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-06-13T07:16:47.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 96789 29.59 MB/sec execute 60 sec latency 64.281 ms 2024-06-13T07:16:48.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 97282 29.50 MB/sec execute 61 sec latency 194.513 ms 2024-06-13T07:16:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:48 smithi026 ceph-mon[32041]: pgmap v998: 129 pgs: 129 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.4 MiB/s wr, 596 op/s 2024-06-13T07:16:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:48 smithi135 ceph-mon[36276]: pgmap v998: 129 pgs: 129 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.4 MiB/s wr, 596 op/s 2024-06-13T07:16:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:48 smithi136 ceph-mon[37229]: pgmap v998: 129 pgs: 129 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.4 MiB/s wr, 596 op/s 2024-06-13T07:16:49.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 97503 29.16 MB/sec execute 62 sec latency 481.200 ms 2024-06-13T07:16:50.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 97888 29.08 MB/sec execute 63 sec latency 156.565 ms 2024-06-13T07:16:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.291+0000 7f0427a00640 1 -- 172.21.15.26:0/1489859936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 msgr2=0x7f04201096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.291+0000 7f0427a00640 1 --2- 172.21.15.26:0/1489859936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 0x7f04201096a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f040c009900 tx=0x7f040c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.291+0000 7f0427a00640 1 -- 172.21.15.26:0/1489859936 shutdown_connections 2024-06-13T07:16:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.291+0000 7f0427a00640 1 --2- 172.21.15.26:0/1489859936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 0x7f04201096a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.291+0000 7f0427a00640 1 --2- 172.21.15.26:0/1489859936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420109be0 0x7f0420110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.291+0000 7f0427a00640 1 --2- 172.21.15.26:0/1489859936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0420108030 0x7f0420108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.291+0000 7f0427a00640 1 -- 172.21.15.26:0/1489859936 >> 172.21.15.26:0/1489859936 conn(0x7f0420076ff0 msgr2=0x7f0420079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.293+0000 7f0427a00640 1 -- 172.21.15.26:0/1489859936 shutdown_connections 2024-06-13T07:16:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.293+0000 7f0427a00640 1 -- 172.21.15.26:0/1489859936 wait complete. 2024-06-13T07:16:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.296+0000 7f0427a00640 1 Processor -- start 2024-06-13T07:16:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.296+0000 7f0427a00640 1 -- start start 2024-06-13T07:16:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.297+0000 7f0427a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420108030 0x7f042019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.298+0000 7f0427a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 0x7f042019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.298+0000 7f0425000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420108030 0x7f042019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.298+0000 7f0427a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0420109be0 0x7f04201a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.299+0000 7f0427a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0420113d00 con 0x7f0420109be0 2024-06-13T07:16:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.299+0000 7f041fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 0x7f042019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.299+0000 7f0425000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420108030 0x7f042019e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41754/0 (socket says 172.21.15.26:41754) 2024-06-13T07:16:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.299+0000 7f041fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 0x7f042019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:58890/0 (socket says 172.21.15.26:58890) 2024-06-13T07:16:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.299+0000 7f0425000640 1 -- 172.21.15.26:0/1503100298 learned_addr learned my addr 172.21.15.26:0/1503100298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.299+0000 7f0425a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0420109be0 0x7f04201a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.299+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0420113b80 con 0x7f0420109220 2024-06-13T07:16:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.299+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0420113e80 con 0x7f0420108030 2024-06-13T07:16:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.300+0000 7f0425a00640 1 -- 172.21.15.26:0/1503100298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420108030 msgr2=0x7f042019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.300+0000 7f0425a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420108030 0x7f042019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.300+0000 7f0425a00640 1 -- 172.21.15.26:0/1503100298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 msgr2=0x7f042019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.300+0000 7f0425a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 0x7f042019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.300+0000 7f0425a00640 1 -- 172.21.15.26:0/1503100298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04201a5ed0 con 0x7f0420109be0 2024-06-13T07:16:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.301+0000 7f041fe00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 0x7f042019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:16:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.301+0000 7f0425000640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420108030 0x7f042019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:16:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.301+0000 7f0425a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0420109be0 0x7f04201a5860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f041800be20 tx=0x7f041800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.302+0000 7f041d600640 1 -- 172.21.15.26:0/1503100298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418019070 con 0x7f0420109be0 2024-06-13T07:16:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.303+0000 7f041d600640 1 -- 172.21.15.26:0/1503100298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0418014920 con 0x7f0420109be0 2024-06-13T07:16:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.303+0000 7f041d600640 1 -- 172.21.15.26:0/1503100298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418012930 con 0x7f0420109be0 2024-06-13T07:16:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.303+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04201a61c0 con 0x7f0420109be0 2024-06-13T07:16:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.303+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04201a6710 con 0x7f0420109be0 2024-06-13T07:16:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.311+0000 7f041d600640 1 -- 172.21.15.26:0/1503100298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f041801c070 con 0x7f0420109be0 2024-06-13T07:16:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.313+0000 7f041d600640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04080a2cb0 0x7f04080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.314+0000 7f041d600640 1 -- 172.21.15.26:0/1503100298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f04180cece0 con 0x7f0420109be0 2024-06-13T07:16:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.314+0000 7f0425000640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04080a2cb0 0x7f04080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.317+0000 7f0425000640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04080a2cb0 0x7f04080a5170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0414009c80 tx=0x7f0414005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.345+0000 7f042a7ed740 1 Processor -- start 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.346+0000 7f042a7ed740 1 -- start start 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.346+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30443c00 0x561e30444020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.346+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 0x561e304490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.346+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3044bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.346+0000 7f042a7ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561e303a7420 con 0x561e304445f0 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.346+0000 7f042a7ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e3034ec50 con 0x561e30449760 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.346+0000 7f042a7ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561e303a33e0 con 0x561e30443c00 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30443c00 0x561e30444020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30443c00 0x561e30444020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41770/0 (socket says 172.21.15.26:41770) 2024-06-13T07:16:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425000640 1 -- 172.21.15.26:0/325669616 learned_addr learned my addr 172.21.15.26:0/325669616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425a00640 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3044bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f041fe00640 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 0x561e304490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425a00640 1 -- 172.21.15.26:0/325669616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30443c00 msgr2=0x561e30444020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425a00640 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30443c00 0x561e30444020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425a00640 1 -- 172.21.15.26:0/325669616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 msgr2=0x561e304490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425a00640 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 0x561e304490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.347+0000 7f0425a00640 1 -- 172.21.15.26:0/325669616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e300bb210 con 0x561e30449760 2024-06-13T07:16:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.348+0000 7f0425a00640 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3044bbf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f04180cc440 tx=0x7f0418012340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=589038cae6c1ad1f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.348+0000 7f0412000640 1 -- 172.21.15.26:0/325669616 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041809f500 con 0x561e30449760 2024-06-13T07:16:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.348+0000 7f0412000640 1 -- 172.21.15.26:0/325669616 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04180a3dc0 con 0x561e30449760 2024-06-13T07:16:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.348+0000 7f0412000640 1 -- 172.21.15.26:0/325669616 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418014ad0 con 0x561e30449760 2024-06-13T07:16:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.350+0000 7f042a7ed740 1 -- 172.21.15.26:0/325669616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 msgr2=0x561e3044bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.350+0000 7f042a7ed740 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3044bbf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f04180cc440 tx=0x7f0418012340 comp rx=0 tx=0).stop 2024-06-13T07:16:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.351+0000 7f042a7ed740 1 -- 172.21.15.26:0/325669616 shutdown_connections 2024-06-13T07:16:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.351+0000 7f042a7ed740 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3044bbf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.351+0000 7f042a7ed740 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 0x561e304490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.351+0000 7f042a7ed740 1 --2- 172.21.15.26:0/325669616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30443c00 0x561e30444020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.351+0000 7f042a7ed740 1 -- 172.21.15.26:0/325669616 >> 172.21.15.26:0/325669616 conn(0x561e3040bfb0 msgr2=0x561e3043f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.351+0000 7f042a7ed740 1 -- 172.21.15.26:0/325669616 shutdown_connections 2024-06-13T07:16:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.352+0000 7f042a7ed740 1 -- 172.21.15.26:0/325669616 wait complete. 2024-06-13T07:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.352+0000 7f042a7ed740 1 -- start start 2024-06-13T07:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.353+0000 7f042a7ed740 1 Processor -- start 2024-06-13T07:16:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e3045f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e30466520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f042a7ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561e3045d720 con 0x561e30443c00 2024-06-13T07:16:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f042a7ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e303620b0 con 0x561e30449760 2024-06-13T07:16:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f042a7ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561e3045d8a0 con 0x561e304445f0 2024-06-13T07:16:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f042a7ed740 10 client.0 _ll_register_callbacks cb 0x561e3034df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:16:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f0425000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f041fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e3045f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f0425000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48012/0 (socket says 172.21.15.26:48012) 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.354+0000 7f0425000640 1 -- 172.21.15.26:0/700863180 learned_addr learned my addr 172.21.15.26:0/700863180 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f0425a00640 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e30466520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f041fe00640 1 -- 172.21.15.26:0/700863180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 msgr2=0x561e30466520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f041fe00640 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e30466520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f041fe00640 1 -- 172.21.15.26:0/700863180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 msgr2=0x561e3045efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f041fe00640 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f041fe00640 1 -- 172.21.15.26:0/700863180 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e30187620 con 0x561e304445f0 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f0425a00640 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e30466520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f041fe00640 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e3045f4e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f040c02a910 tx=0x7f040c0319d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a06a121ea605cc5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.355+0000 7f0406a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:16:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.356+0000 7f0406a00640 1 -- 172.21.15.26:0/700863180 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c03f070 con 0x561e304445f0 2024-06-13T07:16:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.356+0000 7f0406a00640 1 -- 172.21.15.26:0/700863180 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f040c005030 con 0x561e304445f0 2024-06-13T07:16:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.356+0000 7f042a7ed740 1 -- 172.21.15.26:0/700863180 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04180a3dc0 con 0x561e304445f0 2024-06-13T07:16:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.357+0000 7f0406a00640 1 -- 172.21.15.26:0/700863180 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f040c035070 con 0x561e304445f0 2024-06-13T07:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.357+0000 7f042a7ed740 10 client.26734 fetch_fsmap learned FSMap version 45 2024-06-13T07:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.357+0000 7f042a7ed740 1 -- 172.21.15.26:0/700863180 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561e30469900 con 0x561e304445f0 2024-06-13T07:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.357+0000 7f0406a00640 1 -- 172.21.15.26:0/700863180 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04180a3dc0 con 0x561e304445f0 2024-06-13T07:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.359+0000 7f0406a00640 1 -- 172.21.15.26:0/700863180 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f040c0035f0 con 0x561e304445f0 2024-06-13T07:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.359+0000 7f042a7ed740 10 client.26734 fetch_fsmap finished waiting for FSMap version 45 2024-06-13T07:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.359+0000 7f042a7ed740 10 client.26734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.359+0000 7f042a7ed740 20 client.26734 populate_metadata read hostname 'smithi026' 2024-06-13T07:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.359+0000 7f042a7ed740 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e3046aff0 0x561e3046d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.359+0000 7f042a7ed740 4 client.26734 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:16:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.359+0000 7f042a7ed740 1 -- 172.21.15.26:0/700863180 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e303b4b50 con 0x561e3046aff0 2024-06-13T07:16:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.361+0000 7f0425000640 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e3046aff0 0x561e3046d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.362+0000 7f0425000640 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e3046aff0 0x561e3046d440 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8a0d9d02e36049 server_cookie=33a3f6f354c4b528 in_seq=0 out_seq=0 2024-06-13T07:16:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.362+0000 7f0406a00640 10 client.26734 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.366+0000 7f0406a00640 1 -- 172.21.15.26:0/700863180 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e303b4b50 con 0x561e3046aff0 2024-06-13T07:16:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.366+0000 7f0406a00640 10 client.26734 handle_command_reply: tid=0 2024-06-13T07:16:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.367+0000 7f042a7ed740 1 client.26734 shutdown 2024-06-13T07:16:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 -- 172.21.15.26:0/700863180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 msgr2=0x561e3045f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e3045f4e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f040c02a910 tx=0x7f040c0319d0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 -- 172.21.15.26:0/700863180 shutdown_connections 2024-06-13T07:16:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e3046aff0 0x561e3046d440 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e30466520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e3045f4e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 --2- 172.21.15.26:0/700863180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 -- 172.21.15.26:0/700863180 >> 172.21.15.26:0/700863180 conn(0x561e3040bfb0 msgr2=0x561e3043e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f0406a00640 0 client.26734 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.368+0000 7f042a7ed740 1 -- 172.21.15.26:0/700863180 shutdown_connections 2024-06-13T07:16:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.370+0000 7f042a7ed740 1 -- 172.21.15.26:0/700863180 wait complete. 2024-06-13T07:16:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.371+0000 7f042a7ed740 20 client.26734 trim_cache size 0 max 16384 2024-06-13T07:16:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.409+0000 7f042a7ed740 1 Processor -- start 2024-06-13T07:16:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.409+0000 7f042a7ed740 1 -- start start 2024-06-13T07:16:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.410+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e304664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3045f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f042a7ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04201aae40 con 0x561e30443c00 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f042a7ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04201a86e0 con 0x561e30449760 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f042a7ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0420082590 con 0x561e304445f0 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f041fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e304664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f0425a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3045f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f041fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e304664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41796/0 (socket says 172.21.15.26:41796) 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f041fe00640 1 -- 172.21.15.26:0/3325747170 learned_addr learned my addr 172.21.15.26:0/3325747170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.411+0000 7f0425000640 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.412+0000 7f0425a00640 1 -- 172.21.15.26:0/3325747170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 msgr2=0x561e304664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.413+0000 7f0425a00640 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e304664e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.413+0000 7f0425a00640 1 -- 172.21.15.26:0/3325747170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 msgr2=0x561e3045efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.414+0000 7f0425a00640 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.414+0000 7f0425a00640 1 -- 172.21.15.26:0/3325747170 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e304449d0 con 0x561e30449760 2024-06-13T07:16:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.414+0000 7f0425000640 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:16:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.414+0000 7f041fe00640 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e304664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:16:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.414+0000 7f0425a00640 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3045f4e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0418009f90 tx=0x7f041809b210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=553d3cb4f9c5895a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.415+0000 7f0411600640 1 -- 172.21.15.26:0/3325747170 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041809f500 con 0x561e30449760 2024-06-13T07:16:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.415+0000 7f0411600640 1 -- 172.21.15.26:0/3325747170 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04180a3dc0 con 0x561e30449760 2024-06-13T07:16:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.415+0000 7f0411600640 1 -- 172.21.15.26:0/3325747170 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041809f500 con 0x561e30449760 2024-06-13T07:16:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.417+0000 7f042a7ed740 1 -- 172.21.15.26:0/3325747170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 msgr2=0x561e3045f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.417+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3045f4e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0418009f90 tx=0x7f041809b210 comp rx=0 tx=0).stop 2024-06-13T07:16:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.417+0000 7f042a7ed740 1 -- 172.21.15.26:0/3325747170 shutdown_connections 2024-06-13T07:16:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.417+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30449760 0x561e3045f4e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.417+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e304445f0 0x561e304664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.417+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3325747170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e30443c00 0x561e3045efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.417+0000 7f042a7ed740 1 -- 172.21.15.26:0/3325747170 >> 172.21.15.26:0/3325747170 conn(0x561e3040bfb0 msgr2=0x561e3045e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.419+0000 7f042a7ed740 1 -- 172.21.15.26:0/3325747170 shutdown_connections 2024-06-13T07:16:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.419+0000 7f042a7ed740 1 -- 172.21.15.26:0/3325747170 wait complete. 2024-06-13T07:16:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.419+0000 7f042a7ed740 1 -- start start 2024-06-13T07:16:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.420+0000 7f042a7ed740 1 Processor -- start 2024-06-13T07:16:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.421+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30443c00 0x561e304662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.421+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 0x561e30462d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.421+0000 7f042a7ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30449760 0x561e304632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.421+0000 7f042a7ed740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04201a6ba0 con 0x561e304445f0 2024-06-13T07:16:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.421+0000 7f042a7ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04201a8010 con 0x561e30443c00 2024-06-13T07:16:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.421+0000 7f042a7ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04201a7270 con 0x561e30449760 2024-06-13T07:16:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.421+0000 7f042a7ed740 10 client.0 _ll_register_callbacks cb 0x7f040c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:16:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.423+0000 7f0425a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30449760 0x561e304632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.423+0000 7f0425a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30449760 0x561e304632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41812/0 (socket says 172.21.15.26:41812) 2024-06-13T07:16:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.423+0000 7f0425a00640 1 -- 172.21.15.26:0/3993903153 learned_addr learned my addr 172.21.15.26:0/3993903153 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:16:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.423+0000 7f0425000640 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30443c00 0x561e304662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.423+0000 7f041fe00640 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 0x561e30462d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.424+0000 7f0425a00640 1 -- 172.21.15.26:0/3993903153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30443c00 msgr2=0x561e304662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.424+0000 7f0425a00640 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30443c00 0x561e304662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.424+0000 7f0425a00640 1 -- 172.21.15.26:0/3993903153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 msgr2=0x561e30462d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:16:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.424+0000 7f0425a00640 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 0x561e30462d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.424+0000 7f0425a00640 1 -- 172.21.15.26:0/3993903153 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e304e4d20 con 0x561e30449760 2024-06-13T07:16:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.425+0000 7f0406a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:16:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.425+0000 7f0425a00640 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30449760 0x561e304632a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f04180a3890 tx=0x7f04180123d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd75578dcfbbc893 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:16:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.425+0000 7f0406a00640 1 -- 172.21.15.26:0/3993903153 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418014ad0 con 0x561e30449760 2024-06-13T07:16:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.426+0000 7f0406a00640 1 -- 172.21.15.26:0/3993903153 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f04180a3930 con 0x561e30449760 2024-06-13T07:16:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.426+0000 7f042a7ed740 1 -- 172.21.15.26:0/3993903153 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04180a3dc0 con 0x561e30449760 2024-06-13T07:16:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.426+0000 7f0406a00640 1 -- 172.21.15.26:0/3993903153 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0418014c70 con 0x561e30449760 2024-06-13T07:16:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.426+0000 7f042a7ed740 10 client.26740 fetch_fsmap learned FSMap version 45 2024-06-13T07:16:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.426+0000 7f042a7ed740 1 -- 172.21.15.26:0/3993903153 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561e304e5eb0 con 0x561e30449760 2024-06-13T07:16:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.427+0000 7f0406a00640 1 -- 172.21.15.26:0/3993903153 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04180a3dc0 con 0x561e30449760 2024-06-13T07:16:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.428+0000 7f0406a00640 1 -- 172.21.15.26:0/3993903153 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f0418096f30 con 0x561e30449760 2024-06-13T07:16:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.428+0000 7f042a7ed740 10 client.26740 fetch_fsmap finished waiting for FSMap version 45 2024-06-13T07:16:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.428+0000 7f042a7ed740 10 client.26740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:16:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.428+0000 7f042a7ed740 20 client.26740 populate_metadata read hostname 'smithi026' 2024-06-13T07:16:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.428+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e304e6240 0x561e304e8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:16:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.428+0000 7f042a7ed740 4 client.26740 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:16:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.428+0000 7f042a7ed740 1 -- 172.21.15.26:0/3993903153 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e303aeea0 con 0x561e304e6240 2024-06-13T07:16:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.429+0000 7f0425000640 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e304e6240 0x561e304e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:16:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.430+0000 7f0425000640 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e304e6240 0x561e304e8630 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=913d578e5f8b9b37 server_cookie=fb52e9e413ae715 in_seq=0 out_seq=0 2024-06-13T07:16:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.430+0000 7f0406a00640 10 client.26740 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:50.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:50 smithi026 ceph-mon[32041]: pgmap v999: 129 pgs: 129 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 679 op/s 2024-06-13T07:16:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:50 smithi135 ceph-mon[36276]: pgmap v999: 129 pgs: 129 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 679 op/s 2024-06-13T07:16:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:50 smithi136 ceph-mon[37229]: pgmap v999: 129 pgs: 129 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 679 op/s 2024-06-13T07:16:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.632+0000 7f0406a00640 1 -- 172.21.15.26:0/3993903153 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561e303aeea0 con 0x561e304e6240 2024-06-13T07:16:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.632+0000 7f0406a00640 10 client.26740 handle_command_reply: tid=0 2024-06-13T07:16:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.632+0000 7f042a7ed740 1 client.26740 shutdown 2024-06-13T07:16:50.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.634+0000 7f042a7ed740 1 -- 172.21.15.26:0/3993903153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30449760 msgr2=0x561e304632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.634+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30449760 0x561e304632a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f04180a3890 tx=0x7f04180123d0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.636+0000 7f042a7ed740 1 -- 172.21.15.26:0/3993903153 shutdown_connections 2024-06-13T07:16:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.636+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x561e304e6240 0x561e304e8630 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.636+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e30449760 0x561e304632a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.636+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e304445f0 0x561e30462d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.636+0000 7f042a7ed740 1 --2- 172.21.15.26:0/3993903153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e30443c00 0x561e304662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.636+0000 7f042a7ed740 1 -- 172.21.15.26:0/3993903153 >> 172.21.15.26:0/3993903153 conn(0x561e3040bfb0 msgr2=0x561e30445950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:50.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.636+0000 7f0406a00640 0 client.26740 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:16:50.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.636+0000 7f042a7ed740 1 -- 172.21.15.26:0/3993903153 shutdown_connections 2024-06-13T07:16:50.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.639+0000 7f042a7ed740 1 -- 172.21.15.26:0/3993903153 wait complete. 2024-06-13T07:16:50.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.639+0000 7f042a7ed740 20 client.26740 trim_cache size 0 max 16384 2024-06-13T07:16:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:16:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:16:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:16:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.640+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04080a2cb0 msgr2=0x7f04080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.640+0000 7f0427a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04080a2cb0 0x7f04080a5170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0414009c80 tx=0x7f0414005f90 comp rx=0 tx=0).stop 2024-06-13T07:16:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.640+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0420109be0 msgr2=0x7f04201a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:16:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.640+0000 7f0427a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0420109be0 0x7f04201a5860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f041800be20 tx=0x7f041800bf10 comp rx=0 tx=0).stop 2024-06-13T07:16:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.641+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 shutdown_connections 2024-06-13T07:16:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.641+0000 7f0427a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04080a2cb0 0x7f04080a5170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.641+0000 7f0427a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0420109220 0x7f042019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.641+0000 7f0427a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0420109be0 0x7f04201a5860 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.641+0000 7f0427a00640 1 --2- 172.21.15.26:0/1503100298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420108030 0x7f042019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:16:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.641+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 >> 172.21.15.26:0/1503100298 conn(0x7f0420076ff0 msgr2=0x7f042010c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:16:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.641+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 shutdown_connections 2024-06-13T07:16:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:16:50.642+0000 7f0427a00640 1 -- 172.21.15.26:0/1503100298 wait complete. 2024-06-13T07:16:51.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 98370 29.00 MB/sec execute 64 sec latency 155.657 ms 2024-06-13T07:16:51.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:51 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:51.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:51 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:51 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:52.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 98934 28.96 MB/sec execute 65 sec latency 286.959 ms 2024-06-13T07:16:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:52 smithi026 ceph-mon[32041]: pgmap v1000: 129 pgs: 129 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 491 op/s 2024-06-13T07:16:52.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:52 smithi136 ceph-mon[37229]: pgmap v1000: 129 pgs: 129 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 491 op/s 2024-06-13T07:16:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:52 smithi135 ceph-mon[36276]: pgmap v1000: 129 pgs: 129 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 491 op/s 2024-06-13T07:16:53.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 99581 29.10 MB/sec execute 66 sec latency 76.730 ms 2024-06-13T07:16:54.162 INFO:tasks.workunit.client.0.smithi026.stdout: 10 100227 29.24 MB/sec execute 67 sec latency 66.715 ms 2024-06-13T07:16:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:54 smithi026 ceph-mon[32041]: pgmap v1001: 129 pgs: 129 active+clean; 558 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 641 op/s 2024-06-13T07:16:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:54 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:54 smithi135 ceph-mon[36276]: pgmap v1001: 129 pgs: 129 active+clean; 558 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 641 op/s 2024-06-13T07:16:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:54 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:54 smithi136 ceph-mon[37229]: pgmap v1001: 129 pgs: 129 active+clean; 558 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 641 op/s 2024-06-13T07:16:54.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:54 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:55.163 INFO:tasks.workunit.client.0.smithi026.stdout: 10 100755 29.33 MB/sec execute 68 sec latency 67.195 ms 2024-06-13T07:16:56.163 INFO:tasks.workunit.client.0.smithi026.stdout: 10 101258 29.21 MB/sec execute 69 sec latency 63.093 ms 2024-06-13T07:16:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:56 smithi026 ceph-mon[32041]: pgmap v1002: 129 pgs: 129 active+clean; 548 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-06-13T07:16:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:56 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:56 smithi135 ceph-mon[36276]: pgmap v1002: 129 pgs: 129 active+clean; 548 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-06-13T07:16:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:56 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:56 smithi136 ceph-mon[37229]: pgmap v1002: 129 pgs: 129 active+clean; 548 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 560 op/s 2024-06-13T07:16:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:56 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:16:57.163 INFO:tasks.workunit.client.0.smithi026.stdout: 10 101876 29.22 MB/sec execute 70 sec latency 80.726 ms 2024-06-13T07:16:58.163 INFO:tasks.workunit.client.0.smithi026.stdout: 10 102536 29.30 MB/sec execute 71 sec latency 73.931 ms 2024-06-13T07:16:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:16:58 smithi026 ceph-mon[32041]: pgmap v1003: 129 pgs: 129 active+clean; 547 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 530 op/s 2024-06-13T07:16:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:16:58 smithi135 ceph-mon[36276]: pgmap v1003: 129 pgs: 129 active+clean; 547 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 530 op/s 2024-06-13T07:16:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:16:58 smithi136 ceph-mon[37229]: pgmap v1003: 129 pgs: 129 active+clean; 547 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 530 op/s 2024-06-13T07:16:59.163 INFO:tasks.workunit.client.0.smithi026.stdout: 10 103194 29.37 MB/sec execute 72 sec latency 80.604 ms 2024-06-13T07:17:00.164 INFO:tasks.workunit.client.0.smithi026.stdout: 10 103975 29.57 MB/sec execute 73 sec latency 57.964 ms 2024-06-13T07:17:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:00 smithi026 ceph-mon[32041]: pgmap v1004: 129 pgs: 129 active+clean; 554 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 676 op/s 2024-06-13T07:17:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:00 smithi135 ceph-mon[36276]: pgmap v1004: 129 pgs: 129 active+clean; 554 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 676 op/s 2024-06-13T07:17:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:00 smithi136 ceph-mon[37229]: pgmap v1004: 129 pgs: 129 active+clean; 554 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 676 op/s 2024-06-13T07:17:01.164 INFO:tasks.workunit.client.0.smithi026.stdout: 10 104403 29.53 MB/sec execute 74 sec latency 296.221 ms 2024-06-13T07:17:02.164 INFO:tasks.workunit.client.0.smithi026.stdout: 10 105011 29.59 MB/sec execute 75 sec latency 48.390 ms 2024-06-13T07:17:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:02 smithi026 ceph-mon[32041]: pgmap v1005: 129 pgs: 129 active+clean; 555 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 499 op/s 2024-06-13T07:17:02.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:02 smithi135 ceph-mon[36276]: pgmap v1005: 129 pgs: 129 active+clean; 555 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 499 op/s 2024-06-13T07:17:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:02 smithi136 ceph-mon[37229]: pgmap v1005: 129 pgs: 129 active+clean; 555 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 499 op/s 2024-06-13T07:17:03.164 INFO:tasks.workunit.client.0.smithi026.stdout: 10 105707 29.58 MB/sec execute 76 sec latency 45.965 ms 2024-06-13T07:17:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.986+0000 7fedf3600640 1 -- 172.21.15.26:0/3036915084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 msgr2=0x7fedec10add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.986+0000 7fedf3600640 1 --2- 172.21.15.26:0/3036915084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 0x7fedec10add0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fede0009900 tx=0x7fede002a410 comp rx=0 tx=0).stop 2024-06-13T07:17:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.987+0000 7fedf3600640 1 -- 172.21.15.26:0/3036915084 shutdown_connections 2024-06-13T07:17:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.987+0000 7fedf3600640 1 --2- 172.21.15.26:0/3036915084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fedec10b310 0x7fedec10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.987+0000 7fedf3600640 1 --2- 172.21.15.26:0/3036915084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedec10dcc0 0x7fedec10e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.987+0000 7fedf3600640 1 --2- 172.21.15.26:0/3036915084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 0x7fedec10add0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.987+0000 7fedf3600640 1 -- 172.21.15.26:0/3036915084 >> 172.21.15.26:0/3036915084 conn(0x7fedec06ea10 msgr2=0x7fedec070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.987+0000 7fedf3600640 1 -- 172.21.15.26:0/3036915084 shutdown_connections 2024-06-13T07:17:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.988+0000 7fedf3600640 1 -- 172.21.15.26:0/3036915084 wait complete. 2024-06-13T07:17:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.989+0000 7fedf3600640 1 Processor -- start 2024-06-13T07:17:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.989+0000 7fedf3600640 1 -- start start 2024-06-13T07:17:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.990+0000 7fedf3600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 0x7fedec1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.990+0000 7fedf3600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedec10b310 0x7fedec1a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.990+0000 7fedf3600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fedec10dcc0 0x7fedec1adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.990+0000 7fedf3600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fedec11c4a0 con 0x7fedec10b310 2024-06-13T07:17:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.990+0000 7fedf3600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fedec11c320 con 0x7fedec10dcc0 2024-06-13T07:17:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.990+0000 7fedf3600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fedec11c620 con 0x7fedec1089a0 2024-06-13T07:17:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.991+0000 7fedf1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fedec10dcc0 0x7fedec1adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.991+0000 7fedf1600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fedec10dcc0 0x7fedec1adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43572/0 (socket says 172.21.15.26:43572) 2024-06-13T07:17:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.991+0000 7fedf1600640 1 -- 172.21.15.26:0/4275950191 learned_addr learned my addr 172.21.15.26:0/4275950191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.991+0000 7fedf0c00640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 0x7fedec1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.992+0000 7fedf1600640 1 -- 172.21.15.26:0/4275950191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 msgr2=0x7fedec1a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.992+0000 7fedf1600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 0x7fedec1a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.992+0000 7fedf1600640 1 -- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedec10b310 msgr2=0x7fedec1a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.992+0000 7fedf1600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedec10b310 0x7fedec1a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.992+0000 7fedf1600640 1 -- 172.21.15.26:0/4275950191 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec1ae620 con 0x7fedec10dcc0 2024-06-13T07:17:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.992+0000 7fedf0c00640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 0x7fedec1a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:17:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.993+0000 7fedf1600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fedec10dcc0 0x7fedec1adfb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fede400eb10 tx=0x7fede400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.993+0000 7fede9600640 1 -- 172.21.15.26:0/4275950191 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4004b20 con 0x7fedec10dcc0 2024-06-13T07:17:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.993+0000 7fede9600640 1 -- 172.21.15.26:0/4275950191 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede4004cc0 con 0x7fedec10dcc0 2024-06-13T07:17:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.993+0000 7fede9600640 1 -- 172.21.15.26:0/4275950191 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4010760 con 0x7fedec10dcc0 2024-06-13T07:17:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.995+0000 7fedf3600640 1 -- 172.21.15.26:0/4275950191 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec1ae910 con 0x7fedec10dcc0 2024-06-13T07:17:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:03.995+0000 7fedf3600640 1 -- 172.21.15.26:0/4275950191 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec1aede0 con 0x7fedec10dcc0 2024-06-13T07:17:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.002+0000 7fedf65e8740 1 Processor -- start 2024-06-13T07:17:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.003+0000 7fedf65e8740 1 -- start start 2024-06-13T07:17:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.003+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a388020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.003+0000 7fede9600640 1 -- 172.21.15.26:0/4275950191 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fede4005000 con 0x7fedec10dcc0 2024-06-13T07:17:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.003+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a38d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.003+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 0x55fd0a38fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.003+0000 7fedf65e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fd0a2eb420 con 0x55fd0a3885f0 2024-06-13T07:17:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.003+0000 7fedf65e8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fd0a292c50 con 0x55fd0a387c00 2024-06-13T07:17:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.003+0000 7fedf65e8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fd0a2e73e0 con 0x55fd0a38d760 2024-06-13T07:17:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.004+0000 7fedebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a388020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.004+0000 7fedebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a388020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43588/0 (socket says 172.21.15.26:43588) 2024-06-13T07:17:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.004+0000 7fedebe00640 1 -- 172.21.15.26:0/83107440 learned_addr learned my addr 172.21.15.26:0/83107440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.004+0000 7fedebe00640 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 0x55fd0a38fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.004+0000 7fedebe00640 1 -- 172.21.15.26:0/83107440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 msgr2=0x55fd0a38fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.005+0000 7fedebe00640 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 0x55fd0a38fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.005+0000 7fedebe00640 1 -- 172.21.15.26:0/83107440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 msgr2=0x55fd0a38d0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:17:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.005+0000 7fedf0c00640 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a38d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.005+0000 7fedebe00640 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a38d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.005+0000 7fedebe00640 1 -- 172.21.15.26:0/83107440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd09fff210 con 0x55fd0a387c00 2024-06-13T07:17:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.005+0000 7fedf0c00640 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a38d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:17:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.006+0000 7fedebe00640 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a388020 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fedd80137d0 tx=0x7fedd80096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f1401ead8c48285 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.006+0000 7fedddc00640 1 -- 172.21.15.26:0/83107440 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8005220 con 0x55fd0a387c00 2024-06-13T07:17:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.006+0000 7fedddc00640 1 -- 172.21.15.26:0/83107440 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd8013e10 con 0x55fd0a387c00 2024-06-13T07:17:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.006+0000 7fedddc00640 1 -- 172.21.15.26:0/83107440 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd800c930 con 0x55fd0a387c00 2024-06-13T07:17:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.006+0000 7fede9600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fedd40a2d30 0x7fedd40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.007+0000 7fedf0c00640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fedd40a2d30 0x7fedd40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.007+0000 7fede9600640 1 -- 172.21.15.26:0/4275950191 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fede40cfab0 con 0x7fedec10dcc0 2024-06-13T07:17:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.008+0000 7fedf65e8740 1 -- 172.21.15.26:0/83107440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 msgr2=0x55fd0a388020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.008+0000 7fedf65e8740 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a388020 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fedd80137d0 tx=0x7fedd80096e0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.008+0000 7fedf0c00640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fedd40a2d30 0x7fedd40a51f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fede0031440 tx=0x7fede00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.008+0000 7fedf65e8740 1 -- 172.21.15.26:0/83107440 shutdown_connections 2024-06-13T07:17:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.008+0000 7fedf65e8740 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 0x55fd0a38fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.009+0000 7fedf65e8740 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a38d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.009+0000 7fedf65e8740 1 --2- 172.21.15.26:0/83107440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a388020 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.009+0000 7fedf65e8740 1 -- 172.21.15.26:0/83107440 >> 172.21.15.26:0/83107440 conn(0x55fd0a34ffb0 msgr2=0x55fd0a383680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.010+0000 7fedf65e8740 1 -- 172.21.15.26:0/83107440 shutdown_connections 2024-06-13T07:17:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.010+0000 7fedf65e8740 1 -- 172.21.15.26:0/83107440 wait complete. 2024-06-13T07:17:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.011+0000 7fedf65e8740 1 -- start start 2024-06-13T07:17:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.011+0000 7fedf65e8740 1 Processor -- start 2024-06-13T07:17:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.012+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a391f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a392450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 0x55fd0a399490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf65e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fd0a3906b0 con 0x55fd0a3885f0 2024-06-13T07:17:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a391f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a391f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43592/0 (socket says 172.21.15.26:43592) 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf0c00640 1 -- 172.21.15.26:0/477598379 learned_addr learned my addr 172.21.15.26:0/477598379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fd0a2a60b0 con 0x55fd0a387c00 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fd0a390830 con 0x55fd0a38d760 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedf65e8740 10 client.0 _ll_register_callbacks cb 0x55fd0a291f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.013+0000 7fedebe00640 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a392450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.014+0000 7fedf0c00640 1 -- 172.21.15.26:0/477598379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 msgr2=0x55fd0a399490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.014+0000 7fedf0c00640 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 0x55fd0a399490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.014+0000 7fedf0c00640 1 -- 172.21.15.26:0/477598379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 msgr2=0x55fd0a392450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.014+0000 7fedf0c00640 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a392450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.014+0000 7fedf0c00640 1 -- 172.21.15.26:0/477598379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0a0cb620 con 0x55fd0a387c00 2024-06-13T07:17:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.014+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd8013e10 con 0x55fd0a387c00 2024-06-13T07:17:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.014+0000 7fedf0c00640 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a391f10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fede00314e0 tx=0x7fede0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a4b1e8b4a76738 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.014+0000 7fedd2000640 10 client.26729 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:17:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.015+0000 7fedd2000640 1 -- 172.21.15.26:0/477598379 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0057050 con 0x55fd0a387c00 2024-06-13T07:17:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.015+0000 7fedd2000640 1 -- 172.21.15.26:0/477598379 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fede0004b80 con 0x55fd0a387c00 2024-06-13T07:17:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.015+0000 7fedd2000640 1 -- 172.21.15.26:0/477598379 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fede004d070 con 0x55fd0a387c00 2024-06-13T07:17:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.015+0000 7fedf65e8740 10 client.26729 fetch_fsmap learned FSMap version 47 2024-06-13T07:17:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.015+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=47}) -- 0x55fd0a39c730 con 0x55fd0a387c00 2024-06-13T07:17:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.015+0000 7fedd2000640 1 -- 172.21.15.26:0/477598379 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fede00316b0 con 0x55fd0a387c00 2024-06-13T07:17:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.016+0000 7fedf65e8740 10 client.26729 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:17:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.016+0000 7fedf65e8740 10 client.26729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:17:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.016+0000 7fedf65e8740 20 client.26729 populate_metadata read hostname 'smithi026' 2024-06-13T07:17:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.016+0000 7fedf65e8740 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fd0a39dd30 0x55fd0a3a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.016+0000 7fedf65e8740 4 client.26729 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:17:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.016+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd0a2f8b50 con 0x55fd0a39dd30 2024-06-13T07:17:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.017+0000 7fedebe00640 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fd0a39dd30 0x55fd0a3a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.018+0000 7fedebe00640 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fd0a39dd30 0x55fd0a3a01c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc99801fb105c5a server_cookie=b6ec5fc7d55bdde in_seq=0 out_seq=0 2024-06-13T07:17:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.018+0000 7fedd2000640 10 client.26729 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.022+0000 7fedd2000640 1 -- 172.21.15.26:0/477598379 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fd0a2f8b50 con 0x55fd0a39dd30 2024-06-13T07:17:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.022+0000 7fedd2000640 10 client.26729 handle_command_reply: tid=0 2024-06-13T07:17:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.022+0000 7fedf65e8740 1 client.26729 shutdown 2024-06-13T07:17:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 msgr2=0x55fd0a391f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedf65e8740 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a391f10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fede00314e0 tx=0x7fede0009260 comp rx=0 tx=0).stop 2024-06-13T07:17:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 shutdown_connections 2024-06-13T07:17:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedf65e8740 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fd0a39dd30 0x55fd0a3a01c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedf65e8740 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a38d760 0x55fd0a399490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedf65e8740 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a3885f0 0x55fd0a392450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedf65e8740 1 --2- 172.21.15.26:0/477598379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a391f10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedd2000640 0 client.26729 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.023+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 >> 172.21.15.26:0/477598379 conn(0x55fd0a34ffb0 msgr2=0x55fd0a382840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.024+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 shutdown_connections 2024-06-13T07:17:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.024+0000 7fedf65e8740 1 -- 172.21.15.26:0/477598379 wait complete. 2024-06-13T07:17:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.024+0000 7fedf65e8740 20 client.26729 trim_cache size 0 max 16384 2024-06-13T07:17:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.054+0000 7fedf65e8740 1 Processor -- start 2024-06-13T07:17:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.054+0000 7fedf65e8740 1 -- start start 2024-06-13T07:17:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.055+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a386610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.055+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a399450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.055+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a38d760 0x55fd0a386b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.055+0000 7fedf65e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1bbdb0 con 0x55fd0a38d760 2024-06-13T07:17:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.055+0000 7fedf65e8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b9680 con 0x55fd0a387c00 2024-06-13T07:17:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.055+0000 7fedf65e8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b74d0 con 0x55fd0a3885f0 2024-06-13T07:17:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.055+0000 7fedebe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a399450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedebe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a399450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52094/0 (socket says 172.21.15.26:52094) 2024-06-13T07:17:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedf0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a386610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedf0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a386610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43598/0 (socket says 172.21.15.26:43598) 2024-06-13T07:17:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedf0c00640 1 -- 172.21.15.26:0/475133770 learned_addr learned my addr 172.21.15.26:0/475133770 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedf1600640 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a38d760 0x55fd0a386b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedebe00640 1 -- 172.21.15.26:0/475133770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 msgr2=0x55fd0a386610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedebe00640 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a386610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedebe00640 1 -- 172.21.15.26:0/475133770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a38d760 msgr2=0x55fd0a386b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedebe00640 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a38d760 0x55fd0a386b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.056+0000 7fedebe00640 1 -- 172.21.15.26:0/475133770 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0a37f020 con 0x55fd0a3885f0 2024-06-13T07:17:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.057+0000 7fedf0c00640 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a386610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:17:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.057+0000 7fedf1600640 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a38d760 0x55fd0a386b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:17:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.057+0000 7fedebe00640 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a399450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fedd80137a0 tx=0x7fedd8005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5b3c68ec4afa2e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.057+0000 7feddd200640 1 -- 172.21.15.26:0/475133770 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd802d070 con 0x55fd0a3885f0 2024-06-13T07:17:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.058+0000 7feddd200640 1 -- 172.21.15.26:0/475133770 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd80054e0 con 0x55fd0a3885f0 2024-06-13T07:17:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.058+0000 7feddd200640 1 -- 172.21.15.26:0/475133770 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004a00 con 0x55fd0a3885f0 2024-06-13T07:17:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 -- 172.21.15.26:0/475133770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 msgr2=0x55fd0a399450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a399450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fedd80137a0 tx=0x7fedd8005270 comp rx=0 tx=0).stop 2024-06-13T07:17:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 -- 172.21.15.26:0/475133770 shutdown_connections 2024-06-13T07:17:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a38d760 0x55fd0a386b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a399450 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 --2- 172.21.15.26:0/475133770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a387c00 0x55fd0a386610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 -- 172.21.15.26:0/475133770 >> 172.21.15.26:0/475133770 conn(0x55fd0a34ffb0 msgr2=0x55fd0a383100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 -- 172.21.15.26:0/475133770 shutdown_connections 2024-06-13T07:17:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.059+0000 7fedf65e8740 1 -- 172.21.15.26:0/475133770 wait complete. 2024-06-13T07:17:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.060+0000 7fedf65e8740 1 -- start start 2024-06-13T07:17:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.060+0000 7fedf65e8740 1 Processor -- start 2024-06-13T07:17:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.061+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a387c00 0x55fd0a394c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.061+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a395170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.061+0000 7fedf65e8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a38d760 0x55fd0a398190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.061+0000 7fedf65e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b7b40 con 0x55fd0a387c00 2024-06-13T07:17:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.061+0000 7fedf65e8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b8fb0 con 0x55fd0a38d760 2024-06-13T07:17:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.061+0000 7fedf65e8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b8210 con 0x55fd0a3885f0 2024-06-13T07:17:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.061+0000 7fedf65e8740 10 client.0 _ll_register_callbacks cb 0x7fedd8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:17:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.063+0000 7fedebe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a395170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.063+0000 7fedebe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a395170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52102/0 (socket says 172.21.15.26:52102) 2024-06-13T07:17:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.063+0000 7fedebe00640 1 -- 172.21.15.26:0/1312081351 learned_addr learned my addr 172.21.15.26:0/1312081351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.063+0000 7fedf1600640 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a38d760 0x55fd0a398190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.063+0000 7fedf0c00640 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a387c00 0x55fd0a394c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.064+0000 7fedf1600640 1 -- 172.21.15.26:0/1312081351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 msgr2=0x55fd0a395170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.065+0000 7fedf1600640 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a395170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.065+0000 7fedf1600640 1 -- 172.21.15.26:0/1312081351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a387c00 msgr2=0x55fd0a394c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.065+0000 7fedf1600640 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a387c00 0x55fd0a394c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.065+0000 7fedf1600640 1 -- 172.21.15.26:0/1312081351 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0a398780 con 0x55fd0a38d760 2024-06-13T07:17:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.066+0000 7fedf1600640 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a38d760 0x55fd0a398190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fede40049d0 tx=0x7fede409bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa4c7ec85f13370 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.066+0000 7fedd2000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:17:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.066+0000 7fedebe00640 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a395170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:17:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.067+0000 7fedd2000640 1 -- 172.21.15.26:0/1312081351 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede409bec0 con 0x55fd0a38d760 2024-06-13T07:17:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.067+0000 7fedf65e8740 1 -- 172.21.15.26:0/1312081351 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd80054e0 con 0x55fd0a38d760 2024-06-13T07:17:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.067+0000 7fedd2000640 1 -- 172.21.15.26:0/1312081351 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fede40052f0 con 0x55fd0a38d760 2024-06-13T07:17:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.068+0000 7fedd2000640 1 -- 172.21.15.26:0/1312081351 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fede40a0270 con 0x55fd0a38d760 2024-06-13T07:17:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.069+0000 7fedf65e8740 10 client.26735 fetch_fsmap learned FSMap version 47 2024-06-13T07:17:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.069+0000 7fedf65e8740 1 -- 172.21.15.26:0/1312081351 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55fd0a419ba0 con 0x55fd0a38d760 2024-06-13T07:17:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.069+0000 7fedd2000640 1 -- 172.21.15.26:0/1312081351 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede40a4b30 con 0x55fd0a38d760 2024-06-13T07:17:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.070+0000 7fedd2000640 1 -- 172.21.15.26:0/1312081351 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fede4097ca0 con 0x55fd0a38d760 2024-06-13T07:17:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.070+0000 7fedf65e8740 10 client.26735 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:17:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.070+0000 7fedf65e8740 10 client.26735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:17:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.070+0000 7fedf65e8740 20 client.26735 populate_metadata read hostname 'smithi026' 2024-06-13T07:17:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.070+0000 7fedf65e8740 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fd0a41a440 0x55fd0a41c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.070+0000 7fedf65e8740 4 client.26735 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:17:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.070+0000 7fedf65e8740 1 -- 172.21.15.26:0/1312081351 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fedd80054e0 con 0x55fd0a41a440 2024-06-13T07:17:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.072+0000 7fedf0c00640 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fd0a41a440 0x55fd0a41c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.075+0000 7fedd2000640 10 client.26735 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.075+0000 7fedf0c00640 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fd0a41a440 0x55fd0a41c830 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f04c346674eedf1 server_cookie=ce6a4bbb8f7e3aeb in_seq=0 out_seq=0 2024-06-13T07:17:04.164 INFO:tasks.workunit.client.0.smithi026.stdout: 10 106404 29.74 MB/sec execute 77 sec latency 114.684 ms 2024-06-13T07:17:04.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:04 smithi026 ceph-mon[32041]: pgmap v1006: 129 pgs: 129 active+clean; 561 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.4 MiB/s wr, 549 op/s 2024-06-13T07:17:04.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:04 smithi135 ceph-mon[36276]: pgmap v1006: 129 pgs: 129 active+clean; 561 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.4 MiB/s wr, 549 op/s 2024-06-13T07:17:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:04 smithi136 ceph-mon[37229]: pgmap v1006: 129 pgs: 129 active+clean; 561 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.4 MiB/s wr, 549 op/s 2024-06-13T07:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.653+0000 7fedd2000640 1 -- 172.21.15.26:0/1312081351 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fedd80054e0 con 0x55fd0a41a440 2024-06-13T07:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.653+0000 7fedd2000640 10 client.26735 handle_command_reply: tid=0 2024-06-13T07:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.653+0000 7fedf65e8740 1 client.26735 shutdown 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedf65e8740 1 -- 172.21.15.26:0/1312081351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a38d760 msgr2=0x55fd0a398190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedf65e8740 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a38d760 0x55fd0a398190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fede40049d0 tx=0x7fede409bd80 comp rx=0 tx=0).stop 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedf65e8740 1 -- 172.21.15.26:0/1312081351 shutdown_connections 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedf65e8740 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fd0a41a440 0x55fd0a41c830 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedf65e8740 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd0a38d760 0x55fd0a398190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedf65e8740 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0a3885f0 0x55fd0a395170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedf65e8740 1 --2- 172.21.15.26:0/1312081351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0a387c00 0x55fd0a394c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedf65e8740 1 -- 172.21.15.26:0/1312081351 >> 172.21.15.26:0/1312081351 conn(0x55fd0a34ffb0 msgr2=0x55fd0a391b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.654+0000 7fedd2000640 0 client.26735 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.655+0000 7fedf65e8740 1 -- 172.21.15.26:0/1312081351 shutdown_connections 2024-06-13T07:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.655+0000 7fedf65e8740 1 -- 172.21.15.26:0/1312081351 wait complete. 2024-06-13T07:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.655+0000 7fedf65e8740 20 client.26735 trim_cache size 0 max 16384 2024-06-13T07:17:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:17:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:17:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:17:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.657+0000 7fedf3600640 1 -- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fedd40a2d30 msgr2=0x7fedd40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.657+0000 7fedf3600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fedd40a2d30 0x7fedd40a51f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fede0031440 tx=0x7fede00315b0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.657+0000 7fedf3600640 1 -- 172.21.15.26:0/4275950191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fedec10dcc0 msgr2=0x7fedec1adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.657+0000 7fedf3600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fedec10dcc0 0x7fedec1adfb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fede400eb10 tx=0x7fede400c2d0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.658+0000 7fedf3600640 1 -- 172.21.15.26:0/4275950191 shutdown_connections 2024-06-13T07:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.658+0000 7fedf3600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fedd40a2d30 0x7fedd40a51f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.658+0000 7fedf3600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fedec10dcc0 0x7fedec1adfb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.658+0000 7fedf3600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedec10b310 0x7fedec1a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.658+0000 7fedf3600640 1 --2- 172.21.15.26:0/4275950191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedec1089a0 0x7fedec1a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.658+0000 7fedf3600640 1 -- 172.21.15.26:0/4275950191 >> 172.21.15.26:0/4275950191 conn(0x7fedec06ea10 msgr2=0x7fedec10d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.658+0000 7fedf3600640 1 -- 172.21.15.26:0/4275950191 shutdown_connections 2024-06-13T07:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:04.658+0000 7fedf3600640 1 -- 172.21.15.26:0/4275950191 wait complete. 2024-06-13T07:17:05.165 INFO:tasks.workunit.client.0.smithi026.stdout: 10 106897 29.63 MB/sec execute 78 sec latency 555.328 ms 2024-06-13T07:17:06.164 INFO:tasks.workunit.client.0.smithi026.stdout: 10 107469 29.72 MB/sec execute 79 sec latency 125.914 ms 2024-06-13T07:17:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:06 smithi026 ceph-mon[32041]: pgmap v1007: 129 pgs: 129 active+clean; 568 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:17:06.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:06 smithi135 ceph-mon[36276]: pgmap v1007: 129 pgs: 129 active+clean; 568 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:17:06.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:06 smithi136 ceph-mon[37229]: pgmap v1007: 129 pgs: 129 active+clean; 568 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:17:07.165 INFO:tasks.workunit.client.0.smithi026.stdout: 10 108178 29.90 MB/sec execute 80 sec latency 75.740 ms 2024-06-13T07:17:08.165 INFO:tasks.workunit.client.0.smithi026.stdout: 10 108717 29.81 MB/sec execute 81 sec latency 86.123 ms 2024-06-13T07:17:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:08 smithi026 ceph-mon[32041]: pgmap v1008: 129 pgs: 129 active+clean; 569 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-06-13T07:17:08.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:08 smithi135 ceph-mon[36276]: pgmap v1008: 129 pgs: 129 active+clean; 569 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-06-13T07:17:08.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:08 smithi136 ceph-mon[37229]: pgmap v1008: 129 pgs: 129 active+clean; 569 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-06-13T07:17:09.165 INFO:tasks.workunit.client.0.smithi026.stdout: 10 108744 29.45 MB/sec execute 82 sec latency 1049.396 ms 2024-06-13T07:17:10.165 INFO:tasks.workunit.client.0.smithi026.stdout: 10 108883 29.22 MB/sec execute 83 sec latency 1721.206 ms 2024-06-13T07:17:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:10 smithi026 ceph-mon[32041]: pgmap v1009: 129 pgs: 129 active+clean; 594 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-06-13T07:17:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:10 smithi135 ceph-mon[36276]: pgmap v1009: 129 pgs: 129 active+clean; 594 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-06-13T07:17:10.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:10 smithi136 ceph-mon[37229]: pgmap v1009: 129 pgs: 129 active+clean; 594 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-06-13T07:17:11.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 109147 29.03 MB/sec execute 84 sec latency 142.153 ms 2024-06-13T07:17:12.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 109519 28.95 MB/sec execute 85 sec latency 149.121 ms 2024-06-13T07:17:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:12 smithi026 ceph-mon[32041]: pgmap v1010: 129 pgs: 129 active+clean; 607 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.4 MiB/s wr, 543 op/s 2024-06-13T07:17:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:12 smithi135 ceph-mon[36276]: pgmap v1010: 129 pgs: 129 active+clean; 607 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.4 MiB/s wr, 543 op/s 2024-06-13T07:17:12.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:12 smithi136 ceph-mon[37229]: pgmap v1010: 129 pgs: 129 active+clean; 607 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.4 MiB/s wr, 543 op/s 2024-06-13T07:17:13.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 110072 28.94 MB/sec execute 86 sec latency 60.697 ms 2024-06-13T07:17:14.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 110765 29.07 MB/sec execute 87 sec latency 73.978 ms 2024-06-13T07:17:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:14 smithi026 ceph-mon[32041]: pgmap v1011: 129 pgs: 129 active+clean; 614 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.7 MiB/s wr, 655 op/s 2024-06-13T07:17:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:14 smithi135 ceph-mon[36276]: pgmap v1011: 129 pgs: 129 active+clean; 614 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.7 MiB/s wr, 655 op/s 2024-06-13T07:17:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:14 smithi136 ceph-mon[37229]: pgmap v1011: 129 pgs: 129 active+clean; 614 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.7 MiB/s wr, 655 op/s 2024-06-13T07:17:15.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 111276 29.01 MB/sec execute 88 sec latency 135.644 ms 2024-06-13T07:17:16.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 111529 28.82 MB/sec execute 89 sec latency 422.875 ms 2024-06-13T07:17:16.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:16 smithi026 ceph-mon[32041]: pgmap v1012: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-06-13T07:17:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:16 smithi135 ceph-mon[36276]: pgmap v1012: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-06-13T07:17:16.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:16 smithi136 ceph-mon[37229]: pgmap v1012: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-06-13T07:17:17.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 111773 28.65 MB/sec execute 90 sec latency 368.178 ms 2024-06-13T07:17:18.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 112261 28.65 MB/sec execute 91 sec latency 276.413 ms 2024-06-13T07:17:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:18 smithi135 ceph-mon[36276]: pgmap v1013: 129 pgs: 129 active+clean; 575 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-06-13T07:17:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:18 smithi136 ceph-mon[37229]: pgmap v1013: 129 pgs: 129 active+clean; 575 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-06-13T07:17:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:18 smithi026 ceph-mon[32041]: pgmap v1013: 129 pgs: 129 active+clean; 575 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-06-13T07:17:19.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 112939 28.75 MB/sec execute 92 sec latency 78.919 ms 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 -- 172.21.15.26:0/3330756760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738109220 msgr2=0x7fc7381096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 --2- 172.21.15.26:0/3330756760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738109220 0x7fc7381096a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc724009900 tx=0x7fc72402a3f0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 -- 172.21.15.26:0/3330756760 shutdown_connections 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 --2- 172.21.15.26:0/3330756760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc738109be0 0x7fc738110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 --2- 172.21.15.26:0/3330756760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738109220 0x7fc7381096a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 --2- 172.21.15.26:0/3330756760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc738108030 0x7fc738108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 -- 172.21.15.26:0/3330756760 >> 172.21.15.26:0/3330756760 conn(0x7fc738076ff0 msgr2=0x7fc738079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 -- 172.21.15.26:0/3330756760 shutdown_connections 2024-06-13T07:17:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.178+0000 7fc73da00640 1 -- 172.21.15.26:0/3330756760 wait complete. 2024-06-13T07:17:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.180+0000 7fc73da00640 1 Processor -- start 2024-06-13T07:17:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.180+0000 7fc73da00640 1 -- start start 2024-06-13T07:17:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.180+0000 7fc73da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738108030 0x7fc73819e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.180+0000 7fc73da00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc738109220 0x7fc73819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.180+0000 7fc73da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc738109be0 0x7fc7381a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.180+0000 7fc73da00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc738113d10 con 0x7fc738109be0 2024-06-13T07:17:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.180+0000 7fc73da00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc738113b90 con 0x7fc738109220 2024-06-13T07:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.180+0000 7fc73da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc738113e90 con 0x7fc738108030 2024-06-13T07:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738108030 0x7fc73819e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc738109220 0x7fc73819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738108030 0x7fc73819e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50624/0 (socket says 172.21.15.26:50624) 2024-06-13T07:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736a00640 1 -- 172.21.15.26:0/1130154070 learned_addr learned my addr 172.21.15.26:0/1130154070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736a00640 1 -- 172.21.15.26:0/1130154070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc738109220 msgr2=0x7fc73819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736a00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc738109220 0x7fc73819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736a00640 1 -- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc738109be0 msgr2=0x7fc7381a57f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:17:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736a00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc738109be0 0x7fc7381a57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736a00640 1 -- 172.21.15.26:0/1130154070 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7381a5e60 con 0x7fc738108030 2024-06-13T07:17:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.181+0000 7fc736000640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc738109220 0x7fc73819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:17:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.182+0000 7fc736a00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738108030 0x7fc73819e240 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc72800d770 tx=0x7fc72800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.182+0000 7fc72f400640 1 -- 172.21.15.26:0/1130154070 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728017070 con 0x7fc738108030 2024-06-13T07:17:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.182+0000 7fc72f400640 1 -- 172.21.15.26:0/1130154070 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc728009c30 con 0x7fc738108030 2024-06-13T07:17:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.182+0000 7fc72f400640 1 -- 172.21.15.26:0/1130154070 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728003dd0 con 0x7fc738108030 2024-06-13T07:17:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.182+0000 7fc73da00640 1 -- 172.21.15.26:0/1130154070 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7381a6150 con 0x7fc738108030 2024-06-13T07:17:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.183+0000 7fc73da00640 1 -- 172.21.15.26:0/1130154070 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7381a6620 con 0x7fc738108030 2024-06-13T07:17:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.186+0000 7fc72f400640 1 -- 172.21.15.26:0/1130154070 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc728003f70 con 0x7fc738108030 2024-06-13T07:17:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.188+0000 7fc7407ea740 1 Processor -- start 2024-06-13T07:17:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.188+0000 7fc7407ea740 1 -- start start 2024-06-13T07:17:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.188+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f41020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.188+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.188+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f48bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.188+0000 7fc7407ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b694ea4420 con 0x55b694f415f0 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.188+0000 7fc7407ea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b694e4bc50 con 0x55b694f46760 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.188+0000 7fc7407ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b694ea03e0 con 0x55b694f40c00 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.189+0000 7fc737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f41020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.189+0000 7fc72f400640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc71c0a2cb0 0x7fc71c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.189+0000 7fc737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f41020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50634/0 (socket says 172.21.15.26:50634) 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.189+0000 7fc737400640 1 -- 172.21.15.26:0/4248436468 learned_addr learned my addr 172.21.15.26:0/4248436468 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.189+0000 7fc736000640 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f48bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.189+0000 7fc72f400640 1 -- 172.21.15.26:0/1130154070 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc7280cedc0 con 0x7fc738108030 2024-06-13T07:17:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.189+0000 7fc736a00640 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.190+0000 7fc736000640 1 -- 172.21.15.26:0/4248436468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 msgr2=0x55b694f41020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.190+0000 7fc736000640 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f41020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.190+0000 7fc736000640 1 -- 172.21.15.26:0/4248436468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 msgr2=0x55b694f460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.190+0000 7fc737400640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc71c0a2cb0 0x7fc71c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.190+0000 7fc736000640 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.190+0000 7fc736000640 1 -- 172.21.15.26:0/4248436468 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b694bb8210 con 0x55b694f46760 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.190+0000 7fc736a00640 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.190+0000 7fc736000640 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f48bf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc72402a910 tx=0x7fc724004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f772399d7a840ffb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.191+0000 7fc723e00640 1 -- 172.21.15.26:0/4248436468 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724035040 con 0x55b694f46760 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.191+0000 7fc737400640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc71c0a2cb0 0x7fc71c0a5170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b694f41e50 tx=0x7fc730004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.191+0000 7fc723e00640 1 -- 172.21.15.26:0/4248436468 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc724004c50 con 0x55b694f46760 2024-06-13T07:17:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.191+0000 7fc723e00640 1 -- 172.21.15.26:0/4248436468 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724031c10 con 0x55b694f46760 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 -- 172.21.15.26:0/4248436468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 msgr2=0x55b694f48bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f48bf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc72402a910 tx=0x7fc724004530 comp rx=0 tx=0).stop 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 -- 172.21.15.26:0/4248436468 shutdown_connections 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f48bf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 --2- 172.21.15.26:0/4248436468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 -- 172.21.15.26:0/4248436468 >> 172.21.15.26:0/4248436468 conn(0x55b694f08fb0 msgr2=0x55b694f3c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 -- 172.21.15.26:0/4248436468 shutdown_connections 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.192+0000 7fc7407ea740 1 -- 172.21.15.26:0/4248436468 wait complete. 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.193+0000 7fc7407ea740 1 -- start start 2024-06-13T07:17:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.193+0000 7fc7407ea740 1 Processor -- start 2024-06-13T07:17:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f415f0 0x55b694f4b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f46760 0x55b694f524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc7407ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b694f496b0 con 0x55b694f46760 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc7407ea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b694e5f0b0 con 0x55b694f40c00 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc7407ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b694f49830 con 0x55b694f415f0 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc7407ea740 10 client.0 _ll_register_callbacks cb 0x55b694e4af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc737400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f46760 0x55b694f524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc737400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f46760 0x55b694f524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57084/0 (socket says 172.21.15.26:57084) 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.194+0000 7fc737400640 1 -- 172.21.15.26:0/1920256558 learned_addr learned my addr 172.21.15.26:0/1920256558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.195+0000 7fc736000640 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f415f0 0x55b694f4b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.195+0000 7fc736a00640 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.195+0000 7fc736000640 1 -- 172.21.15.26:0/1920256558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 msgr2=0x55b694f4af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.195+0000 7fc736000640 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.195+0000 7fc736000640 1 -- 172.21.15.26:0/1920256558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f46760 msgr2=0x55b694f524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.195+0000 7fc736000640 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f46760 0x55b694f524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.195+0000 7fc736000640 1 -- 172.21.15.26:0/1920256558 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b694c84620 con 0x55b694f415f0 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.196+0000 7fc736000640 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f415f0 0x55b694f4b480 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc7240314e0 tx=0x7fc7240049a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5212385b1000bc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.196+0000 7fc720c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.196+0000 7fc720c00640 1 -- 172.21.15.26:0/1920256558 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724035040 con 0x55b694f415f0 2024-06-13T07:17:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.196+0000 7fc720c00640 1 -- 172.21.15.26:0/1920256558 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc724031990 con 0x55b694f415f0 2024-06-13T07:17:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.196+0000 7fc7407ea740 1 -- 172.21.15.26:0/1920256558 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc724004c50 con 0x55b694f415f0 2024-06-13T07:17:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.197+0000 7fc720c00640 1 -- 172.21.15.26:0/1920256558 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc724009520 con 0x55b694f415f0 2024-06-13T07:17:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.197+0000 7fc7407ea740 10 client.26767 fetch_fsmap learned FSMap version 47 2024-06-13T07:17:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.197+0000 7fc7407ea740 1 -- 172.21.15.26:0/1920256558 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b694f49700 con 0x55b694f415f0 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.197+0000 7fc720c00640 1 -- 172.21.15.26:0/1920256558 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc724005610 con 0x55b694f415f0 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.197+0000 7fc720c00640 1 -- 172.21.15.26:0/1920256558 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fc724055030 con 0x55b694f415f0 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.198+0000 7fc7407ea740 10 client.26767 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.198+0000 7fc7407ea740 10 client.26767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.198+0000 7fc7407ea740 20 client.26767 populate_metadata read hostname 'smithi026' 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.198+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b694f679e0 0x55b694f69dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.198+0000 7fc7407ea740 4 client.26767 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.198+0000 7fc7407ea740 1 -- 172.21.15.26:0/1920256558 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc724004c50 con 0x55b694f679e0 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.198+0000 7fc736a00640 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b694f679e0 0x55b694f69dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.199+0000 7fc720c00640 10 client.26767 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.199+0000 7fc736a00640 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b694f679e0 0x55b694f69dd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=311092f029fbff00 server_cookie=9592f7ae713c48e9 in_seq=0 out_seq=0 2024-06-13T07:17:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.201+0000 7fc720c00640 1 -- 172.21.15.26:0/1920256558 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fc724004c50 con 0x55b694f679e0 2024-06-13T07:17:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.201+0000 7fc720c00640 10 client.26767 handle_command_reply: tid=0 2024-06-13T07:17:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.201+0000 7fc7407ea740 1 client.26767 shutdown 2024-06-13T07:17:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.201+0000 7fc7407ea740 1 -- 172.21.15.26:0/1920256558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f415f0 msgr2=0x55b694f4b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.201+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f415f0 0x55b694f4b480 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc7240314e0 tx=0x7fc7240049a0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc7407ea740 1 -- 172.21.15.26:0/1920256558 shutdown_connections 2024-06-13T07:17:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b694f679e0 0x55b694f69dd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f46760 0x55b694f524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f415f0 0x55b694f4b480 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1920256558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc7407ea740 1 -- 172.21.15.26:0/1920256558 >> 172.21.15.26:0/1920256558 conn(0x55b694f08fb0 msgr2=0x55b694f3b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc720c00640 0 client.26767 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc7407ea740 1 -- 172.21.15.26:0/1920256558 shutdown_connections 2024-06-13T07:17:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.202+0000 7fc7407ea740 1 -- 172.21.15.26:0/1920256558 wait complete. 2024-06-13T07:17:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.204+0000 7fc7407ea740 20 client.26767 trim_cache size 0 max 16384 2024-06-13T07:17:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.230+0000 7fc7407ea740 1 Processor -- start 2024-06-13T07:17:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.230+0000 7fc7407ea740 1 -- start start 2024-06-13T07:17:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f4af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f52480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f4b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc7407ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381aad00 con 0x55b694f415f0 2024-06-13T07:17:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc7407ea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381a85d0 con 0x55b694f46760 2024-06-13T07:17:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc7407ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7380825a0 con 0x55b694f40c00 2024-06-13T07:17:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc736000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f52480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc737400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f4b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc736a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f4af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc737400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f4b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50534/0 (socket says 172.21.15.26:50534) 2024-06-13T07:17:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.231+0000 7fc737400640 1 -- 172.21.15.26:0/1965974595 learned_addr learned my addr 172.21.15.26:0/1965974595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc736a00640 1 -- 172.21.15.26:0/1965974595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 msgr2=0x55b694f4b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc736a00640 1 --2- 172.21.15.26:0/1965974595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f4b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc736a00640 1 -- 172.21.15.26:0/1965974595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 msgr2=0x55b694f52480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc736a00640 1 --2- 172.21.15.26:0/1965974595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f52480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc736a00640 1 -- 172.21.15.26:0/1965974595 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b694f6a400 con 0x55b694f40c00 2024-06-13T07:17:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc736a00640 1 --2- 172.21.15.26:0/1965974595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f4af40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc72809aed0 tx=0x7fc7280cc340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78471013220704 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc723400640 1 -- 172.21.15.26:0/1965974595 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728020040 con 0x55b694f40c00 2024-06-13T07:17:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc723400640 1 -- 172.21.15.26:0/1965974595 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7280a3e40 con 0x55b694f40c00 2024-06-13T07:17:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.232+0000 7fc723400640 1 -- 172.21.15.26:0/1965974595 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7280bb0b0 con 0x55b694f40c00 2024-06-13T07:17:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.233+0000 7fc7407ea740 1 -- 172.21.15.26:0/1965974595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 msgr2=0x55b694f4af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.233+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1965974595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f4af40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc72809aed0 tx=0x7fc7280cc340 comp rx=0 tx=0).stop 2024-06-13T07:17:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.233+0000 7fc7407ea740 1 -- 172.21.15.26:0/1965974595 shutdown_connections 2024-06-13T07:17:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.233+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1965974595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f46760 0x55b694f4b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.233+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1965974595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f52480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.233+0000 7fc7407ea740 1 --2- 172.21.15.26:0/1965974595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f40c00 0x55b694f4af40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.233+0000 7fc7407ea740 1 -- 172.21.15.26:0/1965974595 >> 172.21.15.26:0/1965974595 conn(0x55b694f08fb0 msgr2=0x55b694f3c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.234+0000 7fc7407ea740 1 -- 172.21.15.26:0/1965974595 shutdown_connections 2024-06-13T07:17:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.234+0000 7fc7407ea740 1 -- 172.21.15.26:0/1965974595 wait complete. 2024-06-13T07:17:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.234+0000 7fc7407ea740 1 -- start start 2024-06-13T07:17:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.234+0000 7fc7407ea740 1 Processor -- start 2024-06-13T07:17:19.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.235+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.236+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f4f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.236+0000 7fc7407ea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f46760 0x55b694fd05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.236+0000 7fc7407ea740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381a6a90 con 0x55b694f415f0 2024-06-13T07:17:19.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.236+0000 7fc7407ea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381a7f00 con 0x55b694f40c00 2024-06-13T07:17:19.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.236+0000 7fc7407ea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381a7160 con 0x55b694f46760 2024-06-13T07:17:19.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.237+0000 7fc7407ea740 10 client.0 _ll_register_callbacks cb 0x7fc7280046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:17:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.236+0000 7fc736a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.237+0000 7fc736a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50542/0 (socket says 172.21.15.26:50542) 2024-06-13T07:17:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.237+0000 7fc736a00640 1 -- 172.21.15.26:0/2036126158 learned_addr learned my addr 172.21.15.26:0/2036126158 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.237+0000 7fc737400640 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f46760 0x55b694fd05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.237+0000 7fc736000640 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f4f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.238+0000 7fc736a00640 1 -- 172.21.15.26:0/2036126158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f46760 msgr2=0x55b694fd05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.238+0000 7fc736a00640 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f46760 0x55b694fd05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.238+0000 7fc736a00640 1 -- 172.21.15.26:0/2036126158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 msgr2=0x55b694f4f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.238+0000 7fc736a00640 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f4f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.238+0000 7fc736a00640 1 -- 172.21.15.26:0/2036126158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b694fd0b60 con 0x55b694f40c00 2024-06-13T07:17:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.238+0000 7fc736a00640 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4f460 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc72800d740 tx=0x7fc7280cc3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552970f552c9a56b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.238+0000 7fc720c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:17:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.239+0000 7fc720c00640 1 -- 172.21.15.26:0/2036126158 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7280b3020 con 0x55b694f40c00 2024-06-13T07:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.239+0000 7fc720c00640 1 -- 172.21.15.26:0/2036126158 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc728097840 con 0x55b694f40c00 2024-06-13T07:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.239+0000 7fc7407ea740 1 -- 172.21.15.26:0/2036126158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7280a3e40 con 0x55b694f40c00 2024-06-13T07:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.240+0000 7fc720c00640 1 -- 172.21.15.26:0/2036126158 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc728020070 con 0x55b694f40c00 2024-06-13T07:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.240+0000 7fc7407ea740 10 client.26753 fetch_fsmap learned FSMap version 47 2024-06-13T07:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.240+0000 7fc7407ea740 1 -- 172.21.15.26:0/2036126158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b694fd1ee0 con 0x55b694f40c00 2024-06-13T07:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.240+0000 7fc720c00640 1 -- 172.21.15.26:0/2036126158 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7280a3e40 con 0x55b694f40c00 2024-06-13T07:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.241+0000 7fc720c00640 1 -- 172.21.15.26:0/2036126158 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fc72809f150 con 0x55b694f40c00 2024-06-13T07:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.241+0000 7fc7407ea740 10 client.26753 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.241+0000 7fc7407ea740 10 client.26753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.241+0000 7fc7407ea740 20 client.26753 populate_metadata read hostname 'smithi026' 2024-06-13T07:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.241+0000 7fc7407ea740 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b694fd2890 0x55b694fd4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.241+0000 7fc7407ea740 4 client.26753 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.242+0000 7fc7407ea740 1 -- 172.21.15.26:0/2036126158 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b694eb1b50 con 0x55b694fd2890 2024-06-13T07:17:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.242+0000 7fc736000640 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b694fd2890 0x55b694fd4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.243+0000 7fc720c00640 10 client.26753 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.243+0000 7fc736000640 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b694fd2890 0x55b694fd4cb0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fea6820ad0b5eee server_cookie=894269bd29509ff8 in_seq=0 out_seq=0 2024-06-13T07:17:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.474+0000 7fc720c00640 1 -- 172.21.15.26:0/2036126158 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b694eb1b50 con 0x55b694fd2890 2024-06-13T07:17:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.475+0000 7fc720c00640 10 client.26753 handle_command_reply: tid=0 2024-06-13T07:17:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.475+0000 7fc7407ea740 1 client.26753 shutdown 2024-06-13T07:17:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.476+0000 7fc7407ea740 1 -- 172.21.15.26:0/2036126158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 msgr2=0x55b694f4f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.476+0000 7fc7407ea740 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4f460 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc72800d740 tx=0x7fc7280cc3e0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.476+0000 7fc7407ea740 1 -- 172.21.15.26:0/2036126158 shutdown_connections 2024-06-13T07:17:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.476+0000 7fc7407ea740 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b694fd2890 0x55b694fd4cb0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.477+0000 7fc7407ea740 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b694f46760 0x55b694fd05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.477+0000 7fc7407ea740 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b694f415f0 0x55b694f4f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.477+0000 7fc7407ea740 1 --2- 172.21.15.26:0/2036126158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b694f40c00 0x55b694f4f460 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.477+0000 7fc7407ea740 1 -- 172.21.15.26:0/2036126158 >> 172.21.15.26:0/2036126158 conn(0x55b694f08fb0 msgr2=0x55b694f3f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.477+0000 7fc720c00640 0 client.26753 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.477+0000 7fc7407ea740 1 -- 172.21.15.26:0/2036126158 shutdown_connections 2024-06-13T07:17:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.477+0000 7fc7407ea740 1 -- 172.21.15.26:0/2036126158 wait complete. 2024-06-13T07:17:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.478+0000 7fc7407ea740 20 client.26753 trim_cache size 0 max 16384 2024-06-13T07:17:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:17:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:17:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:17:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.481+0000 7fc73da00640 1 -- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc71c0a2cb0 msgr2=0x7fc71c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.481+0000 7fc73da00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc71c0a2cb0 0x7fc71c0a5170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b694f41e50 tx=0x7fc730004a40 comp rx=0 tx=0).stop 2024-06-13T07:17:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.481+0000 7fc73da00640 1 -- 172.21.15.26:0/1130154070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738108030 msgr2=0x7fc73819e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.481+0000 7fc73da00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738108030 0x7fc73819e240 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc72800d770 tx=0x7fc72800dc30 comp rx=0 tx=0).stop 2024-06-13T07:17:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.482+0000 7fc73da00640 1 -- 172.21.15.26:0/1130154070 shutdown_connections 2024-06-13T07:17:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.482+0000 7fc73da00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fc71c0a2cb0 0x7fc71c0a5170 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.482+0000 7fc73da00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc738109220 0x7fc73819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.482+0000 7fc73da00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc738109be0 0x7fc7381a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.482+0000 7fc73da00640 1 --2- 172.21.15.26:0/1130154070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc738108030 0x7fc73819e240 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.482+0000 7fc73da00640 1 -- 172.21.15.26:0/1130154070 >> 172.21.15.26:0/1130154070 conn(0x7fc738076ff0 msgr2=0x7fc73810c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.482+0000 7fc73da00640 1 -- 172.21.15.26:0/1130154070 shutdown_connections 2024-06-13T07:17:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:19.482+0000 7fc73da00640 1 -- 172.21.15.26:0/1130154070 wait complete. 2024-06-13T07:17:20.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 113518 28.79 MB/sec execute 93 sec latency 154.802 ms 2024-06-13T07:17:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:20 smithi135 ceph-mon[36276]: pgmap v1014: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 804 op/s 2024-06-13T07:17:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:20 smithi136 ceph-mon[37229]: pgmap v1014: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 804 op/s 2024-06-13T07:17:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:20 smithi026 ceph-mon[32041]: pgmap v1014: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 804 op/s 2024-06-13T07:17:21.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 113971 28.70 MB/sec execute 94 sec latency 118.999 ms 2024-06-13T07:17:22.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 114623 28.82 MB/sec execute 95 sec latency 98.624 ms 2024-06-13T07:17:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:22 smithi135 ceph-mon[36276]: pgmap v1015: 129 pgs: 129 active+clean; 589 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 574 op/s 2024-06-13T07:17:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:22 smithi136 ceph-mon[37229]: pgmap v1015: 129 pgs: 129 active+clean; 589 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 574 op/s 2024-06-13T07:17:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:22 smithi026 ceph-mon[32041]: pgmap v1015: 129 pgs: 129 active+clean; 589 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 574 op/s 2024-06-13T07:17:23.166 INFO:tasks.workunit.client.0.smithi026.stdout: 10 115234 28.80 MB/sec execute 96 sec latency 70.869 ms 2024-06-13T07:17:24.167 INFO:tasks.workunit.client.0.smithi026.stdout: 10 115899 28.93 MB/sec execute 97 sec latency 78.346 ms 2024-06-13T07:17:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:24 smithi135 ceph-mon[36276]: pgmap v1016: 129 pgs: 129 active+clean; 589 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 617 op/s 2024-06-13T07:17:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:24 smithi136 ceph-mon[37229]: pgmap v1016: 129 pgs: 129 active+clean; 589 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 617 op/s 2024-06-13T07:17:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:24 smithi026 ceph-mon[32041]: pgmap v1016: 129 pgs: 129 active+clean; 589 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 617 op/s 2024-06-13T07:17:25.167 INFO:tasks.workunit.client.0.smithi026.stdout: 10 116493 28.91 MB/sec execute 98 sec latency 111.695 ms 2024-06-13T07:17:26.167 INFO:tasks.workunit.client.0.smithi026.stdout: 10 116974 28.92 MB/sec execute 99 sec latency 140.014 ms 2024-06-13T07:17:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:26 smithi135 ceph-mon[36276]: pgmap v1017: 129 pgs: 129 active+clean; 586 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 611 op/s 2024-06-13T07:17:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:26 smithi136 ceph-mon[37229]: pgmap v1017: 129 pgs: 129 active+clean; 586 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 611 op/s 2024-06-13T07:17:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:26 smithi026 ceph-mon[32041]: pgmap v1017: 129 pgs: 129 active+clean; 586 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 611 op/s 2024-06-13T07:17:27.167 INFO:tasks.workunit.client.0.smithi026.stdout: 10 117600 28.91 MB/sec execute 100 sec latency 164.599 ms 2024-06-13T07:17:28.169 INFO:tasks.workunit.client.0.smithi026.stdout: 10 118245 28.96 MB/sec execute 101 sec latency 75.495 ms 2024-06-13T07:17:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:28 smithi136 ceph-mon[37229]: pgmap v1018: 129 pgs: 129 active+clean; 585 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.5 MiB/s wr, 537 op/s 2024-06-13T07:17:28.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:28 smithi026 ceph-mon[32041]: pgmap v1018: 129 pgs: 129 active+clean; 585 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.5 MiB/s wr, 537 op/s 2024-06-13T07:17:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:28 smithi135 ceph-mon[36276]: pgmap v1018: 129 pgs: 129 active+clean; 585 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.5 MiB/s wr, 537 op/s 2024-06-13T07:17:29.168 INFO:tasks.workunit.client.0.smithi026.stdout: 10 118991 29.13 MB/sec execute 102 sec latency 66.421 ms 2024-06-13T07:17:30.168 INFO:tasks.workunit.client.0.smithi026.stdout: 10 119501 29.11 MB/sec execute 103 sec latency 122.669 ms 2024-06-13T07:17:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:30 smithi136 ceph-mon[37229]: pgmap v1019: 129 pgs: 129 active+clean; 583 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-06-13T07:17:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:30 smithi026 ceph-mon[32041]: pgmap v1019: 129 pgs: 129 active+clean; 583 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-06-13T07:17:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:30 smithi135 ceph-mon[36276]: pgmap v1019: 129 pgs: 129 active+clean; 583 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-06-13T07:17:31.168 INFO:tasks.workunit.client.0.smithi026.stdout: 10 120107 29.15 MB/sec execute 104 sec latency 57.169 ms 2024-06-13T07:17:32.169 INFO:tasks.workunit.client.0.smithi026.stdout: 10 120711 29.17 MB/sec execute 105 sec latency 76.238 ms 2024-06-13T07:17:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:32 smithi136 ceph-mon[37229]: pgmap v1020: 129 pgs: 129 active+clean; 586 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:17:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.747+0000 7f1e2ac00640 1 -- 172.21.15.26:0/2290497991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c096f70 msgr2=0x7f1e1c0973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.747+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/2290497991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c096f70 0x7f1e1c0973f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e1002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.748+0000 7f1e2ac00640 1 -- 172.21.15.26:0/2290497991 shutdown_connections 2024-06-13T07:17:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.748+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/2290497991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e1c097930 0x7f1e1c09e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.748+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/2290497991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c096f70 0x7f1e1c0973f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.748+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/2290497991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e1c095d80 0x7f1e1c0961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.748+0000 7f1e2ac00640 1 -- 172.21.15.26:0/2290497991 >> 172.21.15.26:0/2290497991 conn(0x7f1e1c0912d0 msgr2=0x7f1e1c093730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.748+0000 7f1e2ac00640 1 -- 172.21.15.26:0/2290497991 shutdown_connections 2024-06-13T07:17:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.748+0000 7f1e2ac00640 1 -- 172.21.15.26:0/2290497991 wait complete. 2024-06-13T07:17:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.752+0000 7f1e2ac00640 1 Processor -- start 2024-06-13T07:17:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.753+0000 7f1e2ac00640 1 -- start start 2024-06-13T07:17:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.754+0000 7f1e2ac00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e1c095d80 0x7f1e1c129f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.755+0000 7f1e2ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c096f70 0x7f1e1c12a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.756+0000 7f1e29800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e1c095d80 0x7f1e1c129f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.756+0000 7f1e29800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e1c095d80 0x7f1e1c129f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59654/0 (socket says 172.21.15.26:59654) 2024-06-13T07:17:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.756+0000 7f1e2ac00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e1c097930 0x7f1e1c142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.756+0000 7f1e29800640 1 -- 172.21.15.26:0/1928777055 learned_addr learned my addr 172.21.15.26:0/1928777055 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.756+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c0a1a30 con 0x7f1e1c095d80 2024-06-13T07:17:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.756+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c0a18b0 con 0x7f1e1c097930 2024-06-13T07:17:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.756+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c0a1bb0 con 0x7f1e1c096f70 2024-06-13T07:17:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.757+0000 7f1e2a200640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e1c097930 0x7f1e1c142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.758+0000 7f1e28e00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c096f70 0x7f1e1c12a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.758+0000 7f1e29800640 1 -- 172.21.15.26:0/1928777055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c096f70 msgr2=0x7f1e1c12a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.758+0000 7f1e29800640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c096f70 0x7f1e1c12a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.758+0000 7f1e29800640 1 -- 172.21.15.26:0/1928777055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e1c097930 msgr2=0x7f1e1c142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.758+0000 7f1e29800640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e1c097930 0x7f1e1c142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.758+0000 7f1e29800640 1 -- 172.21.15.26:0/1928777055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c12f540 con 0x7f1e1c095d80 2024-06-13T07:17:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.759+0000 7f1e29800640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e1c095d80 0x7f1e1c129f10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800d950 tx=0x7f1e1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:32.760 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:32 smithi026 ceph-mon[32041]: pgmap v1020: 129 pgs: 129 active+clean; 586 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:17:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.760+0000 7f1e22000640 1 -- 172.21.15.26:0/1928777055 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18017020 con 0x7f1e1c095d80 2024-06-13T07:17:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.760+0000 7f1e22000640 1 -- 172.21.15.26:0/1928777055 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e18009c30 con 0x7f1e1c095d80 2024-06-13T07:17:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.760+0000 7f1e22000640 1 -- 172.21.15.26:0/1928777055 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800f720 con 0x7f1e1c095d80 2024-06-13T07:17:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.760+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1c142e80 con 0x7f1e1c095d80 2024-06-13T07:17:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.760+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c143230 con 0x7f1e1c095d80 2024-06-13T07:17:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.767+0000 7f1e2f1c7740 1 Processor -- start 2024-06-13T07:17:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.767+0000 7f1e2f1c7740 1 -- start start 2024-06-13T07:17:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.768+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a94b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.769+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 0x55fc5a9502d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.769+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 0x55fc5a952dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.769+0000 7f1e2f1c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fc5a8ae420 con 0x55fc5a950940 2024-06-13T07:17:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.769+0000 7f1e2f1c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fc5a855c50 con 0x55fc5a94b7d0 2024-06-13T07:17:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.769+0000 7f1e2f1c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc5a8aa3e0 con 0x55fc5a94ade0 2024-06-13T07:17:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.769+0000 7f1e2a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a94b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.769+0000 7f1e2a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a94b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41800/0 (socket says 172.21.15.26:41800) 2024-06-13T07:17:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.769+0000 7f1e2a200640 1 -- 172.21.15.26:0/4178600385 learned_addr learned my addr 172.21.15.26:0/4178600385 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.770+0000 7f1e29800640 1 --2- 172.21.15.26:0/4178600385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 0x55fc5a9502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.772+0000 7f1e2a200640 1 -- 172.21.15.26:0/4178600385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 msgr2=0x55fc5a9502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.772+0000 7f1e2a200640 1 --2- 172.21.15.26:0/4178600385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 0x55fc5a9502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.772+0000 7f1e2a200640 1 -- 172.21.15.26:0/4178600385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 msgr2=0x55fc5a952dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.772+0000 7f1e2a200640 1 --2- 172.21.15.26:0/4178600385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 0x55fc5a952dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.772+0000 7f1e2a200640 1 -- 172.21.15.26:0/4178600385 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5a5c2210 con 0x55fc5a94ade0 2024-06-13T07:17:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.772+0000 7f1e2a200640 1 --2- 172.21.15.26:0/4178600385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a94b200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1e24050510 tx=0x7f1e2407e7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31efdc34ad053b32 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.773+0000 7f1e16a00640 1 -- 172.21.15.26:0/4178600385 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2407f440 con 0x55fc5a94ade0 2024-06-13T07:17:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.775+0000 7f1e16a00640 1 -- 172.21.15.26:0/4178600385 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e24083030 con 0x55fc5a94ade0 2024-06-13T07:17:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.776+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4178600385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 msgr2=0x55fc5a94b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.776+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4178600385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a94b200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1e24050510 tx=0x7f1e2407e7a0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.778+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4178600385 shutdown_connections 2024-06-13T07:17:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.778+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4178600385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 0x55fc5a9502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.778+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4178600385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 0x55fc5a952dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.778+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4178600385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a94b200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.778+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4178600385 >> 172.21.15.26:0/4178600385 conn(0x55fc5a912fb0 msgr2=0x55fc5a946860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:32.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:32 smithi135 ceph-mon[36276]: pgmap v1020: 129 pgs: 129 active+clean; 586 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:17:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.787+0000 7f1e22000640 1 -- 172.21.15.26:0/1928777055 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1e1801a020 con 0x7f1e1c095d80 2024-06-13T07:17:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.789+0000 7f1e22000640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e0c0a2d10 0x7f1e0c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.789+0000 7f1e22000640 1 -- 172.21.15.26:0/1928777055 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1e180ceb50 con 0x7f1e1c095d80 2024-06-13T07:17:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.792+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4178600385 shutdown_connections 2024-06-13T07:17:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.792+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4178600385 wait complete. 2024-06-13T07:17:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.794+0000 7f1e2f1c7740 1 -- start start 2024-06-13T07:17:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.794+0000 7f1e2f1c7740 1 Processor -- start 2024-06-13T07:17:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.795+0000 7f1e28e00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e0c0a2d10 0x7f1e0c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.797+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a954fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.798+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 0x55fc5a955500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.798+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 0x55fc5a95c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.798+0000 7f1e29800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a954fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.798+0000 7f1e29800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a954fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41816/0 (socket says 172.21.15.26:41816) 2024-06-13T07:17:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.799+0000 7f1e2a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 0x55fc5a95c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.799+0000 7f1e2f1c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fc5a9538f0 con 0x55fc5a950940 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.799+0000 7f1e29800640 1 -- 172.21.15.26:0/4172512696 learned_addr learned my addr 172.21.15.26:0/4172512696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.799+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fc5a8690b0 con 0x55fc5a94b7d0 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.799+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc5a953a70 con 0x55fc5a94ade0 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.799+0000 7f1e2f1c7740 10 client.0 _ll_register_callbacks cb 0x55fc5a854f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.800+0000 7f1e29800640 1 -- 172.21.15.26:0/4172512696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 msgr2=0x55fc5a955500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.800+0000 7f1e29800640 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 0x55fc5a955500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.800+0000 7f1e29800640 1 -- 172.21.15.26:0/4172512696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 msgr2=0x55fc5a95c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.800+0000 7f1e29800640 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 0x55fc5a95c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.800+0000 7f1e29800640 1 -- 172.21.15.26:0/4172512696 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5a68e620 con 0x55fc5a94ade0 2024-06-13T07:17:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.800+0000 7f1e29800640 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a954fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a3780 tx=0x7f1e1809aaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ae26901012f30b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.800+0000 7f1e28e00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e0c0a2d10 0x7f1e0c0a51d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1e10031440 tx=0x7f1e10005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.800+0000 7f1e0b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:17:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.801+0000 7f1e0b400640 1 -- 172.21.15.26:0/4172512696 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1809f480 con 0x55fc5a94ade0 2024-06-13T07:17:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.801+0000 7f1e0b400640 1 -- 172.21.15.26:0/4172512696 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1e1809f170 con 0x55fc5a94ade0 2024-06-13T07:17:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.801+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e24083030 con 0x55fc5a94ade0 2024-06-13T07:17:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.802+0000 7f1e0b400640 1 -- 172.21.15.26:0/4172512696 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1e180a3cb0 con 0x55fc5a94ade0 2024-06-13T07:17:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.803+0000 7f1e2f1c7740 10 client.26785 fetch_fsmap learned FSMap version 47 2024-06-13T07:17:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.803+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55fc5a95f960 con 0x55fc5a94ade0 2024-06-13T07:17:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.804+0000 7f1e0b400640 1 -- 172.21.15.26:0/4172512696 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e24083030 con 0x55fc5a94ade0 2024-06-13T07:17:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.804+0000 7f1e0b400640 1 -- 172.21.15.26:0/4172512696 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f1e18099f90 con 0x55fc5a94ade0 2024-06-13T07:17:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.805+0000 7f1e2f1c7740 10 client.26785 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:17:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.805+0000 7f1e2f1c7740 10 client.26785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:17:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.805+0000 7f1e2f1c7740 20 client.26785 populate_metadata read hostname 'smithi026' 2024-06-13T07:17:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.805+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fc5a9610a0 0x55fc5a963530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.805+0000 7f1e2f1c7740 4 client.26785 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:17:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.805+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc5a8bbb50 con 0x55fc5a9610a0 2024-06-13T07:17:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.805+0000 7f1e28e00640 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fc5a9610a0 0x55fc5a963530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.809+0000 7f1e28e00640 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fc5a9610a0 0x55fc5a963530 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bbb5740e1cdd9e3 server_cookie=28ab3fc9632539c3 in_seq=0 out_seq=0 2024-06-13T07:17:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.813+0000 7f1e0b400640 10 client.26785 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.841+0000 7f1e0b400640 1 -- 172.21.15.26:0/4172512696 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc5a8bbb50 con 0x55fc5a9610a0 2024-06-13T07:17:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.841+0000 7f1e0b400640 10 client.26785 handle_command_reply: tid=0 2024-06-13T07:17:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.842+0000 7f1e2f1c7740 1 client.26785 shutdown 2024-06-13T07:17:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.842+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 msgr2=0x55fc5a954fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.842+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a954fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a3780 tx=0x7f1e1809aaf0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.843+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 shutdown_connections 2024-06-13T07:17:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.843+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fc5a9610a0 0x55fc5a963530 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.843+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94b7d0 0x55fc5a955500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.843+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a950940 0x55fc5a95c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.843+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/4172512696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94ade0 0x55fc5a954fc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.843+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 >> 172.21.15.26:0/4172512696 conn(0x55fc5a912fb0 msgr2=0x55fc5a9459a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.843+0000 7f1e0b400640 0 client.26785 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.844+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 shutdown_connections 2024-06-13T07:17:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.844+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/4172512696 wait complete. 2024-06-13T07:17:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.845+0000 7f1e2f1c7740 20 client.26785 trim_cache size 0 max 16384 2024-06-13T07:17:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.894+0000 7f1e2f1c7740 1 Processor -- start 2024-06-13T07:17:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.895+0000 7f1e2f1c7740 1 -- start start 2024-06-13T07:17:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.895+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94ade0 0x55fc5a9552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.895+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.895+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a950940 0x55fc5a945110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.895+0000 7f1e2f1c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c150330 con 0x55fc5a94b7d0 2024-06-13T07:17:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.895+0000 7f1e2f1c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c14dc00 con 0x55fc5a94ade0 2024-06-13T07:17:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.895+0000 7f1e2f1c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c14ba50 con 0x55fc5a950940 2024-06-13T07:17:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.896+0000 7f1e29800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94ade0 0x55fc5a9552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.896+0000 7f1e2a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a950940 0x55fc5a945110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.897+0000 7f1e29800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94ade0 0x55fc5a9552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50104/0 (socket says 172.21.15.26:50104) 2024-06-13T07:17:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.897+0000 7f1e29800640 1 -- 172.21.15.26:0/1109863684 learned_addr learned my addr 172.21.15.26:0/1109863684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.898+0000 7f1e28e00640 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.898+0000 7f1e2a200640 1 -- 172.21.15.26:0/1109863684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94ade0 msgr2=0x55fc5a9552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.898+0000 7f1e2a200640 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94ade0 0x55fc5a9552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.899+0000 7f1e2a200640 1 -- 172.21.15.26:0/1109863684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94b7d0 msgr2=0x55fc5a95c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.899+0000 7f1e2a200640 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.899+0000 7f1e2a200640 1 -- 172.21.15.26:0/1109863684 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5a953bf0 con 0x55fc5a950940 2024-06-13T07:17:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.900+0000 7f1e28e00640 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:17:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.900+0000 7f1e29800640 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94ade0 0x55fc5a9552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:17:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.900+0000 7f1e2a200640 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a950940 0x55fc5a945110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1e24051060 tx=0x7f1e24067610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1319bd903171d0de server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.903+0000 7f1e16000640 1 -- 172.21.15.26:0/1109863684 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24098070 con 0x55fc5a950940 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.905+0000 7f1e16000640 1 -- 172.21.15.26:0/1109863684 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e2407fcc0 con 0x55fc5a950940 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.905+0000 7f1e16000640 1 -- 172.21.15.26:0/1109863684 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24088b10 con 0x55fc5a950940 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.906+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/1109863684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a950940 msgr2=0x55fc5a945110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.906+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a950940 0x55fc5a945110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1e24051060 tx=0x7f1e24067610 comp rx=0 tx=0).stop 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.907+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/1109863684 shutdown_connections 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.907+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a950940 0x55fc5a945110 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.907+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.907+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/1109863684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a94ade0 0x55fc5a9552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.907+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/1109863684 >> 172.21.15.26:0/1109863684 conn(0x55fc5a912fb0 msgr2=0x55fc5a954bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.908+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/1109863684 shutdown_connections 2024-06-13T07:17:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.910+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/1109863684 wait complete. 2024-06-13T07:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.910+0000 7f1e2f1c7740 1 -- start start 2024-06-13T07:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.911+0000 7f1e2f1c7740 1 Processor -- start 2024-06-13T07:17:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.912+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94ade0 0x55fc5a959ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.912+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.912+0000 7f1e29800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94ade0 0x55fc5a959ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.913+0000 7f1e2f1c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a950940 0x55fc5a95b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.913+0000 7f1e29800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94ade0 0x55fc5a959ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59684/0 (socket says 172.21.15.26:59684) 2024-06-13T07:17:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.914+0000 7f1e28e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.914+0000 7f1e28e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41832/0 (socket says 172.21.15.26:41832) 2024-06-13T07:17:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.914+0000 7f1e2a200640 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a950940 0x55fc5a95b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.914+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c14c0c0 con 0x55fc5a94ade0 2024-06-13T07:17:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.914+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c14d530 con 0x55fc5a950940 2024-06-13T07:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.915+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c14c790 con 0x55fc5a94b7d0 2024-06-13T07:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.914+0000 7f1e29800640 1 -- 172.21.15.26:0/2067837847 learned_addr learned my addr 172.21.15.26:0/2067837847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.915+0000 7f1e2a200640 1 -- 172.21.15.26:0/2067837847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94b7d0 msgr2=0x55fc5a95a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.915+0000 7f1e2a200640 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.915+0000 7f1e2f1c7740 10 client.0 _ll_register_callbacks cb 0x7f1e180046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.915+0000 7f1e2a200640 1 -- 172.21.15.26:0/2067837847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94ade0 msgr2=0x55fc5a959ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.915+0000 7f1e2a200640 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94ade0 0x55fc5a959ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.915+0000 7f1e2a200640 1 -- 172.21.15.26:0/2067837847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5a95b820 con 0x55fc5a950940 2024-06-13T07:17:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.916+0000 7f1e28e00640 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:17:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.917+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e2407fcc0 con 0x55fc5a950940 2024-06-13T07:17:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.919+0000 7f1e2a200640 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a950940 0x55fc5a95b230 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1e24050f40 tx=0x7f1e24064c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b1acd338cc20f6f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.919+0000 7f1e0b400640 10 client.26768 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.920+0000 7f1e0b400640 1 -- 172.21.15.26:0/2067837847 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24098040 con 0x55fc5a950940 2024-06-13T07:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.920+0000 7f1e0b400640 1 -- 172.21.15.26:0/2067837847 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1e24088420 con 0x55fc5a950940 2024-06-13T07:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.921+0000 7f1e0b400640 1 -- 172.21.15.26:0/2067837847 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1e24098040 con 0x55fc5a950940 2024-06-13T07:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.921+0000 7f1e2f1c7740 10 client.26768 fetch_fsmap learned FSMap version 47 2024-06-13T07:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.921+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=47}) -- 0x55fc5a9cbe10 con 0x55fc5a950940 2024-06-13T07:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.922+0000 7f1e0b400640 1 -- 172.21.15.26:0/2067837847 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f1e24088810 con 0x55fc5a950940 2024-06-13T07:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.922+0000 7f1e2f1c7740 10 client.26768 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.922+0000 7f1e2f1c7740 10 client.26768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:17:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.922+0000 7f1e2f1c7740 20 client.26768 populate_metadata read hostname 'smithi026' 2024-06-13T07:17:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.922+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fc5a9cc510 0x55fc5a9ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.922+0000 7f1e2f1c7740 4 client.26768 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:17:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.923+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc5a8b5ea0 con 0x55fc5a9cc510 2024-06-13T07:17:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.923+0000 7f1e29800640 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fc5a9cc510 0x55fc5a9ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.928+0000 7f1e0b400640 10 client.26768 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:32.928+0000 7f1e29800640 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fc5a9cc510 0x55fc5a9ce930 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c621e52ca1bc19 server_cookie=70816c3b3fcd7759 in_seq=0 out_seq=0 2024-06-13T07:17:33.169 INFO:tasks.workunit.client.0.smithi026.stdout: 10 120986 28.98 MB/sec execute 106 sec latency 370.028 ms 2024-06-13T07:17:34.169 INFO:tasks.workunit.client.0.smithi026.stdout: 10 121012 28.72 MB/sec execute 107 sec latency 1346.829 ms 2024-06-13T07:17:34.560 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:34 smithi026 ceph-mon[32041]: pgmap v1021: 129 pgs: 129 active+clean; 596 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-06-13T07:17:34.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:34 smithi136 ceph-mon[37229]: pgmap v1021: 129 pgs: 129 active+clean; 596 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-06-13T07:17:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.744+0000 7f1e0b400640 1 -- 172.21.15.26:0/2067837847 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fc5a8b5ea0 con 0x55fc5a9cc510 2024-06-13T07:17:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.744+0000 7f1e0b400640 10 client.26768 handle_command_reply: tid=0 2024-06-13T07:17:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.744+0000 7f1e2f1c7740 1 client.26768 shutdown 2024-06-13T07:17:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.745+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a950940 msgr2=0x55fc5a95b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.745+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a950940 0x55fc5a95b230 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1e24050f40 tx=0x7f1e24064c10 comp rx=0 tx=0).stop 2024-06-13T07:17:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 shutdown_connections 2024-06-13T07:17:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fc5a9cc510 0x55fc5a9ce930 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc5a950940 0x55fc5a95b230 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc5a94b7d0 0x55fc5a95a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e2f1c7740 1 --2- 172.21.15.26:0/2067837847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc5a94ade0 0x55fc5a959ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 >> 172.21.15.26:0/2067837847 conn(0x55fc5a912fb0 msgr2=0x55fc5a945110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e0b400640 0 client.26768 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 shutdown_connections 2024-06-13T07:17:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.746+0000 7f1e2f1c7740 1 -- 172.21.15.26:0/2067837847 wait complete. 2024-06-13T07:17:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.747+0000 7f1e2f1c7740 20 client.26768 trim_cache size 0 max 16384 2024-06-13T07:17:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:17:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:17:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:17:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e0c0a2d10 msgr2=0x7f1e0c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e0c0a2d10 0x7f1e0c0a51d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1e10031440 tx=0x7f1e10005fd0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e1c095d80 msgr2=0x7f1e1c129f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e1c095d80 0x7f1e1c129f10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800d950 tx=0x7f1e1800de10 comp rx=0 tx=0).stop 2024-06-13T07:17:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 shutdown_connections 2024-06-13T07:17:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1e0c0a2d10 0x7f1e0c0a51d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e1c097930 0x7f1e1c142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c096f70 0x7f1e1c12a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 --2- 172.21.15.26:0/1928777055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e1c095d80 0x7f1e1c129f10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 >> 172.21.15.26:0/1928777055 conn(0x7f1e1c0912d0 msgr2=0x7f1e1c006dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 shutdown_connections 2024-06-13T07:17:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:34.749+0000 7f1e2ac00640 1 -- 172.21.15.26:0/1928777055 wait complete. 2024-06-13T07:17:34.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:34 smithi135 ceph-mon[36276]: pgmap v1021: 129 pgs: 129 active+clean; 596 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-06-13T07:17:35.170 INFO:tasks.workunit.client.0.smithi026.stdout: 10 121301 28.64 MB/sec execute 108 sec latency 1374.795 ms 2024-06-13T07:17:36.170 INFO:tasks.workunit.client.0.smithi026.stdout: 10 121652 28.52 MB/sec execute 109 sec latency 112.765 ms 2024-06-13T07:17:36.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:36 smithi136 ceph-mon[37229]: pgmap v1022: 129 pgs: 129 active+clean; 627 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.4 MiB/s wr, 514 op/s 2024-06-13T07:17:36.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:36 smithi026 ceph-mon[32041]: pgmap v1022: 129 pgs: 129 active+clean; 627 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.4 MiB/s wr, 514 op/s 2024-06-13T07:17:36.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:36 smithi135 ceph-mon[36276]: pgmap v1022: 129 pgs: 129 active+clean; 627 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.4 MiB/s wr, 514 op/s 2024-06-13T07:17:37.170 INFO:tasks.workunit.client.0.smithi026.stdout: 10 122161 28.52 MB/sec execute 110 sec latency 98.937 ms 2024-06-13T07:17:38.170 INFO:tasks.workunit.client.0.smithi026.stdout: 10 122911 28.70 MB/sec execute 111 sec latency 46.704 ms 2024-06-13T07:17:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:38 smithi136 ceph-mon[37229]: pgmap v1023: 129 pgs: 129 active+clean; 639 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 466 op/s 2024-06-13T07:17:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:38 smithi026 ceph-mon[32041]: pgmap v1023: 129 pgs: 129 active+clean; 639 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 466 op/s 2024-06-13T07:17:38.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:38 smithi135 ceph-mon[36276]: pgmap v1023: 129 pgs: 129 active+clean; 639 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 466 op/s 2024-06-13T07:17:39.170 INFO:tasks.workunit.client.0.smithi026.stdout: 10 123481 28.74 MB/sec execute 112 sec latency 59.820 ms 2024-06-13T07:17:40.170 INFO:tasks.workunit.client.0.smithi026.stdout: 10 123956 28.65 MB/sec execute 113 sec latency 82.599 ms 2024-06-13T07:17:40.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:40 smithi136 ceph-mon[37229]: pgmap v1024: 129 pgs: 129 active+clean; 606 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 686 op/s 2024-06-13T07:17:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:40 smithi026 ceph-mon[32041]: pgmap v1024: 129 pgs: 129 active+clean; 606 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 686 op/s 2024-06-13T07:17:40.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:40 smithi135 ceph-mon[36276]: pgmap v1024: 129 pgs: 129 active+clean; 606 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 686 op/s 2024-06-13T07:17:41.170 INFO:tasks.workunit.client.0.smithi026.stdout: 10 124543 28.62 MB/sec execute 114 sec latency 70.390 ms 2024-06-13T07:17:41.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:41 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:17:41.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:41 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:17:41.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:41 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:17:42.174 INFO:tasks.workunit.client.0.smithi026.stdout: 10 124712 28.42 MB/sec execute 115 sec latency 691.736 ms 2024-06-13T07:17:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:42 smithi135 ceph-mon[36276]: pgmap v1025: 129 pgs: 129 active+clean; 599 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.4 MiB/s wr, 645 op/s 2024-06-13T07:17:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:42 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:42 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:42 smithi136 ceph-mon[37229]: pgmap v1025: 129 pgs: 129 active+clean; 599 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.4 MiB/s wr, 645 op/s 2024-06-13T07:17:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:42 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:42.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:42 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:42.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:42 smithi026 ceph-mon[32041]: pgmap v1025: 129 pgs: 129 active+clean; 599 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.4 MiB/s wr, 645 op/s 2024-06-13T07:17:42.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:42 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:42.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:42 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:43.171 INFO:tasks.workunit.client.0.smithi026.stdout: 10 125056 28.36 MB/sec execute 116 sec latency 459.003 ms 2024-06-13T07:17:44.171 INFO:tasks.workunit.client.0.smithi026.stdout: 10 125848 28.46 MB/sec execute 117 sec latency 75.638 ms 2024-06-13T07:17:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:44 smithi026 ceph-mon[32041]: pgmap v1026: 129 pgs: 129 active+clean; 601 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 701 op/s 2024-06-13T07:17:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:44 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:44 smithi135 ceph-mon[36276]: pgmap v1026: 129 pgs: 129 active+clean; 601 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 701 op/s 2024-06-13T07:17:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:44 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:44 smithi136 ceph-mon[37229]: pgmap v1026: 129 pgs: 129 active+clean; 601 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 701 op/s 2024-06-13T07:17:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:44.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:44 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:45.171 INFO:tasks.workunit.client.0.smithi026.stdout: 10 126571 28.69 MB/sec execute 118 sec latency 52.749 ms 2024-06-13T07:17:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:45 smithi135 ceph-mon[36276]: pgmap v1027: 129 pgs: 129 active+clean; 601 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 679 op/s 2024-06-13T07:17:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:45 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:45 smithi136 ceph-mon[37229]: pgmap v1027: 129 pgs: 129 active+clean; 601 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 679 op/s 2024-06-13T07:17:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:45 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.171 INFO:tasks.workunit.client.0.smithi026.stdout: 10 127005 28.67 MB/sec execute 119 sec latency 58.279 ms 2024-06-13T07:17:46.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:45 smithi026 ceph-mon[32041]: pgmap v1027: 129 pgs: 129 active+clean; 601 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 679 op/s 2024-06-13T07:17:46.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:46.254 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:45 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:47.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:46 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:47.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:47.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:46 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:47.171 INFO:tasks.workunit.client.0.smithi026.stdout: 10 127148 28.48 MB/sec execute 120 sec latency 456.359 ms 2024-06-13T07:17:47.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:47.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:46 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.952+0000 7f6baf200640 1 -- 172.21.15.26:0/1711819925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba80775a0 msgr2=0x7f6ba8077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.952+0000 7f6baf200640 1 --2- 172.21.15.26:0/1711819925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba80775a0 0x7f6ba8077a00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009900 tx=0x7f6b9402a3f0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.952+0000 7f6baf200640 1 -- 172.21.15.26:0/1711819925 shutdown_connections 2024-06-13T07:17:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.952+0000 7f6baf200640 1 --2- 172.21.15.26:0/1711819925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba80775a0 0x7f6ba8077a00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.952+0000 7f6baf200640 1 --2- 172.21.15.26:0/1711819925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba8077f40 0x7f6ba80787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.952+0000 7f6baf200640 1 --2- 172.21.15.26:0/1711819925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba8076360 0x7f6ba8076780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.952+0000 7f6baf200640 1 -- 172.21.15.26:0/1711819925 >> 172.21.15.26:0/1711819925 conn(0x7f6ba80ff190 msgr2=0x7f6ba81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.954+0000 7f6baf200640 1 -- 172.21.15.26:0/1711819925 shutdown_connections 2024-06-13T07:17:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.954+0000 7f6baf200640 1 -- 172.21.15.26:0/1711819925 wait complete. 2024-06-13T07:17:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.955+0000 7f6baf200640 1 Processor -- start 2024-06-13T07:17:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.956+0000 7f6baf200640 1 -- start start 2024-06-13T07:17:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.956+0000 7f6baf200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba8076360 0x7f6ba819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.956+0000 7f6baf200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba80775a0 0x7f6ba819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.956+0000 7f6baf200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba8077f40 0x7f6ba81a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.956+0000 7f6baf200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8113cf0 con 0x7f6ba8076360 2024-06-13T07:17:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.956+0000 7f6baf200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8113b70 con 0x7f6ba8077f40 2024-06-13T07:17:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.956+0000 7f6baf200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8113e70 con 0x7f6ba80775a0 2024-06-13T07:17:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba8076360 0x7f6ba819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6bad200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba8077f40 0x7f6ba81a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6ba7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba80775a0 0x7f6ba819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba8076360 0x7f6ba819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46054/0 (socket says 172.21.15.26:46054) 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6ba7e00640 1 -- 172.21.15.26:0/314892889 learned_addr learned my addr 172.21.15.26:0/314892889 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6bad200640 1 -- 172.21.15.26:0/314892889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba80775a0 msgr2=0x7f6ba819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6bad200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba80775a0 0x7f6ba819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6bad200640 1 -- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba8076360 msgr2=0x7f6ba819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6bad200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba8076360 0x7f6ba819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6bad200640 1 -- 172.21.15.26:0/314892889 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba81a5e40 con 0x7f6ba8077f40 2024-06-13T07:17:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.957+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba8076360 0x7f6ba819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:17:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.958+0000 7f6bad200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba8077f40 0x7f6ba81a57d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000ec50 tx=0x7f6ba000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.960+0000 7f6ba4c00640 1 -- 172.21.15.26:0/314892889 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0003b70 con 0x7f6ba8077f40 2024-06-13T07:17:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.960+0000 7f6ba4c00640 1 -- 172.21.15.26:0/314892889 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba0003d10 con 0x7f6ba8077f40 2024-06-13T07:17:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.960+0000 7f6baf200640 1 -- 172.21.15.26:0/314892889 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba81a6130 con 0x7f6ba8077f40 2024-06-13T07:17:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.960+0000 7f6ba4c00640 1 -- 172.21.15.26:0/314892889 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0010890 con 0x7f6ba8077f40 2024-06-13T07:17:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.960+0000 7f6baf200640 1 -- 172.21.15.26:0/314892889 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81a6650 con 0x7f6ba8077f40 2024-06-13T07:17:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.966+0000 7f6bb1fba740 1 Processor -- start 2024-06-13T07:17:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.966+0000 7f6ba4c00640 1 -- 172.21.15.26:0/314892889 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ba0003fc0 con 0x7f6ba8077f40 2024-06-13T07:17:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6bb1fba740 1 -- start start 2024-06-13T07:17:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06afc00 0x5649f06b0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b05f0 0x5649f06b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b5760 0x5649f06b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5649f0613420 con 0x5649f06afc00 2024-06-13T07:17:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5649f05bac50 con 0x5649f06b05f0 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5649f060f3e0 con 0x5649f06b5760 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b05f0 0x5649f06b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b05f0 0x5649f06b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:36960/0 (socket says 172.21.15.26:36960) 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.967+0000 7f6ba7e00640 1 -- 172.21.15.26:0/1296126256 learned_addr learned my addr 172.21.15.26:0/1296126256 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7400640 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b5760 0x5649f06b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7e00640 1 -- 172.21.15.26:0/1296126256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b5760 msgr2=0x5649f06b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b5760 0x5649f06b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7400640 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06afc00 0x5649f06b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7e00640 1 -- 172.21.15.26:0/1296126256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06afc00 msgr2=0x5649f06b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06afc00 0x5649f06b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7e00640 1 -- 172.21.15.26:0/1296126256 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f0327210 con 0x5649f06b05f0 2024-06-13T07:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7400640 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06afc00 0x5649f06b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:17:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.968+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b05f0 0x5649f06b50f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b98011650 tx=0x7f6b98009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae7386a58b8a1c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.969+0000 7f6b9d600640 1 -- 172.21.15.26:0/1296126256 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9801e070 con 0x5649f06b05f0 2024-06-13T07:17:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.969+0000 7f6b9d600640 1 -- 172.21.15.26:0/1296126256 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b9800fba0 con 0x5649f06b05f0 2024-06-13T07:17:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.969+0000 7f6b9d600640 1 -- 172.21.15.26:0/1296126256 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98019cd0 con 0x5649f06b05f0 2024-06-13T07:17:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.970+0000 7f6bb1fba740 1 -- 172.21.15.26:0/1296126256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b05f0 msgr2=0x5649f06b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.970+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b05f0 0x5649f06b50f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b98011650 tx=0x7f6b98009f90 comp rx=0 tx=0).stop 2024-06-13T07:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.971+0000 7f6bb1fba740 1 -- 172.21.15.26:0/1296126256 shutdown_connections 2024-06-13T07:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.971+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b05f0 0x5649f06b50f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.971+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b5760 0x5649f06b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.971+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/1296126256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06afc00 0x5649f06b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.971+0000 7f6bb1fba740 1 -- 172.21.15.26:0/1296126256 >> 172.21.15.26:0/1296126256 conn(0x5649f0677fb0 msgr2=0x5649f06ab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.971+0000 7f6bb1fba740 1 -- 172.21.15.26:0/1296126256 shutdown_connections 2024-06-13T07:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.971+0000 7f6bb1fba740 1 -- 172.21.15.26:0/1296126256 wait complete. 2024-06-13T07:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.972+0000 7f6bb1fba740 1 -- start start 2024-06-13T07:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.972+0000 7f6bb1fba740 1 Processor -- start 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 0x5649f06c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5649f06b8710 con 0x5649f06b5760 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5649f05ce0b0 con 0x5649f06afc00 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5649f06b8890 con 0x5649f06b05f0 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6bb1fba740 10 client.0 _ll_register_callbacks cb 0x5649f05b9f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6ba4c00640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b8c0a2cb0 0x7f6b8c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06ba4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48656/0 (socket says 172.21.15.26:48656) 2024-06-13T07:17:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.973+0000 7f6ba7e00640 1 -- 172.21.15.26:0/7529414 learned_addr learned my addr 172.21.15.26:0/7529414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba7e00640 1 -- 172.21.15.26:0/7529414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 msgr2=0x5649f06b9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba7400640 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06b9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba7e00640 1 -- 172.21.15.26:0/7529414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 msgr2=0x5649f06c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 0x5649f06c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba7e00640 1 -- 172.21.15.26:0/7529414 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f03f3620 con 0x5649f06b05f0 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba7400640 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6bad200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b8c0a2cb0 0x7f6b8c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba4c00640 1 -- 172.21.15.26:0/314892889 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6ba00cf960 con 0x7f6ba8077f40 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.974+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06ba4a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b9800ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34437b8823b90638 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.975+0000 7f6b92000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:17:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.975+0000 7f6bad200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b8c0a2cb0 0x7f6b8c0a5170 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000ef90 tx=0x7f6ba001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.977+0000 7f6b92000640 1 -- 172.21.15.26:0/7529414 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98027060 con 0x5649f06b05f0 2024-06-13T07:17:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.977+0000 7f6b92000640 1 -- 172.21.15.26:0/7529414 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6b98009cc0 con 0x5649f06b05f0 2024-06-13T07:17:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.977+0000 7f6bb1fba740 1 -- 172.21.15.26:0/7529414 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9800fba0 con 0x5649f06b05f0 2024-06-13T07:17:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.978+0000 7f6b92000640 1 -- 172.21.15.26:0/7529414 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6b9801e040 con 0x5649f06b05f0 2024-06-13T07:17:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.978+0000 7f6bb1fba740 10 client.26800 fetch_fsmap learned FSMap version 47 2024-06-13T07:17:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.978+0000 7f6bb1fba740 1 -- 172.21.15.26:0/7529414 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5649f06c4810 con 0x5649f06b05f0 2024-06-13T07:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.979+0000 7f6b92000640 1 -- 172.21.15.26:0/7529414 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b9800fba0 con 0x5649f06b05f0 2024-06-13T07:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.980+0000 7f6b92000640 1 -- 172.21.15.26:0/7529414 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f6b98021030 con 0x5649f06b05f0 2024-06-13T07:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.980+0000 7f6bb1fba740 10 client.26800 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.980+0000 7f6bb1fba740 10 client.26800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.980+0000 7f6bb1fba740 20 client.26800 populate_metadata read hostname 'smithi026' 2024-06-13T07:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.980+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649f06c5cc0 0x5649f06c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.980+0000 7f6bb1fba740 4 client.26800 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.980+0000 7f6bb1fba740 1 -- 172.21.15.26:0/7529414 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f0620b50 con 0x5649f06c5cc0 2024-06-13T07:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.980+0000 7f6ba7400640 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649f06c5cc0 0x5649f06c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.984+0000 7f6ba7400640 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649f06c5cc0 0x5649f06c8120 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402a07a60e117066 server_cookie=c7c2eaabfc71371d in_seq=0 out_seq=0 2024-06-13T07:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.984+0000 7f6b92000640 10 client.26800 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.987+0000 7f6b92000640 1 -- 172.21.15.26:0/7529414 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5649f0620b50 con 0x5649f06c5cc0 2024-06-13T07:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.987+0000 7f6b92000640 10 client.26800 handle_command_reply: tid=0 2024-06-13T07:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.987+0000 7f6bb1fba740 1 client.26800 shutdown 2024-06-13T07:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.988+0000 7f6bb1fba740 1 -- 172.21.15.26:0/7529414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 msgr2=0x5649f06ba4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.988+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06ba4a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b9800ff80 comp rx=0 tx=0).stop 2024-06-13T07:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.988+0000 7f6bb1fba740 1 -- 172.21.15.26:0/7529414 shutdown_connections 2024-06-13T07:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.988+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649f06c5cc0 0x5649f06c8120 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.988+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 0x5649f06c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.988+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06ba4a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.988+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/7529414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.988+0000 7f6bb1fba740 1 -- 172.21.15.26:0/7529414 >> 172.21.15.26:0/7529414 conn(0x5649f0677fb0 msgr2=0x5649f06aa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.989+0000 7f6b92000640 0 client.26800 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.989+0000 7f6bb1fba740 1 -- 172.21.15.26:0/7529414 shutdown_connections 2024-06-13T07:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.989+0000 7f6bb1fba740 1 -- 172.21.15.26:0/7529414 wait complete. 2024-06-13T07:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:47.989+0000 7f6bb1fba740 20 client.26800 trim_cache size 0 max 16384 2024-06-13T07:17:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.031+0000 7f6bb1fba740 1 Processor -- start 2024-06-13T07:17:48.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:47 smithi135 ceph-mon[36276]: pgmap v1028: 129 pgs: 129 active+clean; 607 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 687 op/s 2024-06-13T07:17:48.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:17:48.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:17:48.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:47 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.034+0000 7f6bb1fba740 1 -- start start 2024-06-13T07:17:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.035+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06afc00 0x5649f06b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.035+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b05f0 0x5649f06c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.035+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b5760 0x5649f06b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.035+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b36e0 con 0x5649f06b05f0 2024-06-13T07:17:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.035+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b0fb0 con 0x5649f06b5760 2024-06-13T07:17:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.035+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81aee00 con 0x5649f06afc00 2024-06-13T07:17:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.035+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b05f0 0x5649f06c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.035+0000 7f6ba7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06afc00 0x5649f06b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.036+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b05f0 0x5649f06c14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46086/0 (socket says 172.21.15.26:46086) 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.036+0000 7f6ba7e00640 1 -- 172.21.15.26:0/3726649867 learned_addr learned my addr 172.21.15.26:0/3726649867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.036+0000 7f6ba7400640 1 --2- 172.21.15.26:0/3726649867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b5760 0x5649f06b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.036+0000 7f6ba7e00640 1 -- 172.21.15.26:0/3726649867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06afc00 msgr2=0x5649f06b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.036+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/3726649867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06afc00 0x5649f06b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.036+0000 7f6ba7e00640 1 -- 172.21.15.26:0/3726649867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b5760 msgr2=0x5649f06b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.036+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/3726649867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b5760 0x5649f06b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.036+0000 7f6ba7e00640 1 -- 172.21.15.26:0/3726649867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f06b7b50 con 0x5649f06b05f0 2024-06-13T07:17:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.037+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/3726649867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b05f0 0x5649f06c14a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6b98010010 tx=0x7f6b9800fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f892bc84f0e65e8e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.037+0000 7f6b9cc00640 1 -- 172.21.15.26:0/3726649867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98027040 con 0x5649f06b05f0 2024-06-13T07:17:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.038+0000 7f6b9cc00640 1 -- 172.21.15.26:0/3726649867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98004550 con 0x5649f06b05f0 2024-06-13T07:17:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.038+0000 7f6b9cc00640 1 -- 172.21.15.26:0/3726649867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98027040 con 0x5649f06b05f0 2024-06-13T07:17:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.039+0000 7f6bb1fba740 1 -- 172.21.15.26:0/3726649867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b05f0 msgr2=0x5649f06c14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.039+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/3726649867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b05f0 0x5649f06c14a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6b98010010 tx=0x7f6b9800fcd0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.040+0000 7f6bb1fba740 1 -- 172.21.15.26:0/3726649867 shutdown_connections 2024-06-13T07:17:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.040+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/3726649867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06b5760 0x5649f06b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.040+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/3726649867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b05f0 0x5649f06c14a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.040+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/3726649867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06afc00 0x5649f06b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.040+0000 7f6bb1fba740 1 -- 172.21.15.26:0/3726649867 >> 172.21.15.26:0/3726649867 conn(0x5649f0677fb0 msgr2=0x5649f06b9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.040+0000 7f6bb1fba740 1 -- 172.21.15.26:0/3726649867 shutdown_connections 2024-06-13T07:17:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.041+0000 7f6bb1fba740 1 -- 172.21.15.26:0/3726649867 wait complete. 2024-06-13T07:17:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.042+0000 7f6bb1fba740 1 -- start start 2024-06-13T07:17:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.042+0000 7f6bb1fba740 1 Processor -- start 2024-06-13T07:17:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.043+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.043+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06bda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.043+0000 7f6bb1fba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 0x5649f06bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.043+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81af470 con 0x5649f06b5760 2024-06-13T07:17:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.043+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b08e0 con 0x5649f06afc00 2024-06-13T07:17:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.043+0000 7f6bb1fba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81afb40 con 0x5649f06b05f0 2024-06-13T07:17:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.043+0000 7f6bb1fba740 10 client.0 _ll_register_callbacks cb 0x7f6b9801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:17:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.043+0000 7f6ba7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.044+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06bda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.044+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06bda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48670/0 (socket says 172.21.15.26:48670) 2024-06-13T07:17:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.044+0000 7f6ba7e00640 1 -- 172.21.15.26:0/2519804633 learned_addr learned my addr 172.21.15.26:0/2519804633 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:17:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.044+0000 7f6ba7400640 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 0x5649f06bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.044+0000 7f6ba7e00640 1 -- 172.21.15.26:0/2519804633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 msgr2=0x5649f06c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.044+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.044+0000 7f6ba7e00640 1 -- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 msgr2=0x5649f06bdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:17:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.045+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 0x5649f06bdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.045+0000 7f6ba7e00640 1 -- 172.21.15.26:0/2519804633 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f06c1620 con 0x5649f06b05f0 2024-06-13T07:17:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.045+0000 7f6ba7400640 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 0x5649f06bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:17:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.046+0000 7f6ba7e00640 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06bda90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b9801afc0 tx=0x7f6b98005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2edda630781d8ee4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:17:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.047+0000 7f6b92000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:17:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.047+0000 7f6b92000640 1 -- 172.21.15.26:0/2519804633 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98027050 con 0x5649f06b05f0 2024-06-13T07:17:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.047+0000 7f6b92000640 1 -- 172.21.15.26:0/2519804633 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6b98031040 con 0x5649f06b05f0 2024-06-13T07:17:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.047+0000 7f6bb1fba740 1 -- 172.21.15.26:0/2519804633 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b98004550 con 0x5649f06b05f0 2024-06-13T07:17:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.048+0000 7f6b92000640 1 -- 172.21.15.26:0/2519804633 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6b9801a020 con 0x5649f06b05f0 2024-06-13T07:17:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.048+0000 7f6bb1fba740 10 client.26806 fetch_fsmap learned FSMap version 47 2024-06-13T07:17:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.048+0000 7f6bb1fba740 1 -- 172.21.15.26:0/2519804633 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5649f072f860 con 0x5649f06b05f0 2024-06-13T07:17:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.048+0000 7f6b92000640 1 -- 172.21.15.26:0/2519804633 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98004550 con 0x5649f06b05f0 2024-06-13T07:17:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.049+0000 7f6b92000640 1 -- 172.21.15.26:0/2519804633 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f6b98021070 con 0x5649f06b05f0 2024-06-13T07:17:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.049+0000 7f6bb1fba740 10 client.26806 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:17:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.049+0000 7f6bb1fba740 10 client.26806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:17:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.049+0000 7f6bb1fba740 20 client.26806 populate_metadata read hostname 'smithi026' 2024-06-13T07:17:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.049+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649f0730200 0x5649f0732620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:17:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.049+0000 7f6bb1fba740 4 client.26806 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:17:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.049+0000 7f6bb1fba740 1 -- 172.21.15.26:0/2519804633 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649f061aea0 con 0x5649f0730200 2024-06-13T07:17:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.050+0000 7f6ba7400640 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649f0730200 0x5649f0732620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:17:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.051+0000 7f6ba7400640 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649f0730200 0x5649f0732620 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6275fd95d6e07af4 server_cookie=455fb27885fe8e17 in_seq=0 out_seq=0 2024-06-13T07:17:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.051+0000 7f6b92000640 10 client.26806 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:48.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:47 smithi136 ceph-mon[37229]: pgmap v1028: 129 pgs: 129 active+clean; 607 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 687 op/s 2024-06-13T07:17:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:17:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:17:48.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:47 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:48.171 INFO:tasks.workunit.client.0.smithi026.stdout: 10 127475 28.34 MB/sec execute 121 sec latency 210.682 ms 2024-06-13T07:17:48.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:47 smithi026 ceph-mon[32041]: pgmap v1028: 129 pgs: 129 active+clean; 607 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 687 op/s 2024-06-13T07:17:48.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:17:48.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:17:48.253 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:47 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:17:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.433+0000 7f6b92000640 1 -- 172.21.15.26:0/2519804633 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5649f061aea0 con 0x5649f0730200 2024-06-13T07:17:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.433+0000 7f6b92000640 10 client.26806 handle_command_reply: tid=0 2024-06-13T07:17:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.433+0000 7f6bb1fba740 1 client.26806 shutdown 2024-06-13T07:17:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 -- 172.21.15.26:0/2519804633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 msgr2=0x5649f06bda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06bda90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b9801afc0 tx=0x7f6b98005770 comp rx=0 tx=0).stop 2024-06-13T07:17:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 -- 172.21.15.26:0/2519804633 shutdown_connections 2024-06-13T07:17:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649f0730200 0x5649f0732620 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649f06b5760 0x5649f06bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649f06b05f0 0x5649f06bda90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 --2- 172.21.15.26:0/2519804633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649f06afc00 0x5649f06c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 -- 172.21.15.26:0/2519804633 >> 172.21.15.26:0/2519804633 conn(0x5649f0677fb0 msgr2=0x5649f06b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6b92000640 0 client.26806 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:17:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.434+0000 7f6bb1fba740 1 -- 172.21.15.26:0/2519804633 shutdown_connections 2024-06-13T07:17:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.435+0000 7f6bb1fba740 1 -- 172.21.15.26:0/2519804633 wait complete. 2024-06-13T07:17:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.435+0000 7f6bb1fba740 20 client.26806 trim_cache size 0 max 16384 2024-06-13T07:17:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:17:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:17:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:17:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.439+0000 7f6baf200640 1 -- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b8c0a2cb0 msgr2=0x7f6b8c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.439+0000 7f6baf200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b8c0a2cb0 0x7f6b8c0a5170 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000ef90 tx=0x7f6ba001d000 comp rx=0 tx=0).stop 2024-06-13T07:17:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.439+0000 7f6baf200640 1 -- 172.21.15.26:0/314892889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba8077f40 msgr2=0x7f6ba81a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:17:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.439+0000 7f6baf200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba8077f40 0x7f6ba81a57d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000ec50 tx=0x7f6ba000c6a0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.441+0000 7f6baf200640 1 -- 172.21.15.26:0/314892889 shutdown_connections 2024-06-13T07:17:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.441+0000 7f6baf200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6b8c0a2cb0 0x7f6b8c0a5170 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.441+0000 7f6baf200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba8077f40 0x7f6ba81a57d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.441+0000 7f6baf200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba80775a0 0x7f6ba819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.441+0000 7f6baf200640 1 --2- 172.21.15.26:0/314892889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba8076360 0x7f6ba819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:17:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.441+0000 7f6baf200640 1 -- 172.21.15.26:0/314892889 >> 172.21.15.26:0/314892889 conn(0x7f6ba80ff190 msgr2=0x7f6ba8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:17:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.441+0000 7f6baf200640 1 -- 172.21.15.26:0/314892889 shutdown_connections 2024-06-13T07:17:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:17:48.442+0000 7f6baf200640 1 -- 172.21.15.26:0/314892889 wait complete. 2024-06-13T07:17:49.171 INFO:tasks.workunit.client.0.smithi026.stdout: 10 127711 28.19 MB/sec execute 122 sec latency 160.721 ms 2024-06-13T07:17:50.172 INFO:tasks.workunit.client.0.smithi026.stdout: 10 128198 28.10 MB/sec execute 123 sec latency 254.363 ms 2024-06-13T07:17:50.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:50 smithi136 ceph-mon[37229]: pgmap v1029: 129 pgs: 129 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 746 op/s 2024-06-13T07:17:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:50 smithi026 ceph-mon[32041]: pgmap v1029: 129 pgs: 129 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 746 op/s 2024-06-13T07:17:50.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:50 smithi135 ceph-mon[36276]: pgmap v1029: 129 pgs: 129 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 746 op/s 2024-06-13T07:17:51.172 INFO:tasks.workunit.client.0.smithi026.stdout: 10 129144 28.29 MB/sec execute 124 sec latency 65.381 ms 2024-06-13T07:17:52.172 INFO:tasks.workunit.client.0.smithi026.stdout: 10 129956 28.51 MB/sec execute 125 sec latency 37.397 ms 2024-06-13T07:17:52.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:52 smithi136 ceph-mon[37229]: pgmap v1030: 129 pgs: 129 active+clean; 614 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.3 MiB/s wr, 547 op/s 2024-06-13T07:17:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:52 smithi026 ceph-mon[32041]: pgmap v1030: 129 pgs: 129 active+clean; 614 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.3 MiB/s wr, 547 op/s 2024-06-13T07:17:52.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:52 smithi135 ceph-mon[36276]: pgmap v1030: 129 pgs: 129 active+clean; 614 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.3 MiB/s wr, 547 op/s 2024-06-13T07:17:53.172 INFO:tasks.workunit.client.0.smithi026.stdout: 10 130540 28.58 MB/sec execute 126 sec latency 77.889 ms 2024-06-13T07:17:54.172 INFO:tasks.workunit.client.0.smithi026.stdout: 10 130801 28.44 MB/sec execute 127 sec latency 204.722 ms 2024-06-13T07:17:54.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:54 smithi136 ceph-mon[37229]: pgmap v1031: 129 pgs: 129 active+clean; 608 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.6 MiB/s wr, 563 op/s 2024-06-13T07:17:54.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:54 smithi026 ceph-mon[32041]: pgmap v1031: 129 pgs: 129 active+clean; 608 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.6 MiB/s wr, 563 op/s 2024-06-13T07:17:54.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:54 smithi135 ceph-mon[36276]: pgmap v1031: 129 pgs: 129 active+clean; 608 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.6 MiB/s wr, 563 op/s 2024-06-13T07:17:55.172 INFO:tasks.workunit.client.0.smithi026.stdout: 10 131269 28.39 MB/sec execute 128 sec latency 90.070 ms 2024-06-13T07:17:56.173 INFO:tasks.workunit.client.0.smithi026.stdout: 10 131891 28.37 MB/sec execute 129 sec latency 86.745 ms 2024-06-13T07:17:56.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:56 smithi136 ceph-mon[37229]: pgmap v1032: 129 pgs: 129 active+clean; 607 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.0 MiB/s wr, 570 op/s 2024-06-13T07:17:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:56 smithi026 ceph-mon[32041]: pgmap v1032: 129 pgs: 129 active+clean; 607 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.0 MiB/s wr, 570 op/s 2024-06-13T07:17:56.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:56 smithi135 ceph-mon[36276]: pgmap v1032: 129 pgs: 129 active+clean; 607 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.0 MiB/s wr, 570 op/s 2024-06-13T07:17:57.173 INFO:tasks.workunit.client.0.smithi026.stdout: 10 132549 28.43 MB/sec execute 130 sec latency 91.812 ms 2024-06-13T07:17:58.173 INFO:tasks.workunit.client.0.smithi026.stdout: 10 133253 28.56 MB/sec execute 131 sec latency 47.656 ms 2024-06-13T07:17:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:17:58 smithi136 ceph-mon[37229]: pgmap v1033: 129 pgs: 129 active+clean; 612 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 551 op/s 2024-06-13T07:17:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:17:58 smithi026 ceph-mon[32041]: pgmap v1033: 129 pgs: 129 active+clean; 612 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 551 op/s 2024-06-13T07:17:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:17:58 smithi135 ceph-mon[36276]: pgmap v1033: 129 pgs: 129 active+clean; 612 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 551 op/s 2024-06-13T07:17:59.173 INFO:tasks.workunit.client.0.smithi026.stdout: 10 133828 28.56 MB/sec execute 132 sec latency 57.801 ms 2024-06-13T07:18:00.174 INFO:tasks.workunit.client.0.smithi026.stdout: 10 134310 28.56 MB/sec execute 133 sec latency 92.713 ms 2024-06-13T07:18:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:00 smithi136 ceph-mon[37229]: pgmap v1034: 129 pgs: 129 active+clean; 616 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-06-13T07:18:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:00 smithi026 ceph-mon[32041]: pgmap v1034: 129 pgs: 129 active+clean; 616 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-06-13T07:18:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:00 smithi135 ceph-mon[36276]: pgmap v1034: 129 pgs: 129 active+clean; 616 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-06-13T07:18:01.173 INFO:tasks.workunit.client.0.smithi026.stdout: 10 134928 28.55 MB/sec execute 134 sec latency 72.823 ms 2024-06-13T07:18:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.631+0000 7f4ec4a00640 1 -- 172.21.15.26:0/201248855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec0104c70 msgr2=0x7f4ec0107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.631+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/201248855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec0104c70 0x7f4ec0107060 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009900 tx=0x7f4eac02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.631+0000 7f4ec4a00640 1 -- 172.21.15.26:0/201248855 shutdown_connections 2024-06-13T07:18:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.631+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/201248855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec01075a0 0x7f4ec0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.631+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/201248855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec0104c70 0x7f4ec0107060 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009900 tx=0x7f4eac02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.631+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/201248855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec0102340 0x7f4ec0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.631+0000 7f4ec4a00640 1 -- 172.21.15.26:0/201248855 >> 172.21.15.26:0/201248855 conn(0x7f4ec00fbc90 msgr2=0x7f4ec00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.632+0000 7f4ec4a00640 1 -- 172.21.15.26:0/201248855 shutdown_connections 2024-06-13T07:18:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.632+0000 7f4ec4a00640 1 -- 172.21.15.26:0/201248855 wait complete. 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.633+0000 7f4ec4a00640 1 Processor -- start 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.633+0000 7f4ec4a00640 1 -- start start 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.633+0000 7f4ec4a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec0102340 0x7f4ec0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.633+0000 7f4ec4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec01075a0 0x7f4ec019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.633+0000 7f4ec4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec019f4e0 0x7f4ec01a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.633+0000 7f4ec4a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010d470 con 0x7f4ec019f4e0 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.633+0000 7f4ec4a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010d2f0 con 0x7f4ec0102340 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.633+0000 7f4ec4a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010d5f0 con 0x7f4ec01075a0 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec0102340 0x7f4ec0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec0102340 0x7f4ec0199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53336/0 (socket says 172.21.15.26:53336) 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebe800640 1 -- 172.21.15.26:0/3684667665 learned_addr learned my addr 172.21.15.26:0/3684667665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebde00640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec01075a0 0x7f4ec019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebf200640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec019f4e0 0x7f4ec01a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebe800640 1 -- 172.21.15.26:0/3684667665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec01075a0 msgr2=0x7f4ec019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebe800640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec01075a0 0x7f4ec019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebe800640 1 -- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec019f4e0 msgr2=0x7f4ec01a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebe800640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec019f4e0 0x7f4ec01a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.634+0000 7f4ebe800640 1 -- 172.21.15.26:0/3684667665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec01a1f40 con 0x7f4ec0102340 2024-06-13T07:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.635+0000 7f4ebe800640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec0102340 0x7f4ec0199f40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000d950 tx=0x7f4eb000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.635+0000 7f4eb7400640 1 -- 172.21.15.26:0/3684667665 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0017030 con 0x7f4ec0102340 2024-06-13T07:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.635+0000 7f4eb7400640 1 -- 172.21.15.26:0/3684667665 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb0004020 con 0x7f4ec0102340 2024-06-13T07:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.635+0000 7f4eb7400640 1 -- 172.21.15.26:0/3684667665 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00047a0 con 0x7f4ec0102340 2024-06-13T07:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.635+0000 7f4ec4a00640 1 -- 172.21.15.26:0/3684667665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec01a2230 con 0x7f4ec0102340 2024-06-13T07:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.635+0000 7f4ec4a00640 1 -- 172.21.15.26:0/3684667665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec01a2700 con 0x7f4ec0102340 2024-06-13T07:18:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.642+0000 7f4eb7400640 1 -- 172.21.15.26:0/3684667665 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4eb0009a30 con 0x7f4ec0102340 2024-06-13T07:18:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.643+0000 7f4ec81c2740 1 Processor -- start 2024-06-13T07:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.643+0000 7f4ec81c2740 1 -- start start 2024-06-13T07:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.643+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d34dc0 0x559d95d351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.643+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d357b0 0x559d95d3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.643+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d3a920 0x559d95d3cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.643+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d95c98420 con 0x559d95d34dc0 2024-06-13T07:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.644+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559d95c3fc50 con 0x559d95d3a920 2024-06-13T07:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.644+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d95c943e0 con 0x559d95d357b0 2024-06-13T07:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.644+0000 7f4eb7400640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ea40a2c00 0x7f4ea40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.644+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d3a920 0x559d95d3cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.644+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d3a920 0x559d95d3cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53342/0 (socket says 172.21.15.26:53342) 2024-06-13T07:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.644+0000 7f4eb7400640 1 -- 172.21.15.26:0/3684667665 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4eb00cef00 con 0x7f4ec0102340 2024-06-13T07:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.644+0000 7f4ebde00640 1 -- 172.21.15.26:0/2391126545 learned_addr learned my addr 172.21.15.26:0/2391126545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.645+0000 7f4ebde00640 1 -- 172.21.15.26:0/2391126545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d357b0 msgr2=0x559d95d3a2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.645+0000 7f4ebde00640 1 --2- 172.21.15.26:0/2391126545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d357b0 0x559d95d3a2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.645+0000 7f4ebde00640 1 -- 172.21.15.26:0/2391126545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d34dc0 msgr2=0x559d95d351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.645+0000 7f4ebde00640 1 --2- 172.21.15.26:0/2391126545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d34dc0 0x559d95d351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.645+0000 7f4ebde00640 1 -- 172.21.15.26:0/2391126545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d959ac210 con 0x559d95d3a920 2024-06-13T07:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.645+0000 7f4ebde00640 1 --2- 172.21.15.26:0/2391126545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d3a920 0x559d95d3cdb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0329c7014b47ef server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.645+0000 7f4ebf200640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ea40a2c00 0x7f4ea40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.646+0000 7f4eab400640 1 -- 172.21.15.26:0/2391126545 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac035020 con 0x559d95d3a920 2024-06-13T07:18:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.646+0000 7f4eab400640 1 -- 172.21.15.26:0/2391126545 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac005320 con 0x559d95d3a920 2024-06-13T07:18:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.646+0000 7f4eab400640 1 -- 172.21.15.26:0/2391126545 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0046c0 con 0x559d95d3a920 2024-06-13T07:18:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.648+0000 7f4ec81c2740 1 -- 172.21.15.26:0/2391126545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d3a920 msgr2=0x559d95d3cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.648+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/2391126545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d3a920 0x559d95d3cdb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac005dd0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.648+0000 7f4ebf200640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ea40a2c00 0x7f4ea40a50c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8005c40 tx=0x7f4eb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.648+0000 7f4ec81c2740 1 -- 172.21.15.26:0/2391126545 shutdown_connections 2024-06-13T07:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.648+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/2391126545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d3a920 0x559d95d3cdb0 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac005dd0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.648+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/2391126545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d357b0 0x559d95d3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.648+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/2391126545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d34dc0 0x559d95d351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.648+0000 7f4ec81c2740 1 -- 172.21.15.26:0/2391126545 >> 172.21.15.26:0/2391126545 conn(0x559d95cfcfb0 msgr2=0x559d95d30840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.650+0000 7f4ec81c2740 1 -- 172.21.15.26:0/2391126545 shutdown_connections 2024-06-13T07:18:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.650+0000 7f4ec81c2740 1 -- 172.21.15.26:0/2391126545 wait complete. 2024-06-13T07:18:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.651+0000 7f4ec81c2740 1 -- start start 2024-06-13T07:18:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.651+0000 7f4ec81c2740 1 Processor -- start 2024-06-13T07:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d34dc0 0x559d95d3f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d357b0 0x559d95d3f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d44880 0x559d95d46c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d95d3d870 con 0x559d95d34dc0 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559d95c530b0 con 0x559d95d357b0 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d95d3d9f0 con 0x559d95d44880 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ec81c2740 10 client.0 _ll_register_callbacks cb 0x559d95c3ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d357b0 0x559d95d3f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d357b0 0x559d95d3f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53348/0 (socket says 172.21.15.26:53348) 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ebde00640 1 -- 172.21.15.26:0/235668700 learned_addr learned my addr 172.21.15.26:0/235668700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.652+0000 7f4ebf200640 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d44880 0x559d95d46c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.653+0000 7f4ebde00640 1 -- 172.21.15.26:0/235668700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d44880 msgr2=0x559d95d46c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.653+0000 7f4ebde00640 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d44880 0x559d95d46c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.653+0000 7f4ebde00640 1 -- 172.21.15.26:0/235668700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d34dc0 msgr2=0x559d95d3f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.653+0000 7f4ebde00640 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d34dc0 0x559d95d3f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.653+0000 7f4ebde00640 1 -- 172.21.15.26:0/235668700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d95a78620 con 0x559d95d357b0 2024-06-13T07:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.653+0000 7f4ebde00640 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d357b0 0x559d95d3f850 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac0319d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f13c5bb39e68ca4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.653+0000 7f4e9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:18:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.656+0000 7f4e9fe00640 1 -- 172.21.15.26:0/235668700 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac04d070 con 0x559d95d357b0 2024-06-13T07:18:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.656+0000 7f4e9fe00640 1 -- 172.21.15.26:0/235668700 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4eac004ad0 con 0x559d95d357b0 2024-06-13T07:18:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.656+0000 7f4ec81c2740 1 -- 172.21.15.26:0/235668700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eac005320 con 0x559d95d357b0 2024-06-13T07:18:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.657+0000 7f4e9fe00640 1 -- 172.21.15.26:0/235668700 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4eac035040 con 0x559d95d357b0 2024-06-13T07:18:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.657+0000 7f4ec81c2740 10 client.26795 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.657+0000 7f4ec81c2740 1 -- 172.21.15.26:0/235668700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559d95d5aff0 con 0x559d95d357b0 2024-06-13T07:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.657+0000 7f4e9fe00640 1 -- 172.21.15.26:0/235668700 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac005320 con 0x559d95d357b0 2024-06-13T07:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.658+0000 7f4e9fe00640 1 -- 172.21.15.26:0/235668700 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f4eac03a040 con 0x559d95d357b0 2024-06-13T07:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.658+0000 7f4ec81c2740 10 client.26795 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.658+0000 7f4ec81c2740 10 client.26795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.658+0000 7f4ec81c2740 20 client.26795 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.658+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559d95d5c340 0x559d95d5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.658+0000 7f4ec81c2740 4 client.26795 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.658+0000 7f4ec81c2740 1 -- 172.21.15.26:0/235668700 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d95ca5b50 con 0x559d95d5c340 2024-06-13T07:18:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.659+0000 7f4ebe800640 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559d95d5c340 0x559d95d5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.661+0000 7f4ebe800640 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559d95d5c340 0x559d95d5e730 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1990ab41eb74a2f4 server_cookie=94056a2d8ee85d07 in_seq=0 out_seq=0 2024-06-13T07:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.661+0000 7f4e9fe00640 10 client.26795 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.671+0000 7f4e9fe00640 1 -- 172.21.15.26:0/235668700 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d95ca5b50 con 0x559d95d5c340 2024-06-13T07:18:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.671+0000 7f4e9fe00640 10 client.26795 handle_command_reply: tid=0 2024-06-13T07:18:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 client.26795 shutdown 2024-06-13T07:18:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 -- 172.21.15.26:0/235668700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d357b0 msgr2=0x559d95d3f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d357b0 0x559d95d3f850 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac0319d0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 -- 172.21.15.26:0/235668700 shutdown_connections 2024-06-13T07:18:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559d95d5c340 0x559d95d5e730 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d357b0 0x559d95d3f850 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d44880 0x559d95d46c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/235668700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d34dc0 0x559d95d3f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.672+0000 7f4ec81c2740 1 -- 172.21.15.26:0/235668700 >> 172.21.15.26:0/235668700 conn(0x559d95cfcfb0 msgr2=0x559d95d2f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.674+0000 7f4e9fe00640 0 client.26795 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.674+0000 7f4ec81c2740 1 -- 172.21.15.26:0/235668700 shutdown_connections 2024-06-13T07:18:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.674+0000 7f4ec81c2740 1 -- 172.21.15.26:0/235668700 wait complete. 2024-06-13T07:18:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.675+0000 7f4ec81c2740 20 client.26795 trim_cache size 0 max 16384 2024-06-13T07:18:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.703+0000 7f4ec81c2740 1 Processor -- start 2024-06-13T07:18:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.703+0000 7f4ec81c2740 1 -- start start 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.703+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d34dc0 0x559d95d36cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.703+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 0x559d95d371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.703+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d5e9c0 0x559d95d2f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.703+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01a6e70 con 0x559d95d44880 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.703+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01a4740 con 0x559d95d34dc0 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.703+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00737a0 con 0x559d95d5e9c0 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebf200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d5e9c0 0x559d95d2f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebf200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d5e9c0 0x559d95d2f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53560/0 (socket says 172.21.15.26:53560) 2024-06-13T07:18:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebf200640 1 -- 172.21.15.26:0/3041502830 learned_addr learned my addr 172.21.15.26:0/3041502830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebf200640 1 -- 172.21.15.26:0/3041502830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d34dc0 msgr2=0x559d95d36cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebe800640 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d34dc0 0x559d95d36cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebf200640 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d34dc0 0x559d95d36cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebf200640 1 -- 172.21.15.26:0/3041502830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 msgr2=0x559d95d371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebf200640 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 0x559d95d371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebf200640 1 -- 172.21.15.26:0/3041502830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d95d2c000 con 0x559d95d5e9c0 2024-06-13T07:18:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.704+0000 7f4ebe800640 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d34dc0 0x559d95d36cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.705+0000 7f4ebf200640 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d5e9c0 0x559d95d2f110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8011570 tx=0x7f4eb8012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e00e94c23ae2ad server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.705+0000 7f4eaaa00640 1 -- 172.21.15.26:0/3041502830 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8015070 con 0x559d95d5e9c0 2024-06-13T07:18:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.705+0000 7f4eaaa00640 1 -- 172.21.15.26:0/3041502830 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb801d8f0 con 0x559d95d5e9c0 2024-06-13T07:18:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.705+0000 7f4eaaa00640 1 -- 172.21.15.26:0/3041502830 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8010970 con 0x559d95d5e9c0 2024-06-13T07:18:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.707+0000 7f4ec81c2740 1 -- 172.21.15.26:0/3041502830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d5e9c0 msgr2=0x559d95d2f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.707+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d5e9c0 0x559d95d2f110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8011570 tx=0x7f4eb8012000 comp rx=0 tx=0).stop 2024-06-13T07:18:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.707+0000 7f4ec81c2740 1 -- 172.21.15.26:0/3041502830 shutdown_connections 2024-06-13T07:18:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.707+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d5e9c0 0x559d95d2f110 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.707+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 0x559d95d371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.707+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/3041502830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d34dc0 0x559d95d36cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.707+0000 7f4ec81c2740 1 -- 172.21.15.26:0/3041502830 >> 172.21.15.26:0/3041502830 conn(0x559d95cfcfb0 msgr2=0x559d95d3f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.708+0000 7f4ec81c2740 1 -- 172.21.15.26:0/3041502830 shutdown_connections 2024-06-13T07:18:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.708+0000 7f4ec81c2740 1 -- 172.21.15.26:0/3041502830 wait complete. 2024-06-13T07:18:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.708+0000 7f4ec81c2740 1 -- start start 2024-06-13T07:18:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.709+0000 7f4ec81c2740 1 Processor -- start 2024-06-13T07:18:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.710+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d34dc0 0x559d95d6d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.710+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 0x559d95d6ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.710+0000 7f4ec81c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d5e9c0 0x559d95d6edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.710+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01a2c00 con 0x559d95d44880 2024-06-13T07:18:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.710+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01a4070 con 0x559d95d5e9c0 2024-06-13T07:18:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.710+0000 7f4ec81c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01a32d0 con 0x559d95d34dc0 2024-06-13T07:18:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.710+0000 7f4ec81c2740 10 client.0 _ll_register_callbacks cb 0x7f4eac031660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d34dc0 0x559d95d6d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d34dc0 0x559d95d6d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53568/0 (socket says 172.21.15.26:53568) 2024-06-13T07:18:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebe800640 1 -- 172.21.15.26:0/1566740122 learned_addr learned my addr 172.21.15.26:0/1566740122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebde00640 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 0x559d95d6ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebe800640 1 -- 172.21.15.26:0/1566740122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d5e9c0 msgr2=0x559d95d6edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebe800640 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d5e9c0 0x559d95d6edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebe800640 1 -- 172.21.15.26:0/1566740122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 msgr2=0x559d95d6ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebe800640 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 0x559d95d6ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.712+0000 7f4ebe800640 1 -- 172.21.15.26:0/1566740122 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d95d6f3a0 con 0x559d95d34dc0 2024-06-13T07:18:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.713+0000 7f4ebe800640 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d34dc0 0x559d95d6d8b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009610 tx=0x7f4eb0004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba0470df8b0dcb1a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.713+0000 7f4e9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:18:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.714+0000 7f4e9fe00640 1 -- 172.21.15.26:0/1566740122 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb009f720 con 0x559d95d34dc0 2024-06-13T07:18:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.714+0000 7f4e9fe00640 1 -- 172.21.15.26:0/1566740122 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4eb0099670 con 0x559d95d34dc0 2024-06-13T07:18:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.714+0000 7f4ec81c2740 1 -- 172.21.15.26:0/1566740122 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb801d8f0 con 0x559d95d34dc0 2024-06-13T07:18:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.716+0000 7f4e9fe00640 1 -- 172.21.15.26:0/1566740122 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4eb00a3fe0 con 0x559d95d34dc0 2024-06-13T07:18:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.716+0000 7f4ec81c2740 10 client.26821 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.716+0000 7f4ec81c2740 1 -- 172.21.15.26:0/1566740122 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559d95d70690 con 0x559d95d34dc0 2024-06-13T07:18:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.717+0000 7f4e9fe00640 1 -- 172.21.15.26:0/1566740122 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb00b3020 con 0x559d95d34dc0 2024-06-13T07:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.717+0000 7f4e9fe00640 1 -- 172.21.15.26:0/1566740122 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f4eb00979e0 con 0x559d95d34dc0 2024-06-13T07:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.717+0000 7f4ec81c2740 10 client.26821 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.717+0000 7f4ec81c2740 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.717+0000 7f4ec81c2740 20 client.26821 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.717+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559d95dc13a0 0x559d95dc3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.717+0000 7f4ec81c2740 4 client.26821 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.717+0000 7f4ec81c2740 1 -- 172.21.15.26:0/1566740122 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d95c9fea0 con 0x559d95dc13a0 2024-06-13T07:18:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.718+0000 7f4ebde00640 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559d95dc13a0 0x559d95dc3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.720+0000 7f4e9fe00640 10 client.26821 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:01.720+0000 7f4ebde00640 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559d95dc13a0 0x559d95dc3790 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad934e7b0dc05fca server_cookie=e2f9a614dc2a3a61 in_seq=0 out_seq=0 2024-06-13T07:18:02.173 INFO:tasks.workunit.client.0.smithi026.stdout: 10 135294 28.51 MB/sec execute 135 sec latency 337.198 ms 2024-06-13T07:18:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.346+0000 7f4e9fe00640 1 -- 172.21.15.26:0/1566740122 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559d95c9fea0 con 0x559d95dc13a0 2024-06-13T07:18:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.346+0000 7f4e9fe00640 10 client.26821 handle_command_reply: tid=0 2024-06-13T07:18:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.346+0000 7f4ec81c2740 1 client.26821 shutdown 2024-06-13T07:18:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:18:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:18:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:18:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.346+0000 7f4ec81c2740 1 -- 172.21.15.26:0/1566740122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d34dc0 msgr2=0x559d95d6d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d34dc0 0x559d95d6d8b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009610 tx=0x7f4eb0004520 comp rx=0 tx=0).stop 2024-06-13T07:18:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 -- 172.21.15.26:0/1566740122 shutdown_connections 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559d95dc13a0 0x559d95dc3790 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d95d5e9c0 0x559d95d6edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d95d44880 0x559d95d6ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 --2- 172.21.15.26:0/1566740122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d95d34dc0 0x559d95d6d8b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 -- 172.21.15.26:0/1566740122 >> 172.21.15.26:0/1566740122 conn(0x559d95cfcfb0 msgr2=0x559d95d3f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4e9fe00640 0 client.26821 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 -- 172.21.15.26:0/1566740122 shutdown_connections 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 1 -- 172.21.15.26:0/1566740122 wait complete. 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.347+0000 7f4ec81c2740 20 client.26821 trim_cache size 0 max 16384 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.348+0000 7f4ec4a00640 1 -- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ea40a2c00 msgr2=0x7f4ea40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.348+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ea40a2c00 0x7f4ea40a50c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8005c40 tx=0x7f4eb8009260 comp rx=0 tx=0).stop 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.348+0000 7f4ec4a00640 1 -- 172.21.15.26:0/3684667665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec0102340 msgr2=0x7f4ec0199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.348+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec0102340 0x7f4ec0199f40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000d950 tx=0x7f4eb000de10 comp rx=0 tx=0).stop 2024-06-13T07:18:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.349+0000 7f4ec4a00640 1 -- 172.21.15.26:0/3684667665 shutdown_connections 2024-06-13T07:18:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.349+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ea40a2c00 0x7f4ea40a50c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.349+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec019f4e0 0x7f4ec01a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.349+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec01075a0 0x7f4ec019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.349+0000 7f4ec4a00640 1 --2- 172.21.15.26:0/3684667665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec0102340 0x7f4ec0199f40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.349+0000 7f4ec4a00640 1 -- 172.21.15.26:0/3684667665 >> 172.21.15.26:0/3684667665 conn(0x7f4ec00fbc90 msgr2=0x7f4ec0101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.349+0000 7f4ec4a00640 1 -- 172.21.15.26:0/3684667665 shutdown_connections 2024-06-13T07:18:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:02.349+0000 7f4ec4a00640 1 -- 172.21.15.26:0/3684667665 wait complete. 2024-06-13T07:18:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:02 smithi136 ceph-mon[37229]: pgmap v1035: 129 pgs: 129 active+clean; 621 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-06-13T07:18:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:02 smithi026 ceph-mon[32041]: pgmap v1035: 129 pgs: 129 active+clean; 621 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-06-13T07:18:02.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:02 smithi135 ceph-mon[36276]: pgmap v1035: 129 pgs: 129 active+clean; 621 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-06-13T07:18:03.173 INFO:tasks.workunit.client.0.smithi026.stdout: 10 135902 28.57 MB/sec execute 136 sec latency 109.254 ms 2024-06-13T07:18:04.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 136452 28.57 MB/sec execute 137 sec latency 55.385 ms 2024-06-13T07:18:04.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:04 smithi136 ceph-mon[37229]: pgmap v1036: 129 pgs: 129 active+clean; 626 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.6 MiB/s wr, 472 op/s 2024-06-13T07:18:04.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:04 smithi026 ceph-mon[32041]: pgmap v1036: 129 pgs: 129 active+clean; 626 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.6 MiB/s wr, 472 op/s 2024-06-13T07:18:04.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:04 smithi135 ceph-mon[36276]: pgmap v1036: 129 pgs: 129 active+clean; 626 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.6 MiB/s wr, 472 op/s 2024-06-13T07:18:05.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 136989 28.54 MB/sec execute 138 sec latency 100.530 ms 2024-06-13T07:18:06.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 137667 28.60 MB/sec execute 139 sec latency 68.169 ms 2024-06-13T07:18:06.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:06 smithi136 ceph-mon[37229]: pgmap v1037: 129 pgs: 129 active+clean; 626 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 475 op/s 2024-06-13T07:18:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:06 smithi026 ceph-mon[32041]: pgmap v1037: 129 pgs: 129 active+clean; 626 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 475 op/s 2024-06-13T07:18:06.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:06 smithi135 ceph-mon[36276]: pgmap v1037: 129 pgs: 129 active+clean; 626 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 475 op/s 2024-06-13T07:18:07.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 138226 28.59 MB/sec execute 140 sec latency 195.844 ms 2024-06-13T07:18:08.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 138445 28.47 MB/sec execute 141 sec latency 424.935 ms 2024-06-13T07:18:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:08 smithi136 ceph-mon[37229]: pgmap v1038: 129 pgs: 129 active+clean; 629 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 434 op/s 2024-06-13T07:18:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:08 smithi026 ceph-mon[32041]: pgmap v1038: 129 pgs: 129 active+clean; 629 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 434 op/s 2024-06-13T07:18:08.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:08 smithi135 ceph-mon[36276]: pgmap v1038: 129 pgs: 129 active+clean; 629 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 434 op/s 2024-06-13T07:18:09.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 138947 28.51 MB/sec execute 142 sec latency 108.442 ms 2024-06-13T07:18:10.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 139393 28.48 MB/sec execute 143 sec latency 276.609 ms 2024-06-13T07:18:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:10 smithi136 ceph-mon[37229]: pgmap v1039: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.0 MiB/s wr, 572 op/s 2024-06-13T07:18:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:10 smithi026 ceph-mon[32041]: pgmap v1039: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.0 MiB/s wr, 572 op/s 2024-06-13T07:18:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:10 smithi135 ceph-mon[36276]: pgmap v1039: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.0 MiB/s wr, 572 op/s 2024-06-13T07:18:11.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 139868 28.48 MB/sec execute 144 sec latency 64.866 ms 2024-06-13T07:18:12.175 INFO:tasks.workunit.client.0.smithi026.stdout: 10 140186 28.38 MB/sec execute 145 sec latency 101.129 ms 2024-06-13T07:18:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:12 smithi136 ceph-mon[37229]: pgmap v1040: 129 pgs: 129 active+clean; 626 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.9 MiB/s wr, 534 op/s 2024-06-13T07:18:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:12 smithi026 ceph-mon[32041]: pgmap v1040: 129 pgs: 129 active+clean; 626 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.9 MiB/s wr, 534 op/s 2024-06-13T07:18:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:12 smithi135 ceph-mon[36276]: pgmap v1040: 129 pgs: 129 active+clean; 626 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.9 MiB/s wr, 534 op/s 2024-06-13T07:18:13.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 140210 28.18 MB/sec execute 146 sec latency 1043.149 ms 2024-06-13T07:18:14.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 140280 28.01 MB/sec execute 147 sec latency 1984.874 ms 2024-06-13T07:18:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:14 smithi136 ceph-mon[37229]: pgmap v1041: 129 pgs: 129 active+clean; 638 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 580 op/s 2024-06-13T07:18:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:14 smithi026 ceph-mon[32041]: pgmap v1041: 129 pgs: 129 active+clean; 638 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 580 op/s 2024-06-13T07:18:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:14 smithi135 ceph-mon[36276]: pgmap v1041: 129 pgs: 129 active+clean; 638 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 580 op/s 2024-06-13T07:18:15.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 140545 27.90 MB/sec execute 148 sec latency 1727.621 ms 2024-06-13T07:18:16.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 140800 27.79 MB/sec execute 149 sec latency 166.503 ms 2024-06-13T07:18:16.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:16 smithi026 ceph-mon[32041]: pgmap v1042: 129 pgs: 129 active+clean; 670 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.4 MiB/s wr, 593 op/s 2024-06-13T07:18:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:16 smithi136 ceph-mon[37229]: pgmap v1042: 129 pgs: 129 active+clean; 670 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.4 MiB/s wr, 593 op/s 2024-06-13T07:18:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.391+0000 7f1004800640 1 -- 172.21.15.26:0/2350804719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc102bf0 msgr2=0x7f0ffc103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.391+0000 7f1004800640 1 --2- 172.21.15.26:0/2350804719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc102bf0 0x7f0ffc103070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800fb10 tx=0x7f0fe8032990 comp rx=0 tx=0).stop 2024-06-13T07:18:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.392+0000 7f1004800640 1 -- 172.21.15.26:0/2350804719 shutdown_connections 2024-06-13T07:18:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.392+0000 7f1004800640 1 --2- 172.21.15.26:0/2350804719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc1035b0 0x7f0ffc109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.394+0000 7f1004800640 1 --2- 172.21.15.26:0/2350804719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc102bf0 0x7f0ffc103070 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.394+0000 7f1004800640 1 --2- 172.21.15.26:0/2350804719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ffc101a00 0x7f0ffc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.394+0000 7f1004800640 1 -- 172.21.15.26:0/2350804719 >> 172.21.15.26:0/2350804719 conn(0x7f0ffc0fcf90 msgr2=0x7f0ffc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.394+0000 7f1004800640 1 -- 172.21.15.26:0/2350804719 shutdown_connections 2024-06-13T07:18:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.395+0000 7f1004800640 1 -- 172.21.15.26:0/2350804719 wait complete. 2024-06-13T07:18:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.396+0000 7f1004800640 1 Processor -- start 2024-06-13T07:18:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.396+0000 7f1004800640 1 -- start start 2024-06-13T07:18:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1004800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc101a00 0x7f0ffc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1004800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ffc102bf0 0x7f0ffc070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1004800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc1035b0 0x7f0ffc1a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1004800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10b550 con 0x7f0ffc101a00 2024-06-13T07:18:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1004800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10b3d0 con 0x7f0ffc102bf0 2024-06-13T07:18:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1004800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10b6d0 con 0x7f0ffc1035b0 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1002800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc1035b0 0x7f0ffc1a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1001400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ffc102bf0 0x7f0ffc070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1002800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc1035b0 0x7f0ffc1a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39608/0 (socket says 172.21.15.26:39608) 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1001400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ffc102bf0 0x7f0ffc070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:46174/0 (socket says 172.21.15.26:46174) 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.397+0000 7f1002800640 1 -- 172.21.15.26:0/1778894481 learned_addr learned my addr 172.21.15.26:0/1778894481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1001e00640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc101a00 0x7f0ffc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1002800640 1 -- 172.21.15.26:0/1778894481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ffc102bf0 msgr2=0x7f0ffc070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1002800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ffc102bf0 0x7f0ffc070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1002800640 1 -- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc101a00 msgr2=0x7f0ffc070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1002800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc101a00 0x7f0ffc070220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1002800640 1 -- 172.21.15.26:0/1778894481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc0738e0 con 0x7f0ffc1035b0 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1001e00640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc101a00 0x7f0ffc070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:18:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1001400640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ffc102bf0 0x7f0ffc070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:18:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.398+0000 7f1002800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc1035b0 0x7f0ffc1a5c60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800b970 tx=0x7f0ff800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.399+0000 7f0ff6a00640 1 -- 172.21.15.26:0/1778894481 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8019070 con 0x7f0ffc1035b0 2024-06-13T07:18:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.399+0000 7f0ff6a00640 1 -- 172.21.15.26:0/1778894481 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff800cb40 con 0x7f0ffc1035b0 2024-06-13T07:18:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.399+0000 7f0ff6a00640 1 -- 172.21.15.26:0/1778894481 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80076a0 con 0x7f0ffc1035b0 2024-06-13T07:18:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.400+0000 7f1004800640 1 -- 172.21.15.26:0/1778894481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc1a61a0 con 0x7f0ffc1035b0 2024-06-13T07:18:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.403+0000 7f1004800640 1 -- 172.21.15.26:0/1778894481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a66f0 con 0x7f0ffc1035b0 2024-06-13T07:18:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.411+0000 7f0ff6a00640 1 -- 172.21.15.26:0/1778894481 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ff801c070 con 0x7f0ffc1035b0 2024-06-13T07:18:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.414+0000 7f0ff6a00640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0fe40a2d00 0x7f0fe40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.414+0000 7f100764e740 1 Processor -- start 2024-06-13T07:18:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.414+0000 7f0ff6a00640 1 -- 172.21.15.26:0/1778894481 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0ff80ce4d0 con 0x7f0ffc1035b0 2024-06-13T07:18:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.414+0000 7f1001e00640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0fe40a2d00 0x7f0fe40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.416+0000 7f1001e00640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0fe40a2d00 0x7f0fe40a51c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc102a50 tx=0x7f0ff0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.417+0000 7f100764e740 1 -- start start 2024-06-13T07:18:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.417+0000 7f100764e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380de020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.417+0000 7f100764e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.417+0000 7f100764e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.417+0000 7f100764e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555d38041420 con 0x555d380e3760 2024-06-13T07:18:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.417+0000 7f100764e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d37fe8c50 con 0x555d380ddc00 2024-06-13T07:18:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.417+0000 7f100764e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555d3803d3e0 con 0x555d380de5f0 2024-06-13T07:18:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.418+0000 7f1002800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.418+0000 7f1002800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380e5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57392/0 (socket says 172.21.15.26:57392) 2024-06-13T07:18:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.418+0000 7f1002800640 1 -- 172.21.15.26:0/2069867471 learned_addr learned my addr 172.21.15.26:0/2069867471 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.419+0000 7f1001e00640 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.420+0000 7f1001400640 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.420+0000 7f1001e00640 1 -- 172.21.15.26:0/2069867471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 msgr2=0x555d380e30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.420+0000 7f1001e00640 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.420+0000 7f1001e00640 1 -- 172.21.15.26:0/2069867471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 msgr2=0x555d380e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.420+0000 7f1001e00640 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.420+0000 7f1001e00640 1 -- 172.21.15.26:0/2069867471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d37d55210 con 0x555d380ddc00 2024-06-13T07:18:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.422+0000 7f1001e00640 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380de020 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0012000 tx=0x7f0ff0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8ce12121efa556 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.423+0000 7f0fef000640 1 -- 172.21.15.26:0/2069867471 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0003d70 con 0x555d380ddc00 2024-06-13T07:18:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.423+0000 7f0fef000640 1 -- 172.21.15.26:0/2069867471 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff0025070 con 0x555d380ddc00 2024-06-13T07:18:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.423+0000 7f0fef000640 1 -- 172.21.15.26:0/2069867471 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0020890 con 0x555d380ddc00 2024-06-13T07:18:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.424+0000 7f100764e740 1 -- 172.21.15.26:0/2069867471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 msgr2=0x555d380de020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.424+0000 7f100764e740 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380de020 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0012000 tx=0x7f0ff0014000 comp rx=0 tx=0).stop 2024-06-13T07:18:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.425+0000 7f100764e740 1 -- 172.21.15.26:0/2069867471 shutdown_connections 2024-06-13T07:18:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.425+0000 7f100764e740 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.425+0000 7f100764e740 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.425+0000 7f100764e740 1 --2- 172.21.15.26:0/2069867471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380de020 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.425+0000 7f100764e740 1 -- 172.21.15.26:0/2069867471 >> 172.21.15.26:0/2069867471 conn(0x555d380a5fb0 msgr2=0x555d380d9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.425+0000 7f100764e740 1 -- 172.21.15.26:0/2069867471 shutdown_connections 2024-06-13T07:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.425+0000 7f100764e740 1 -- 172.21.15.26:0/2069867471 wait complete. 2024-06-13T07:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.426+0000 7f100764e740 1 -- start start 2024-06-13T07:18:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.427+0000 7f100764e740 1 Processor -- start 2024-06-13T07:18:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.429+0000 7f100764e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.429+0000 7f100764e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.429+0000 7f100764e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.429+0000 7f100764e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555d380e6710 con 0x555d380e3760 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.429+0000 7f100764e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d37ffc0b0 con 0x555d380ddc00 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.429+0000 7f100764e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555d380e6890 con 0x555d380de5f0 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.429+0000 7f100764e740 10 client.0 _ll_register_callbacks cb 0x555d37fe7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1001400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1001400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39632/0 (socket says 172.21.15.26:39632) 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1001400640 1 -- 172.21.15.26:0/1749690115 learned_addr learned my addr 172.21.15.26:0/1749690115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1001e00640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1001400640 1 -- 172.21.15.26:0/1749690115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 msgr2=0x555d380e7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1001400640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380e7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1001400640 1 -- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 msgr2=0x555d380ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1002800640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.431+0000 7f1001400640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380ef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.432+0000 7f1001400640 1 -- 172.21.15.26:0/1749690115 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d37e21620 con 0x555d380de5f0 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.432+0000 7f1002800640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380ef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.432+0000 7f1001e00640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380e7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.432+0000 7f1001400640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e8450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800fa60 tx=0x7f0fe8004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb90bac1477b7fd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.432+0000 7f0fe3400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.433+0000 7f0fe3400640 1 -- 172.21.15.26:0/1749690115 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8005630 con 0x555d380de5f0 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.433+0000 7f0fe3400640 1 -- 172.21.15.26:0/1749690115 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0fe8004050 con 0x555d380de5f0 2024-06-13T07:18:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.433+0000 7f100764e740 1 -- 172.21.15.26:0/1749690115 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff0025070 con 0x555d380de5f0 2024-06-13T07:18:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.435+0000 7f0fe3400640 1 -- 172.21.15.26:0/1749690115 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0fe8004250 con 0x555d380de5f0 2024-06-13T07:18:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.435+0000 7f100764e740 10 client.26833 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.435+0000 7f100764e740 1 -- 172.21.15.26:0/1749690115 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555d380f28c0 con 0x555d380de5f0 2024-06-13T07:18:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.435+0000 7f0fe3400640 1 -- 172.21.15.26:0/1749690115 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff0025070 con 0x555d380de5f0 2024-06-13T07:18:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.436+0000 7f0fe3400640 1 -- 172.21.15.26:0/1749690115 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f0fe800e660 con 0x555d380de5f0 2024-06-13T07:18:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.437+0000 7f100764e740 10 client.26833 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.437+0000 7f100764e740 10 client.26833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.437+0000 7f100764e740 20 client.26833 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.437+0000 7f100764e740 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d380f4310 0x555d380f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.437+0000 7f100764e740 4 client.26833 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.437+0000 7f100764e740 1 -- 172.21.15.26:0/1749690115 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d3804eb50 con 0x555d380f4310 2024-06-13T07:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.437+0000 7f1001e00640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d380f4310 0x555d380f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.438+0000 7f0fe3400640 10 client.26833 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.438+0000 7f1001e00640 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d380f4310 0x555d380f67b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2e71602972ef05 server_cookie=494e3408353c741a in_seq=0 out_seq=0 2024-06-13T07:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.441+0000 7f0fe3400640 1 -- 172.21.15.26:0/1749690115 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555d3804eb50 con 0x555d380f4310 2024-06-13T07:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f0fe3400640 10 client.26833 handle_command_reply: tid=0 2024-06-13T07:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 client.26833 shutdown 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 -- 172.21.15.26:0/1749690115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 msgr2=0x555d380e8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e8450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800fa60 tx=0x7f0fe8004be0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 -- 172.21.15.26:0/1749690115 shutdown_connections 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d380f4310 0x555d380f67b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380e3760 0x555d380ef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380de5f0 0x555d380e8450 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 --2- 172.21.15.26:0/1749690115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380e7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f100764e740 1 -- 172.21.15.26:0/1749690115 >> 172.21.15.26:0/1749690115 conn(0x555d380a5fb0 msgr2=0x555d380d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.442+0000 7f0fe3400640 0 client.26833 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.443+0000 7f100764e740 1 -- 172.21.15.26:0/1749690115 shutdown_connections 2024-06-13T07:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.443+0000 7f100764e740 1 -- 172.21.15.26:0/1749690115 wait complete. 2024-06-13T07:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.443+0000 7f100764e740 20 client.26833 trim_cache size 0 max 16384 2024-06-13T07:18:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.491+0000 7f100764e740 1 Processor -- start 2024-06-13T07:18:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.491+0000 7f100764e740 1 -- start start 2024-06-13T07:18:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.491+0000 7f100764e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380d86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.491+0000 7f100764e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.491+0000 7f100764e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380fae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.491+0000 7f100764e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b3670 con 0x555d380de5f0 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.491+0000 7f100764e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b0f40 con 0x555d380ddc00 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.491+0000 7f100764e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1aed50 con 0x555d380e3760 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.492+0000 7f1001400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.492+0000 7f1001400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57414/0 (socket says 172.21.15.26:57414) 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.492+0000 7f1001400640 1 -- 172.21.15.26:0/639153316 learned_addr learned my addr 172.21.15.26:0/639153316 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.492+0000 7f1002800640 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.492+0000 7f1001e00640 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380d86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.493+0000 7f1002800640 1 -- 172.21.15.26:0/639153316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 msgr2=0x555d380d86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.493+0000 7f1002800640 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380d86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.493+0000 7f1002800640 1 -- 172.21.15.26:0/639153316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 msgr2=0x555d380ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.493+0000 7f1002800640 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.493+0000 7f1002800640 1 -- 172.21.15.26:0/639153316 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d380e2f40 con 0x555d380e3760 2024-06-13T07:18:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.494+0000 7f1002800640 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380fae50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80970a0 tx=0x7f0ff8009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2b14809bf1f69e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.495+0000 7f0fee600640 1 -- 172.21.15.26:0/639153316 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff809ac40 con 0x555d380e3760 2024-06-13T07:18:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.495+0000 7f0fee600640 1 -- 172.21.15.26:0/639153316 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff809eff0 con 0x555d380e3760 2024-06-13T07:18:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.496+0000 7f0fee600640 1 -- 172.21.15.26:0/639153316 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800ccf0 con 0x555d380e3760 2024-06-13T07:18:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.497+0000 7f100764e740 1 -- 172.21.15.26:0/639153316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 msgr2=0x555d380fae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.497+0000 7f100764e740 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380fae50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80970a0 tx=0x7f0ff8009930 comp rx=0 tx=0).stop 2024-06-13T07:18:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.497+0000 7f100764e740 1 -- 172.21.15.26:0/639153316 shutdown_connections 2024-06-13T07:18:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.497+0000 7f100764e740 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380fae50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.497+0000 7f100764e740 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.497+0000 7f100764e740 1 --2- 172.21.15.26:0/639153316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380d86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.497+0000 7f100764e740 1 -- 172.21.15.26:0/639153316 >> 172.21.15.26:0/639153316 conn(0x555d380a5fb0 msgr2=0x555d380d7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.498+0000 7f100764e740 1 -- 172.21.15.26:0/639153316 shutdown_connections 2024-06-13T07:18:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.498+0000 7f100764e740 1 -- 172.21.15.26:0/639153316 wait complete. 2024-06-13T07:18:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.500+0000 7f100764e740 1 -- start start 2024-06-13T07:18:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.500+0000 7f100764e740 1 Processor -- start 2024-06-13T07:18:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.501+0000 7f100764e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.501+0000 7f100764e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.501+0000 7f100764e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380ee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.501+0000 7f100764e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1af400 con 0x555d380de5f0 2024-06-13T07:18:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.501+0000 7f100764e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b0870 con 0x555d380ddc00 2024-06-13T07:18:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.501+0000 7f100764e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1afad0 con 0x555d380e3760 2024-06-13T07:18:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.501+0000 7f100764e740 10 client.0 _ll_register_callbacks cb 0x7f0fe8005580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.502+0000 7f1002800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.502+0000 7f1001e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.502+0000 7f1002800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380ee500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:39642/0 (socket says 172.21.15.26:39642) 2024-06-13T07:18:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.502+0000 7f1002800640 1 -- 172.21.15.26:0/1052647695 learned_addr learned my addr 172.21.15.26:0/1052647695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.502+0000 7f1001400640 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.503+0000 7f1001e00640 1 -- 172.21.15.26:0/1052647695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 msgr2=0x555d380ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.503+0000 7f1001e00640 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380ee500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.503+0000 7f1001e00640 1 -- 172.21.15.26:0/1052647695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 msgr2=0x555d380ed4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.503+0000 7f1001e00640 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ed4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.503+0000 7f1001e00640 1 -- 172.21.15.26:0/1052647695 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d380eeaf0 con 0x555d380ddc00 2024-06-13T07:18:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.504+0000 7f1001e00640 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380ecfb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x555d380e67a0 tx=0x7f0ff00119c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bed642f0ce4677a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.504+0000 7f0fe3400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:18:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.505+0000 7f0fe3400640 1 -- 172.21.15.26:0/1052647695 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00355a0 con 0x555d380ddc00 2024-06-13T07:18:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.505+0000 7f0fe3400640 1 -- 172.21.15.26:0/1052647695 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0ff0035740 con 0x555d380ddc00 2024-06-13T07:18:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.506+0000 7f100764e740 1 -- 172.21.15.26:0/1052647695 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff809eff0 con 0x555d380ddc00 2024-06-13T07:18:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.506+0000 7f0fe3400640 1 -- 172.21.15.26:0/1052647695 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0ff0035940 con 0x555d380ddc00 2024-06-13T07:18:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.508+0000 7f100764e740 10 client.26807 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.509+0000 7f100764e740 1 -- 172.21.15.26:0/1052647695 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555d3815f3c0 con 0x555d380ddc00 2024-06-13T07:18:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.509+0000 7f0fe3400640 1 -- 172.21.15.26:0/1052647695 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff809eff0 con 0x555d380ddc00 2024-06-13T07:18:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.509+0000 7f0fe3400640 1 -- 172.21.15.26:0/1052647695 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f0ff003d030 con 0x555d380ddc00 2024-06-13T07:18:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.510+0000 7f100764e740 10 client.26807 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.510+0000 7f100764e740 10 client.26807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.510+0000 7f100764e740 20 client.26807 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.510+0000 7f100764e740 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d3815fb60 0x555d38161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.510+0000 7f100764e740 4 client.26807 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:18:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.510+0000 7f100764e740 1 -- 172.21.15.26:0/1052647695 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d38048ea0 con 0x555d3815fb60 2024-06-13T07:18:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.510+0000 7f1001400640 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d3815fb60 0x555d38161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.513+0000 7f0fe3400640 10 client.26807 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.513+0000 7f1001400640 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d3815fb60 0x555d38161f50 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2bd495487557d5 server_cookie=98caac48e0c1905f in_seq=0 out_seq=0 2024-06-13T07:18:16.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:16 smithi135 ceph-mon[36276]: pgmap v1042: 129 pgs: 129 active+clean; 670 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.4 MiB/s wr, 593 op/s 2024-06-13T07:18:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.856+0000 7f0fe3400640 1 -- 172.21.15.26:0/1052647695 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555d38048ea0 con 0x555d3815fb60 2024-06-13T07:18:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.856+0000 7f0fe3400640 10 client.26807 handle_command_reply: tid=0 2024-06-13T07:18:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.856+0000 7f100764e740 1 client.26807 shutdown 2024-06-13T07:18:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f100764e740 1 -- 172.21.15.26:0/1052647695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 msgr2=0x555d380ecfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f100764e740 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380ecfb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x555d380e67a0 tx=0x7f0ff00119c0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f100764e740 1 -- 172.21.15.26:0/1052647695 shutdown_connections 2024-06-13T07:18:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f100764e740 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555d3815fb60 0x555d38161f50 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f0fe3400640 0 client.26807 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f100764e740 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555d380e3760 0x555d380ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f100764e740 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d380de5f0 0x555d380ed4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f100764e740 1 --2- 172.21.15.26:0/1052647695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d380ddc00 0x555d380ecfb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.857+0000 7f100764e740 1 -- 172.21.15.26:0/1052647695 >> 172.21.15.26:0/1052647695 conn(0x555d380a5fb0 msgr2=0x555d380d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.858+0000 7f100764e740 1 -- 172.21.15.26:0/1052647695 shutdown_connections 2024-06-13T07:18:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.858+0000 7f100764e740 1 -- 172.21.15.26:0/1052647695 wait complete. 2024-06-13T07:18:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.858+0000 7f100764e740 20 client.26807 trim_cache size 0 max 16384 2024-06-13T07:18:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:18:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:18:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.859+0000 7f1004800640 1 -- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0fe40a2d00 msgr2=0x7f0fe40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.859+0000 7f1004800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0fe40a2d00 0x7f0fe40a51c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc102a50 tx=0x7f0ff0005f90 comp rx=0 tx=0).stop 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.859+0000 7f1004800640 1 -- 172.21.15.26:0/1778894481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc1035b0 msgr2=0x7f0ffc1a5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.859+0000 7f1004800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc1035b0 0x7f0ffc1a5c60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800b970 tx=0x7f0ff800be30 comp rx=0 tx=0).stop 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.860+0000 7f1004800640 1 -- 172.21.15.26:0/1778894481 shutdown_connections 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.860+0000 7f1004800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f0fe40a2d00 0x7f0fe40a51c0 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc102a50 tx=0x7f0ff0005f90 comp rx=0 tx=0).stop 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.860+0000 7f1004800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ffc102bf0 0x7f0ffc070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.860+0000 7f1004800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ffc1035b0 0x7f0ffc1a5c60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.860+0000 7f1004800640 1 --2- 172.21.15.26:0/1778894481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc101a00 0x7f0ffc070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.860+0000 7f1004800640 1 -- 172.21.15.26:0/1778894481 >> 172.21.15.26:0/1778894481 conn(0x7f0ffc0fcf90 msgr2=0x7f0ffc105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.860+0000 7f1004800640 1 -- 172.21.15.26:0/1778894481 shutdown_connections 2024-06-13T07:18:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:16.860+0000 7f1004800640 1 -- 172.21.15.26:0/1778894481 wait complete. 2024-06-13T07:18:17.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 141196 27.69 MB/sec execute 150 sec latency 133.542 ms 2024-06-13T07:18:18.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 141983 27.87 MB/sec execute 151 sec latency 67.578 ms 2024-06-13T07:18:18.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:18 smithi136 ceph-mon[37229]: pgmap v1043: 129 pgs: 129 active+clean; 682 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 562 op/s 2024-06-13T07:18:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:18 smithi026 ceph-mon[32041]: pgmap v1043: 129 pgs: 129 active+clean; 682 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 562 op/s 2024-06-13T07:18:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:18 smithi135 ceph-mon[36276]: pgmap v1043: 129 pgs: 129 active+clean; 682 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 562 op/s 2024-06-13T07:18:19.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 142781 28.01 MB/sec execute 152 sec latency 59.729 ms 2024-06-13T07:18:20.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 143305 28.04 MB/sec execute 153 sec latency 103.605 ms 2024-06-13T07:18:20.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:20 smithi136 ceph-mon[37229]: pgmap v1044: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-06-13T07:18:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:20 smithi026 ceph-mon[32041]: pgmap v1044: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-06-13T07:18:20.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:20 smithi135 ceph-mon[36276]: pgmap v1044: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-06-13T07:18:21.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 143714 27.97 MB/sec execute 154 sec latency 136.087 ms 2024-06-13T07:18:22.176 INFO:tasks.workunit.client.0.smithi026.stdout: 10 144284 27.95 MB/sec execute 155 sec latency 72.844 ms 2024-06-13T07:18:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:22 smithi136 ceph-mon[37229]: pgmap v1045: 129 pgs: 129 active+clean; 621 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 581 op/s 2024-06-13T07:18:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:22 smithi026 ceph-mon[32041]: pgmap v1045: 129 pgs: 129 active+clean; 621 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 581 op/s 2024-06-13T07:18:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:22 smithi135 ceph-mon[36276]: pgmap v1045: 129 pgs: 129 active+clean; 621 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 581 op/s 2024-06-13T07:18:23.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 145165 28.03 MB/sec execute 156 sec latency 54.119 ms 2024-06-13T07:18:24.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 146027 28.22 MB/sec execute 157 sec latency 58.596 ms 2024-06-13T07:18:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:24 smithi136 ceph-mon[37229]: pgmap v1046: 129 pgs: 129 active+clean; 624 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-06-13T07:18:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:24 smithi026 ceph-mon[32041]: pgmap v1046: 129 pgs: 129 active+clean; 624 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-06-13T07:18:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:24 smithi135 ceph-mon[36276]: pgmap v1046: 129 pgs: 129 active+clean; 624 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-06-13T07:18:25.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 146586 28.24 MB/sec execute 158 sec latency 54.037 ms 2024-06-13T07:18:26.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 147207 28.27 MB/sec execute 159 sec latency 49.242 ms 2024-06-13T07:18:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:26 smithi026 ceph-mon[32041]: pgmap v1047: 129 pgs: 129 active+clean; 626 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.7 MiB/s wr, 615 op/s 2024-06-13T07:18:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:26 smithi135 ceph-mon[36276]: pgmap v1047: 129 pgs: 129 active+clean; 626 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.7 MiB/s wr, 615 op/s 2024-06-13T07:18:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:26 smithi136 ceph-mon[37229]: pgmap v1047: 129 pgs: 129 active+clean; 626 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.7 MiB/s wr, 615 op/s 2024-06-13T07:18:27.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 147823 28.26 MB/sec execute 160 sec latency 98.515 ms 2024-06-13T07:18:28.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 148513 28.30 MB/sec execute 161 sec latency 73.054 ms 2024-06-13T07:18:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:28 smithi026 ceph-mon[32041]: pgmap v1048: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-06-13T07:18:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:28 smithi135 ceph-mon[36276]: pgmap v1048: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-06-13T07:18:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:28 smithi136 ceph-mon[37229]: pgmap v1048: 129 pgs: 129 active+clean; 630 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-06-13T07:18:29.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 149282 28.45 MB/sec execute 162 sec latency 49.808 ms 2024-06-13T07:18:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.047+0000 7f35d2c00640 1 -- 172.21.15.26:0/85007802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc100a50 msgr2=0x7f35cc100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.047+0000 7f35d2c00640 1 --2- 172.21.15.26:0/85007802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc100a50 0x7f35cc100eb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009900 tx=0x7f35b802a410 comp rx=0 tx=0).stop 2024-06-13T07:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.048+0000 7f35d2c00640 1 -- 172.21.15.26:0/85007802 shutdown_connections 2024-06-13T07:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.048+0000 7f35d2c00640 1 --2- 172.21.15.26:0/85007802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35cc1013f0 0x7f35cc101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.048+0000 7f35d2c00640 1 --2- 172.21.15.26:0/85007802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc100a50 0x7f35cc100eb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.048+0000 7f35d2c00640 1 --2- 172.21.15.26:0/85007802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35cc1000f0 0x7f35cc100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.048+0000 7f35d2c00640 1 -- 172.21.15.26:0/85007802 >> 172.21.15.26:0/85007802 conn(0x7f35cc0fbc90 msgr2=0x7f35cc0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.048+0000 7f35d2c00640 1 -- 172.21.15.26:0/85007802 shutdown_connections 2024-06-13T07:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.048+0000 7f35d2c00640 1 -- 172.21.15.26:0/85007802 wait complete. 2024-06-13T07:18:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.049+0000 7f35d2c00640 1 Processor -- start 2024-06-13T07:18:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.050+0000 7f35d2c00640 1 -- start start 2024-06-13T07:18:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.050+0000 7f35d2c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc1000f0 0x7f35cc110c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.050+0000 7f35d2c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35cc100a50 0x7f35cc116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.050+0000 7f35d2c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35cc1013f0 0x7f35cc1111c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.050+0000 7f35d2c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1039d0 con 0x7f35cc1013f0 2024-06-13T07:18:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.050+0000 7f35d2c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc103850 con 0x7f35cc100a50 2024-06-13T07:18:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.050+0000 7f35d2c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc103b50 con 0x7f35cc1000f0 2024-06-13T07:18:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.051+0000 7f35d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc1000f0 0x7f35cc110c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.051+0000 7f35d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc1000f0 0x7f35cc110c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43016/0 (socket says 172.21.15.26:43016) 2024-06-13T07:18:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.051+0000 7f35d1800640 1 -- 172.21.15.26:0/87675487 learned_addr learned my addr 172.21.15.26:0/87675487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.051+0000 7f35d0e00640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35cc100a50 0x7f35cc116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.051+0000 7f35d2200640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35cc1013f0 0x7f35cc1111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.052+0000 7f35d1800640 1 -- 172.21.15.26:0/87675487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35cc100a50 msgr2=0x7f35cc116220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.052+0000 7f35d1800640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35cc100a50 0x7f35cc116220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.052+0000 7f35d1800640 1 -- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35cc1013f0 msgr2=0x7f35cc1111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.052+0000 7f35d1800640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35cc1013f0 0x7f35cc1111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.052+0000 7f35d1800640 1 -- 172.21.15.26:0/87675487 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc1ae5f0 con 0x7f35cc1000f0 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.052+0000 7f35d0e00640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35cc100a50 0x7f35cc116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.052+0000 7f35d1800640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc1000f0 0x7f35cc110c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35c000db80 tx=0x7f35c000b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.054+0000 7f35ca000640 1 -- 172.21.15.26:0/87675487 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0017070 con 0x7f35cc1000f0 2024-06-13T07:18:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.054+0000 7f35ca000640 1 -- 172.21.15.26:0/87675487 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c0003b70 con 0x7f35cc1000f0 2024-06-13T07:18:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.054+0000 7f35ca000640 1 -- 172.21.15.26:0/87675487 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c00106c0 con 0x7f35cc1000f0 2024-06-13T07:18:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.055+0000 7f35d2c00640 1 -- 172.21.15.26:0/87675487 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc1ae8e0 con 0x7f35cc1000f0 2024-06-13T07:18:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.056+0000 7f35d2c00640 1 -- 172.21.15.26:0/87675487 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc1aedb0 con 0x7f35cc1000f0 2024-06-13T07:18:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.060+0000 7f35d71cd740 1 Processor -- start 2024-06-13T07:18:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.060+0000 7f35ca000640 1 -- 172.21.15.26:0/87675487 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35c0004020 con 0x7f35cc1000f0 2024-06-13T07:18:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.060+0000 7f35d71cd740 1 -- start start 2024-06-13T07:18:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850e5dc0 0x55a2850e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e67b0 0x55a2850eb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850eb920 0x55a2850eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d71cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a285049420 con 0x55a2850eb920 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d71cd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a284ff0c50 con 0x55a2850e5dc0 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d71cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a2850453e0 con 0x55a2850e67b0 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850eb920 0x55a2850eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850eb920 0x55a2850eddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58682/0 (socket says 172.21.15.26:58682) 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d0e00640 1 -- 172.21.15.26:0/733565193 learned_addr learned my addr 172.21.15.26:0/733565193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.061+0000 7f35d2200640 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850e5dc0 0x55a2850e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.062+0000 7f35ca000640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f35b40a2d90 0x7f35b40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.062+0000 7f35d1800640 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e67b0 0x55a2850eb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.063+0000 7f35d2200640 1 -- 172.21.15.26:0/733565193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e67b0 msgr2=0x55a2850eb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.063+0000 7f35d2200640 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e67b0 0x55a2850eb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.063+0000 7f35d2200640 1 -- 172.21.15.26:0/733565193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850eb920 msgr2=0x55a2850eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.063+0000 7f35ca000640 1 -- 172.21.15.26:0/87675487 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f35c00cf030 con 0x7f35cc1000f0 2024-06-13T07:18:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.063+0000 7f35d2200640 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850eb920 0x55a2850eddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.064+0000 7f35d2200640 1 -- 172.21.15.26:0/733565193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a284d5d210 con 0x55a2850e5dc0 2024-06-13T07:18:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.064+0000 7f35d1800640 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e67b0 0x55a2850eb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.064+0000 7f35d0e00640 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850eb920 0x55a2850eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:18:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.064+0000 7f35d2200640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f35b40a2d90 0x7f35b40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.064+0000 7f35d2200640 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850e5dc0 0x55a2850e61e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35c4006fd0 tx=0x7f35c4009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a75d9355469b98 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.065+0000 7f35bea00640 1 -- 172.21.15.26:0/733565193 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4023070 con 0x55a2850e5dc0 2024-06-13T07:18:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.065+0000 7f35bea00640 1 -- 172.21.15.26:0/733565193 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c401b870 con 0x55a2850e5dc0 2024-06-13T07:18:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 -- 172.21.15.26:0/733565193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850e5dc0 msgr2=0x55a2850e61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850e5dc0 0x55a2850e61e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35c4006fd0 tx=0x7f35c4009f10 comp rx=0 tx=0).stop 2024-06-13T07:18:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d2200640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f35b40a2d90 0x7f35b40a5250 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f35c4020000 tx=0x7f35c401d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 -- 172.21.15.26:0/733565193 shutdown_connections 2024-06-13T07:18:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850eb920 0x55a2850eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e67b0 0x55a2850eb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 --2- 172.21.15.26:0/733565193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850e5dc0 0x55a2850e61e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 -- 172.21.15.26:0/733565193 >> 172.21.15.26:0/733565193 conn(0x55a2850adfb0 msgr2=0x55a2850e1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 -- 172.21.15.26:0/733565193 shutdown_connections 2024-06-13T07:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.066+0000 7f35d71cd740 1 -- 172.21.15.26:0/733565193 wait complete. 2024-06-13T07:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.067+0000 7f35d71cd740 1 -- start start 2024-06-13T07:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.067+0000 7f35d71cd740 1 Processor -- start 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850f0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 0x55a2850f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d71cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2850ee970 con 0x55a2850e67b0 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d71cd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a2850040b0 con 0x55a2850eb920 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d71cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a2850eeaf0 con 0x55a2850e5dc0 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d71cd740 10 client.0 _ll_register_callbacks cb 0x55a284feff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43036/0 (socket says 172.21.15.26:43036) 2024-06-13T07:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.068+0000 7f35d1800640 1 -- 172.21.15.26:0/1018325588 learned_addr learned my addr 172.21.15.26:0/1018325588 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.069+0000 7f35d1800640 1 -- 172.21.15.26:0/1018325588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 msgr2=0x55a2850f76d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.069+0000 7f35d1800640 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 0x55a2850f76d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.069+0000 7f35d1800640 1 -- 172.21.15.26:0/1018325588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 msgr2=0x55a2850f0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.069+0000 7f35d1800640 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.069+0000 7f35d1800640 1 -- 172.21.15.26:0/1018325588 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a284e29620 con 0x55a2850e5dc0 2024-06-13T07:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.070+0000 7f35d1800640 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f0150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f35c00a3c00 tx=0x7f35c009b0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ed23110381a65c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.070+0000 7f35b3400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:18:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.070+0000 7f35b3400640 1 -- 172.21.15.26:0/1018325588 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c009b4c0 con 0x55a2850e5dc0 2024-06-13T07:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.070+0000 7f35b3400640 1 -- 172.21.15.26:0/1018325588 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f35c0003d20 con 0x55a2850e5dc0 2024-06-13T07:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.071+0000 7f35d71cd740 1 -- 172.21.15.26:0/1018325588 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c401b870 con 0x55a2850e5dc0 2024-06-13T07:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.071+0000 7f35b3400640 1 -- 172.21.15.26:0/1018325588 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f35c009f870 con 0x55a2850e5dc0 2024-06-13T07:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.072+0000 7f35d71cd740 10 client.26848 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.072+0000 7f35d71cd740 1 -- 172.21.15.26:0/1018325588 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a2850faa70 con 0x55a2850e5dc0 2024-06-13T07:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.072+0000 7f35b3400640 1 -- 172.21.15.26:0/1018325588 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c401b870 con 0x55a2850e5dc0 2024-06-13T07:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.072+0000 7f35b3400640 1 -- 172.21.15.26:0/1018325588 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f35c00ccba0 con 0x55a2850e5dc0 2024-06-13T07:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.072+0000 7f35d71cd740 10 client.26848 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.072+0000 7f35d71cd740 10 client.26848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.073+0000 7f35d71cd740 20 client.26848 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.073+0000 7f35d71cd740 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a2850fc5c0 0x55a2850fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.073+0000 7f35d71cd740 4 client.26848 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.073+0000 7f35d71cd740 1 -- 172.21.15.26:0/1018325588 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a285056b50 con 0x55a2850fc5c0 2024-06-13T07:18:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.075+0000 7f35d0e00640 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a2850fc5c0 0x55a2850fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.076+0000 7f35d0e00640 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a2850fc5c0 0x55a2850fea50 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb3ca312c70da70 server_cookie=bdb234230633cae3 in_seq=0 out_seq=0 2024-06-13T07:18:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.076+0000 7f35b3400640 10 client.26848 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.080+0000 7f35b3400640 1 -- 172.21.15.26:0/1018325588 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a285056b50 con 0x55a2850fc5c0 2024-06-13T07:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.080+0000 7f35b3400640 10 client.26848 handle_command_reply: tid=0 2024-06-13T07:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.080+0000 7f35d71cd740 1 client.26848 shutdown 2024-06-13T07:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35d71cd740 1 -- 172.21.15.26:0/1018325588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 msgr2=0x55a2850f0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35d71cd740 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f0150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f35c00a3c00 tx=0x7f35c009b0c0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35d71cd740 1 -- 172.21.15.26:0/1018325588 shutdown_connections 2024-06-13T07:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35d71cd740 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a2850fc5c0 0x55a2850fea50 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35b3400640 0 client.26848 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35d71cd740 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 0x55a2850f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35d71cd740 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35d71cd740 1 --2- 172.21.15.26:0/1018325588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f0150 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.081+0000 7f35d71cd740 1 -- 172.21.15.26:0/1018325588 >> 172.21.15.26:0/1018325588 conn(0x55a2850adfb0 msgr2=0x55a2850e09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.082+0000 7f35d71cd740 1 -- 172.21.15.26:0/1018325588 shutdown_connections 2024-06-13T07:18:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.082+0000 7f35d71cd740 1 -- 172.21.15.26:0/1018325588 wait complete. 2024-06-13T07:18:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.082+0000 7f35d71cd740 20 client.26848 trim_cache size 0 max 16384 2024-06-13T07:18:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.111+0000 7f35d71cd740 1 Processor -- start 2024-06-13T07:18:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.111+0000 7f35d71cd740 1 -- start start 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.111+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850adfb0 0x55a2850f0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850e0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d71cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1bbba0 con 0x55a2850e67b0 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d71cd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b9470 con 0x55a2850adfb0 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d71cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b72c0 con 0x55a2850e5dc0 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43038/0 (socket says 172.21.15.26:43038) 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d0e00640 1 -- 172.21.15.26:0/3144202621 learned_addr learned my addr 172.21.15.26:0/3144202621 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d1800640 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850adfb0 0x55a2850f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d0e00640 1 -- 172.21.15.26:0/3144202621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850adfb0 msgr2=0x55a2850f0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d0e00640 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850adfb0 0x55a2850f0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d0e00640 1 -- 172.21.15.26:0/3144202621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 msgr2=0x55a2850e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d2200640 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d0e00640 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850e0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.112+0000 7f35d0e00640 1 -- 172.21.15.26:0/3144202621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2850e6b90 con 0x55a2850e5dc0 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.113+0000 7f35d0e00640 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f7690 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f35b802aa00 tx=0x7f35b8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a2d595925ff7e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.113+0000 7f35be000640 1 -- 172.21.15.26:0/3144202621 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8035040 con 0x55a2850e5dc0 2024-06-13T07:18:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.113+0000 7f35be000640 1 -- 172.21.15.26:0/3144202621 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b8005400 con 0x55a2850e5dc0 2024-06-13T07:18:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.113+0000 7f35be000640 1 -- 172.21.15.26:0/3144202621 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8004bf0 con 0x55a2850e5dc0 2024-06-13T07:18:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 -- 172.21.15.26:0/3144202621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 msgr2=0x55a2850f7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f7690 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f35b802aa00 tx=0x7f35b8009560 comp rx=0 tx=0).stop 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 -- 172.21.15.26:0/3144202621 shutdown_connections 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850e5dc0 0x55a2850f7690 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 --2- 172.21.15.26:0/3144202621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850adfb0 0x55a2850f0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 -- 172.21.15.26:0/3144202621 >> 172.21.15.26:0/3144202621 conn(0x55a2850eb3f0 msgr2=0x55a2850efd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 -- 172.21.15.26:0/3144202621 shutdown_connections 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.114+0000 7f35d71cd740 1 -- 172.21.15.26:0/3144202621 wait complete. 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.115+0000 7f35d71cd740 1 -- start start 2024-06-13T07:18:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.115+0000 7f35d71cd740 1 Processor -- start 2024-06-13T07:18:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850adfb0 0x55a2850f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d71cd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 0x55a2850f6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d71cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b7930 con 0x55a2850e67b0 2024-06-13T07:18:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d71cd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b8da0 con 0x55a2850eb920 2024-06-13T07:18:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d71cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b8000 con 0x55a2850adfb0 2024-06-13T07:18:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d71cd740 10 client.0 _ll_register_callbacks cb 0x7f35c009b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850adfb0 0x55a2850f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850adfb0 0x55a2850f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43042/0 (socket says 172.21.15.26:43042) 2024-06-13T07:18:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d1800640 1 -- 172.21.15.26:0/2095863094 learned_addr learned my addr 172.21.15.26:0/2095863094 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.116+0000 7f35d0e00640 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.117+0000 7f35d1800640 1 -- 172.21.15.26:0/2095863094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 msgr2=0x55a2850f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.117+0000 7f35d2200640 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 0x55a2850f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.117+0000 7f35d1800640 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 0x55a2850f6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.117+0000 7f35d1800640 1 -- 172.21.15.26:0/2095863094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 msgr2=0x55a2850f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.117+0000 7f35d1800640 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.117+0000 7f35d1800640 1 -- 172.21.15.26:0/2095863094 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2850f6710 con 0x55a2850adfb0 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.117+0000 7f35d1800640 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850adfb0 0x55a2850f4bd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f35c000bd70 tx=0x7f35c0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de860e87ec33400e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.117+0000 7f35b3400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.118+0000 7f35b3400640 1 -- 172.21.15.26:0/2095863094 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c009b4c0 con 0x55a2850adfb0 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.118+0000 7f35b3400640 1 -- 172.21.15.26:0/2095863094 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f35c0097b30 con 0x55a2850adfb0 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.118+0000 7f35d71cd740 1 -- 172.21.15.26:0/2095863094 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b8005400 con 0x55a2850adfb0 2024-06-13T07:18:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.118+0000 7f35b3400640 1 -- 172.21.15.26:0/2095863094 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f35c00a4130 con 0x55a2850adfb0 2024-06-13T07:18:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.119+0000 7f35d71cd740 10 client.26854 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.119+0000 7f35d71cd740 1 -- 172.21.15.26:0/2095863094 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a285167330 con 0x55a2850adfb0 2024-06-13T07:18:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.119+0000 7f35b3400640 1 -- 172.21.15.26:0/2095863094 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b8005400 con 0x55a2850adfb0 2024-06-13T07:18:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.120+0000 7f35b3400640 1 -- 172.21.15.26:0/2095863094 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f35c009a4a0 con 0x55a2850adfb0 2024-06-13T07:18:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.120+0000 7f35d71cd740 10 client.26854 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.120+0000 7f35d71cd740 10 client.26854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.120+0000 7f35d71cd740 20 client.26854 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.120+0000 7f35d71cd740 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a285167b60 0x55a285169f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.120+0000 7f35d71cd740 4 client.26854 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:18:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.120+0000 7f35d71cd740 1 -- 172.21.15.26:0/2095863094 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a285050ea0 con 0x55a285167b60 2024-06-13T07:18:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.121+0000 7f35d0e00640 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a285167b60 0x55a285169f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.122+0000 7f35d0e00640 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a285167b60 0x55a285169f80 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a13d83eb16b019 server_cookie=7e4f45191347cfae in_seq=0 out_seq=0 2024-06-13T07:18:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.122+0000 7f35b3400640 10 client.26854 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:30.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 149758 28.42 MB/sec execute 163 sec latency 92.893 ms 2024-06-13T07:18:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:30 smithi026 ceph-mon[32041]: pgmap v1049: 129 pgs: 129 active+clean; 634 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 627 op/s 2024-06-13T07:18:30.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:30 smithi135 ceph-mon[36276]: pgmap v1049: 129 pgs: 129 active+clean; 634 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 627 op/s 2024-06-13T07:18:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:30 smithi136 ceph-mon[37229]: pgmap v1049: 129 pgs: 129 active+clean; 634 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 627 op/s 2024-06-13T07:18:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.818+0000 7f35b3400640 1 -- 172.21.15.26:0/2095863094 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a285050ea0 con 0x55a285167b60 2024-06-13T07:18:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35b3400640 10 client.26854 handle_command_reply: tid=0 2024-06-13T07:18:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 client.26854 shutdown 2024-06-13T07:18:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 -- 172.21.15.26:0/2095863094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850adfb0 msgr2=0x55a2850f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850adfb0 0x55a2850f4bd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f35c000bd70 tx=0x7f35c0000f40 comp rx=0 tx=0).stop 2024-06-13T07:18:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 -- 172.21.15.26:0/2095863094 shutdown_connections 2024-06-13T07:18:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55a285167b60 0x55a285169f80 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35b3400640 0 client.26854 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2850eb920 0x55a2850f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2850e67b0 0x55a2850f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 --2- 172.21.15.26:0/2095863094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2850adfb0 0x55a2850f4bd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.819+0000 7f35d71cd740 1 -- 172.21.15.26:0/2095863094 >> 172.21.15.26:0/2095863094 conn(0x55a2850eb3f0 msgr2=0x55a2850e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.820+0000 7f35d71cd740 1 -- 172.21.15.26:0/2095863094 shutdown_connections 2024-06-13T07:18:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.820+0000 7f35d71cd740 1 -- 172.21.15.26:0/2095863094 wait complete. 2024-06-13T07:18:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.820+0000 7f35d71cd740 20 client.26854 trim_cache size 0 max 16384 2024-06-13T07:18:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:18:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:18:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:18:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.821+0000 7f35d2c00640 1 -- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f35b40a2d90 msgr2=0x7f35b40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.821+0000 7f35d2c00640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f35b40a2d90 0x7f35b40a5250 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f35c4020000 tx=0x7f35c401d000 comp rx=0 tx=0).stop 2024-06-13T07:18:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.821+0000 7f35d2c00640 1 -- 172.21.15.26:0/87675487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc1000f0 msgr2=0x7f35cc110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.821+0000 7f35d2c00640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc1000f0 0x7f35cc110c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35c000db80 tx=0x7f35c000b630 comp rx=0 tx=0).stop 2024-06-13T07:18:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.822+0000 7f35d2c00640 1 -- 172.21.15.26:0/87675487 shutdown_connections 2024-06-13T07:18:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.822+0000 7f35d2c00640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f35b40a2d90 0x7f35b40a5250 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.822+0000 7f35d2c00640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35cc100a50 0x7f35cc116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.822+0000 7f35d2c00640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35cc1013f0 0x7f35cc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.822+0000 7f35d2c00640 1 --2- 172.21.15.26:0/87675487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35cc1000f0 0x7f35cc110c80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.822+0000 7f35d2c00640 1 -- 172.21.15.26:0/87675487 >> 172.21.15.26:0/87675487 conn(0x7f35cc0fbc90 msgr2=0x7f35cc0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.823+0000 7f35d2c00640 1 -- 172.21.15.26:0/87675487 shutdown_connections 2024-06-13T07:18:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:30.823+0000 7f35d2c00640 1 -- 172.21.15.26:0/87675487 wait complete. 2024-06-13T07:18:31.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 150142 28.32 MB/sec execute 164 sec latency 493.753 ms 2024-06-13T07:18:32.177 INFO:tasks.workunit.client.0.smithi026.stdout: 10 150652 28.36 MB/sec execute 165 sec latency 94.297 ms 2024-06-13T07:18:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:32 smithi026 ceph-mon[32041]: pgmap v1050: 129 pgs: 129 active+clean; 635 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2024-06-13T07:18:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:32 smithi135 ceph-mon[36276]: pgmap v1050: 129 pgs: 129 active+clean; 635 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2024-06-13T07:18:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:32 smithi136 ceph-mon[37229]: pgmap v1050: 129 pgs: 129 active+clean; 635 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2024-06-13T07:18:33.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 151283 28.38 MB/sec execute 166 sec latency 105.797 ms 2024-06-13T07:18:34.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 151911 28.38 MB/sec execute 167 sec latency 144.173 ms 2024-06-13T07:18:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:34 smithi026 ceph-mon[32041]: pgmap v1051: 129 pgs: 129 active+clean; 641 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.8 MiB/s wr, 496 op/s 2024-06-13T07:18:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:34 smithi135 ceph-mon[36276]: pgmap v1051: 129 pgs: 129 active+clean; 641 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.8 MiB/s wr, 496 op/s 2024-06-13T07:18:34.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:34 smithi136 ceph-mon[37229]: pgmap v1051: 129 pgs: 129 active+clean; 641 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.8 MiB/s wr, 496 op/s 2024-06-13T07:18:35.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 152445 28.45 MB/sec execute 168 sec latency 94.907 ms 2024-06-13T07:18:36.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 152972 28.45 MB/sec execute 169 sec latency 80.026 ms 2024-06-13T07:18:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:36 smithi026 ceph-mon[32041]: pgmap v1052: 129 pgs: 129 active+clean; 645 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 508 op/s 2024-06-13T07:18:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:36 smithi135 ceph-mon[36276]: pgmap v1052: 129 pgs: 129 active+clean; 645 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 508 op/s 2024-06-13T07:18:36.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:36 smithi136 ceph-mon[37229]: pgmap v1052: 129 pgs: 129 active+clean; 645 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 508 op/s 2024-06-13T07:18:37.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 153401 28.38 MB/sec execute 170 sec latency 112.020 ms 2024-06-13T07:18:38.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 154012 28.37 MB/sec execute 171 sec latency 80.181 ms 2024-06-13T07:18:38.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:38 smithi026 ceph-mon[32041]: pgmap v1053: 129 pgs: 129 active+clean; 644 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 538 op/s 2024-06-13T07:18:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:38 smithi135 ceph-mon[36276]: pgmap v1053: 129 pgs: 129 active+clean; 644 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 538 op/s 2024-06-13T07:18:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:38 smithi136 ceph-mon[37229]: pgmap v1053: 129 pgs: 129 active+clean; 644 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 538 op/s 2024-06-13T07:18:39.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 154887 28.49 MB/sec execute 172 sec latency 46.445 ms 2024-06-13T07:18:40.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 155445 28.51 MB/sec execute 173 sec latency 263.772 ms 2024-06-13T07:18:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:40 smithi026 ceph-mon[32041]: pgmap v1054: 129 pgs: 129 active+clean; 648 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.7 MiB/s wr, 534 op/s 2024-06-13T07:18:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:40 smithi135 ceph-mon[36276]: pgmap v1054: 129 pgs: 129 active+clean; 648 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.7 MiB/s wr, 534 op/s 2024-06-13T07:18:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:40 smithi136 ceph-mon[37229]: pgmap v1054: 129 pgs: 129 active+clean; 648 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.7 MiB/s wr, 534 op/s 2024-06-13T07:18:41.179 INFO:tasks.workunit.client.0.smithi026.stdout: 10 155849 28.53 MB/sec execute 174 sec latency 407.867 ms 2024-06-13T07:18:42.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 155862 28.37 MB/sec execute 175 sec latency 1407.939 ms 2024-06-13T07:18:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:42 smithi026 ceph-mon[32041]: pgmap v1055: 129 pgs: 129 active+clean; 651 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-06-13T07:18:42.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:42 smithi135 ceph-mon[36276]: pgmap v1055: 129 pgs: 129 active+clean; 651 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-06-13T07:18:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:42 smithi136 ceph-mon[37229]: pgmap v1055: 129 pgs: 129 active+clean; 651 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-06-13T07:18:43.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 155998 28.25 MB/sec execute 176 sec latency 2102.115 ms 2024-06-13T07:18:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.061+0000 7f47f4400640 1 -- 172.21.15.26:0/845380686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109be0 msgr2=0x7f47ec110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.061+0000 7f47f4400640 1 --2- 172.21.15.26:0/845380686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109be0 0x7f47ec110470 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009ca0 tx=0x7f47e402a8c0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.062+0000 7f47f4400640 1 -- 172.21.15.26:0/845380686 shutdown_connections 2024-06-13T07:18:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.062+0000 7f47f4400640 1 --2- 172.21.15.26:0/845380686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109be0 0x7f47ec110470 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.062+0000 7f47f4400640 1 --2- 172.21.15.26:0/845380686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec109220 0x7f47ec1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.062+0000 7f47f4400640 1 --2- 172.21.15.26:0/845380686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47ec108030 0x7f47ec108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.062+0000 7f47f4400640 1 -- 172.21.15.26:0/845380686 >> 172.21.15.26:0/845380686 conn(0x7f47ec076ff0 msgr2=0x7f47ec079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.062+0000 7f47f4400640 1 -- 172.21.15.26:0/845380686 shutdown_connections 2024-06-13T07:18:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.062+0000 7f47f4400640 1 -- 172.21.15.26:0/845380686 wait complete. 2024-06-13T07:18:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.063+0000 7f47f4400640 1 Processor -- start 2024-06-13T07:18:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.064+0000 7f47f4400640 1 -- start start 2024-06-13T07:18:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.065+0000 7f47f4400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47ec108030 0x7f47ec19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.065+0000 7f47f4400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109220 0x7f47ec19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.065+0000 7f47f4400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec109be0 0x7f47ec1a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.065+0000 7f47f4400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec113d10 con 0x7f47ec109be0 2024-06-13T07:18:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.065+0000 7f47f4400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec113b90 con 0x7f47ec108030 2024-06-13T07:18:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.065+0000 7f47f4400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec113e90 con 0x7f47ec109220 2024-06-13T07:18:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.065+0000 7f47f1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109220 0x7f47ec19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.066+0000 7f47f1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109220 0x7f47ec19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60678/0 (socket says 172.21.15.26:60678) 2024-06-13T07:18:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.066+0000 7f47f1000640 1 -- 172.21.15.26:0/142668064 learned_addr learned my addr 172.21.15.26:0/142668064 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.066+0000 7f47f2400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec109be0 0x7f47ec1a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.066+0000 7f47f1000640 1 -- 172.21.15.26:0/142668064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47ec108030 msgr2=0x7f47ec19e240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:18:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.066+0000 7f47f1000640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47ec108030 0x7f47ec19e240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.066+0000 7f47f1000640 1 -- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec109be0 msgr2=0x7f47ec1a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.066+0000 7f47f1000640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec109be0 0x7f47ec1a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.066+0000 7f47f1000640 1 -- 172.21.15.26:0/142668064 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec1a5e60 con 0x7f47ec109220 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.067+0000 7f47f1000640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109220 0x7f47ec19e780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f47d800d950 tx=0x7f47d800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.067+0000 7f47ea000640 1 -- 172.21.15.26:0/142668064 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8017070 con 0x7f47ec109220 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.067+0000 7f47ea000640 1 -- 172.21.15.26:0/142668064 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d8009c30 con 0x7f47ec109220 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.067+0000 7f47f4400640 1 -- 172.21.15.26:0/142668064 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec07a1f0 con 0x7f47ec109220 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.067+0000 7f47ea000640 1 -- 172.21.15.26:0/142668064 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d800f690 con 0x7f47ec109220 2024-06-13T07:18:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.067+0000 7f47f4400640 1 -- 172.21.15.26:0/142668064 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec07a740 con 0x7f47ec109220 2024-06-13T07:18:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.075+0000 7f47ea000640 1 -- 172.21.15.26:0/142668064 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f47d800f830 con 0x7f47ec109220 2024-06-13T07:18:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.075+0000 7f47f7384740 1 Processor -- start 2024-06-13T07:18:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.076+0000 7f47f7384740 1 -- start start 2024-06-13T07:18:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.076+0000 7f47f7384740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208016e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.076+0000 7f47f7384740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x5620801730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.076+0000 7f47f7384740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 0x562080175bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.076+0000 7f47f7384740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5620800d1420 con 0x56208016dc00 2024-06-13T07:18:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.076+0000 7f47f7384740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562080078c50 con 0x56208016e5f0 2024-06-13T07:18:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.076+0000 7f47f7384740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5620800cd3e0 con 0x562080173760 2024-06-13T07:18:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.077+0000 7f47f1a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x5620801730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.077+0000 7f47f1a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x5620801730f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59922/0 (socket says 172.21.15.26:59922) 2024-06-13T07:18:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.077+0000 7f47f1a00640 1 -- 172.21.15.26:0/2816323683 learned_addr learned my addr 172.21.15.26:0/2816323683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.077+0000 7f47f1a00640 1 -- 172.21.15.26:0/2816323683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 msgr2=0x562080175bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.077+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2816323683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 0x562080175bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.077+0000 7f47f1a00640 1 -- 172.21.15.26:0/2816323683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 msgr2=0x56208016e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.077+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2816323683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208016e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.077+0000 7f47f1a00640 1 -- 172.21.15.26:0/2816323683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207fde5210 con 0x56208016e5f0 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.078+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2816323683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x5620801730f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f47e000ba80 tx=0x7f47e000bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9af36d1b5cda1a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.078+0000 7f47ea000640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f47d40a2e20 0x7f47d40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.078+0000 7f47ea000640 1 -- 172.21.15.26:0/142668064 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f47d80d00e0 con 0x7f47ec109220 2024-06-13T07:18:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.078+0000 7f47f2400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f47d40a2e20 0x7f47d40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.079+0000 7f47dea00640 1 -- 172.21.15.26:0/2816323683 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0016070 con 0x56208016e5f0 2024-06-13T07:18:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.079+0000 7f47dea00640 1 -- 172.21.15.26:0/2816323683 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e000dca0 con 0x56208016e5f0 2024-06-13T07:18:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.079+0000 7f47dea00640 1 -- 172.21.15.26:0/2816323683 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0016070 con 0x56208016e5f0 2024-06-13T07:18:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f7384740 1 -- 172.21.15.26:0/2816323683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 msgr2=0x5620801730f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f7384740 1 --2- 172.21.15.26:0/2816323683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x5620801730f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f47e000ba80 tx=0x7f47e000bf40 comp rx=0 tx=0).stop 2024-06-13T07:18:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f2400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f47d40a2e20 0x7f47d40a52e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x56208016ee50 tx=0x7f47e4031bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f7384740 1 -- 172.21.15.26:0/2816323683 shutdown_connections 2024-06-13T07:18:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f7384740 1 --2- 172.21.15.26:0/2816323683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x5620801730f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f7384740 1 --2- 172.21.15.26:0/2816323683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 0x562080175bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f7384740 1 --2- 172.21.15.26:0/2816323683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208016e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f7384740 1 -- 172.21.15.26:0/2816323683 >> 172.21.15.26:0/2816323683 conn(0x562080135fb0 msgr2=0x562080169680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.080+0000 7f47f7384740 1 -- 172.21.15.26:0/2816323683 shutdown_connections 2024-06-13T07:18:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.081+0000 7f47f7384740 1 -- 172.21.15.26:0/2816323683 wait complete. 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.081+0000 7f47f7384740 1 -- start start 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.081+0000 7f47f7384740 1 Processor -- start 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.082+0000 7f47f7384740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x562080177f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.082+0000 7f47f7384740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016e5f0 0x562080178470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.082+0000 7f47f7384740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208017f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.082+0000 7f47f7384740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562080176710 con 0x56208016dc00 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.082+0000 7f47f7384740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56208008c0b0 con 0x562080173760 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.082+0000 7f47f7384740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562080176890 con 0x56208016e5f0 2024-06-13T07:18:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.082+0000 7f47f7384740 10 client.0 _ll_register_callbacks cb 0x562080077f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f2400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208017f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f2400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208017f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59936/0 (socket says 172.21.15.26:59936) 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f2400640 1 -- 172.21.15.26:0/2979648553 learned_addr learned my addr 172.21.15.26:0/2979648553 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x562080177f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f2400640 1 -- 172.21.15.26:0/2979648553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016e5f0 msgr2=0x562080178470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f1000640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016e5f0 0x562080178470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f2400640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016e5f0 0x562080178470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f2400640 1 -- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 msgr2=0x562080177f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f2400640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x562080177f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f2400640 1 -- 172.21.15.26:0/2979648553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207feb1620 con 0x562080173760 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x562080177f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.083+0000 7f47f1000640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016e5f0 0x562080178470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.084+0000 7f47f2400640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208017f4b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f47e4035000 tx=0x7f47e402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3991fe33594893bd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.084+0000 7f47d3400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.084+0000 7f47d3400640 1 -- 172.21.15.26:0/2979648553 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4004740 con 0x562080173760 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.084+0000 7f47f7384740 1 -- 172.21.15.26:0/2979648553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e000dca0 con 0x562080173760 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.084+0000 7f47d3400640 1 -- 172.21.15.26:0/2979648553 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f47e402f610 con 0x562080173760 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.084+0000 7f47d3400640 1 -- 172.21.15.26:0/2979648553 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f47e40098d0 con 0x562080173760 2024-06-13T07:18:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.084+0000 7f47f7384740 10 client.26828 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.084+0000 7f47f7384740 1 -- 172.21.15.26:0/2979648553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562080182890 con 0x562080173760 2024-06-13T07:18:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.085+0000 7f47d3400640 1 -- 172.21.15.26:0/2979648553 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e000dca0 con 0x562080173760 2024-06-13T07:18:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.086+0000 7f47d3400640 1 -- 172.21.15.26:0/2979648553 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f47e402f810 con 0x562080173760 2024-06-13T07:18:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.086+0000 7f47f7384740 10 client.26828 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.086+0000 7f47f7384740 10 client.26828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.086+0000 7f47f7384740 20 client.26828 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.086+0000 7f47f7384740 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562080183f70 0x562080186360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.086+0000 7f47f7384740 4 client.26828 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:18:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.086+0000 7f47f7384740 1 -- 172.21.15.26:0/2979648553 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620800deb50 con 0x562080183f70 2024-06-13T07:18:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.087+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562080183f70 0x562080186360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.088+0000 7f47d3400640 10 client.26828 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.088+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562080183f70 0x562080186360 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d78c971d41474c server_cookie=bc4f485800f2bfc9 in_seq=0 out_seq=0 2024-06-13T07:18:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.095+0000 7f47d3400640 1 -- 172.21.15.26:0/2979648553 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5620800deb50 con 0x562080183f70 2024-06-13T07:18:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.095+0000 7f47d3400640 10 client.26828 handle_command_reply: tid=0 2024-06-13T07:18:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.095+0000 7f47f7384740 1 client.26828 shutdown 2024-06-13T07:18:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.096+0000 7f47f7384740 1 -- 172.21.15.26:0/2979648553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 msgr2=0x56208017f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.097+0000 7f47f7384740 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208017f4b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f47e4035000 tx=0x7f47e402a8c0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.098+0000 7f47f7384740 1 -- 172.21.15.26:0/2979648553 shutdown_connections 2024-06-13T07:18:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.098+0000 7f47f7384740 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562080183f70 0x562080186360 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.099+0000 7f47f7384740 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208017f4b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.099+0000 7f47d3400640 0 client.26828 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.099+0000 7f47f7384740 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016e5f0 0x562080178470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.099+0000 7f47f7384740 1 --2- 172.21.15.26:0/2979648553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x562080177f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.099+0000 7f47f7384740 1 -- 172.21.15.26:0/2979648553 >> 172.21.15.26:0/2979648553 conn(0x562080135fb0 msgr2=0x562080168840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.099+0000 7f47f7384740 1 -- 172.21.15.26:0/2979648553 shutdown_connections 2024-06-13T07:18:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.099+0000 7f47f7384740 1 -- 172.21.15.26:0/2979648553 wait complete. 2024-06-13T07:18:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.099+0000 7f47f7384740 20 client.26828 trim_cache size 0 max 16384 2024-06-13T07:18:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.157+0000 7f47f7384740 1 Processor -- start 2024-06-13T07:18:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.159+0000 7f47f7384740 1 -- start start 2024-06-13T07:18:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.159+0000 7f47f7384740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016dc00 0x56208016cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.159+0000 7f47f7384740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016e5f0 0x56208017f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.159+0000 7f47f7384740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208018ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.159+0000 7f47f7384740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1bbda0 con 0x56208016e5f0 2024-06-13T07:18:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.159+0000 7f47f7384740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1b9670 con 0x562080173760 2024-06-13T07:18:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.159+0000 7f47f7384740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1b74e0 con 0x56208016dc00 2024-06-13T07:18:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.159+0000 7f47f2400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208018ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.160+0000 7f47f1a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016dc00 0x56208016cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.160+0000 7f47f2400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208018ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59938/0 (socket says 172.21.15.26:59938) 2024-06-13T07:18:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.160+0000 7f47f2400640 1 -- 172.21.15.26:0/1349247412 learned_addr learned my addr 172.21.15.26:0/1349247412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.160+0000 7f47f2400640 1 -- 172.21.15.26:0/1349247412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016dc00 msgr2=0x56208016cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.160+0000 7f47f2400640 1 --2- 172.21.15.26:0/1349247412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016dc00 0x56208016cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.160+0000 7f47f2400640 1 -- 172.21.15.26:0/1349247412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016e5f0 msgr2=0x56208017f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.160+0000 7f47f2400640 1 --2- 172.21.15.26:0/1349247412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016e5f0 0x56208017f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.160+0000 7f47f2400640 1 -- 172.21.15.26:0/1349247412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620801866d0 con 0x562080173760 2024-06-13T07:18:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.161+0000 7f47f2400640 1 --2- 172.21.15.26:0/1349247412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208018ac60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f47e402a900 tx=0x7f47e404a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c5d52db54619ae1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.162+0000 7f47de000640 1 -- 172.21.15.26:0/1349247412 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e402fcc0 con 0x562080173760 2024-06-13T07:18:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.162+0000 7f47de000640 1 -- 172.21.15.26:0/1349247412 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4036040 con 0x562080173760 2024-06-13T07:18:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.162+0000 7f47de000640 1 -- 172.21.15.26:0/1349247412 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4030c00 con 0x562080173760 2024-06-13T07:18:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.163+0000 7f47f7384740 1 -- 172.21.15.26:0/1349247412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 msgr2=0x56208018ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.163+0000 7f47f7384740 1 --2- 172.21.15.26:0/1349247412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208018ac60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f47e402a900 tx=0x7f47e404a000 comp rx=0 tx=0).stop 2024-06-13T07:18:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.163+0000 7f47f7384740 1 -- 172.21.15.26:0/1349247412 shutdown_connections 2024-06-13T07:18:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.164+0000 7f47f7384740 1 --2- 172.21.15.26:0/1349247412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562080173760 0x56208018ac60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.164+0000 7f47f7384740 1 --2- 172.21.15.26:0/1349247412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016e5f0 0x56208017f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.164+0000 7f47f7384740 1 --2- 172.21.15.26:0/1349247412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56208016dc00 0x56208016cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.164+0000 7f47f7384740 1 -- 172.21.15.26:0/1349247412 >> 172.21.15.26:0/1349247412 conn(0x562080135fb0 msgr2=0x56208016c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.166+0000 7f47f7384740 1 -- 172.21.15.26:0/1349247412 shutdown_connections 2024-06-13T07:18:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.166+0000 7f47f7384740 1 -- 172.21.15.26:0/1349247412 wait complete. 2024-06-13T07:18:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.166+0000 7f47f7384740 1 -- start start 2024-06-13T07:18:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.167+0000 7f47f7384740 1 Processor -- start 2024-06-13T07:18:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.168+0000 7f47f7384740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208017c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.168+0000 7f47f7384740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x56208017ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.168+0000 7f47f7384740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 0x56208017dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f1000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x56208017ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f1000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x56208017ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59954/0 (socket says 172.21.15.26:59954) 2024-06-13T07:18:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f1a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208017c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f2400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 0x56208017dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f1000640 1 -- 172.21.15.26:0/2525091787 learned_addr learned my addr 172.21.15.26:0/2525091787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f1a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208017c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59638/0 (socket says 172.21.15.26:59638) 2024-06-13T07:18:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1b7b30 con 0x56208016dc00 2024-06-13T07:18:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1b8fa0 con 0x56208016e5f0 2024-06-13T07:18:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1b8200 con 0x562080173760 2024-06-13T07:18:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.169+0000 7f47f7384740 10 client.0 _ll_register_callbacks cb 0x7f47e4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.171+0000 7f47f2400640 1 -- 172.21.15.26:0/2525091787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 msgr2=0x56208017ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.171+0000 7f47f2400640 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x56208017ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.171+0000 7f47f2400640 1 -- 172.21.15.26:0/2525091787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 msgr2=0x56208017c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.171+0000 7f47f2400640 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208017c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.171+0000 7f47f2400640 1 -- 172.21.15.26:0/2525091787 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56208017e2a0 con 0x562080173760 2024-06-13T07:18:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.171+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208017c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:18:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.171+0000 7f47f1000640 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x56208017ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:18:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.174+0000 7f47f2400640 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 0x56208017dcb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47e4006010 tx=0x7f47e402fef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d62dd0f95b47d04 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.174+0000 7f47d3400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:18:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.176+0000 7f47d3400640 1 -- 172.21.15.26:0/2525091787 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4030950 con 0x562080173760 2024-06-13T07:18:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.176+0000 7f47d3400640 1 -- 172.21.15.26:0/2525091787 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f47e4030af0 con 0x562080173760 2024-06-13T07:18:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.178+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e4036040 con 0x562080173760 2024-06-13T07:18:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.178+0000 7f47d3400640 1 -- 172.21.15.26:0/2525091787 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f47e4030cf0 con 0x562080173760 2024-06-13T07:18:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.178+0000 7f47f7384740 10 client.26866 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.178+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56208017f5b0 con 0x562080173760 2024-06-13T07:18:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.179+0000 7f47d3400640 1 -- 172.21.15.26:0/2525091787 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4036040 con 0x562080173760 2024-06-13T07:18:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.179+0000 7f47d3400640 1 -- 172.21.15.26:0/2525091787 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f47e406f030 con 0x562080173760 2024-06-13T07:18:44.179 INFO:tasks.workunit.client.0.smithi026.stdout: 10 156228 28.18 MB/sec execute 177 sec latency 142.523 ms 2024-06-13T07:18:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.179+0000 7f47f7384740 10 client.26866 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.179+0000 7f47f7384740 10 client.26866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.179+0000 7f47f7384740 20 client.26866 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.179+0000 7f47f7384740 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562080183f70 0x5620801f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.179+0000 7f47f7384740 4 client.26866 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:18:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.179+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620800d8ea0 con 0x562080183f70 2024-06-13T07:18:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.181+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562080183f70 0x5620801f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.183+0000 7f47f1a00640 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562080183f70 0x5620801f0ef0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f881e115cbbd2367 server_cookie=6bdfd3e238219c39 in_seq=0 out_seq=0 2024-06-13T07:18:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:44.183+0000 7f47d3400640 10 client.26866 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:44 smithi026 ceph-mon[32041]: pgmap v1056: 129 pgs: 129 active+clean; 690 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 577 op/s 2024-06-13T07:18:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:44 smithi135 ceph-mon[36276]: pgmap v1056: 129 pgs: 129 active+clean; 690 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 577 op/s 2024-06-13T07:18:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:44 smithi136 ceph-mon[37229]: pgmap v1056: 129 pgs: 129 active+clean; 690 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 577 op/s 2024-06-13T07:18:45.178 INFO:tasks.workunit.client.0.smithi026.stdout: 10 156298 28.03 MB/sec execute 178 sec latency 865.501 ms 2024-06-13T07:18:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.641+0000 7f47d3400640 1 -- 172.21.15.26:0/2525091787 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5620800d8ea0 con 0x562080183f70 2024-06-13T07:18:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.641+0000 7f47d3400640 10 client.26866 handle_command_reply: tid=0 2024-06-13T07:18:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.641+0000 7f47f7384740 1 client.26866 shutdown 2024-06-13T07:18:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.641+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 msgr2=0x56208017dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.641+0000 7f47f7384740 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 0x56208017dcb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47e4006010 tx=0x7f47e402fef0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 shutdown_connections 2024-06-13T07:18:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47f7384740 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562080183f70 0x5620801f0ef0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47d3400640 0 client.26866 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47f7384740 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56208016e5f0 0x56208017ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47f7384740 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562080173760 0x56208017dcb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47f7384740 1 --2- 172.21.15.26:0/2525091787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56208016dc00 0x56208017c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 >> 172.21.15.26:0/2525091787 conn(0x562080135fb0 msgr2=0x56208016c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 shutdown_connections 2024-06-13T07:18:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.642+0000 7f47f7384740 1 -- 172.21.15.26:0/2525091787 wait complete. 2024-06-13T07:18:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.643+0000 7f47f7384740 20 client.26866 trim_cache size 0 max 16384 2024-06-13T07:18:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:18:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:18:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:18:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 -- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f47d40a2e20 msgr2=0x7f47d40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f47d40a2e20 0x7f47d40a52e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x56208016ee50 tx=0x7f47e4031bb0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 -- 172.21.15.26:0/142668064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109220 msgr2=0x7f47ec19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109220 0x7f47ec19e780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f47d800d950 tx=0x7f47d800de10 comp rx=0 tx=0).stop 2024-06-13T07:18:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 -- 172.21.15.26:0/142668064 shutdown_connections 2024-06-13T07:18:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f47d40a2e20 0x7f47d40a52e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec109be0 0x7f47ec1a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec109220 0x7f47ec19e780 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 --2- 172.21.15.26:0/142668064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47ec108030 0x7f47ec19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 -- 172.21.15.26:0/142668064 >> 172.21.15.26:0/142668064 conn(0x7f47ec076ff0 msgr2=0x7f47ec10c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 -- 172.21.15.26:0/142668064 shutdown_connections 2024-06-13T07:18:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:45.645+0000 7f47f4400640 1 -- 172.21.15.26:0/142668064 wait complete. 2024-06-13T07:18:46.179 INFO:tasks.workunit.client.0.smithi026.stdout: 10 156504 27.90 MB/sec execute 179 sec latency 889.161 ms 2024-06-13T07:18:46.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:46 smithi026 ceph-mon[32041]: pgmap v1057: 129 pgs: 129 active+clean; 712 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.5 MiB/s wr, 554 op/s 2024-06-13T07:18:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:46 smithi135 ceph-mon[36276]: pgmap v1057: 129 pgs: 129 active+clean; 712 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.5 MiB/s wr, 554 op/s 2024-06-13T07:18:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:46 smithi136 ceph-mon[37229]: pgmap v1057: 129 pgs: 129 active+clean; 712 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.5 MiB/s wr, 554 op/s 2024-06-13T07:18:47.180 INFO:tasks.workunit.client.0.smithi026.stdout: 10 156734 27.80 MB/sec execute 180 sec latency 132.508 ms 2024-06-13T07:18:48.180 INFO:tasks.workunit.client.0.smithi026.stdout: 10 157035 27.74 MB/sec execute 181 sec latency 110.052 ms 2024-06-13T07:18:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:48 smithi026 ceph-mon[32041]: pgmap v1058: 129 pgs: 129 active+clean; 726 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-06-13T07:18:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:48 smithi135 ceph-mon[36276]: pgmap v1058: 129 pgs: 129 active+clean; 726 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-06-13T07:18:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:48 smithi136 ceph-mon[37229]: pgmap v1058: 129 pgs: 129 active+clean; 726 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-06-13T07:18:49.180 INFO:tasks.workunit.client.0.smithi026.stdout: 10 157565 27.71 MB/sec execute 182 sec latency 114.565 ms 2024-06-13T07:18:49.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:49 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:18:49.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:49 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:18:49.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:49 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:18:50.181 INFO:tasks.workunit.client.0.smithi026.stdout: 10 158153 27.70 MB/sec execute 183 sec latency 267.815 ms 2024-06-13T07:18:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:50 smithi026 ceph-mon[32041]: pgmap v1059: 129 pgs: 129 active+clean; 671 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 692 op/s 2024-06-13T07:18:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:50 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:50.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:50 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:50 smithi135 ceph-mon[36276]: pgmap v1059: 129 pgs: 129 active+clean; 671 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 692 op/s 2024-06-13T07:18:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:50 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:50 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:50 smithi136 ceph-mon[37229]: pgmap v1059: 129 pgs: 129 active+clean; 671 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 692 op/s 2024-06-13T07:18:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:50 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:50 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:51.180 INFO:tasks.workunit.client.0.smithi026.stdout: 10 158899 27.82 MB/sec execute 184 sec latency 58.334 ms 2024-06-13T07:18:52.126 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:52.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:52.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:51 smithi136 ceph-mon[37229]: pgmap v1060: 129 pgs: 129 active+clean; 657 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 759 op/s 2024-06-13T07:18:52.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:18:52.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:18:52.127 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:51 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:52.180 INFO:tasks.workunit.client.0.smithi026.stdout: 10 159610 27.93 MB/sec execute 185 sec latency 91.948 ms 2024-06-13T07:18:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:51 smithi026 ceph-mon[32041]: pgmap v1060: 129 pgs: 129 active+clean; 657 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 759 op/s 2024-06-13T07:18:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:18:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:18:52.252 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:51 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:51 smithi135 ceph-mon[36276]: pgmap v1060: 129 pgs: 129 active+clean; 657 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 759 op/s 2024-06-13T07:18:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:18:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:18:52.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:51 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:18:53.180 INFO:tasks.workunit.client.0.smithi026.stdout: 10 160040 27.91 MB/sec execute 186 sec latency 76.247 ms 2024-06-13T07:18:54.181 INFO:tasks.workunit.client.0.smithi026.stdout: 10 160598 27.89 MB/sec execute 187 sec latency 81.210 ms 2024-06-13T07:18:54.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:54 smithi136 ceph-mon[37229]: pgmap v1061: 129 pgs: 129 active+clean; 637 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-06-13T07:18:54.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:54 smithi026 ceph-mon[32041]: pgmap v1061: 129 pgs: 129 active+clean; 637 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-06-13T07:18:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:54 smithi135 ceph-mon[36276]: pgmap v1061: 129 pgs: 129 active+clean; 637 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-06-13T07:18:55.181 INFO:tasks.workunit.client.0.smithi026.stdout: 10 161081 27.85 MB/sec execute 188 sec latency 63.959 ms 2024-06-13T07:18:56.181 INFO:tasks.workunit.client.0.smithi026.stdout: 10 161705 27.89 MB/sec execute 189 sec latency 103.937 ms 2024-06-13T07:18:56.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:56 smithi136 ceph-mon[37229]: pgmap v1062: 129 pgs: 129 active+clean; 640 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 708 op/s 2024-06-13T07:18:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:56 smithi026 ceph-mon[32041]: pgmap v1062: 129 pgs: 129 active+clean; 640 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 708 op/s 2024-06-13T07:18:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:56 smithi135 ceph-mon[36276]: pgmap v1062: 129 pgs: 129 active+clean; 640 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 708 op/s 2024-06-13T07:18:57.181 INFO:tasks.workunit.client.0.smithi026.stdout: 10 162391 27.92 MB/sec execute 190 sec latency 88.083 ms 2024-06-13T07:18:58.181 INFO:tasks.workunit.client.0.smithi026.stdout: 10 163069 27.97 MB/sec execute 191 sec latency 76.730 ms 2024-06-13T07:18:58.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:18:58 smithi136 ceph-mon[37229]: pgmap v1063: 129 pgs: 129 active+clean; 642 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.7 MiB/s wr, 632 op/s 2024-06-13T07:18:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:18:58 smithi026 ceph-mon[32041]: pgmap v1063: 129 pgs: 129 active+clean; 642 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.7 MiB/s wr, 632 op/s 2024-06-13T07:18:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:18:58 smithi135 ceph-mon[36276]: pgmap v1063: 129 pgs: 129 active+clean; 642 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.7 MiB/s wr, 632 op/s 2024-06-13T07:18:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.878+0000 7fb071800640 1 -- 172.21.15.26:0/2668821979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c1089a0 msgr2=0x7fb06c10add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.878+0000 7fb071800640 1 --2- 172.21.15.26:0/2668821979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c1089a0 0x7fb06c10add0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb064009be0 tx=0x7fb06402a6d0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.881+0000 7fb071800640 1 -- 172.21.15.26:0/2668821979 shutdown_connections 2024-06-13T07:18:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.881+0000 7fb071800640 1 --2- 172.21.15.26:0/2668821979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06c10dcc0 0x7fb06c10e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.881+0000 7fb071800640 1 --2- 172.21.15.26:0/2668821979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb06c10b310 0x7fb06c10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.881+0000 7fb071800640 1 --2- 172.21.15.26:0/2668821979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c1089a0 0x7fb06c10add0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.881+0000 7fb071800640 1 -- 172.21.15.26:0/2668821979 >> 172.21.15.26:0/2668821979 conn(0x7fb06c06ea10 msgr2=0x7fb06c070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.881+0000 7fb071800640 1 -- 172.21.15.26:0/2668821979 shutdown_connections 2024-06-13T07:18:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.882+0000 7fb071800640 1 -- 172.21.15.26:0/2668821979 wait complete. 2024-06-13T07:18:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.884+0000 7fb071800640 1 Processor -- start 2024-06-13T07:18:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.884+0000 7fb071800640 1 -- start start 2024-06-13T07:18:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.885+0000 7fb071800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb06c1089a0 0x7fb06c117030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.885+0000 7fb071800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c10b310 0x7fb06c117570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.885+0000 7fb071800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06c10dcc0 0x7fb06c1b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.885+0000 7fb06a000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c10b310 0x7fb06c117570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.885+0000 7fb06a000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c10b310 0x7fb06c117570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54926/0 (socket says 172.21.15.26:54926) 2024-06-13T07:18:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.886+0000 7fb06a000640 1 -- 172.21.15.26:0/1696188389 learned_addr learned my addr 172.21.15.26:0/1696188389 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.886+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb06c1089a0 0x7fb06c117030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.886+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c11c450 con 0x7fb06c10dcc0 2024-06-13T07:18:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.886+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c11c2d0 con 0x7fb06c10b310 2024-06-13T07:18:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.886+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c11c5d0 con 0x7fb06c1089a0 2024-06-13T07:18:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.886+0000 7fb06b400640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06c10dcc0 0x7fb06c1b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.888+0000 7fb06a000640 1 -- 172.21.15.26:0/1696188389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb06c1089a0 msgr2=0x7fb06c117030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.888+0000 7fb06a000640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb06c1089a0 0x7fb06c117030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.888+0000 7fb06a000640 1 -- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06c10dcc0 msgr2=0x7fb06c1b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.888+0000 7fb06a000640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06c10dcc0 0x7fb06c1b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.888+0000 7fb06a000640 1 -- 172.21.15.26:0/1696188389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c1b7080 con 0x7fb06c10b310 2024-06-13T07:18:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.889+0000 7fb06b400640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06c10dcc0 0x7fb06c1b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:18:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.889+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb06c1089a0 0x7fb06c117030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:18:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.890+0000 7fb06a000640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c10b310 0x7fb06c117570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb05800d950 tx=0x7fb05800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.891+0000 7fb063400640 1 -- 172.21.15.26:0/1696188389 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058017070 con 0x7fb06c10b310 2024-06-13T07:18:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.891+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c1b7250 con 0x7fb06c10b310 2024-06-13T07:18:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.891+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1b7740 con 0x7fb06c10b310 2024-06-13T07:18:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.891+0000 7fb063400640 1 -- 172.21.15.26:0/1696188389 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb058009c30 con 0x7fb06c10b310 2024-06-13T07:18:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.891+0000 7fb063400640 1 -- 172.21.15.26:0/1696188389 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05800f6b0 con 0x7fb06c10b310 2024-06-13T07:18:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.914+0000 7fb063400640 1 -- 172.21.15.26:0/1696188389 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb05800f850 con 0x7fb06c10b310 2024-06-13T07:18:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.915+0000 7fb074578740 1 Processor -- start 2024-06-13T07:18:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.915+0000 7fb074578740 1 -- start start 2024-06-13T07:18:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.916+0000 7fb074578740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c574e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.916+0000 7fb074578740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574e5f0 0x5606c57530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.916+0000 7fb074578740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5755bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.916+0000 7fb074578740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5606c56b1420 con 0x5606c574dc00 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.916+0000 7fb074578740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606c5658c50 con 0x5606c5753760 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.916+0000 7fb074578740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5606c56ad3e0 con 0x5606c574e5f0 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.917+0000 7fb06a000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5755bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.917+0000 7fb06a000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5755bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54940/0 (socket says 172.21.15.26:54940) 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.917+0000 7fb06a000640 1 -- 172.21.15.26:0/1675801339 learned_addr learned my addr 172.21.15.26:0/1675801339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.917+0000 7fb06b400640 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c574e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.918+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574e5f0 0x5606c57530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.918+0000 7fb06aa00640 1 -- 172.21.15.26:0/1675801339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 msgr2=0x5606c5755bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.918+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5755bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.918+0000 7fb06aa00640 1 -- 172.21.15.26:0/1675801339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 msgr2=0x5606c574e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.919+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c574e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.919+0000 7fb06a000640 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5755bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-13T07:18:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.919+0000 7fb06aa00640 1 -- 172.21.15.26:0/1675801339 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c53c5210 con 0x5606c574e5f0 2024-06-13T07:18:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.920+0000 7fb063400640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb0500a2d00 0x7fb0500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.920+0000 7fb063400640 1 -- 172.21.15.26:0/1696188389 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb0580cef20 con 0x7fb06c10b310 2024-06-13T07:18:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.922+0000 7fb06b400640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb0500a2d00 0x7fb0500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.923+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574e5f0 0x5606c57530f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb064009b30 tx=0x7fb064005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=211edd6c5e3aea7d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.923+0000 7fb057e00640 1 -- 172.21.15.26:0/1675801339 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064041030 con 0x5606c574e5f0 2024-06-13T07:18:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.924+0000 7fb057e00640 1 -- 172.21.15.26:0/1675801339 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb064003e50 con 0x5606c574e5f0 2024-06-13T07:18:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.924+0000 7fb057e00640 1 -- 172.21.15.26:0/1675801339 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064036040 con 0x5606c574e5f0 2024-06-13T07:18:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.925+0000 7fb074578740 1 -- 172.21.15.26:0/1675801339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574e5f0 msgr2=0x5606c57530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.925+0000 7fb074578740 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574e5f0 0x5606c57530f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb064009b30 tx=0x7fb064005ec0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.925+0000 7fb06b400640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb0500a2d00 0x7fb0500a51c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5606c574ee50 tx=0x7fb05c00d540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.926+0000 7fb074578740 1 -- 172.21.15.26:0/1675801339 shutdown_connections 2024-06-13T07:18:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.926+0000 7fb074578740 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5755bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.926+0000 7fb074578740 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574e5f0 0x5606c57530f0 secure :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb064009b30 tx=0x7fb064005ec0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.926+0000 7fb074578740 1 --2- 172.21.15.26:0/1675801339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c574e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.926+0000 7fb074578740 1 -- 172.21.15.26:0/1675801339 >> 172.21.15.26:0/1675801339 conn(0x5606c5715fb0 msgr2=0x5606c5749680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.926+0000 7fb074578740 1 -- 172.21.15.26:0/1675801339 shutdown_connections 2024-06-13T07:18:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.927+0000 7fb074578740 1 -- 172.21.15.26:0/1675801339 wait complete. 2024-06-13T07:18:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.927+0000 7fb074578740 1 -- start start 2024-06-13T07:18:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.928+0000 7fb074578740 1 Processor -- start 2024-06-13T07:18:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.929+0000 7fb074578740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c5758130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.929+0000 7fb074578740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5758670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.929+0000 7fb074578740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c575d6a0 0x5606c575fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.929+0000 7fb074578740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5606c57566b0 con 0x5606c574dc00 2024-06-13T07:18:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.929+0000 7fb074578740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606c566c0b0 con 0x5606c5753760 2024-06-13T07:18:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.929+0000 7fb074578740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5606c5756830 con 0x5606c575d6a0 2024-06-13T07:18:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.930+0000 7fb06b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c575d6a0 0x5606c575fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.930+0000 7fb06b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c575d6a0 0x5606c575fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38118/0 (socket says 172.21.15.26:38118) 2024-06-13T07:18:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.930+0000 7fb06b400640 1 -- 172.21.15.26:0/1862070000 learned_addr learned my addr 172.21.15.26:0/1862070000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.930+0000 7fb074578740 10 client.0 _ll_register_callbacks cb 0x5606c5657f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.930+0000 7fb06a000640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5758670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.931+0000 7fb06b400640 1 -- 172.21.15.26:0/1862070000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 msgr2=0x5606c5758670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.931+0000 7fb06b400640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5758670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.931+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c5758130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.931+0000 7fb06b400640 1 -- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 msgr2=0x5606c5758130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.931+0000 7fb06b400640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c5758130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.931+0000 7fb06b400640 1 -- 172.21.15.26:0/1862070000 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c5491620 con 0x5606c575d6a0 2024-06-13T07:18:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.931+0000 7fb06a000640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5758670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.932+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c5758130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.932+0000 7fb06b400640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c575d6a0 0x5606c575fa90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb05c013fd0 tx=0x7fb05c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2e04cdc9c2e659f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.932+0000 7fb054c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:18:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.937+0000 7fb054c00640 1 -- 172.21.15.26:0/1862070000 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c01e820 con 0x5606c575d6a0 2024-06-13T07:18:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.937+0000 7fb074578740 1 -- 172.21.15.26:0/1862070000 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb064003e50 con 0x5606c575d6a0 2024-06-13T07:18:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.937+0000 7fb054c00640 1 -- 172.21.15.26:0/1862070000 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb05c01e620 con 0x5606c575d6a0 2024-06-13T07:18:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.938+0000 7fb054c00640 1 -- 172.21.15.26:0/1862070000 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb05c01e820 con 0x5606c575d6a0 2024-06-13T07:18:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.938+0000 7fb074578740 10 client.26878 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.938+0000 7fb074578740 1 -- 172.21.15.26:0/1862070000 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5606c5762ec0 con 0x5606c575d6a0 2024-06-13T07:18:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.938+0000 7fb054c00640 1 -- 172.21.15.26:0/1862070000 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb064003e50 con 0x5606c575d6a0 2024-06-13T07:18:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.938+0000 7fb054c00640 1 -- 172.21.15.26:0/1862070000 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fb05c029c90 con 0x5606c575d6a0 2024-06-13T07:18:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.939+0000 7fb074578740 10 client.26878 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.939+0000 7fb074578740 10 client.26878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.939+0000 7fb074578740 20 client.26878 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.939+0000 7fb074578740 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606c5764600 0x5606c5766aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.939+0000 7fb074578740 4 client.26878 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:18:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.939+0000 7fb074578740 1 -- 172.21.15.26:0/1862070000 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606c56beb50 con 0x5606c5764600 2024-06-13T07:18:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.941+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606c5764600 0x5606c5766aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.944+0000 7fb06aa00640 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606c5764600 0x5606c5766aa0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91e09a4c99e420ca server_cookie=efed34d43c9d8985 in_seq=0 out_seq=0 2024-06-13T07:18:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.944+0000 7fb054c00640 10 client.26878 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.953+0000 7fb054c00640 1 -- 172.21.15.26:0/1862070000 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5606c56beb50 con 0x5606c5764600 2024-06-13T07:18:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.954+0000 7fb054c00640 10 client.26878 handle_command_reply: tid=0 2024-06-13T07:18:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.954+0000 7fb074578740 1 client.26878 shutdown 2024-06-13T07:18:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.957+0000 7fb074578740 1 -- 172.21.15.26:0/1862070000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c575d6a0 msgr2=0x5606c575fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.957+0000 7fb074578740 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c575d6a0 0x5606c575fa90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb05c013fd0 tx=0x7fb05c0046c0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.958+0000 7fb074578740 1 -- 172.21.15.26:0/1862070000 shutdown_connections 2024-06-13T07:18:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.958+0000 7fb074578740 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606c5764600 0x5606c5766aa0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.958+0000 7fb074578740 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c5753760 0x5606c5758670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.958+0000 7fb074578740 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c575d6a0 0x5606c575fa90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.958+0000 7fb074578740 1 --2- 172.21.15.26:0/1862070000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c574dc00 0x5606c5758130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.958+0000 7fb054c00640 0 client.26878 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.958+0000 7fb074578740 1 -- 172.21.15.26:0/1862070000 >> 172.21.15.26:0/1862070000 conn(0x5606c5715fb0 msgr2=0x5606c5748840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.960+0000 7fb074578740 1 -- 172.21.15.26:0/1862070000 shutdown_connections 2024-06-13T07:18:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.960+0000 7fb074578740 1 -- 172.21.15.26:0/1862070000 wait complete. 2024-06-13T07:18:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:58.960+0000 7fb074578740 20 client.26878 trim_cache size 0 max 16384 2024-06-13T07:18:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.005+0000 7fb074578740 1 Processor -- start 2024-06-13T07:18:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.005+0000 7fb074578740 1 -- start start 2024-06-13T07:18:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.005+0000 7fb074578740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5749870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.005+0000 7fb074578740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c57446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.005+0000 7fb074578740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c574c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.006+0000 7fb06aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5749870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.006+0000 7fb06aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5749870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38124/0 (socket says 172.21.15.26:38124) 2024-06-13T07:18:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.006+0000 7fb06aa00640 1 -- 172.21.15.26:0/3840763511 learned_addr learned my addr 172.21.15.26:0/3840763511 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.006+0000 7fb06b400640 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c574c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.006+0000 7fb06a000640 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c57446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.007+0000 7fb074578740 1 -- 172.21.15.26:0/3840763511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c012670 con 0x5606c5753760 2024-06-13T07:18:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.007+0000 7fb074578740 1 -- 172.21.15.26:0/3840763511 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c00ff40 con 0x5606c575d6a0 2024-06-13T07:18:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.007+0000 7fb074578740 1 -- 172.21.15.26:0/3840763511 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c00dd90 con 0x5606c574dc00 2024-06-13T07:18:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.008+0000 7fb06b400640 1 -- 172.21.15.26:0/3840763511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 msgr2=0x5606c5749870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.008+0000 7fb06b400640 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5749870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.008+0000 7fb06b400640 1 -- 172.21.15.26:0/3840763511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 msgr2=0x5606c57446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.008+0000 7fb06b400640 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c57446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.008+0000 7fb06b400640 1 -- 172.21.15.26:0/3840763511 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c5745190 con 0x5606c575d6a0 2024-06-13T07:18:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.009+0000 7fb06b400640 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c574c340 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb05c01b3e0 tx=0x7fb05c01dc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8431d5f3b71785e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.010+0000 7fb057400640 1 -- 172.21.15.26:0/3840763511 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c03a030 con 0x5606c575d6a0 2024-06-13T07:18:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.010+0000 7fb057400640 1 -- 172.21.15.26:0/3840763511 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c01e620 con 0x5606c575d6a0 2024-06-13T07:18:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.010+0000 7fb057400640 1 -- 172.21.15.26:0/3840763511 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c004990 con 0x5606c575d6a0 2024-06-13T07:18:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 -- 172.21.15.26:0/3840763511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 msgr2=0x5606c574c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:18:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c574c340 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb05c01b3e0 tx=0x7fb05c01dc70 comp rx=0 tx=0).stop 2024-06-13T07:18:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 -- 172.21.15.26:0/3840763511 shutdown_connections 2024-06-13T07:18:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c574c340 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c57446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 --2- 172.21.15.26:0/3840763511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5749870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 -- 172.21.15.26:0/3840763511 >> 172.21.15.26:0/3840763511 conn(0x5606c5715fb0 msgr2=0x5606c5749160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:18:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 -- 172.21.15.26:0/3840763511 shutdown_connections 2024-06-13T07:18:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.012+0000 7fb074578740 1 -- 172.21.15.26:0/3840763511 wait complete. 2024-06-13T07:18:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.014+0000 7fb074578740 1 -- start start 2024-06-13T07:18:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.014+0000 7fb074578740 1 Processor -- start 2024-06-13T07:18:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.019+0000 7fb074578740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5775d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.019+0000 7fb074578740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c5776290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.020+0000 7fb074578740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c57772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.020+0000 7fb06aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5775d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.020+0000 7fb06aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5775d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:38130/0 (socket says 172.21.15.26:38130) 2024-06-13T07:18:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.021+0000 7fb06aa00640 1 -- 172.21.15.26:0/4254726111 learned_addr learned my addr 172.21.15.26:0/4254726111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:18:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.021+0000 7fb06b400640 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c57772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.021+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c00e400 con 0x5606c5753760 2024-06-13T07:18:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.021+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c00f870 con 0x5606c575d6a0 2024-06-13T07:18:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.021+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c00ead0 con 0x5606c574dc00 2024-06-13T07:18:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.021+0000 7fb074578740 10 client.0 _ll_register_callbacks cb 0x7fb05c00daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:18:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.022+0000 7fb06b400640 1 -- 172.21.15.26:0/4254726111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 msgr2=0x5606c5775d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.022+0000 7fb06b400640 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5775d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.022+0000 7fb06b400640 1 -- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 msgr2=0x5606c5776290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:18:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.021+0000 7fb06a000640 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c5776290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.022+0000 7fb06b400640 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c5776290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:18:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.022+0000 7fb06b400640 1 -- 172.21.15.26:0/4254726111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c5777890 con 0x5606c575d6a0 2024-06-13T07:18:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.023+0000 7fb06a000640 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c5776290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:18:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.026+0000 7fb06b400640 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c57772a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb05c012cb0 tx=0x7fb05c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a785fdaffe1fc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:18:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.026+0000 7fb054c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:18:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.026+0000 7fb054c00640 1 -- 172.21.15.26:0/4254726111 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c043020 con 0x5606c575d6a0 2024-06-13T07:18:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.026+0000 7fb054c00640 1 -- 172.21.15.26:0/4254726111 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb05c004680 con 0x5606c575d6a0 2024-06-13T07:18:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.027+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c01e620 con 0x5606c575d6a0 2024-06-13T07:18:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.027+0000 7fb054c00640 1 -- 172.21.15.26:0/4254726111 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb05c03a040 con 0x5606c575d6a0 2024-06-13T07:18:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.028+0000 7fb074578740 10 client.26846 fetch_fsmap learned FSMap version 47 2024-06-13T07:18:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.028+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5606c5778ba0 con 0x5606c575d6a0 2024-06-13T07:18:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.028+0000 7fb054c00640 1 -- 172.21.15.26:0/4254726111 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c01e620 con 0x5606c575d6a0 2024-06-13T07:18:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.029+0000 7fb054c00640 1 -- 172.21.15.26:0/4254726111 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fb05c01baf0 con 0x5606c575d6a0 2024-06-13T07:18:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.030+0000 7fb074578740 10 client.26846 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:18:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.030+0000 7fb074578740 10 client.26846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:18:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.030+0000 7fb074578740 20 client.26846 populate_metadata read hostname 'smithi026' 2024-06-13T07:18:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.030+0000 7fb074578740 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606c57cf720 0x5606c57d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:18:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.030+0000 7fb074578740 4 client.26846 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:18:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.030+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606c5744db0 con 0x5606c57cf720 2024-06-13T07:18:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.031+0000 7fb06aa00640 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606c57cf720 0x5606c57d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:18:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.032+0000 7fb06aa00640 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606c57cf720 0x5606c57d1b10 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a6d688444000 server_cookie=c589f206b0bef36 in_seq=0 out_seq=0 2024-06-13T07:18:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:18:59.032+0000 7fb054c00640 10 client.26846 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:18:59.181 INFO:tasks.workunit.client.0.smithi026.stdout: 10 163549 28.02 MB/sec execute 192 sec latency 243.136 ms 2024-06-13T07:19:00.181 INFO:tasks.workunit.client.0.smithi026.stdout: 10 163645 27.88 MB/sec execute 193 sec latency 1219.954 ms 2024-06-13T07:19:00.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:00 smithi136 ceph-mon[37229]: pgmap v1064: 129 pgs: 129 active+clean; 652 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-06-13T07:19:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:00 smithi026 ceph-mon[32041]: pgmap v1064: 129 pgs: 129 active+clean; 652 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-06-13T07:19:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:00 smithi135 ceph-mon[36276]: pgmap v1064: 129 pgs: 129 active+clean; 652 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-06-13T07:19:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.002+0000 7fb054c00640 1 -- 172.21.15.26:0/4254726111 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5606c5744db0 con 0x5606c57cf720 2024-06-13T07:19:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.002+0000 7fb054c00640 10 client.26846 handle_command_reply: tid=0 2024-06-13T07:19:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.002+0000 7fb074578740 1 client.26846 shutdown 2024-06-13T07:19:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.003+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 msgr2=0x5606c57772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.003+0000 7fb074578740 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c57772a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb05c012cb0 tx=0x7fb05c015000 comp rx=0 tx=0).stop 2024-06-13T07:19:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.003+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 shutdown_connections 2024-06-13T07:19:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.003+0000 7fb074578740 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5606c57cf720 0x5606c57d1b10 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.003+0000 7fb074578740 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606c575d6a0 0x5606c57772a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.003+0000 7fb074578740 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606c5753760 0x5606c5776290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.003+0000 7fb054c00640 0 client.26846 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.003+0000 7fb074578740 1 --2- 172.21.15.26:0/4254726111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606c574dc00 0x5606c5775d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.004+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 >> 172.21.15.26:0/4254726111 conn(0x5606c5715fb0 msgr2=0x5606c574cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.005+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 shutdown_connections 2024-06-13T07:19:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.005+0000 7fb074578740 1 -- 172.21.15.26:0/4254726111 wait complete. 2024-06-13T07:19:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.006+0000 7fb074578740 20 client.26846 trim_cache size 0 max 16384 2024-06-13T07:19:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:19:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:19:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:19:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb0500a2d00 msgr2=0x7fb0500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb0500a2d00 0x7fb0500a51c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5606c574ee50 tx=0x7fb05c00d540 comp rx=0 tx=0).stop 2024-06-13T07:19:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c10b310 msgr2=0x7fb06c117570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c10b310 0x7fb06c117570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb05800d950 tx=0x7fb05800de10 comp rx=0 tx=0).stop 2024-06-13T07:19:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 shutdown_connections 2024-06-13T07:19:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fb0500a2d00 0x7fb0500a51c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb06c10b310 0x7fb06c117570 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06c10dcc0 0x7fb06c1b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 --2- 172.21.15.26:0/1696188389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb06c1089a0 0x7fb06c117030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.008+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 >> 172.21.15.26:0/1696188389 conn(0x7fb06c06ea10 msgr2=0x7fb06c1090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.009+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 shutdown_connections 2024-06-13T07:19:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:01.009+0000 7fb071800640 1 -- 172.21.15.26:0/1696188389 wait complete. 2024-06-13T07:19:01.182 INFO:tasks.workunit.client.0.smithi026.stdout: 10 163866 27.78 MB/sec execute 194 sec latency 1240.751 ms 2024-06-13T07:19:02.182 INFO:tasks.workunit.client.0.smithi026.stdout: 10 164173 27.74 MB/sec execute 195 sec latency 201.478 ms 2024-06-13T07:19:02.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:02 smithi136 ceph-mon[37229]: pgmap v1065: 129 pgs: 129 active+clean; 664 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-06-13T07:19:02.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:02 smithi026 ceph-mon[32041]: pgmap v1065: 129 pgs: 129 active+clean; 664 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-06-13T07:19:02.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:02 smithi135 ceph-mon[36276]: pgmap v1065: 129 pgs: 129 active+clean; 664 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-06-13T07:19:03.182 INFO:tasks.workunit.client.0.smithi026.stdout: 10 164729 27.77 MB/sec execute 196 sec latency 104.218 ms 2024-06-13T07:19:04.182 INFO:tasks.workunit.client.0.smithi026.stdout: 10 165330 27.75 MB/sec execute 197 sec latency 44.465 ms 2024-06-13T07:19:04.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:04 smithi136 ceph-mon[37229]: pgmap v1066: 129 pgs: 129 active+clean; 666 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 517 op/s 2024-06-13T07:19:04.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:04 smithi026 ceph-mon[32041]: pgmap v1066: 129 pgs: 129 active+clean; 666 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 517 op/s 2024-06-13T07:19:04.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:04 smithi135 ceph-mon[36276]: pgmap v1066: 129 pgs: 129 active+clean; 666 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 517 op/s 2024-06-13T07:19:05.182 INFO:tasks.workunit.client.0.smithi026.stdout: 10 165635 27.69 MB/sec execute 198 sec latency 185.182 ms 2024-06-13T07:19:06.182 INFO:tasks.workunit.client.0.smithi026.stdout: 10 166168 27.73 MB/sec execute 199 sec latency 324.888 ms 2024-06-13T07:19:06.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:06 smithi136 ceph-mon[37229]: pgmap v1067: 129 pgs: 129 active+clean; 666 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-06-13T07:19:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:06 smithi026 ceph-mon[32041]: pgmap v1067: 129 pgs: 129 active+clean; 666 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-06-13T07:19:06.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:06 smithi135 ceph-mon[36276]: pgmap v1067: 129 pgs: 129 active+clean; 666 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-06-13T07:19:07.182 INFO:tasks.workunit.client.0.smithi026.stdout: 10 166781 27.71 MB/sec execute 200 sec latency 119.882 ms 2024-06-13T07:19:08.183 INFO:tasks.workunit.client.0.smithi026.stdout: 10 167339 27.73 MB/sec execute 201 sec latency 118.340 ms 2024-06-13T07:19:08.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:08 smithi136 ceph-mon[37229]: pgmap v1068: 129 pgs: 129 active+clean; 640 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.2 MiB/s wr, 585 op/s 2024-06-13T07:19:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:08 smithi026 ceph-mon[32041]: pgmap v1068: 129 pgs: 129 active+clean; 640 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.2 MiB/s wr, 585 op/s 2024-06-13T07:19:08.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:08 smithi135 ceph-mon[36276]: pgmap v1068: 129 pgs: 129 active+clean; 640 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.2 MiB/s wr, 585 op/s 2024-06-13T07:19:09.183 INFO:tasks.workunit.client.0.smithi026.stdout: 10 168046 27.78 MB/sec execute 202 sec latency 46.987 ms 2024-06-13T07:19:10.183 INFO:tasks.workunit.client.0.smithi026.stdout: 10 168631 27.82 MB/sec execute 203 sec latency 93.494 ms 2024-06-13T07:19:10.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:10 smithi136 ceph-mon[37229]: pgmap v1069: 129 pgs: 129 active+clean; 639 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 651 op/s 2024-06-13T07:19:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:10 smithi026 ceph-mon[32041]: pgmap v1069: 129 pgs: 129 active+clean; 639 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 651 op/s 2024-06-13T07:19:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:10 smithi135 ceph-mon[36276]: pgmap v1069: 129 pgs: 129 active+clean; 639 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 651 op/s 2024-06-13T07:19:11.183 INFO:tasks.workunit.client.0.smithi026.stdout: 10 169298 27.87 MB/sec execute 204 sec latency 125.794 ms 2024-06-13T07:19:12.183 INFO:tasks.workunit.client.0.smithi026.stdout: 10 169879 27.88 MB/sec execute 205 sec latency 92.193 ms 2024-06-13T07:19:12.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:12 smithi136 ceph-mon[37229]: pgmap v1070: 129 pgs: 129 active+clean; 642 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 608 op/s 2024-06-13T07:19:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:12 smithi026 ceph-mon[32041]: pgmap v1070: 129 pgs: 129 active+clean; 642 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 608 op/s 2024-06-13T07:19:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:12 smithi135 ceph-mon[36276]: pgmap v1070: 129 pgs: 129 active+clean; 642 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 608 op/s 2024-06-13T07:19:13.183 INFO:tasks.workunit.client.0.smithi026.stdout: 10 170275 27.86 MB/sec execute 206 sec latency 79.934 ms 2024-06-13T07:19:14.183 INFO:tasks.workunit.client.0.smithi026.stdout: 10 170785 27.84 MB/sec execute 207 sec latency 122.649 ms 2024-06-13T07:19:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.244+0000 7f04b1600640 1 -- 172.21.15.26:0/163634762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4030950 con 0x7f04b4109300 2024-06-13T07:19:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.244+0000 7f04b9c00640 1 -- 172.21.15.26:0/163634762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4109300 msgr2=0x7f04b410fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.244+0000 7f04b9c00640 1 --2- 172.21.15.26:0/163634762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4109300 0x7f04b410fce0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009860 tx=0x7f04a402a340 comp rx=0 tx=0).stop 2024-06-13T07:19:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.249+0000 7f04b9c00640 1 -- 172.21.15.26:0/163634762 shutdown_connections 2024-06-13T07:19:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.249+0000 7f04b9c00640 1 --2- 172.21.15.26:0/163634762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b4110220 0x7f04b4112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.249+0000 7f04b9c00640 1 --2- 172.21.15.26:0/163634762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4109300 0x7f04b410fce0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.249+0000 7f04b9c00640 1 --2- 172.21.15.26:0/163634762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b41089a0 0x7f04b4108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.249+0000 7f04b9c00640 1 -- 172.21.15.26:0/163634762 >> 172.21.15.26:0/163634762 conn(0x7f04b406ea10 msgr2=0x7f04b4070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.253+0000 7f04b9c00640 1 -- 172.21.15.26:0/163634762 shutdown_connections 2024-06-13T07:19:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.253+0000 7f04b9c00640 1 -- 172.21.15.26:0/163634762 wait complete. 2024-06-13T07:19:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.255+0000 7f04b9c00640 1 Processor -- start 2024-06-13T07:19:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.255+0000 7f04b9c00640 1 -- start start 2024-06-13T07:19:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.255+0000 7f04b9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b41089a0 0x7f04b41a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.255+0000 7f04b9c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b4109300 0x7f04b41a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4110220 0x7f04b41a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b9c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41180c0 con 0x7f04b4110220 2024-06-13T07:19:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b9c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4117f40 con 0x7f04b4109300 2024-06-13T07:19:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b9c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4118240 con 0x7f04b41089a0 2024-06-13T07:19:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b41089a0 0x7f04b41a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b41089a0 0x7f04b41a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45380/0 (socket says 172.21.15.26:45380) 2024-06-13T07:19:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2a00640 1 -- 172.21.15.26:0/516358994 learned_addr learned my addr 172.21.15.26:0/516358994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2000640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b4109300 0x7f04b41a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2a00640 1 -- 172.21.15.26:0/516358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b4109300 msgr2=0x7f04b41a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2a00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b4109300 0x7f04b41a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2a00640 1 -- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4110220 msgr2=0x7f04b41a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2a00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4110220 0x7f04b41a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.256+0000 7f04b2a00640 1 -- 172.21.15.26:0/516358994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b41aa200 con 0x7f04b41089a0 2024-06-13T07:19:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.257+0000 7f04b2a00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b41089a0 0x7f04b41a25e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e9c0 tx=0x7f049c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.257+0000 7f04ab400640 1 -- 172.21.15.26:0/516358994 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c00cde0 con 0x7f04b41089a0 2024-06-13T07:19:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.257+0000 7f04ab400640 1 -- 172.21.15.26:0/516358994 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f049c0035f0 con 0x7f04b41089a0 2024-06-13T07:19:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.257+0000 7f04ab400640 1 -- 172.21.15.26:0/516358994 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c010670 con 0x7f04b41089a0 2024-06-13T07:19:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.257+0000 7f04b9c00640 1 -- 172.21.15.26:0/516358994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b41aa4f0 con 0x7f04b41089a0 2024-06-13T07:19:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.257+0000 7f04b9c00640 1 -- 172.21.15.26:0/516358994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b41aa9f0 con 0x7f04b41089a0 2024-06-13T07:19:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.264+0000 7f04bca5b740 1 Processor -- start 2024-06-13T07:19:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04bca5b740 1 -- start start 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe40870830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870e00 0x55fe40875900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 0x55fe40878400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04bca5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fe407cb420 con 0x55fe40870e00 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04bca5b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fe40772c50 con 0x55fe40875f70 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04bca5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fe407c73e0 con 0x55fe40870410 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe40870830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe40870830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45396/0 (socket says 172.21.15.26:45396) 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.265+0000 7f04b3400640 1 -- 172.21.15.26:0/876069321 learned_addr learned my addr 172.21.15.26:0/876069321 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.266+0000 7f04ab400640 1 -- 172.21.15.26:0/516358994 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f049c0037a0 con 0x7f04b41089a0 2024-06-13T07:19:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.267+0000 7f04b3400640 1 -- 172.21.15.26:0/876069321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 msgr2=0x55fe40878400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:19:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.267+0000 7f04b3400640 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 0x55fe40878400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.267+0000 7f04b2a00640 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870e00 0x55fe40875900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.267+0000 7f04b3400640 1 -- 172.21.15.26:0/876069321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870e00 msgr2=0x55fe40875900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.267+0000 7f04b3400640 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870e00 0x55fe40875900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.267+0000 7f04b3400640 1 -- 172.21.15.26:0/876069321 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe404df210 con 0x55fe40870410 2024-06-13T07:19:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.268+0000 7f04b2a00640 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870e00 0x55fe40875900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:19:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.268+0000 7f04b3400640 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe40870830 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009d80 tx=0x7f04ac00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38290258b80ef5a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.268+0000 7f04a3e00640 1 -- 172.21.15.26:0/876069321 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac016070 con 0x55fe40870410 2024-06-13T07:19:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.269+0000 7f04a3e00640 1 -- 172.21.15.26:0/876069321 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04ac005020 con 0x55fe40870410 2024-06-13T07:19:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.269+0000 7f04a3e00640 1 -- 172.21.15.26:0/876069321 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0116d0 con 0x55fe40870410 2024-06-13T07:19:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.270+0000 7f04bca5b740 1 -- 172.21.15.26:0/876069321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 msgr2=0x55fe40870830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.270+0000 7f04ab400640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04980a2cb0 0x7f04980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.270+0000 7f04ab400640 1 -- 172.21.15.26:0/516358994 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f049c0cf880 con 0x7f04b41089a0 2024-06-13T07:19:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.270+0000 7f04bca5b740 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe40870830 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009d80 tx=0x7f04ac00e6c0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.271+0000 7f04b2a00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04980a2cb0 0x7f04980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.272+0000 7f04b2a00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04980a2cb0 0x7f04980a5170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c8e0 tx=0x7f049c00c850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.272+0000 7f04bca5b740 1 -- 172.21.15.26:0/876069321 shutdown_connections 2024-06-13T07:19:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.272+0000 7f04bca5b740 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 0x55fe40878400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.272+0000 7f04bca5b740 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870e00 0x55fe40875900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.272+0000 7f04bca5b740 1 --2- 172.21.15.26:0/876069321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe40870830 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.272+0000 7f04bca5b740 1 -- 172.21.15.26:0/876069321 >> 172.21.15.26:0/876069321 conn(0x55fe4082ffb0 msgr2=0x55fe4086be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.272+0000 7f04bca5b740 1 -- 172.21.15.26:0/876069321 shutdown_connections 2024-06-13T07:19:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.273+0000 7f04bca5b740 1 -- 172.21.15.26:0/876069321 wait complete. 2024-06-13T07:19:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.274+0000 7f04bca5b740 1 -- start start 2024-06-13T07:19:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.274+0000 7f04bca5b740 1 Processor -- start 2024-06-13T07:19:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870410 0x55fe4087a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe4087ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 0x55fe40881d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04bca5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fe40878f20 con 0x55fe40875f70 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04bca5b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fe407860b0 con 0x55fe40870410 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04bca5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fe408790a0 con 0x55fe40870e00 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04bca5b740 10 client.0 _ll_register_callbacks cb 0x55fe40771f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04b2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe4087ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04b2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe4087ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45402/0 (socket says 172.21.15.26:45402) 2024-06-13T07:19:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04b2000640 1 -- 172.21.15.26:0/1857982274 learned_addr learned my addr 172.21.15.26:0/1857982274 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.275+0000 7f04b3400640 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870410 0x55fe4087a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04b3400640 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 0x55fe40881d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04b2000640 1 -- 172.21.15.26:0/1857982274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870410 msgr2=0x55fe4087a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04b2000640 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870410 0x55fe4087a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04b2000640 1 -- 172.21.15.26:0/1857982274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 msgr2=0x55fe40881d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04b2000640 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 0x55fe40881d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04b2000640 1 -- 172.21.15.26:0/1857982274 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe405ab620 con 0x55fe40870e00 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04b2000640 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe4087ad10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009250 tx=0x7f04a4030860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9ca8b6159e1fa73 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04a0c00640 1 -- 172.21.15.26:0/1857982274 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4037040 con 0x55fe40870e00 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04a0c00640 1 -- 172.21.15.26:0/1857982274 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f04a4004240 con 0x55fe40870e00 2024-06-13T07:19:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.276+0000 7f04bca5b740 1 -- 172.21.15.26:0/1857982274 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ac005020 con 0x55fe40870e00 2024-06-13T07:19:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.277+0000 7f04a0c00640 1 -- 172.21.15.26:0/1857982274 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f04a4049050 con 0x55fe40870e00 2024-06-13T07:19:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.277+0000 7f04bca5b740 10 client.26896 fetch_fsmap learned FSMap version 47 2024-06-13T07:19:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.277+0000 7f04bca5b740 1 -- 172.21.15.26:0/1857982274 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55fe408850a0 con 0x55fe40870e00 2024-06-13T07:19:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.277+0000 7f04a0c00640 1 -- 172.21.15.26:0/1857982274 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04ac005020 con 0x55fe40870e00 2024-06-13T07:19:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.277+0000 7f04a0c00640 1 -- 172.21.15.26:0/1857982274 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f04a403a070 con 0x55fe40870e00 2024-06-13T07:19:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.278+0000 7f04bca5b740 10 client.26896 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:19:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.278+0000 7f04bca5b740 10 client.26896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:19:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.278+0000 7f04bca5b740 20 client.26896 populate_metadata read hostname 'smithi026' 2024-06-13T07:19:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.278+0000 7f04bca5b740 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe40886830 0x55fe40888c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.278+0000 7f04bca5b740 4 client.26896 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:19:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.278+0000 7f04bca5b740 1 -- 172.21.15.26:0/1857982274 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe407d8b50 con 0x55fe40886830 2024-06-13T07:19:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.279+0000 7f04b3400640 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe40886830 0x55fe40888c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.280+0000 7f04b3400640 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe40886830 0x55fe40888c20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11cce851002043d server_cookie=2bf43db0302b9542 in_seq=0 out_seq=0 2024-06-13T07:19:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.280+0000 7f04a0c00640 10 client.26896 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.281+0000 7f04a0c00640 1 -- 172.21.15.26:0/1857982274 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe407d8b50 con 0x55fe40886830 2024-06-13T07:19:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.281+0000 7f04a0c00640 10 client.26896 handle_command_reply: tid=0 2024-06-13T07:19:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.281+0000 7f04bca5b740 1 client.26896 shutdown 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.281+0000 7f04bca5b740 1 -- 172.21.15.26:0/1857982274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 msgr2=0x55fe4087ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.281+0000 7f04bca5b740 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe4087ad10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009250 tx=0x7f04a4030860 comp rx=0 tx=0).stop 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04bca5b740 1 -- 172.21.15.26:0/1857982274 shutdown_connections 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04bca5b740 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe40886830 0x55fe40888c20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04bca5b740 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 0x55fe40881d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04bca5b740 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe4087ad10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04bca5b740 1 --2- 172.21.15.26:0/1857982274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870410 0x55fe4087a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04a0c00640 0 client.26896 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04bca5b740 1 -- 172.21.15.26:0/1857982274 >> 172.21.15.26:0/1857982274 conn(0x55fe4082ffb0 msgr2=0x55fe40862840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04bca5b740 1 -- 172.21.15.26:0/1857982274 shutdown_connections 2024-06-13T07:19:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.282+0000 7f04bca5b740 1 -- 172.21.15.26:0/1857982274 wait complete. 2024-06-13T07:19:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.283+0000 7f04bca5b740 20 client.26896 trim_cache size 0 max 16384 2024-06-13T07:19:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.312+0000 7f04bca5b740 1 Processor -- start 2024-06-13T07:19:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.313+0000 7f04bca5b740 1 -- start start 2024-06-13T07:19:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.313+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870410 0x55fe408757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.313+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe40881d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.313+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 0x55fe40872160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.313+0000 7f04bca5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b3600 con 0x55fe40870410 2024-06-13T07:19:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.313+0000 7f04bca5b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b0ed0 con 0x55fe40875f70 2024-06-13T07:19:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.313+0000 7f04bca5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41aed20 con 0x55fe40870e00 2024-06-13T07:19:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.314+0000 7f04b3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870410 0x55fe408757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.314+0000 7f04b2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe40881d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.314+0000 7f04b2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe40881d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45418/0 (socket says 172.21.15.26:45418) 2024-06-13T07:19:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.314+0000 7f04b2000640 1 -- 172.21.15.26:0/429402233 learned_addr learned my addr 172.21.15.26:0/429402233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.314+0000 7f04b3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870410 0x55fe408757f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45410/0 (socket says 172.21.15.26:45410) 2024-06-13T07:19:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.314+0000 7f04b3400640 1 --2- 172.21.15.26:0/429402233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 0x55fe40872160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.315+0000 7f04b3400640 1 -- 172.21.15.26:0/429402233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 msgr2=0x55fe40881d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.315+0000 7f04b3400640 1 --2- 172.21.15.26:0/429402233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe40881d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.315+0000 7f04b3400640 1 -- 172.21.15.26:0/429402233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 msgr2=0x55fe40872160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.315+0000 7f04b3400640 1 --2- 172.21.15.26:0/429402233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 0x55fe40872160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.315+0000 7f04b3400640 1 -- 172.21.15.26:0/429402233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4086cb40 con 0x55fe40870410 2024-06-13T07:19:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.316+0000 7f04b3400640 1 --2- 172.21.15.26:0/429402233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870410 0x55fe408757f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005bd0 tx=0x7f04ac009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91201c9561eb241f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.316+0000 7f04a3400640 1 -- 172.21.15.26:0/429402233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac016040 con 0x55fe40870410 2024-06-13T07:19:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.316+0000 7f04a3400640 1 -- 172.21.15.26:0/429402233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04ac011470 con 0x55fe40870410 2024-06-13T07:19:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.316+0000 7f04a3400640 1 -- 172.21.15.26:0/429402233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac01a660 con 0x55fe40870410 2024-06-13T07:19:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.317+0000 7f04bca5b740 1 -- 172.21.15.26:0/429402233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870410 msgr2=0x55fe408757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.317+0000 7f04bca5b740 1 --2- 172.21.15.26:0/429402233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870410 0x55fe408757f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005bd0 tx=0x7f04ac009680 comp rx=0 tx=0).stop 2024-06-13T07:19:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.318+0000 7f04bca5b740 1 -- 172.21.15.26:0/429402233 shutdown_connections 2024-06-13T07:19:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.318+0000 7f04bca5b740 1 --2- 172.21.15.26:0/429402233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40875f70 0x55fe40872160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.318+0000 7f04bca5b740 1 --2- 172.21.15.26:0/429402233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870e00 0x55fe40881d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.318+0000 7f04bca5b740 1 --2- 172.21.15.26:0/429402233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40870410 0x55fe408757f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.318+0000 7f04bca5b740 1 -- 172.21.15.26:0/429402233 >> 172.21.15.26:0/429402233 conn(0x55fe4082ffb0 msgr2=0x55fe4087a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.318+0000 7f04bca5b740 1 -- 172.21.15.26:0/429402233 shutdown_connections 2024-06-13T07:19:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.318+0000 7f04bca5b740 1 -- 172.21.15.26:0/429402233 wait complete. 2024-06-13T07:19:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.319+0000 7f04bca5b740 1 -- start start 2024-06-13T07:19:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.319+0000 7f04bca5b740 1 Processor -- start 2024-06-13T07:19:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.320+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe4087dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.320+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870e00 0x55fe4087e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.320+0000 7f04bca5b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 0x55fe40881420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.320+0000 7f04bca5b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41af390 con 0x55fe40875f70 2024-06-13T07:19:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.320+0000 7f04bca5b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b0800 con 0x55fe40870e00 2024-06-13T07:19:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.320+0000 7f04bca5b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41afa60 con 0x55fe40870410 2024-06-13T07:19:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.320+0000 7f04bca5b740 10 client.0 _ll_register_callbacks cb 0x7f04a4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:19:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.321+0000 7f04b2000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870e00 0x55fe4087e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.321+0000 7f04b2000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870e00 0x55fe4087e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:41214/0 (socket says 172.21.15.26:41214) 2024-06-13T07:19:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.321+0000 7f04b2000640 1 -- 172.21.15.26:0/3997812281 learned_addr learned my addr 172.21.15.26:0/3997812281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.322+0000 7f04b3400640 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe4087dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.322+0000 7f04b3400640 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 0x55fe40881420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.324+0000 7f04b3400640 1 -- 172.21.15.26:0/3997812281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870e00 msgr2=0x55fe4087e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.324+0000 7f04b3400640 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870e00 0x55fe4087e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.324+0000 7f04b3400640 1 -- 172.21.15.26:0/3997812281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 msgr2=0x55fe40881420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.324+0000 7f04b3400640 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 0x55fe40881420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.324+0000 7f04b3400640 1 -- 172.21.15.26:0/3997812281 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe40881a10 con 0x55fe40870410 2024-06-13T07:19:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.326+0000 7f04b3400640 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe4087dec0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009cd0 tx=0x7f04ac00efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1d9bc4cd44e98e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.326+0000 7f04a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:19:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.326+0000 7f04a0c00640 1 -- 172.21.15.26:0/3997812281 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac016050 con 0x55fe40870410 2024-06-13T07:19:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.326+0000 7f04a0c00640 1 -- 172.21.15.26:0/3997812281 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f04ac004d20 con 0x55fe40870410 2024-06-13T07:19:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.327+0000 7f04bca5b740 1 -- 172.21.15.26:0/3997812281 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ac011470 con 0x55fe40870410 2024-06-13T07:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.328+0000 7f04a0c00640 1 -- 172.21.15.26:0/3997812281 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f04ac011620 con 0x55fe40870410 2024-06-13T07:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.328+0000 7f04bca5b740 10 client.26902 fetch_fsmap learned FSMap version 47 2024-06-13T07:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.328+0000 7f04bca5b740 1 -- 172.21.15.26:0/3997812281 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55fe40866a00 con 0x55fe40870410 2024-06-13T07:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.328+0000 7f04a0c00640 1 -- 172.21.15.26:0/3997812281 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04ac011470 con 0x55fe40870410 2024-06-13T07:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.328+0000 7f04a0c00640 1 -- 172.21.15.26:0/3997812281 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f04ac02f040 con 0x55fe40870410 2024-06-13T07:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.328+0000 7f04bca5b740 10 client.26902 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.328+0000 7f04bca5b740 10 client.26902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.328+0000 7f04bca5b740 20 client.26902 populate_metadata read hostname 'smithi026' 2024-06-13T07:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.329+0000 7f04bca5b740 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe40867060 0x55fe40869450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.329+0000 7f04bca5b740 4 client.26902 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.329+0000 7f04bca5b740 1 -- 172.21.15.26:0/3997812281 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe407d2ea0 con 0x55fe40867060 2024-06-13T07:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.329+0000 7f04b2000640 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe40867060 0x55fe40869450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.330+0000 7f04b2000640 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe40867060 0x55fe40869450 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e69b226c0b99040 server_cookie=b63877d391a6a5bd in_seq=0 out_seq=0 2024-06-13T07:19:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.330+0000 7f04a0c00640 10 client.26902 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:14.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:14 smithi136 ceph-mon[37229]: pgmap v1071: 129 pgs: 129 active+clean; 648 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.5 MiB/s wr, 611 op/s 2024-06-13T07:19:14.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:14 smithi026 ceph-mon[32041]: pgmap v1071: 129 pgs: 129 active+clean; 648 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.5 MiB/s wr, 611 op/s 2024-06-13T07:19:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:14 smithi135 ceph-mon[36276]: pgmap v1071: 129 pgs: 129 active+clean; 648 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.5 MiB/s wr, 611 op/s 2024-06-13T07:19:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.922+0000 7f04a0c00640 1 -- 172.21.15.26:0/3997812281 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fe407d2ea0 con 0x55fe40867060 2024-06-13T07:19:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04a0c00640 10 client.26902 handle_command_reply: tid=0 2024-06-13T07:19:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 client.26902 shutdown 2024-06-13T07:19:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 -- 172.21.15.26:0/3997812281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 msgr2=0x55fe4087dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe4087dec0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009cd0 tx=0x7f04ac00efd0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 -- 172.21.15.26:0/3997812281 shutdown_connections 2024-06-13T07:19:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55fe40867060 0x55fe40869450 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe40870e00 0x55fe4087e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe40875f70 0x55fe40881420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 --2- 172.21.15.26:0/3997812281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe40870410 0x55fe4087dec0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.923+0000 7f04bca5b740 1 -- 172.21.15.26:0/3997812281 >> 172.21.15.26:0/3997812281 conn(0x55fe4082ffb0 msgr2=0x55fe4086ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.924+0000 7f04a0c00640 0 client.26902 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.924+0000 7f04bca5b740 1 -- 172.21.15.26:0/3997812281 shutdown_connections 2024-06-13T07:19:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.924+0000 7f04bca5b740 1 -- 172.21.15.26:0/3997812281 wait complete. 2024-06-13T07:19:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.924+0000 7f04bca5b740 20 client.26902 trim_cache size 0 max 16384 2024-06-13T07:19:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:19:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:19:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:19:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.925+0000 7f04b9c00640 1 -- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04980a2cb0 msgr2=0x7f04980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.925+0000 7f04b9c00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04980a2cb0 0x7f04980a5170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c8e0 tx=0x7f049c00c850 comp rx=0 tx=0).stop 2024-06-13T07:19:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.925+0000 7f04b9c00640 1 -- 172.21.15.26:0/516358994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b41089a0 msgr2=0x7f04b41a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.925+0000 7f04b9c00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b41089a0 0x7f04b41a25e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e9c0 tx=0x7f049c00ee80 comp rx=0 tx=0).stop 2024-06-13T07:19:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.928+0000 7f04b9c00640 1 -- 172.21.15.26:0/516358994 shutdown_connections 2024-06-13T07:19:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.928+0000 7f04b9c00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f04980a2cb0 0x7f04980a5170 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.928+0000 7f04b9c00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b4109300 0x7f04b41a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.928+0000 7f04b9c00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b4110220 0x7f04b41a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.928+0000 7f04b9c00640 1 --2- 172.21.15.26:0/516358994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b41089a0 0x7f04b41a25e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.928+0000 7f04b9c00640 1 -- 172.21.15.26:0/516358994 >> 172.21.15.26:0/516358994 conn(0x7f04b406ea10 msgr2=0x7f04b410bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.928+0000 7f04b9c00640 1 -- 172.21.15.26:0/516358994 shutdown_connections 2024-06-13T07:19:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:14.928+0000 7f04b9c00640 1 -- 172.21.15.26:0/516358994 wait complete. 2024-06-13T07:19:15.183 INFO:tasks.workunit.client.0.smithi026.stdout: 10 171308 27.77 MB/sec execute 208 sec latency 303.074 ms 2024-06-13T07:19:16.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 171965 27.84 MB/sec execute 209 sec latency 226.672 ms 2024-06-13T07:19:16.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:16 smithi136 ceph-mon[37229]: pgmap v1072: 129 pgs: 129 active+clean; 653 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-06-13T07:19:16.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:16 smithi026 ceph-mon[32041]: pgmap v1072: 129 pgs: 129 active+clean; 653 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-06-13T07:19:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:16 smithi135 ceph-mon[36276]: pgmap v1072: 129 pgs: 129 active+clean; 653 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-06-13T07:19:17.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 172743 27.93 MB/sec execute 210 sec latency 74.392 ms 2024-06-13T07:19:18.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 173318 27.99 MB/sec execute 211 sec latency 87.594 ms 2024-06-13T07:19:18.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:18 smithi026 ceph-mon[32041]: pgmap v1073: 129 pgs: 129 active+clean; 658 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-06-13T07:19:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:18 smithi135 ceph-mon[36276]: pgmap v1073: 129 pgs: 129 active+clean; 658 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-06-13T07:19:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:18 smithi136 ceph-mon[37229]: pgmap v1073: 129 pgs: 129 active+clean; 658 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-06-13T07:19:19.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 173704 27.96 MB/sec execute 212 sec latency 99.803 ms 2024-06-13T07:19:20.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 173913 27.88 MB/sec execute 213 sec latency 427.222 ms 2024-06-13T07:19:20.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:20 smithi026 ceph-mon[32041]: pgmap v1074: 129 pgs: 129 active+clean; 660 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-06-13T07:19:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:20 smithi135 ceph-mon[36276]: pgmap v1074: 129 pgs: 129 active+clean; 660 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-06-13T07:19:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:20 smithi136 ceph-mon[37229]: pgmap v1074: 129 pgs: 129 active+clean; 660 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-06-13T07:19:21.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 173963 27.75 MB/sec execute 214 sec latency 1273.944 ms 2024-06-13T07:19:22.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 174243 27.65 MB/sec execute 215 sec latency 135.919 ms 2024-06-13T07:19:22.427 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:22 smithi136 ceph-mon[37229]: pgmap v1075: 129 pgs: 129 active+clean; 670 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 479 op/s 2024-06-13T07:19:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:22 smithi026 ceph-mon[32041]: pgmap v1075: 129 pgs: 129 active+clean; 670 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 479 op/s 2024-06-13T07:19:22.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:22 smithi135 ceph-mon[36276]: pgmap v1075: 129 pgs: 129 active+clean; 670 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 479 op/s 2024-06-13T07:19:23.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 174503 27.57 MB/sec execute 216 sec latency 384.240 ms 2024-06-13T07:19:24.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 175278 27.65 MB/sec execute 217 sec latency 151.129 ms 2024-06-13T07:19:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:24 smithi026 ceph-mon[32041]: pgmap v1076: 129 pgs: 129 active+clean; 673 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.6 MiB/s wr, 524 op/s 2024-06-13T07:19:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:24 smithi135 ceph-mon[36276]: pgmap v1076: 129 pgs: 129 active+clean; 673 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.6 MiB/s wr, 524 op/s 2024-06-13T07:19:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:24 smithi136 ceph-mon[37229]: pgmap v1076: 129 pgs: 129 active+clean; 673 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.6 MiB/s wr, 524 op/s 2024-06-13T07:19:25.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 176012 27.71 MB/sec execute 218 sec latency 49.299 ms 2024-06-13T07:19:26.184 INFO:tasks.workunit.client.0.smithi026.stdout: 10 176175 27.65 MB/sec execute 219 sec latency 854.757 ms 2024-06-13T07:19:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:26 smithi026 ceph-mon[32041]: pgmap v1077: 129 pgs: 129 active+clean; 674 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.8 MiB/s wr, 560 op/s 2024-06-13T07:19:26.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:26 smithi135 ceph-mon[36276]: pgmap v1077: 129 pgs: 129 active+clean; 674 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.8 MiB/s wr, 560 op/s 2024-06-13T07:19:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:26 smithi136 ceph-mon[37229]: pgmap v1077: 129 pgs: 129 active+clean; 674 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.8 MiB/s wr, 560 op/s 2024-06-13T07:19:27.185 INFO:tasks.workunit.client.0.smithi026.stdout: 10 176501 27.61 MB/sec execute 220 sec latency 892.586 ms 2024-06-13T07:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.174+0000 7f15ebe00640 1 -- 172.21.15.26:0/127216010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e410dcc0 msgr2=0x7f15e410e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.174+0000 7f15ebe00640 1 --2- 172.21.15.26:0/127216010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e410dcc0 0x7f15e410e140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009900 tx=0x7f15d002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.175+0000 7f15ebe00640 1 -- 172.21.15.26:0/127216010 shutdown_connections 2024-06-13T07:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.175+0000 7f15ebe00640 1 --2- 172.21.15.26:0/127216010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e410b310 0x7f15e410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.175+0000 7f15ebe00640 1 --2- 172.21.15.26:0/127216010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e410dcc0 0x7f15e410e140 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.175+0000 7f15ebe00640 1 --2- 172.21.15.26:0/127216010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e41089a0 0x7f15e410add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.175+0000 7f15ebe00640 1 -- 172.21.15.26:0/127216010 >> 172.21.15.26:0/127216010 conn(0x7f15e406ea10 msgr2=0x7f15e4070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.179+0000 7f15ebe00640 1 -- 172.21.15.26:0/127216010 shutdown_connections 2024-06-13T07:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.179+0000 7f15ebe00640 1 -- 172.21.15.26:0/127216010 wait complete. 2024-06-13T07:19:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.180+0000 7f15ebe00640 1 Processor -- start 2024-06-13T07:19:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.181+0000 7f15ebe00640 1 -- start start 2024-06-13T07:19:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.181+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e41089a0 0x7f15e41a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.181+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e410b310 0x7f15e41a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.181+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e410dcc0 0x7f15e41adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.182+0000 7f15e9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e41089a0 0x7f15e41a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.182+0000 7f15ebe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15e411c490 con 0x7f15e410b310 2024-06-13T07:19:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.182+0000 7f15e8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e410b310 0x7f15e41a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.182+0000 7f15ebe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15e411c310 con 0x7f15e410dcc0 2024-06-13T07:19:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.182+0000 7f15ebe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15e411c610 con 0x7f15e41089a0 2024-06-13T07:19:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.182+0000 7f15e9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e41089a0 0x7f15e41a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51418/0 (socket says 172.21.15.26:51418) 2024-06-13T07:19:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.182+0000 7f15e9400640 1 -- 172.21.15.26:0/1006764493 learned_addr learned my addr 172.21.15.26:0/1006764493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.182+0000 7f15e9e00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e410dcc0 0x7f15e41adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.183+0000 7f15e9400640 1 -- 172.21.15.26:0/1006764493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e410dcc0 msgr2=0x7f15e41adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.183+0000 7f15e9400640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e410dcc0 0x7f15e41adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.183+0000 7f15e9400640 1 -- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e410b310 msgr2=0x7f15e41a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.183+0000 7f15e9400640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e410b310 0x7f15e41a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.183+0000 7f15e9400640 1 -- 172.21.15.26:0/1006764493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e41ae590 con 0x7f15e41089a0 2024-06-13T07:19:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.183+0000 7f15e9400640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e41089a0 0x7f15e41a69c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00eab0 tx=0x7f15dc00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.184+0000 7f15e2000640 1 -- 172.21.15.26:0/1006764493 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00ce40 con 0x7f15e41089a0 2024-06-13T07:19:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.184+0000 7f15ebe00640 1 -- 172.21.15.26:0/1006764493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e41ae880 con 0x7f15e41089a0 2024-06-13T07:19:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.186+0000 7f15e2000640 1 -- 172.21.15.26:0/1006764493 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc011950 con 0x7f15e41089a0 2024-06-13T07:19:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.186+0000 7f15e2000640 1 -- 172.21.15.26:0/1006764493 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc01ab70 con 0x7f15e41089a0 2024-06-13T07:19:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.186+0000 7f15ebe00640 1 -- 172.21.15.26:0/1006764493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41aedd0 con 0x7f15e41089a0 2024-06-13T07:19:28.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 176700 27.52 MB/sec execute 221 sec latency 157.803 ms 2024-06-13T07:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.196+0000 7f15eec1b740 1 Processor -- start 2024-06-13T07:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.196+0000 7f15eec1b740 1 -- start start 2024-06-13T07:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.197+0000 7f15e2000640 1 -- 172.21.15.26:0/1006764493 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15dc01ad10 con 0x7f15e41089a0 2024-06-13T07:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.197+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x5643075ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075ff5f0 0x5643076040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564307604760 0x564307606bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15eec1b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564307562420 con 0x5643075fec00 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15eec1b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564307509c50 con 0x5643075ff5f0 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15eec1b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56430755e3e0 con 0x564307604760 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15e9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x5643075ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15e8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564307604760 0x564307606bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15e9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x5643075ff020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53702/0 (socket says 172.21.15.26:53702) 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.198+0000 7f15e9e00640 1 -- 172.21.15.26:0/2786900651 learned_addr learned my addr 172.21.15.26:0/2786900651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.199+0000 7f15e8a00640 1 -- 172.21.15.26:0/2786900651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075ff5f0 msgr2=0x5643076040f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:19:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.199+0000 7f15e8a00640 1 --2- 172.21.15.26:0/2786900651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075ff5f0 0x5643076040f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.199+0000 7f15e8a00640 1 -- 172.21.15.26:0/2786900651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 msgr2=0x5643075ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.199+0000 7f15e8a00640 1 --2- 172.21.15.26:0/2786900651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x5643075ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.199+0000 7f15e8a00640 1 -- 172.21.15.26:0/2786900651 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564307276210 con 0x564307604760 2024-06-13T07:19:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.199+0000 7f15e8a00640 1 --2- 172.21.15.26:0/2786900651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564307604760 0x564307606bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f15d80058c0 tx=0x7f15d800be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf89a20ecbc20f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.200+0000 7f15e2000640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.201+0000 7f15e2000640 1 -- 172.21.15.26:0/1006764493 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f15dc0cf9b0 con 0x7f15e41089a0 2024-06-13T07:19:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.201+0000 7f15e9e00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.203+0000 7f15d6600640 1 -- 172.21.15.26:0/2786900651 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d800ddf0 con 0x564307604760 2024-06-13T07:19:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.203+0000 7f15d6600640 1 -- 172.21.15.26:0/2786900651 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d8017900 con 0x564307604760 2024-06-13T07:19:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.203+0000 7f15d6600640 1 -- 172.21.15.26:0/2786900651 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d800fa90 con 0x564307604760 2024-06-13T07:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.204+0000 7f15eec1b740 1 -- 172.21.15.26:0/2786900651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564307604760 msgr2=0x564307606bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.204+0000 7f15eec1b740 1 --2- 172.21.15.26:0/2786900651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564307604760 0x564307606bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f15d80058c0 tx=0x7f15d800be70 comp rx=0 tx=0).stop 2024-06-13T07:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.204+0000 7f15e9e00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f15d00096c0 tx=0x7f15d00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.207+0000 7f15eec1b740 1 -- 172.21.15.26:0/2786900651 shutdown_connections 2024-06-13T07:19:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.207+0000 7f15eec1b740 1 --2- 172.21.15.26:0/2786900651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075ff5f0 0x5643076040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.207+0000 7f15eec1b740 1 --2- 172.21.15.26:0/2786900651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564307604760 0x564307606bf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.207+0000 7f15eec1b740 1 --2- 172.21.15.26:0/2786900651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x5643075ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.207+0000 7f15eec1b740 1 -- 172.21.15.26:0/2786900651 >> 172.21.15.26:0/2786900651 conn(0x5643075c6fb0 msgr2=0x5643075fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.207+0000 7f15eec1b740 1 -- 172.21.15.26:0/2786900651 shutdown_connections 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.208+0000 7f15eec1b740 1 -- 172.21.15.26:0/2786900651 wait complete. 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.209+0000 7f15eec1b740 1 -- start start 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.209+0000 7f15eec1b740 1 Processor -- start 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.210+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307608f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.210+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.210+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x564307610510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.211+0000 7f15eec1b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564307607710 con 0x5643075fec00 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.211+0000 7f15eec1b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56430751d0b0 con 0x564307604760 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.211+0000 7f15eec1b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564307607890 con 0x5643075ff5f0 2024-06-13T07:19:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.211+0000 7f15eec1b740 10 client.0 _ll_register_callbacks cb 0x564307508f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.211+0000 7f15e9e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x564307610510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.211+0000 7f15e9e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x564307610510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:47360/0 (socket says 172.21.15.26:47360) 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.211+0000 7f15e9e00640 1 -- 172.21.15.26:0/1698167618 learned_addr learned my addr 172.21.15.26:0/1698167618 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.212+0000 7f15e8a00640 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.212+0000 7f15e9e00640 1 -- 172.21.15.26:0/1698167618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 msgr2=0x5643076094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.212+0000 7f15e9e00640 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076094d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.212+0000 7f15e9e00640 1 -- 172.21.15.26:0/1698167618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 msgr2=0x564307608f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.212+0000 7f15e9e00640 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307608f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.212+0000 7f15e9e00640 1 -- 172.21.15.26:0/1698167618 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564307342620 con 0x564307604760 2024-06-13T07:19:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.212+0000 7f15e8a00640 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.213+0000 7f15e9e00640 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x564307610510 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f15d00314e0 tx=0x7f15d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecea1b95c7cd3edd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.214+0000 7f15caa00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.214+0000 7f15caa00640 1 -- 172.21.15.26:0/1698167618 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d004d070 con 0x564307604760 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.214+0000 7f15eec1b740 1 -- 172.21.15.26:0/1698167618 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d8017900 con 0x564307604760 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.214+0000 7f15caa00640 1 -- 172.21.15.26:0/1698167618 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f15d00328d0 con 0x564307604760 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.214+0000 7f15caa00640 1 -- 172.21.15.26:0/1698167618 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f15d0059050 con 0x564307604760 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.215+0000 7f15eec1b740 10 client.26864 fetch_fsmap learned FSMap version 47 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.215+0000 7f15eec1b740 1 -- 172.21.15.26:0/1698167618 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564307613940 con 0x564307604760 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.215+0000 7f15caa00640 1 -- 172.21.15.26:0/1698167618 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d8017900 con 0x564307604760 2024-06-13T07:19:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.216+0000 7f15caa00640 1 -- 172.21.15.26:0/1698167618 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f15d0003d20 con 0x564307604760 2024-06-13T07:19:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.219+0000 7f15eec1b740 10 client.26864 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:19:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.220+0000 7f15eec1b740 10 client.26864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:19:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.220+0000 7f15eec1b740 20 client.26864 populate_metadata read hostname 'smithi026' 2024-06-13T07:19:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.220+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564307614fd0 0x564307617440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.220+0000 7f15eec1b740 4 client.26864 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:19:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.220+0000 7f15eec1b740 1 -- 172.21.15.26:0/1698167618 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56430756fb50 con 0x564307614fd0 2024-06-13T07:19:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.221+0000 7f15e9400640 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564307614fd0 0x564307617440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.222+0000 7f15caa00640 10 client.26864 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.222+0000 7f15e9400640 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564307614fd0 0x564307617440 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fadc2e732f6e5b58 server_cookie=6b1eb4572d2a5c45 in_seq=0 out_seq=0 2024-06-13T07:19:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.226+0000 7f15caa00640 1 -- 172.21.15.26:0/1698167618 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56430756fb50 con 0x564307614fd0 2024-06-13T07:19:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.226+0000 7f15caa00640 10 client.26864 handle_command_reply: tid=0 2024-06-13T07:19:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.226+0000 7f15eec1b740 1 client.26864 shutdown 2024-06-13T07:19:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15eec1b740 1 -- 172.21.15.26:0/1698167618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 msgr2=0x564307610510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x564307610510 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f15d00314e0 tx=0x7f15d0009260 comp rx=0 tx=0).stop 2024-06-13T07:19:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15eec1b740 1 -- 172.21.15.26:0/1698167618 shutdown_connections 2024-06-13T07:19:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564307614fd0 0x564307617440 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x564307610510 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1698167618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307608f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15eec1b740 1 -- 172.21.15.26:0/1698167618 >> 172.21.15.26:0/1698167618 conn(0x5643075c6fb0 msgr2=0x5643075f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.230+0000 7f15caa00640 0 client.26864 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.231+0000 7f15eec1b740 1 -- 172.21.15.26:0/1698167618 shutdown_connections 2024-06-13T07:19:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.232+0000 7f15eec1b740 1 -- 172.21.15.26:0/1698167618 wait complete. 2024-06-13T07:19:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.232+0000 7f15eec1b740 20 client.26864 trim_cache size 0 max 16384 2024-06-13T07:19:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.286+0000 7f15eec1b740 1 Processor -- start 2024-06-13T07:19:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.286+0000 7f15eec1b740 1 -- start start 2024-06-13T07:19:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.287+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075fec00 0x5643076085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.287+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.288+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564307604760 0x564307608f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.288+0000 7f15eec1b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41bbda0 con 0x564307604760 2024-06-13T07:19:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.288+0000 7f15eec1b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b9670 con 0x5643075fec00 2024-06-13T07:19:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.288+0000 7f15eec1b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b74c0 con 0x5643075ff5f0 2024-06-13T07:19:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.288+0000 7f15e8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.288+0000 7f15e8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51444/0 (socket says 172.21.15.26:51444) 2024-06-13T07:19:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.288+0000 7f15e8a00640 1 -- 172.21.15.26:0/1302298884 learned_addr learned my addr 172.21.15.26:0/1302298884 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.289+0000 7f15e9400640 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075fec00 0x5643076085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.289+0000 7f15e8a00640 1 -- 172.21.15.26:0/1302298884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075fec00 msgr2=0x5643076085b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.289+0000 7f15e8a00640 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075fec00 0x5643076085b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.289+0000 7f15e8a00640 1 -- 172.21.15.26:0/1302298884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564307604760 msgr2=0x564307608f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.289+0000 7f15e8a00640 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564307604760 0x564307608f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.289+0000 7f15e8a00640 1 -- 172.21.15.26:0/1302298884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643075fb330 con 0x5643075ff5f0 2024-06-13T07:19:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.289+0000 7f15e9400640 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075fec00 0x5643076085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:19:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.289+0000 7f15e8a00640 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076104d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f15d800b930 tx=0x7f15d800b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4857d57b5ba2f75b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.290+0000 7f15d5c00640 1 -- 172.21.15.26:0/1302298884 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8026070 con 0x5643075ff5f0 2024-06-13T07:19:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.290+0000 7f15d5c00640 1 -- 172.21.15.26:0/1302298884 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d800ddf0 con 0x5643075ff5f0 2024-06-13T07:19:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.291+0000 7f15d5c00640 1 -- 172.21.15.26:0/1302298884 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8004e40 con 0x5643075ff5f0 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.291+0000 7f15eec1b740 1 -- 172.21.15.26:0/1302298884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 msgr2=0x5643076104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.292+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076104d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f15d800b930 tx=0x7f15d800b6c0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.292+0000 7f15eec1b740 1 -- 172.21.15.26:0/1302298884 shutdown_connections 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.292+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564307604760 0x564307608f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.292+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x5643076104d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.292+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1302298884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643075fec00 0x5643076085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.292+0000 7f15eec1b740 1 -- 172.21.15.26:0/1302298884 >> 172.21.15.26:0/1302298884 conn(0x5643075c6fb0 msgr2=0x5643075fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.292+0000 7f15eec1b740 1 -- 172.21.15.26:0/1302298884 shutdown_connections 2024-06-13T07:19:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.292+0000 7f15eec1b740 1 -- 172.21.15.26:0/1302298884 wait complete. 2024-06-13T07:19:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.293+0000 7f15eec1b740 1 -- start start 2024-06-13T07:19:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.296+0000 7f15eec1b740 1 Processor -- start 2024-06-13T07:19:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.297+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307610460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x56430760cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15e9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307610460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15eec1b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x56430760d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15eec1b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b7b30 con 0x5643075fec00 2024-06-13T07:19:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15eec1b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b8fa0 con 0x564307604760 2024-06-13T07:19:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15eec1b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b8200 con 0x5643075ff5f0 2024-06-13T07:19:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15eec1b740 10 client.0 _ll_register_callbacks cb 0x7f15d0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:19:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15e9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307610460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53722/0 (socket says 172.21.15.26:53722) 2024-06-13T07:19:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.298+0000 7f15e9400640 1 -- 172.21.15.26:0/1228867965 learned_addr learned my addr 172.21.15.26:0/1228867965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.299+0000 7f15e8a00640 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x56430760cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.300+0000 7f15e9400640 1 -- 172.21.15.26:0/1228867965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 msgr2=0x56430760cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.300+0000 7f15e9400640 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x56430760cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.300+0000 7f15e9400640 1 -- 172.21.15.26:0/1228867965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 msgr2=0x56430760d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.300+0000 7f15e9400640 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x56430760d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.300+0000 7f15e9400640 1 -- 172.21.15.26:0/1228867965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56430767ddc0 con 0x5643075fec00 2024-06-13T07:19:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.302+0000 7f15e9400640 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307610460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00c700 tx=0x7f15dc09c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455af1c996054e7c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.302+0000 7f15caa00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:19:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.302+0000 7f15caa00640 1 -- 172.21.15.26:0/1228867965 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0a4a90 con 0x5643075fec00 2024-06-13T07:19:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.303+0000 7f15caa00640 1 -- 172.21.15.26:0/1228867965 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f15dc011b00 con 0x5643075fec00 2024-06-13T07:19:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.303+0000 7f15eec1b740 1 -- 172.21.15.26:0/1228867965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d800ddf0 con 0x5643075fec00 2024-06-13T07:19:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.303+0000 7f15caa00640 1 -- 172.21.15.26:0/1228867965 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f15dc098490 con 0x5643075fec00 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.304+0000 7f15eec1b740 10 client.16659 fetch_fsmap learned FSMap version 47 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.304+0000 7f15eec1b740 1 -- 172.21.15.26:0/1228867965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56430767f0b0 con 0x5643075fec00 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.305+0000 7f15caa00640 1 -- 172.21.15.26:0/1228867965 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d800ddf0 con 0x5643075fec00 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.305+0000 7f15caa00640 1 -- 172.21.15.26:0/1228867965 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f15dc022030 con 0x5643075fec00 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.305+0000 7f15eec1b740 10 client.16659 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.305+0000 7f15eec1b740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.305+0000 7f15eec1b740 20 client.16659 populate_metadata read hostname 'smithi026' 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.305+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56430767f910 0x564307681d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.305+0000 7f15eec1b740 4 client.16659 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:19:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.305+0000 7f15eec1b740 1 -- 172.21.15.26:0/1228867965 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564307569ea0 con 0x56430767f910 2024-06-13T07:19:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.306+0000 7f15e8a00640 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56430767f910 0x564307681d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.307+0000 7f15caa00640 10 client.16659 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:28.308+0000 7f15e8a00640 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56430767f910 0x564307681d30 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8594ad3bb1a1da43 server_cookie=cdbb67af5e3e2ac0 in_seq=0 out_seq=0 2024-06-13T07:19:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:28 smithi026 ceph-mon[32041]: pgmap v1078: 129 pgs: 129 active+clean; 672 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 551 op/s 2024-06-13T07:19:28.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:28 smithi135 ceph-mon[36276]: pgmap v1078: 129 pgs: 129 active+clean; 672 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 551 op/s 2024-06-13T07:19:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:28 smithi136 ceph-mon[37229]: pgmap v1078: 129 pgs: 129 active+clean; 672 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 551 op/s 2024-06-13T07:19:29.187 INFO:tasks.workunit.client.0.smithi026.stdout: 10 176756 27.41 MB/sec execute 222 sec latency 990.140 ms 2024-06-13T07:19:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.293+0000 7f15caa00640 1 -- 172.21.15.26:0/1228867965 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564307569ea0 con 0x56430767f910 2024-06-13T07:19:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.293+0000 7f15caa00640 10 client.16659 handle_command_reply: tid=0 2024-06-13T07:19:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.293+0000 7f15eec1b740 1 client.16659 shutdown 2024-06-13T07:19:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.293+0000 7f15eec1b740 1 -- 172.21.15.26:0/1228867965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 msgr2=0x564307610460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.293+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307610460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00c700 tx=0x7f15dc09c670 comp rx=0 tx=0).stop 2024-06-13T07:19:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.294+0000 7f15eec1b740 1 -- 172.21.15.26:0/1228867965 shutdown_connections 2024-06-13T07:19:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.294+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56430767f910 0x564307681d30 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.294+0000 7f15caa00640 0 client.16659 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.294+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564307604760 0x56430760d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.294+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643075ff5f0 0x56430760cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.294+0000 7f15eec1b740 1 --2- 172.21.15.26:0/1228867965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5643075fec00 0x564307610460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.294+0000 7f15eec1b740 1 -- 172.21.15.26:0/1228867965 >> 172.21.15.26:0/1228867965 conn(0x5643075c6fb0 msgr2=0x5643075fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.294+0000 7f15eec1b740 1 -- 172.21.15.26:0/1228867965 shutdown_connections 2024-06-13T07:19:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.295+0000 7f15eec1b740 1 -- 172.21.15.26:0/1228867965 wait complete. 2024-06-13T07:19:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.295+0000 7f15eec1b740 20 client.16659 trim_cache size 0 max 16384 2024-06-13T07:19:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:19:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:19:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:19:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.296+0000 7f15ebe00640 1 -- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15cc0a2d00 msgr2=0x7f15cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.296+0000 7f15ebe00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f15d00096c0 tx=0x7f15d00315b0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.296+0000 7f15ebe00640 1 -- 172.21.15.26:0/1006764493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e41089a0 msgr2=0x7f15e41a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.296+0000 7f15ebe00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e41089a0 0x7f15e41a69c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00eab0 tx=0x7f15dc00ef70 comp rx=0 tx=0).stop 2024-06-13T07:19:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.297+0000 7f15ebe00640 1 -- 172.21.15.26:0/1006764493 shutdown_connections 2024-06-13T07:19:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.297+0000 7f15ebe00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.297+0000 7f15ebe00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e410dcc0 0x7f15e41adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.297+0000 7f15ebe00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e410b310 0x7f15e41a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.297+0000 7f15ebe00640 1 --2- 172.21.15.26:0/1006764493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15e41089a0 0x7f15e41a69c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.297+0000 7f15ebe00640 1 -- 172.21.15.26:0/1006764493 >> 172.21.15.26:0/1006764493 conn(0x7f15e406ea10 msgr2=0x7f15e410aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.298+0000 7f15ebe00640 1 -- 172.21.15.26:0/1006764493 shutdown_connections 2024-06-13T07:19:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:29.298+0000 7f15ebe00640 1 -- 172.21.15.26:0/1006764493 wait complete. 2024-06-13T07:19:30.187 INFO:tasks.workunit.client.0.smithi026.stdout: 10 176994 27.36 MB/sec execute 223 sec latency 573.543 ms 2024-06-13T07:19:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:30 smithi026 ceph-mon[32041]: pgmap v1079: 129 pgs: 129 active+clean; 684 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.9 MiB/s wr, 611 op/s 2024-06-13T07:19:30.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:30 smithi135 ceph-mon[36276]: pgmap v1079: 129 pgs: 129 active+clean; 684 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.9 MiB/s wr, 611 op/s 2024-06-13T07:19:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:30 smithi136 ceph-mon[37229]: pgmap v1079: 129 pgs: 129 active+clean; 684 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.9 MiB/s wr, 611 op/s 2024-06-13T07:19:31.187 INFO:tasks.workunit.client.0.smithi026.stdout: 10 177485 27.36 MB/sec execute 224 sec latency 94.727 ms 2024-06-13T07:19:32.188 INFO:tasks.workunit.client.0.smithi026.stdout: 10 178037 27.33 MB/sec execute 225 sec latency 93.037 ms 2024-06-13T07:19:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:32 smithi026 ceph-mon[32041]: pgmap v1080: 129 pgs: 129 active+clean; 680 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-06-13T07:19:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:32 smithi135 ceph-mon[36276]: pgmap v1080: 129 pgs: 129 active+clean; 680 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-06-13T07:19:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:32 smithi136 ceph-mon[37229]: pgmap v1080: 129 pgs: 129 active+clean; 680 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-06-13T07:19:33.187 INFO:tasks.workunit.client.0.smithi026.stdout: 10 178756 27.38 MB/sec execute 226 sec latency 62.511 ms 2024-06-13T07:19:34.188 INFO:tasks.workunit.client.0.smithi026.stdout: 10 179413 27.44 MB/sec execute 227 sec latency 68.675 ms 2024-06-13T07:19:34.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:34 smithi026 ceph-mon[32041]: pgmap v1081: 129 pgs: 129 active+clean; 676 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 680 op/s 2024-06-13T07:19:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:34 smithi135 ceph-mon[36276]: pgmap v1081: 129 pgs: 129 active+clean; 676 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 680 op/s 2024-06-13T07:19:34.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:34 smithi136 ceph-mon[37229]: pgmap v1081: 129 pgs: 129 active+clean; 676 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 680 op/s 2024-06-13T07:19:35.188 INFO:tasks.workunit.client.0.smithi026.stdout: 10 179887 27.46 MB/sec execute 228 sec latency 116.146 ms 2024-06-13T07:19:36.188 INFO:tasks.workunit.client.0.smithi026.stdout: 10 180163 27.40 MB/sec execute 229 sec latency 418.479 ms 2024-06-13T07:19:36.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:36 smithi026 ceph-mon[32041]: pgmap v1082: 129 pgs: 129 active+clean; 658 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 660 op/s 2024-06-13T07:19:36.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:36 smithi135 ceph-mon[36276]: pgmap v1082: 129 pgs: 129 active+clean; 658 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 660 op/s 2024-06-13T07:19:36.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:36 smithi136 ceph-mon[37229]: pgmap v1082: 129 pgs: 129 active+clean; 658 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 660 op/s 2024-06-13T07:19:37.188 INFO:tasks.workunit.client.0.smithi026.stdout: 10 180584 27.35 MB/sec execute 230 sec latency 106.078 ms 2024-06-13T07:19:38.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 181246 27.35 MB/sec execute 231 sec latency 69.297 ms 2024-06-13T07:19:38.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:38 smithi026 ceph-mon[32041]: pgmap v1083: 129 pgs: 129 active+clean; 652 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.7 MiB/s wr, 579 op/s 2024-06-13T07:19:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:38 smithi135 ceph-mon[36276]: pgmap v1083: 129 pgs: 129 active+clean; 652 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.7 MiB/s wr, 579 op/s 2024-06-13T07:19:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:38 smithi136 ceph-mon[37229]: pgmap v1083: 129 pgs: 129 active+clean; 652 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.7 MiB/s wr, 579 op/s 2024-06-13T07:19:39.188 INFO:tasks.workunit.client.0.smithi026.stdout: 10 181980 27.44 MB/sec execute 232 sec latency 78.127 ms 2024-06-13T07:19:40.188 INFO:tasks.workunit.client.0.smithi026.stdout: 10 182681 27.50 MB/sec execute 233 sec latency 102.859 ms 2024-06-13T07:19:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:40 smithi026 ceph-mon[32041]: pgmap v1084: 129 pgs: 129 active+clean; 659 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-06-13T07:19:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:40 smithi135 ceph-mon[36276]: pgmap v1084: 129 pgs: 129 active+clean; 659 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-06-13T07:19:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:40 smithi136 ceph-mon[37229]: pgmap v1084: 129 pgs: 129 active+clean; 659 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-06-13T07:19:41.189 INFO:tasks.workunit.client.0.smithi026.stdout: 10 183231 27.50 MB/sec execute 234 sec latency 65.276 ms 2024-06-13T07:19:42.189 INFO:tasks.workunit.client.0.smithi026.stdout: 10 183589 27.47 MB/sec execute 235 sec latency 118.730 ms 2024-06-13T07:19:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.490+0000 7fad71e00640 1 -- 172.21.15.26:0/380126191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c0759e0 msgr2=0x7fad6c073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.490+0000 7fad71e00640 1 --2- 172.21.15.26:0/380126191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c0759e0 0x7fad6c073e30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fad6401ace0 tx=0x7fad6403b960 comp rx=0 tx=0).stop 2024-06-13T07:19:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.491+0000 7fad71e00640 1 -- 172.21.15.26:0/380126191 shutdown_connections 2024-06-13T07:19:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.491+0000 7fad71e00640 1 --2- 172.21.15.26:0/380126191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad6c10de00 0x7fad6c1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.491+0000 7fad71e00640 1 --2- 172.21.15.26:0/380126191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6c074400 0x7fad6c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.491+0000 7fad71e00640 1 --2- 172.21.15.26:0/380126191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c0759e0 0x7fad6c073e30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.491+0000 7fad71e00640 1 -- 172.21.15.26:0/380126191 >> 172.21.15.26:0/380126191 conn(0x7fad6c0fcda0 msgr2=0x7fad6c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.492+0000 7fad71e00640 1 -- 172.21.15.26:0/380126191 shutdown_connections 2024-06-13T07:19:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.492+0000 7fad71e00640 1 -- 172.21.15.26:0/380126191 wait complete. 2024-06-13T07:19:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.493+0000 7fad71e00640 1 Processor -- start 2024-06-13T07:19:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.494+0000 7fad71e00640 1 -- start start 2024-06-13T07:19:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.494+0000 7fad71e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c074400 0x7fad6c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.494+0000 7fad71e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad6c0759e0 0x7fad6c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.494+0000 7fad71e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6c10de00 0x7fad6c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.494+0000 7fad71e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c108270 con 0x7fad6c10de00 2024-06-13T07:19:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.494+0000 7fad71e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1080f0 con 0x7fad6c074400 2024-06-13T07:19:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.494+0000 7fad71e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1083f0 con 0x7fad6c0759e0 2024-06-13T07:19:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.495+0000 7fad6be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad6c0759e0 0x7fad6c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.495+0000 7fad6be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad6c0759e0 0x7fad6c196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44982/0 (socket says 172.21.15.26:44982) 2024-06-13T07:19:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.495+0000 7fad6be00640 1 -- 172.21.15.26:0/3680072286 learned_addr learned my addr 172.21.15.26:0/3680072286 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.495+0000 7fad71400640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6c10de00 0x7fad6c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.496+0000 7fad70a00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c074400 0x7fad6c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.496+0000 7fad6be00640 1 -- 172.21.15.26:0/3680072286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c074400 msgr2=0x7fad6c195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.496+0000 7fad6be00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c074400 0x7fad6c195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.496+0000 7fad6be00640 1 -- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6c10de00 msgr2=0x7fad6c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.496+0000 7fad6be00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6c10de00 0x7fad6c19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.496+0000 7fad6be00640 1 -- 172.21.15.26:0/3680072286 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c19d5a0 con 0x7fad6c0759e0 2024-06-13T07:19:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.497+0000 7fad70a00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c074400 0x7fad6c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:19:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.497+0000 7fad6be00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad6c0759e0 0x7fad6c196300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00d950 tx=0x7fad5c00dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.498+0000 7fad69600640 1 -- 172.21.15.26:0/3680072286 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c017070 con 0x7fad6c0759e0 2024-06-13T07:19:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.498+0000 7fad69600640 1 -- 172.21.15.26:0/3680072286 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad5c00de80 con 0x7fad6c0759e0 2024-06-13T07:19:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.498+0000 7fad69600640 1 -- 172.21.15.26:0/3680072286 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c00f620 con 0x7fad6c0759e0 2024-06-13T07:19:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.498+0000 7fad71e00640 1 -- 172.21.15.26:0/3680072286 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c19d890 con 0x7fad6c0759e0 2024-06-13T07:19:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.498+0000 7fad71e00640 1 -- 172.21.15.26:0/3680072286 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c19dd90 con 0x7fad6c0759e0 2024-06-13T07:19:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.503+0000 7fad69600640 1 -- 172.21.15.26:0/3680072286 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad5c00f7c0 con 0x7fad6c0759e0 2024-06-13T07:19:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.505+0000 7fad69600640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fad500a2cd0 0x7fad500a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.506+0000 7fad69600640 1 -- 172.21.15.26:0/3680072286 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fad5c0cf060 con 0x7fad6c0759e0 2024-06-13T07:19:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.506+0000 7fad70a00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fad500a2cd0 0x7fad500a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:42 smithi026 ceph-mon[32041]: pgmap v1085: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 508 op/s 2024-06-13T07:19:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.509+0000 7fad761f6740 1 Processor -- start 2024-06-13T07:19:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.510+0000 7fad70a00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fad500a2cd0 0x7fad500a5190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fad64006010 tx=0x7fad64040900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.511+0000 7fad761f6740 1 -- start start 2024-06-13T07:19:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.512+0000 7fad761f6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd64dc0 0x560dedd651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.512+0000 7fad761f6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd657b0 0x560dedd6a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.512+0000 7fad761f6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd6cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.512+0000 7fad761f6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560dedcc8420 con 0x560dedd657b0 2024-06-13T07:19:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.512+0000 7fad761f6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560dedc6fc50 con 0x560dedd64dc0 2024-06-13T07:19:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.512+0000 7fad761f6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560dedcc43e0 con 0x560dedd6a920 2024-06-13T07:19:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.512+0000 7fad71400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd6cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.513+0000 7fad71400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd6cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:44986/0 (socket says 172.21.15.26:44986) 2024-06-13T07:19:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.513+0000 7fad71400640 1 -- 172.21.15.26:0/26977952 learned_addr learned my addr 172.21.15.26:0/26977952 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.513+0000 7fad71400640 1 -- 172.21.15.26:0/26977952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd64dc0 msgr2=0x560dedd651e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.513+0000 7fad71400640 1 --2- 172.21.15.26:0/26977952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd64dc0 0x560dedd651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.513+0000 7fad71400640 1 -- 172.21.15.26:0/26977952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd657b0 msgr2=0x560dedd6a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.513+0000 7fad71400640 1 --2- 172.21.15.26:0/26977952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd657b0 0x560dedd6a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.513+0000 7fad71400640 1 -- 172.21.15.26:0/26977952 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ded9dc210 con 0x560dedd6a920 2024-06-13T07:19:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.514+0000 7fad71400640 1 --2- 172.21.15.26:0/26977952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd6cdb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fad58009c90 tx=0x7fad58011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b261c3a597675e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.515+0000 7fad61c00640 1 -- 172.21.15.26:0/26977952 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5801e070 con 0x560dedd6a920 2024-06-13T07:19:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.515+0000 7fad61c00640 1 -- 172.21.15.26:0/26977952 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad58009e40 con 0x560dedd6a920 2024-06-13T07:19:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.515+0000 7fad61c00640 1 -- 172.21.15.26:0/26977952 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5800eab0 con 0x560dedd6a920 2024-06-13T07:19:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.516+0000 7fad761f6740 1 -- 172.21.15.26:0/26977952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 msgr2=0x560dedd6cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.516+0000 7fad761f6740 1 --2- 172.21.15.26:0/26977952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd6cdb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fad58009c90 tx=0x7fad58011810 comp rx=0 tx=0).stop 2024-06-13T07:19:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.517+0000 7fad761f6740 1 -- 172.21.15.26:0/26977952 shutdown_connections 2024-06-13T07:19:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.517+0000 7fad761f6740 1 --2- 172.21.15.26:0/26977952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd6cdb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.517+0000 7fad761f6740 1 --2- 172.21.15.26:0/26977952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd657b0 0x560dedd6a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.517+0000 7fad761f6740 1 --2- 172.21.15.26:0/26977952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd64dc0 0x560dedd651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.517+0000 7fad761f6740 1 -- 172.21.15.26:0/26977952 >> 172.21.15.26:0/26977952 conn(0x560dedd2cfb0 msgr2=0x560dedd60840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.517+0000 7fad761f6740 1 -- 172.21.15.26:0/26977952 shutdown_connections 2024-06-13T07:19:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.517+0000 7fad761f6740 1 -- 172.21.15.26:0/26977952 wait complete. 2024-06-13T07:19:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.518+0000 7fad761f6740 1 -- start start 2024-06-13T07:19:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.518+0000 7fad761f6740 1 Processor -- start 2024-06-13T07:19:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad761f6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd64dc0 0x560dedd6f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad761f6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd6f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad761f6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd76710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad761f6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560dedd6d8d0 con 0x560dedd64dc0 2024-06-13T07:19:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad761f6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560dedc830b0 con 0x560dedd657b0 2024-06-13T07:19:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad761f6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560dedd6da50 con 0x560dedd6a920 2024-06-13T07:19:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad761f6740 10 client.0 _ll_register_callbacks cb 0x560dedc6ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:19:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad6be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd6f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad6be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd6f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38176/0 (socket says 172.21.15.26:38176) 2024-06-13T07:19:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad6be00640 1 -- 172.21.15.26:0/1292595099 learned_addr learned my addr 172.21.15.26:0/1292595099 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad6be00640 1 -- 172.21.15.26:0/1292595099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 msgr2=0x560dedd76710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:19:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad6be00640 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd76710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.519+0000 7fad6be00640 1 -- 172.21.15.26:0/1292595099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd64dc0 msgr2=0x560dedd6f190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad6be00640 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd64dc0 0x560dedd6f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad6be00640 1 -- 172.21.15.26:0/1292595099 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dedaa8620 con 0x560dedd657b0 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad6be00640 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd6f6d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0a3bb0 tx=0x7fad5c0a40e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddf48b25cbe81f25 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad56000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad56000640 1 -- 172.21.15.26:0/1292595099 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0b4060 con 0x560dedd657b0 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad56000640 1 -- 172.21.15.26:0/1292595099 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fad5c0b4200 con 0x560dedd657b0 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad761f6740 1 -- 172.21.15.26:0/1292595099 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560dedcd5b50 con 0x560dedd657b0 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad56000640 1 -- 172.21.15.26:0/1292595099 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fad5c020040 con 0x560dedd657b0 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad761f6740 10 client.26873 fetch_fsmap learned FSMap version 47 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.520+0000 7fad761f6740 1 -- 172.21.15.26:0/1292595099 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560dedd79af0 con 0x560dedd657b0 2024-06-13T07:19:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.521+0000 7fad56000640 1 -- 172.21.15.26:0/1292595099 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560dedcd5b50 con 0x560dedd657b0 2024-06-13T07:19:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.524+0000 7fad56000640 1 -- 172.21.15.26:0/1292595099 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fad5c00fb40 con 0x560dedd657b0 2024-06-13T07:19:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.524+0000 7fad761f6740 10 client.26873 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:19:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.524+0000 7fad761f6740 10 client.26873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:19:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.524+0000 7fad761f6740 20 client.26873 populate_metadata read hostname 'smithi026' 2024-06-13T07:19:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.524+0000 7fad761f6740 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560dedd7b140 0x560dedd7d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.524+0000 7fad761f6740 4 client.26873 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:19:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.524+0000 7fad761f6740 1 -- 172.21.15.26:0/1292595099 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dedccfea0 con 0x560dedd7b140 2024-06-13T07:19:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.524+0000 7fad70a00640 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560dedd7b140 0x560dedd7d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.526+0000 7fad70a00640 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560dedd7b140 0x560dedd7d5d0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89acd6c13e1765c server_cookie=a974a8ed2e2cd684 in_seq=0 out_seq=0 2024-06-13T07:19:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.526+0000 7fad56000640 10 client.26873 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:42.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:42 smithi135 ceph-mon[36276]: pgmap v1085: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 508 op/s 2024-06-13T07:19:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.533+0000 7fad56000640 1 -- 172.21.15.26:0/1292595099 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560dedccfea0 con 0x560dedd7b140 2024-06-13T07:19:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.533+0000 7fad56000640 10 client.26873 handle_command_reply: tid=0 2024-06-13T07:19:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.534+0000 7fad761f6740 1 client.26873 shutdown 2024-06-13T07:19:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.534+0000 7fad761f6740 1 -- 172.21.15.26:0/1292595099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 msgr2=0x560dedd6f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.534+0000 7fad761f6740 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd6f6d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0a3bb0 tx=0x7fad5c0a40e0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad761f6740 1 -- 172.21.15.26:0/1292595099 shutdown_connections 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad761f6740 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560dedd7b140 0x560dedd7d5d0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad761f6740 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd6f6d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad761f6740 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd6a920 0x560dedd76710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad761f6740 1 --2- 172.21.15.26:0/1292595099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd64dc0 0x560dedd6f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad761f6740 1 -- 172.21.15.26:0/1292595099 >> 172.21.15.26:0/1292595099 conn(0x560dedd2cfb0 msgr2=0x560dedd5f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad56000640 0 client.26873 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad761f6740 1 -- 172.21.15.26:0/1292595099 shutdown_connections 2024-06-13T07:19:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.535+0000 7fad761f6740 1 -- 172.21.15.26:0/1292595099 wait complete. 2024-06-13T07:19:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.536+0000 7fad761f6740 20 client.26873 trim_cache size 0 max 16384 2024-06-13T07:19:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.569+0000 7fad761f6740 1 Processor -- start 2024-06-13T07:19:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.573+0000 7fad761f6740 1 -- start start 2024-06-13T07:19:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.573+0000 7fad761f6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd63e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.573+0000 7fad761f6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.573+0000 7fad761f6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 0x560dedd81ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.573+0000 7fad761f6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c070180 con 0x560dedd6a920 2024-06-13T07:19:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.573+0000 7fad761f6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c06da50 con 0x560dedd657b0 2024-06-13T07:19:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.573+0000 7fad761f6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c06b8a0 con 0x560dedd64dc0 2024-06-13T07:19:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.574+0000 7fad6be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.574+0000 7fad6be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd766d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38192/0 (socket says 172.21.15.26:38192) 2024-06-13T07:19:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.574+0000 7fad6be00640 1 -- 172.21.15.26:0/2106251759 learned_addr learned my addr 172.21.15.26:0/2106251759 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.574+0000 7fad6be00640 1 -- 172.21.15.26:0/2106251759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 msgr2=0x560dedd63e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:19:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.574+0000 7fad6be00640 1 --2- 172.21.15.26:0/2106251759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd63e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.574+0000 7fad6be00640 1 -- 172.21.15.26:0/2106251759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 msgr2=0x560dedd81ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.574+0000 7fad6be00640 1 --2- 172.21.15.26:0/2106251759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 0x560dedd81ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.574+0000 7fad6be00640 1 -- 172.21.15.26:0/2106251759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dedd5c000 con 0x560dedd657b0 2024-06-13T07:19:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.575+0000 7fad6be00640 1 --2- 172.21.15.26:0/2106251759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd766d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fad5c099b60 tx=0x7fad5c097e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ab0df9df9c65fa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.575+0000 7fad61200640 1 -- 172.21.15.26:0/2106251759 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0aa070 con 0x560dedd657b0 2024-06-13T07:19:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.575+0000 7fad61200640 1 -- 172.21.15.26:0/2106251759 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad5c0b43e0 con 0x560dedd657b0 2024-06-13T07:19:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.576+0000 7fad761f6740 1 -- 172.21.15.26:0/2106251759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 msgr2=0x560dedd766d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.576+0000 7fad761f6740 1 --2- 172.21.15.26:0/2106251759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd766d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fad5c099b60 tx=0x7fad5c097e30 comp rx=0 tx=0).stop 2024-06-13T07:19:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.577+0000 7fad761f6740 1 -- 172.21.15.26:0/2106251759 shutdown_connections 2024-06-13T07:19:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.577+0000 7fad761f6740 1 --2- 172.21.15.26:0/2106251759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd766d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.577+0000 7fad761f6740 1 --2- 172.21.15.26:0/2106251759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 0x560dedd81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.577+0000 7fad761f6740 1 --2- 172.21.15.26:0/2106251759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd63e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.577+0000 7fad761f6740 1 -- 172.21.15.26:0/2106251759 >> 172.21.15.26:0/2106251759 conn(0x560dedd2cfb0 msgr2=0x560dedd63730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.577+0000 7fad761f6740 1 -- 172.21.15.26:0/2106251759 shutdown_connections 2024-06-13T07:19:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.577+0000 7fad761f6740 1 -- 172.21.15.26:0/2106251759 wait complete. 2024-06-13T07:19:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.577+0000 7fad761f6740 1 -- start start 2024-06-13T07:19:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.578+0000 7fad761f6740 1 Processor -- start 2024-06-13T07:19:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.579+0000 7fad761f6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd73c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.579+0000 7fad761f6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.580+0000 7fad761f6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 0x560dedd75190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.580+0000 7fad70a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd73c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.581+0000 7fad70a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd73c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45004/0 (socket says 172.21.15.26:45004) 2024-06-13T07:19:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.580+0000 7fad6be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.581+0000 7fad6be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38198/0 (socket says 172.21.15.26:38198) 2024-06-13T07:19:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.581+0000 7fad70a00640 1 -- 172.21.15.26:0/1013268337 learned_addr learned my addr 172.21.15.26:0/1013268337 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.581+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c06bf10 con 0x560dedd6a920 2024-06-13T07:19:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.581+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c06d380 con 0x560dedd657b0 2024-06-13T07:19:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.581+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c06c5e0 con 0x560dedd64dc0 2024-06-13T07:19:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.581+0000 7fad761f6740 10 client.0 _ll_register_callbacks cb 0x7fad5c009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:19:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.581+0000 7fad71400640 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 0x560dedd75190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.582+0000 7fad6be00640 1 -- 172.21.15.26:0/1013268337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 msgr2=0x560dedd73c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.582+0000 7fad6be00640 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd73c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.582+0000 7fad6be00640 1 -- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 msgr2=0x560dedd75190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.582+0000 7fad6be00640 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 0x560dedd75190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.582+0000 7fad6be00640 1 -- 172.21.15.26:0/1013268337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dedd75780 con 0x560dedd657b0 2024-06-13T07:19:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.582+0000 7fad70a00640 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd73c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:19:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.582+0000 7fad71400640 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 0x560dedd75190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:19:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.583+0000 7fad6be00640 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd741d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0b7040 tx=0x7fad5c097760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec10ce68d6b12b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.583+0000 7fad56000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:19:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.583+0000 7fad56000640 1 -- 172.21.15.26:0/1013268337 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0aa070 con 0x560dedd657b0 2024-06-13T07:19:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.583+0000 7fad56000640 1 -- 172.21.15.26:0/1013268337 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fad5c09a700 con 0x560dedd657b0 2024-06-13T07:19:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.583+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad5c0b43e0 con 0x560dedd657b0 2024-06-13T07:19:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.584+0000 7fad56000640 1 -- 172.21.15.26:0/1013268337 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fad5c00fcc0 con 0x560dedd657b0 2024-06-13T07:19:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.584+0000 7fad761f6740 10 client.26879 fetch_fsmap learned FSMap version 47 2024-06-13T07:19:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.584+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560dedde5eb0 con 0x560dedd657b0 2024-06-13T07:19:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.585+0000 7fad56000640 1 -- 172.21.15.26:0/1013268337 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad5c0b43e0 con 0x560dedd657b0 2024-06-13T07:19:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.585+0000 7fad56000640 1 -- 172.21.15.26:0/1013268337 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fad5c09f410 con 0x560dedd657b0 2024-06-13T07:19:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.585+0000 7fad761f6740 10 client.26879 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:19:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.585+0000 7fad761f6740 10 client.26879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:19:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.585+0000 7fad761f6740 20 client.26879 populate_metadata read hostname 'smithi026' 2024-06-13T07:19:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.585+0000 7fad761f6740 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560dedde6460 0x560dedde8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.585+0000 7fad761f6740 4 client.26879 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:19:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.585+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dedd6ef80 con 0x560dedde6460 2024-06-13T07:19:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.586+0000 7fad70a00640 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560dedde6460 0x560dedde8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.587+0000 7fad56000640 10 client.26879 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:42.587+0000 7fad70a00640 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560dedde6460 0x560dedde8880 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653bbe5055a9a335 server_cookie=69b6c899177dfbe8 in_seq=0 out_seq=0 2024-06-13T07:19:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:42 smithi136 ceph-mon[37229]: pgmap v1085: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 508 op/s 2024-06-13T07:19:43.189 INFO:tasks.workunit.client.0.smithi026.stdout: 10 183909 27.43 MB/sec execute 236 sec latency 418.006 ms 2024-06-13T07:19:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.190+0000 7fad56000640 1 -- 172.21.15.26:0/1013268337 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560dedd6ef80 con 0x560dedde6460 2024-06-13T07:19:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.190+0000 7fad56000640 10 client.26879 handle_command_reply: tid=0 2024-06-13T07:19:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.190+0000 7fad761f6740 1 client.26879 shutdown 2024-06-13T07:19:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 msgr2=0x560dedd741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad761f6740 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd741d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0b7040 tx=0x7fad5c097760 comp rx=0 tx=0).stop 2024-06-13T07:19:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 shutdown_connections 2024-06-13T07:19:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad761f6740 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560dedde6460 0x560dedde8880 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad761f6740 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dedd657b0 0x560dedd741d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad56000640 0 client.26879 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad761f6740 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560dedd6a920 0x560dedd75190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad761f6740 1 --2- 172.21.15.26:0/1013268337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dedd64dc0 0x560dedd73c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.191+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 >> 172.21.15.26:0/1013268337 conn(0x560dedd2cfb0 msgr2=0x560dedd5f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.193+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 shutdown_connections 2024-06-13T07:19:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.193+0000 7fad761f6740 1 -- 172.21.15.26:0/1013268337 wait complete. 2024-06-13T07:19:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.194+0000 7fad761f6740 20 client.26879 trim_cache size 0 max 16384 2024-06-13T07:19:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:19:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:19:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:19:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 -- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fad500a2cd0 msgr2=0x7fad500a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fad500a2cd0 0x7fad500a5190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fad64006010 tx=0x7fad64040900 comp rx=0 tx=0).stop 2024-06-13T07:19:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 -- 172.21.15.26:0/3680072286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad6c0759e0 msgr2=0x7fad6c196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad6c0759e0 0x7fad6c196300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00d950 tx=0x7fad5c00dd50 comp rx=0 tx=0).stop 2024-06-13T07:19:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 -- 172.21.15.26:0/3680072286 shutdown_connections 2024-06-13T07:19:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fad500a2cd0 0x7fad500a5190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad6c10de00 0x7fad6c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad6c0759e0 0x7fad6c196300 secure :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00d950 tx=0x7fad5c00dd50 comp rx=0 tx=0).stop 2024-06-13T07:19:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 --2- 172.21.15.26:0/3680072286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c074400 0x7fad6c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.195+0000 7fad71e00640 1 -- 172.21.15.26:0/3680072286 >> 172.21.15.26:0/3680072286 conn(0x7fad6c0fcda0 msgr2=0x7fad6c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.197+0000 7fad71e00640 1 -- 172.21.15.26:0/3680072286 shutdown_connections 2024-06-13T07:19:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:43.197+0000 7fad71e00640 1 -- 172.21.15.26:0/3680072286 wait complete. 2024-06-13T07:19:44.189 INFO:tasks.workunit.client.0.smithi026.stdout: 10 184502 27.40 MB/sec execute 237 sec latency 96.557 ms 2024-06-13T07:19:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:44 smithi026 ceph-mon[32041]: pgmap v1086: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-06-13T07:19:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:44 smithi135 ceph-mon[36276]: pgmap v1086: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-06-13T07:19:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:44 smithi136 ceph-mon[37229]: pgmap v1086: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-06-13T07:19:45.189 INFO:tasks.workunit.client.0.smithi026.stdout: 10 185206 27.46 MB/sec execute 238 sec latency 91.069 ms 2024-06-13T07:19:46.189 INFO:tasks.workunit.client.0.smithi026.stdout: 10 185911 27.54 MB/sec execute 239 sec latency 86.641 ms 2024-06-13T07:19:46.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:46 smithi026 ceph-mon[32041]: pgmap v1087: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 460 op/s 2024-06-13T07:19:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:46 smithi135 ceph-mon[36276]: pgmap v1087: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 460 op/s 2024-06-13T07:19:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:46 smithi136 ceph-mon[37229]: pgmap v1087: 129 pgs: 129 active+clean; 663 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 460 op/s 2024-06-13T07:19:47.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 186466 27.56 MB/sec execute 240 sec latency 78.583 ms 2024-06-13T07:19:48.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 186811 27.51 MB/sec execute 241 sec latency 72.185 ms 2024-06-13T07:19:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:48 smithi026 ceph-mon[32041]: pgmap v1088: 129 pgs: 129 active+clean; 653 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 415 op/s 2024-06-13T07:19:48.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:48 smithi135 ceph-mon[36276]: pgmap v1088: 129 pgs: 129 active+clean; 653 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 415 op/s 2024-06-13T07:19:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:48 smithi136 ceph-mon[37229]: pgmap v1088: 129 pgs: 129 active+clean; 653 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 415 op/s 2024-06-13T07:19:49.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 187429 27.52 MB/sec execute 242 sec latency 52.273 ms 2024-06-13T07:19:50.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 188120 27.54 MB/sec execute 243 sec latency 46.933 ms 2024-06-13T07:19:50.504 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:50 smithi026 ceph-mon[32041]: pgmap v1089: 129 pgs: 129 active+clean; 657 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 510 op/s 2024-06-13T07:19:50.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:50 smithi135 ceph-mon[36276]: pgmap v1089: 129 pgs: 129 active+clean; 657 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 510 op/s 2024-06-13T07:19:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:50 smithi136 ceph-mon[37229]: pgmap v1089: 129 pgs: 129 active+clean; 657 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 510 op/s 2024-06-13T07:19:51.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 188788 27.59 MB/sec execute 244 sec latency 161.281 ms 2024-06-13T07:19:52.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 189264 27.60 MB/sec execute 245 sec latency 119.394 ms 2024-06-13T07:19:52.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:52 smithi026 ceph-mon[32041]: pgmap v1090: 129 pgs: 129 active+clean; 662 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.2 MiB/s wr, 479 op/s 2024-06-13T07:19:52.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:52 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:19:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:52 smithi135 ceph-mon[36276]: pgmap v1090: 129 pgs: 129 active+clean; 662 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.2 MiB/s wr, 479 op/s 2024-06-13T07:19:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:52 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:19:52.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:52 smithi136 ceph-mon[37229]: pgmap v1090: 129 pgs: 129 active+clean; 662 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.2 MiB/s wr, 479 op/s 2024-06-13T07:19:52.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:52 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:19:53.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 189817 27.59 MB/sec execute 246 sec latency 133.885 ms 2024-06-13T07:19:54.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 190167 27.56 MB/sec execute 247 sec latency 181.081 ms 2024-06-13T07:19:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:54 smithi026 ceph-mon[32041]: pgmap v1091: 129 pgs: 129 active+clean; 666 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:19:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:19:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:19:54.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:54 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:19:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:54 smithi135 ceph-mon[36276]: pgmap v1091: 129 pgs: 129 active+clean; 666 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:19:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:19:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:19:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:54 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:19:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:54 smithi136 ceph-mon[37229]: pgmap v1091: 129 pgs: 129 active+clean; 666 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:19:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:19:54.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:19:54.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:54 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:19:55.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 190474 27.51 MB/sec execute 248 sec latency 307.089 ms 2024-06-13T07:19:56.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 191037 27.51 MB/sec execute 249 sec latency 126.401 ms 2024-06-13T07:19:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.393+0000 7f2251200640 1 -- 172.21.15.26:0/3437492973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 msgr2=0x7f224c1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.393+0000 7f2251200640 1 --2- 172.21.15.26:0/3437492973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 0x7f224c1052a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2238009900 tx=0x7f223802a440 comp rx=0 tx=0).stop 2024-06-13T07:19:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.394+0000 7f2251200640 1 -- 172.21.15.26:0/3437492973 shutdown_connections 2024-06-13T07:19:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.394+0000 7f2251200640 1 --2- 172.21.15.26:0/3437492973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c1057e0 0x7f224c10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.394+0000 7f2251200640 1 --2- 172.21.15.26:0/3437492973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 0x7f224c1052a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.394+0000 7f2251200640 1 --2- 172.21.15.26:0/3437492973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f224c103c30 0x7f224c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.394+0000 7f2251200640 1 -- 172.21.15.26:0/3437492973 >> 172.21.15.26:0/3437492973 conn(0x7f224c0ff1c0 msgr2=0x7f224c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.394+0000 7f2251200640 1 -- 172.21.15.26:0/3437492973 shutdown_connections 2024-06-13T07:19:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.395+0000 7f2251200640 1 -- 172.21.15.26:0/3437492973 wait complete. 2024-06-13T07:19:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.396+0000 7f2251200640 1 Processor -- start 2024-06-13T07:19:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.397+0000 7f2251200640 1 -- start start 2024-06-13T07:19:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.398+0000 7f2251200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f224c103c30 0x7f224c195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.398+0000 7f2251200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 0x7f224c196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.398+0000 7f2251200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c1057e0 0x7f224c19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.398+0000 7f2251200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10f920 con 0x7f224c1057e0 2024-06-13T07:19:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.398+0000 7f2251200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10f7a0 con 0x7f224c103c30 2024-06-13T07:19:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.398+0000 7f2251200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10faa0 con 0x7f224c104e20 2024-06-13T07:19:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f2249400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 0x7f224c196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f2249400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 0x7f224c196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57044/0 (socket says 172.21.15.26:57044) 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f2249400640 1 -- 172.21.15.26:0/2303708418 learned_addr learned my addr 172.21.15.26:0/2303708418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f224a800640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c1057e0 0x7f224c19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f2249400640 1 -- 172.21.15.26:0/2303708418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f224c103c30 msgr2=0x7f224c195bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f2249e00640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f224c103c30 0x7f224c195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f2249400640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f224c103c30 0x7f224c195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f2249400640 1 -- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c1057e0 msgr2=0x7f224c19d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.399+0000 7f2249400640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c1057e0 0x7f224c19d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.400+0000 7f2249400640 1 -- 172.21.15.26:0/2303708418 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1ae820 con 0x7f224c104e20 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.400+0000 7f224a800640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c1057e0 0x7f224c19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.400+0000 7f2249400640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 0x7f224c196100 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2238009a20 tx=0x7f2238031af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.400+0000 7f2242a00640 1 -- 172.21.15.26:0/2303708418 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223803f030 con 0x7f224c104e20 2024-06-13T07:19:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.400+0000 7f2242a00640 1 -- 172.21.15.26:0/2303708418 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2238003ce0 con 0x7f224c104e20 2024-06-13T07:19:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.400+0000 7f2242a00640 1 -- 172.21.15.26:0/2303708418 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238035070 con 0x7f224c104e20 2024-06-13T07:19:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.400+0000 7f2251200640 1 -- 172.21.15.26:0/2303708418 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c1ae9f0 con 0x7f224c104e20 2024-06-13T07:19:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.402+0000 7f2251200640 1 -- 172.21.15.26:0/2303708418 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1aee60 con 0x7f224c104e20 2024-06-13T07:19:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.404+0000 7f2242a00640 1 -- 172.21.15.26:0/2303708418 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2238042030 con 0x7f224c104e20 2024-06-13T07:19:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.406+0000 7f2242a00640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22300a2c30 0x7f22300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.406+0000 7f2242a00640 1 -- 172.21.15.26:0/2303708418 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f22380ef3e0 con 0x7f224c104e20 2024-06-13T07:19:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.408+0000 7f2249e00640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22300a2c30 0x7f22300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.408+0000 7f2254020740 1 Processor -- start 2024-06-13T07:19:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.409+0000 7f2254020740 1 -- start start 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.409+0000 7f2249e00640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22300a2c30 0x7f22300a50f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f224c104c80 tx=0x7f223c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.409+0000 7f2254020740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f896020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.409+0000 7f2254020740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f8965f0 0x558a6f89b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.409+0000 7f2254020740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f89dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.409+0000 7f2254020740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558a6f7f9420 con 0x558a6f895c00 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.409+0000 7f2254020740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a6f7a0c50 con 0x558a6f8965f0 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.409+0000 7f2254020740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558a6f7f53e0 con 0x558a6f89b760 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f2249400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f8965f0 0x558a6f89b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f2249400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f8965f0 0x558a6f89b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:52926/0 (socket says 172.21.15.26:52926) 2024-06-13T07:19:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f2249400640 1 -- 172.21.15.26:0/3822531017 learned_addr learned my addr 172.21.15.26:0/3822531017 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f224a800640 1 --2- 172.21.15.26:0/3822531017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f89dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f2249400640 1 -- 172.21.15.26:0/3822531017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 msgr2=0x558a6f89dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f2249400640 1 --2- 172.21.15.26:0/3822531017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f89dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f2249400640 1 -- 172.21.15.26:0/3822531017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 msgr2=0x558a6f896020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f2249400640 1 --2- 172.21.15.26:0/3822531017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f896020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.410+0000 7f2249400640 1 -- 172.21.15.26:0/3822531017 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6f50d210 con 0x558a6f8965f0 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.411+0000 7f2249400640 1 --2- 172.21.15.26:0/3822531017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f8965f0 0x558a6f89b0f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f22380b8220 tx=0x7f22380bb390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d830025f0f1cc7f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.411+0000 7f2237400640 1 -- 172.21.15.26:0/3822531017 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22380bb850 con 0x558a6f8965f0 2024-06-13T07:19:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.411+0000 7f2237400640 1 -- 172.21.15.26:0/3822531017 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22380bfc00 con 0x558a6f8965f0 2024-06-13T07:19:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.411+0000 7f2237400640 1 -- 172.21.15.26:0/3822531017 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223803b460 con 0x558a6f8965f0 2024-06-13T07:19:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.415+0000 7f2254020740 1 -- 172.21.15.26:0/3822531017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f8965f0 msgr2=0x558a6f89b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.415+0000 7f2254020740 1 --2- 172.21.15.26:0/3822531017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f8965f0 0x558a6f89b0f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f22380b8220 tx=0x7f22380bb390 comp rx=0 tx=0).stop 2024-06-13T07:19:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.416+0000 7f2254020740 1 -- 172.21.15.26:0/3822531017 shutdown_connections 2024-06-13T07:19:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.416+0000 7f2254020740 1 --2- 172.21.15.26:0/3822531017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f8965f0 0x558a6f89b0f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.416+0000 7f2254020740 1 --2- 172.21.15.26:0/3822531017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f89dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.416+0000 7f2254020740 1 --2- 172.21.15.26:0/3822531017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f896020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.416+0000 7f2254020740 1 -- 172.21.15.26:0/3822531017 >> 172.21.15.26:0/3822531017 conn(0x558a6f85dfb0 msgr2=0x558a6f891680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.416+0000 7f2254020740 1 -- 172.21.15.26:0/3822531017 shutdown_connections 2024-06-13T07:19:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.416+0000 7f2254020740 1 -- 172.21.15.26:0/3822531017 wait complete. 2024-06-13T07:19:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.417+0000 7f2254020740 1 -- start start 2024-06-13T07:19:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.417+0000 7f2254020740 1 Processor -- start 2024-06-13T07:19:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.418+0000 7f2254020740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f89fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.418+0000 7f2254020740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.418+0000 7f2254020740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.418+0000 7f2254020740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558a6f89e4d0 con 0x558a6f895c00 2024-06-13T07:19:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.418+0000 7f2254020740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a6f7b40b0 con 0x558a6f89b760 2024-06-13T07:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.418+0000 7f2254020740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558a6f89e650 con 0x558a6f8965f0 2024-06-13T07:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2254020740 10 client.0 _ll_register_callbacks cb 0x558a6f79ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57068/0 (socket says 172.21.15.26:57068) 2024-06-13T07:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 -- 172.21.15.26:0/3604275893 learned_addr learned my addr 172.21.15.26:0/3604275893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249e00640 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f89fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 -- 172.21.15.26:0/3604275893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 msgr2=0x558a6f8a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 -- 172.21.15.26:0/3604275893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 msgr2=0x558a6f89fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f89fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 -- 172.21.15.26:0/3604275893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6f5d9620 con 0x558a6f8965f0 2024-06-13T07:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.419+0000 7f2249400640 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a0220 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2238032b90 tx=0x7f223803be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1655485818ac49ff server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.420+0000 7f222be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.420+0000 7f222be00640 1 -- 172.21.15.26:0/3604275893 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22380d9060 con 0x558a6f8965f0 2024-06-13T07:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.420+0000 7f222be00640 1 -- 172.21.15.26:0/3604275893 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f22380b98f0 con 0x558a6f8965f0 2024-06-13T07:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.421+0000 7f2254020740 1 -- 172.21.15.26:0/3604275893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22380bfc00 con 0x558a6f8965f0 2024-06-13T07:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.421+0000 7f222be00640 1 -- 172.21.15.26:0/3604275893 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f22380d3030 con 0x558a6f8965f0 2024-06-13T07:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.421+0000 7f2254020740 10 client.26938 fetch_fsmap learned FSMap version 47 2024-06-13T07:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.421+0000 7f2254020740 1 -- 172.21.15.26:0/3604275893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558a6f89e4d0 con 0x558a6f8965f0 2024-06-13T07:19:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.422+0000 7f222be00640 1 -- 172.21.15.26:0/3604275893 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22380bfc00 con 0x558a6f8965f0 2024-06-13T07:19:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.423+0000 7f222be00640 1 -- 172.21.15.26:0/3604275893 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f22380c4030 con 0x558a6f8965f0 2024-06-13T07:19:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.423+0000 7f2254020740 10 client.26938 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:19:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.423+0000 7f2254020740 10 client.26938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:19:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.423+0000 7f2254020740 20 client.26938 populate_metadata read hostname 'smithi026' 2024-06-13T07:19:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.423+0000 7f2254020740 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a6f8aba20 0x558a6f8ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.423+0000 7f2254020740 4 client.26938 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:19:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.423+0000 7f2254020740 1 -- 172.21.15.26:0/3604275893 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a6f806b50 con 0x558a6f8aba20 2024-06-13T07:19:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.425+0000 7f2249e00640 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a6f8aba20 0x558a6f8ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.427+0000 7f2249e00640 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a6f8aba20 0x558a6f8ade40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215b0b913b3d306d server_cookie=f1c1ce510e9ce088 in_seq=0 out_seq=0 2024-06-13T07:19:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.427+0000 7f222be00640 10 client.26938 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.431+0000 7f222be00640 1 -- 172.21.15.26:0/3604275893 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558a6f806b50 con 0x558a6f8aba20 2024-06-13T07:19:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.431+0000 7f222be00640 10 client.26938 handle_command_reply: tid=0 2024-06-13T07:19:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.432+0000 7f2254020740 1 client.26938 shutdown 2024-06-13T07:19:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.432+0000 7f2254020740 1 -- 172.21.15.26:0/3604275893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 msgr2=0x558a6f8a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.432+0000 7f2254020740 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a0220 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2238032b90 tx=0x7f223803be50 comp rx=0 tx=0).stop 2024-06-13T07:19:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.433+0000 7f2254020740 1 -- 172.21.15.26:0/3604275893 shutdown_connections 2024-06-13T07:19:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.433+0000 7f2254020740 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a6f8aba20 0x558a6f8ade40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.433+0000 7f222be00640 0 client.26938 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.433+0000 7f2254020740 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.433+0000 7f2254020740 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a0220 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.433+0000 7f2254020740 1 --2- 172.21.15.26:0/3604275893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f89fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.433+0000 7f2254020740 1 -- 172.21.15.26:0/3604275893 >> 172.21.15.26:0/3604275893 conn(0x558a6f85dfb0 msgr2=0x558a6f890840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.434+0000 7f2254020740 1 -- 172.21.15.26:0/3604275893 shutdown_connections 2024-06-13T07:19:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.434+0000 7f2254020740 1 -- 172.21.15.26:0/3604275893 wait complete. 2024-06-13T07:19:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.434+0000 7f2254020740 20 client.26938 trim_cache size 0 max 16384 2024-06-13T07:19:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.459+0000 7f2254020740 1 Processor -- start 2024-06-13T07:19:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.459+0000 7f2254020740 1 -- start start 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.459+0000 7f2254020740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f895c00 0x558a6f88c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.459+0000 7f2254020740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f8965f0 0x558a6f8a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.459+0000 7f2254020740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f890fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.460+0000 7f2254020740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1bbed0 con 0x558a6f8965f0 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.460+0000 7f2254020740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b97a0 con 0x558a6f895c00 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.460+0000 7f2254020740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b75f0 con 0x558a6f89b760 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.460+0000 7f224a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f890fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.460+0000 7f224a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f890fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57080/0 (socket says 172.21.15.26:57080) 2024-06-13T07:19:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.460+0000 7f224a800640 1 -- 172.21.15.26:0/3995636914 learned_addr learned my addr 172.21.15.26:0/3995636914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.461+0000 7f2249e00640 1 --2- 172.21.15.26:0/3995636914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f895c00 0x558a6f88c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.461+0000 7f224a800640 1 -- 172.21.15.26:0/3995636914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f895c00 msgr2=0x558a6f88c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.461+0000 7f224a800640 1 --2- 172.21.15.26:0/3995636914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f895c00 0x558a6f88c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.461+0000 7f224a800640 1 -- 172.21.15.26:0/3995636914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f8965f0 msgr2=0x558a6f8a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.461+0000 7f224a800640 1 --2- 172.21.15.26:0/3995636914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f8965f0 0x558a6f8a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.461+0000 7f224a800640 1 -- 172.21.15.26:0/3995636914 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6f892330 con 0x558a6f89b760 2024-06-13T07:19:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.462+0000 7f224a800640 1 --2- 172.21.15.26:0/3995636914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f890fc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2244009f20 tx=0x7f22440117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa64b9cdec35f286 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.462+0000 7f2236a00640 1 -- 172.21.15.26:0/3995636914 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244003b70 con 0x558a6f89b760 2024-06-13T07:19:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.463+0000 7f2236a00640 1 -- 172.21.15.26:0/3995636914 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2244003d10 con 0x558a6f89b760 2024-06-13T07:19:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.464+0000 7f2254020740 1 -- 172.21.15.26:0/3995636914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 msgr2=0x558a6f890fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.464+0000 7f2254020740 1 --2- 172.21.15.26:0/3995636914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f890fc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2244009f20 tx=0x7f22440117b0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.464+0000 7f2254020740 1 -- 172.21.15.26:0/3995636914 shutdown_connections 2024-06-13T07:19:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.464+0000 7f2254020740 1 --2- 172.21.15.26:0/3995636914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f89b760 0x558a6f890fc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.464+0000 7f2254020740 1 --2- 172.21.15.26:0/3995636914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f8965f0 0x558a6f8a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.464+0000 7f2254020740 1 --2- 172.21.15.26:0/3995636914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f895c00 0x558a6f88c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.464+0000 7f2254020740 1 -- 172.21.15.26:0/3995636914 >> 172.21.15.26:0/3995636914 conn(0x558a6f85dfb0 msgr2=0x558a6f897970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.465+0000 7f2254020740 1 -- 172.21.15.26:0/3995636914 shutdown_connections 2024-06-13T07:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.465+0000 7f2254020740 1 -- 172.21.15.26:0/3995636914 wait complete. 2024-06-13T07:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.465+0000 7f2254020740 1 -- start start 2024-06-13T07:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.465+0000 7f2254020740 1 Processor -- start 2024-06-13T07:19:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.466+0000 7f2254020740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f8a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.466+0000 7f2254020740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.466+0000 7f2254020740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.466+0000 7f2254020740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b7c60 con 0x558a6f895c00 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.466+0000 7f2254020740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b90d0 con 0x558a6f89b760 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.466+0000 7f2254020740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b8330 con 0x558a6f8965f0 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.466+0000 7f2254020740 10 client.0 _ll_register_callbacks cb 0x7f22380b7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f2249400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:52962/0 (socket says 172.21.15.26:52962) 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 -- 172.21.15.26:0/3768655525 learned_addr learned my addr 172.21.15.26:0/3768655525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f2249e00640 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f8a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 -- 172.21.15.26:0/3768655525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 msgr2=0x558a6f8a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 -- 172.21.15.26:0/3768655525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 msgr2=0x558a6f8a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f8a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 -- 172.21.15.26:0/3768655525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6f8a6170 con 0x558a6f89b760 2024-06-13T07:19:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f224a800640 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a5b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2244006010 tx=0x7f2244009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba607f3941096f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:19:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.467+0000 7f222be00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:19:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.468+0000 7f222be00640 1 -- 172.21.15.26:0/3768655525 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244003970 con 0x558a6f89b760 2024-06-13T07:19:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.468+0000 7f222be00640 1 -- 172.21.15.26:0/3768655525 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2244003b10 con 0x558a6f89b760 2024-06-13T07:19:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.468+0000 7f2254020740 1 -- 172.21.15.26:0/3768655525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2244003d10 con 0x558a6f89b760 2024-06-13T07:19:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.468+0000 7f222be00640 1 -- 172.21.15.26:0/3768655525 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2244004000 con 0x558a6f89b760 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.468+0000 7f2254020740 10 client.26888 fetch_fsmap learned FSMap version 47 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.468+0000 7f2254020740 1 -- 172.21.15.26:0/3768655525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558a6f8a7480 con 0x558a6f89b760 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f222be00640 1 -- 172.21.15.26:0/3768655525 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2244003d10 con 0x558a6f89b760 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f222be00640 1 -- 172.21.15.26:0/3768655525 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f224401bb00 con 0x558a6f89b760 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f2254020740 10 client.26888 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f2254020740 10 client.26888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f2254020740 20 client.26888 populate_metadata read hostname 'smithi026' 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f2254020740 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a6f8aba20 0x558a6f918d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f2254020740 4 client.26888 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f2254020740 1 -- 172.21.15.26:0/3768655525 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a6f800ea0 con 0x558a6f8aba20 2024-06-13T07:19:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.469+0000 7f2249e00640 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a6f8aba20 0x558a6f918d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:19:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.471+0000 7f2249e00640 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a6f8aba20 0x558a6f918d20 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb511dcaa7ca58d server_cookie=a6bc99fab5532f2 in_seq=0 out_seq=0 2024-06-13T07:19:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:56.471+0000 7f222be00640 10 client.26888 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:56 smithi026 ceph-mon[32041]: pgmap v1092: 129 pgs: 129 active+clean; 670 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-06-13T07:19:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:56 smithi135 ceph-mon[36276]: pgmap v1092: 129 pgs: 129 active+clean; 670 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-06-13T07:19:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:56 smithi136 ceph-mon[37229]: pgmap v1092: 129 pgs: 129 active+clean; 670 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-06-13T07:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.169+0000 7f222be00640 1 -- 172.21.15.26:0/3768655525 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558a6f800ea0 con 0x558a6f8aba20 2024-06-13T07:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.169+0000 7f222be00640 10 client.26888 handle_command_reply: tid=0 2024-06-13T07:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.169+0000 7f2254020740 1 client.26888 shutdown 2024-06-13T07:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.169+0000 7f2254020740 1 -- 172.21.15.26:0/3768655525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 msgr2=0x558a6f8a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.169+0000 7f2254020740 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a5b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2244006010 tx=0x7f2244009520 comp rx=0 tx=0).stop 2024-06-13T07:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f2254020740 1 -- 172.21.15.26:0/3768655525 shutdown_connections 2024-06-13T07:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f2254020740 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558a6f8aba20 0x558a6f918d20 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f2254020740 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a6f89b760 0x558a6f8a5b80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f2254020740 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a6f8965f0 0x558a6f8a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f2254020740 1 --2- 172.21.15.26:0/3768655525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a6f895c00 0x558a6f8a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f2254020740 1 -- 172.21.15.26:0/3768655525 >> 172.21.15.26:0/3768655525 conn(0x558a6f85dfb0 msgr2=0x558a6f894650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f222be00640 0 client.26888 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f2254020740 1 -- 172.21.15.26:0/3768655525 shutdown_connections 2024-06-13T07:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.170+0000 7f2254020740 1 -- 172.21.15.26:0/3768655525 wait complete. 2024-06-13T07:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.171+0000 7f2254020740 20 client.26888 trim_cache size 0 max 16384 2024-06-13T07:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.172+0000 7f2251200640 1 -- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22300a2c30 msgr2=0x7f22300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.172+0000 7f2251200640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22300a2c30 0x7f22300a50f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f224c104c80 tx=0x7f223c008040 comp rx=0 tx=0).stop 2024-06-13T07:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.172+0000 7f2251200640 1 -- 172.21.15.26:0/2303708418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 msgr2=0x7f224c196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.172+0000 7f2251200640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 0x7f224c196100 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2238009a20 tx=0x7f2238031af0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.173+0000 7f2251200640 1 -- 172.21.15.26:0/2303708418 shutdown_connections 2024-06-13T07:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.173+0000 7f2251200640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f22300a2c30 0x7f22300a50f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.173+0000 7f2251200640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f224c1057e0 0x7f224c19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.173+0000 7f2251200640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f224c104e20 0x7f224c196100 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.173+0000 7f2251200640 1 --2- 172.21.15.26:0/2303708418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f224c103c30 0x7f224c195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:19:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.173+0000 7f2251200640 1 -- 172.21.15.26:0/2303708418 >> 172.21.15.26:0/2303708418 conn(0x7f224c0ff1c0 msgr2=0x7f224c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.173+0000 7f2251200640 1 -- 172.21.15.26:0/2303708418 shutdown_connections 2024-06-13T07:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:19:57.177+0000 7f2251200640 1 -- 172.21.15.26:0/2303708418 wait complete. 2024-06-13T07:19:57.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 191325 27.47 MB/sec execute 250 sec latency 418.155 ms 2024-06-13T07:19:58.190 INFO:tasks.workunit.client.0.smithi026.stdout: 10 191928 27.50 MB/sec execute 251 sec latency 74.003 ms 2024-06-13T07:19:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:19:58 smithi026 ceph-mon[32041]: pgmap v1093: 129 pgs: 129 active+clean; 677 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.4 MiB/s wr, 400 op/s 2024-06-13T07:19:58.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:19:58 smithi135 ceph-mon[36276]: pgmap v1093: 129 pgs: 129 active+clean; 677 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.4 MiB/s wr, 400 op/s 2024-06-13T07:19:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:19:58 smithi136 ceph-mon[37229]: pgmap v1093: 129 pgs: 129 active+clean; 677 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.4 MiB/s wr, 400 op/s 2024-06-13T07:19:59.191 INFO:tasks.workunit.client.0.smithi026.stdout: 10 192414 27.50 MB/sec execute 252 sec latency 178.921 ms 2024-06-13T07:20:00.192 INFO:tasks.workunit.client.0.smithi026.stdout: 10 192452 27.41 MB/sec execute 253 sec latency 833.199 ms 2024-06-13T07:20:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:20:00.002+0000 7fc609800700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 2 MDSs report damaged metadata 2024-06-13T07:20:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:00 smithi026 ceph-mon[32041]: pgmap v1094: 129 pgs: 129 active+clean; 675 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:20:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:00 smithi026 ceph-mon[32041]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-06-13T07:20:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:00 smithi135 ceph-mon[36276]: pgmap v1094: 129 pgs: 129 active+clean; 675 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:20:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:00 smithi135 ceph-mon[36276]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-06-13T07:20:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:00 smithi136 ceph-mon[37229]: pgmap v1094: 129 pgs: 129 active+clean; 675 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:20:00.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:00 smithi136 ceph-mon[37229]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-06-13T07:20:01.192 INFO:tasks.workunit.client.0.smithi026.stdout: 10 192565 27.32 MB/sec execute 254 sec latency 1619.883 ms 2024-06-13T07:20:02.192 INFO:tasks.workunit.client.0.smithi026.stdout: 10 192763 27.26 MB/sec execute 255 sec latency 866.442 ms 2024-06-13T07:20:02.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:02 smithi026 ceph-mon[32041]: pgmap v1095: 129 pgs: 129 active+clean; 683 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.1 MiB/s wr, 489 op/s 2024-06-13T07:20:02.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:02 smithi135 ceph-mon[36276]: pgmap v1095: 129 pgs: 129 active+clean; 683 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.1 MiB/s wr, 489 op/s 2024-06-13T07:20:02.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:02 smithi136 ceph-mon[37229]: pgmap v1095: 129 pgs: 129 active+clean; 683 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.1 MiB/s wr, 489 op/s 2024-06-13T07:20:03.192 INFO:tasks.workunit.client.0.smithi026.stdout: 10 192974 27.20 MB/sec execute 256 sec latency 154.957 ms 2024-06-13T07:20:04.192 INFO:tasks.workunit.client.0.smithi026.stdout: 10 193196 27.14 MB/sec execute 257 sec latency 217.802 ms 2024-06-13T07:20:04.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:04 smithi026 ceph-mon[32041]: pgmap v1096: 129 pgs: 129 active+clean; 714 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 592 op/s 2024-06-13T07:20:04.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:04 smithi135 ceph-mon[36276]: pgmap v1096: 129 pgs: 129 active+clean; 714 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 592 op/s 2024-06-13T07:20:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:04 smithi136 ceph-mon[37229]: pgmap v1096: 129 pgs: 129 active+clean; 714 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 592 op/s 2024-06-13T07:20:05.192 INFO:tasks.workunit.client.0.smithi026.stdout: 10 193474 27.07 MB/sec execute 258 sec latency 176.867 ms 2024-06-13T07:20:06.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 194019 27.03 MB/sec execute 259 sec latency 99.907 ms 2024-06-13T07:20:06.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:06 smithi135 ceph-mon[36276]: pgmap v1097: 129 pgs: 129 active+clean; 723 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 633 op/s 2024-06-13T07:20:06.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:06 smithi136 ceph-mon[37229]: pgmap v1097: 129 pgs: 129 active+clean; 723 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 633 op/s 2024-06-13T07:20:06.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:06 smithi026 ceph-mon[32041]: pgmap v1097: 129 pgs: 129 active+clean; 723 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 633 op/s 2024-06-13T07:20:07.192 INFO:tasks.workunit.client.0.smithi026.stdout: 10 194919 27.16 MB/sec execute 260 sec latency 60.660 ms 2024-06-13T07:20:08.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 195659 27.24 MB/sec execute 261 sec latency 45.276 ms 2024-06-13T07:20:08.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:08 smithi135 ceph-mon[36276]: pgmap v1098: 129 pgs: 129 active+clean; 707 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 685 op/s 2024-06-13T07:20:08.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:08 smithi136 ceph-mon[37229]: pgmap v1098: 129 pgs: 129 active+clean; 707 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 685 op/s 2024-06-13T07:20:08.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:08 smithi026 ceph-mon[32041]: pgmap v1098: 129 pgs: 129 active+clean; 707 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 685 op/s 2024-06-13T07:20:09.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 196076 27.23 MB/sec execute 262 sec latency 76.551 ms 2024-06-13T07:20:10.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 196567 27.21 MB/sec execute 263 sec latency 45.731 ms 2024-06-13T07:20:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.366+0000 7f3064200640 1 -- 172.21.15.26:0/1009262717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c109210 msgr2=0x7f305c109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.366+0000 7f3064200640 1 --2- 172.21.15.26:0/1009262717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c109210 0x7f305c109690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3050009900 tx=0x7f305002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.369+0000 7f3064200640 1 -- 172.21.15.26:0/1009262717 shutdown_connections 2024-06-13T07:20:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.369+0000 7f3064200640 1 --2- 172.21.15.26:0/1009262717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f305c109bd0 0x7f305c110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.369+0000 7f3064200640 1 --2- 172.21.15.26:0/1009262717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c109210 0x7f305c109690 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.369+0000 7f3064200640 1 --2- 172.21.15.26:0/1009262717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f305c108020 0x7f305c108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.369+0000 7f3064200640 1 -- 172.21.15.26:0/1009262717 >> 172.21.15.26:0/1009262717 conn(0x7f305c076fc0 msgr2=0x7f305c079400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.369+0000 7f3064200640 1 -- 172.21.15.26:0/1009262717 shutdown_connections 2024-06-13T07:20:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.369+0000 7f3064200640 1 -- 172.21.15.26:0/1009262717 wait complete. 2024-06-13T07:20:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.370+0000 7f3064200640 1 Processor -- start 2024-06-13T07:20:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.370+0000 7f3064200640 1 -- start start 2024-06-13T07:20:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.371+0000 7f3064200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c108020 0x7f305c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.371+0000 7f3064200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f305c109210 0x7f305c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.371+0000 7f3064200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f305c109bd0 0x7f305c1a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.371+0000 7f3064200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f305c113d00 con 0x7f305c109210 2024-06-13T07:20:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.371+0000 7f3064200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f305c113b80 con 0x7f305c109bd0 2024-06-13T07:20:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.371+0000 7f3064200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f305c113e80 con 0x7f305c108020 2024-06-13T07:20:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3061800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c108020 0x7f305c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3061800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c108020 0x7f305c19e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48902/0 (socket says 172.21.15.26:48902) 2024-06-13T07:20:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3061800640 1 -- 172.21.15.26:0/2558519910 learned_addr learned my addr 172.21.15.26:0/2558519910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3060e00640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f305c109210 0x7f305c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3062200640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f305c109bd0 0x7f305c1a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3061800640 1 -- 172.21.15.26:0/2558519910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f305c109bd0 msgr2=0x7f305c1a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3061800640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f305c109bd0 0x7f305c1a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3061800640 1 -- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f305c109210 msgr2=0x7f305c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3061800640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f305c109210 0x7f305c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.372+0000 7f3061800640 1 -- 172.21.15.26:0/2558519910 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c1a5ec0 con 0x7f305c108020 2024-06-13T07:20:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.374+0000 7f3061800640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c108020 0x7f305c19e2a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f304800e9c0 tx=0x7f304800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.374+0000 7f305a000640 1 -- 172.21.15.26:0/2558519910 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304800cd80 con 0x7f305c108020 2024-06-13T07:20:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.374+0000 7f305a000640 1 -- 172.21.15.26:0/2558519910 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30480035f0 con 0x7f305c108020 2024-06-13T07:20:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.374+0000 7f3064200640 1 -- 172.21.15.26:0/2558519910 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c1a61b0 con 0x7f305c108020 2024-06-13T07:20:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.374+0000 7f3064200640 1 -- 172.21.15.26:0/2558519910 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c1a6720 con 0x7f305c108020 2024-06-13T07:20:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.375+0000 7f305a000640 1 -- 172.21.15.26:0/2558519910 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048010690 con 0x7f305c108020 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.380+0000 7f305a000640 1 -- 172.21.15.26:0/2558519910 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3048010830 con 0x7f305c108020 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.380+0000 7f3066ff8740 1 Processor -- start 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3066ff8740 1 -- start start 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e914020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e9145f0 0x564c0e9190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e91bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3066ff8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564c0e877420 con 0x564c0e9145f0 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3066ff8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564c0e81ec50 con 0x564c0e913c00 2024-06-13T07:20:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3066ff8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564c0e8733e0 con 0x564c0e919760 2024-06-13T07:20:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3060e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e91bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3060e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e91bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48910/0 (socket says 172.21.15.26:48910) 2024-06-13T07:20:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3060e00640 1 -- 172.21.15.26:0/4002126861 learned_addr learned my addr 172.21.15.26:0/4002126861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.382+0000 7f3062200640 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e914020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3061800640 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e9145f0 0x564c0e9190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3062200640 1 -- 172.21.15.26:0/4002126861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 msgr2=0x564c0e91bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3062200640 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e91bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3062200640 1 -- 172.21.15.26:0/4002126861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e9145f0 msgr2=0x564c0e9190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3062200640 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e9145f0 0x564c0e9190f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3062200640 1 -- 172.21.15.26:0/4002126861 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0e58b210 con 0x564c0e913c00 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3060e00640 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e91bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3061800640 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e9145f0 0x564c0e9190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.383+0000 7f3062200640 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e914020 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3054009f00 tx=0x7f3054011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86db899852e0e3cf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.384+0000 7f304ea00640 1 -- 172.21.15.26:0/4002126861 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054004d20 con 0x564c0e913c00 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.384+0000 7f304ea00640 1 -- 172.21.15.26:0/4002126861 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3054004030 con 0x564c0e913c00 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.384+0000 7f304ea00640 1 -- 172.21.15.26:0/4002126861 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054005280 con 0x564c0e913c00 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.384+0000 7f305a000640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30440a2d30 0x7f30440a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.384+0000 7f3061800640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30440a2d30 0x7f30440a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.384+0000 7f305a000640 1 -- 172.21.15.26:0/2558519910 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f30480cf830 con 0x7f305c108020 2024-06-13T07:20:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.384+0000 7f3066ff8740 1 -- 172.21.15.26:0/4002126861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 msgr2=0x564c0e914020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.384+0000 7f3066ff8740 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e914020 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3054009f00 tx=0x7f3054011810 comp rx=0 tx=0).stop 2024-06-13T07:20:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.385+0000 7f3066ff8740 1 -- 172.21.15.26:0/4002126861 shutdown_connections 2024-06-13T07:20:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.385+0000 7f3066ff8740 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e91bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.385+0000 7f3066ff8740 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e9145f0 0x564c0e9190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.385+0000 7f3066ff8740 1 --2- 172.21.15.26:0/4002126861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e914020 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.385+0000 7f3066ff8740 1 -- 172.21.15.26:0/4002126861 >> 172.21.15.26:0/4002126861 conn(0x564c0e8dbfb0 msgr2=0x564c0e90f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.385+0000 7f3066ff8740 1 -- 172.21.15.26:0/4002126861 shutdown_connections 2024-06-13T07:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.385+0000 7f3066ff8740 1 -- 172.21.15.26:0/4002126861 wait complete. 2024-06-13T07:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.385+0000 7f3061800640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30440a2d30 0x7f30440a51f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3048004770 tx=0x7f30480d3330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.386+0000 7f3066ff8740 1 -- start start 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.386+0000 7f3066ff8740 1 Processor -- start 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e91df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e9145f0 0x564c0e91e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e919760 0x564c0e9254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3066ff8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564c0e91c710 con 0x564c0e919760 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3066ff8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564c0e8320b0 con 0x564c0e913c00 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3066ff8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564c0e91c890 con 0x564c0e9145f0 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3066ff8740 10 client.0 _ll_register_callbacks cb 0x564c0e81df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3060e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e9145f0 0x564c0e91e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3060e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e9145f0 0x564c0e91e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48916/0 (socket says 172.21.15.26:48916) 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.387+0000 7f3060e00640 1 -- 172.21.15.26:0/2537502779 learned_addr learned my addr 172.21.15.26:0/2537502779 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3060e00640 1 -- 172.21.15.26:0/2537502779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 msgr2=0x564c0e91df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3062200640 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e91df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3060e00640 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e91df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3060e00640 1 -- 172.21.15.26:0/2537502779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e919760 msgr2=0x564c0e9254e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3060e00640 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e919760 0x564c0e9254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3060e00640 1 -- 172.21.15.26:0/2537502779 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c0e657620 con 0x564c0e9145f0 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3062200640 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e91df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3060e00640 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e9145f0 0x564c0e91e4a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f305002a910 tx=0x7f3050005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f94653578afdaf9f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3043400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3043400640 1 -- 172.21.15.26:0/2537502779 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050035040 con 0x564c0e9145f0 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.388+0000 7f3043400640 1 -- 172.21.15.26:0/2537502779 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f30500318b0 con 0x564c0e9145f0 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.389+0000 7f3066ff8740 1 -- 172.21.15.26:0/2537502779 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3054004030 con 0x564c0e9145f0 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.389+0000 7f3043400640 1 -- 172.21.15.26:0/2537502779 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3050031ab0 con 0x564c0e9145f0 2024-06-13T07:20:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.389+0000 7f3066ff8740 10 client.26956 fetch_fsmap learned FSMap version 47 2024-06-13T07:20:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.389+0000 7f3066ff8740 1 -- 172.21.15.26:0/2537502779 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564c0e928830 con 0x564c0e9145f0 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.389+0000 7f3043400640 1 -- 172.21.15.26:0/2537502779 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3054004030 con 0x564c0e9145f0 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.389+0000 7f3043400640 1 -- 172.21.15.26:0/2537502779 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f30500035f0 con 0x564c0e9145f0 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.390+0000 7f3066ff8740 10 client.26956 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.390+0000 7f3066ff8740 10 client.26956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.390+0000 7f3066ff8740 20 client.26956 populate_metadata read hostname 'smithi026' 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.390+0000 7f3066ff8740 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564c0e929f70 0x564c0e92c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.390+0000 7f3066ff8740 4 client.26956 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.390+0000 7f3066ff8740 1 -- 172.21.15.26:0/2537502779 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c0e884b50 con 0x564c0e929f70 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.391+0000 7f3062200640 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564c0e929f70 0x564c0e92c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.391+0000 7f3062200640 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564c0e929f70 0x564c0e92c3b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=249ac9fe4a1c0644 server_cookie=2a15340383bf81fa in_seq=0 out_seq=0 2024-06-13T07:20:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.392+0000 7f3043400640 10 client.26956 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.397+0000 7f3043400640 1 -- 172.21.15.26:0/2537502779 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564c0e884b50 con 0x564c0e929f70 2024-06-13T07:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.397+0000 7f3043400640 10 client.26956 handle_command_reply: tid=0 2024-06-13T07:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.397+0000 7f3066ff8740 1 client.26956 shutdown 2024-06-13T07:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3066ff8740 1 -- 172.21.15.26:0/2537502779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e9145f0 msgr2=0x564c0e91e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3066ff8740 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e9145f0 0x564c0e91e4a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f305002a910 tx=0x7f3050005270 comp rx=0 tx=0).stop 2024-06-13T07:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3066ff8740 1 -- 172.21.15.26:0/2537502779 shutdown_connections 2024-06-13T07:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3066ff8740 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564c0e929f70 0x564c0e92c3b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3066ff8740 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e919760 0x564c0e9254e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3066ff8740 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e9145f0 0x564c0e91e4a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3066ff8740 1 --2- 172.21.15.26:0/2537502779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e913c00 0x564c0e91df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3066ff8740 1 -- 172.21.15.26:0/2537502779 >> 172.21.15.26:0/2537502779 conn(0x564c0e8dbfb0 msgr2=0x564c0e90e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.398+0000 7f3043400640 0 client.26956 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.399+0000 7f3066ff8740 1 -- 172.21.15.26:0/2537502779 shutdown_connections 2024-06-13T07:20:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.399+0000 7f3066ff8740 1 -- 172.21.15.26:0/2537502779 wait complete. 2024-06-13T07:20:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.399+0000 7f3066ff8740 20 client.26956 trim_cache size 0 max 16384 2024-06-13T07:20:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3066ff8740 1 Processor -- start 2024-06-13T07:20:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3066ff8740 1 -- start start 2024-06-13T07:20:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 0x564c0e91d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e9254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e90f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3066ff8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f303c0125c0 con 0x564c0e913c00 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3066ff8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f303c00fe90 con 0x564c0e9145f0 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3066ff8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f303c00dce0 con 0x564c0e919760 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3060e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e9254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3060e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e9254a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:42498/0 (socket says 172.21.15.26:42498) 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3060e00640 1 -- 172.21.15.26:0/1209049115 learned_addr learned my addr 172.21.15.26:0/1209049115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.434+0000 7f3062200640 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 0x564c0e91d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.435+0000 7f3062200640 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e90f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.435+0000 7f3060e00640 1 -- 172.21.15.26:0/1209049115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 msgr2=0x564c0e90f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.435+0000 7f3060e00640 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e90f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.435+0000 7f3060e00640 1 -- 172.21.15.26:0/1209049115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 msgr2=0x564c0e91d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.435+0000 7f3060e00640 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 0x564c0e91d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.435+0000 7f3060e00640 1 -- 172.21.15.26:0/1209049115 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0e91cff0 con 0x564c0e9145f0 2024-06-13T07:20:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.435+0000 7f3060e00640 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e9254a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3050004600 tx=0x7f3050009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c61f468a46265d9a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.435+0000 7f304e000640 1 -- 172.21.15.26:0/1209049115 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050035070 con 0x564c0e9145f0 2024-06-13T07:20:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.436+0000 7f304e000640 1 -- 172.21.15.26:0/1209049115 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3050040070 con 0x564c0e9145f0 2024-06-13T07:20:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.436+0000 7f304e000640 1 -- 172.21.15.26:0/1209049115 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500316b0 con 0x564c0e9145f0 2024-06-13T07:20:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.436+0000 7f3066ff8740 1 -- 172.21.15.26:0/1209049115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 msgr2=0x564c0e9254a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.436+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e9254a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3050004600 tx=0x7f3050009780 comp rx=0 tx=0).stop 2024-06-13T07:20:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.437+0000 7f3066ff8740 1 -- 172.21.15.26:0/1209049115 shutdown_connections 2024-06-13T07:20:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.437+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e9254a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.437+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e90f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.437+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1209049115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 0x564c0e91d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.437+0000 7f3066ff8740 1 -- 172.21.15.26:0/1209049115 >> 172.21.15.26:0/1209049115 conn(0x564c0e8dbfb0 msgr2=0x564c0e91d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.437+0000 7f3066ff8740 1 -- 172.21.15.26:0/1209049115 shutdown_connections 2024-06-13T07:20:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.437+0000 7f3066ff8740 1 -- 172.21.15.26:0/1209049115 wait complete. 2024-06-13T07:20:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.441+0000 7f3066ff8740 1 -- start start 2024-06-13T07:20:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.442+0000 7f3066ff8740 1 Processor -- start 2024-06-13T07:20:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.443+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 0x564c0e9223e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e922920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3066ff8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e992c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3066ff8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f303c00e350 con 0x564c0e913c00 2024-06-13T07:20:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3066ff8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f303c00f7c0 con 0x564c0e9145f0 2024-06-13T07:20:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3066ff8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f303c00ea20 con 0x564c0e919760 2024-06-13T07:20:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3066ff8740 10 client.0 _ll_register_callbacks cb 0x7f30540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:20:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3060e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e922920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3060e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e922920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:42506/0 (socket says 172.21.15.26:42506) 2024-06-13T07:20:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.446+0000 7f3060e00640 1 -- 172.21.15.26:0/1096763438 learned_addr learned my addr 172.21.15.26:0/1096763438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.447+0000 7f3060e00640 1 -- 172.21.15.26:0/1096763438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 msgr2=0x564c0e992c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.447+0000 7f3060e00640 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e992c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.447+0000 7f3060e00640 1 -- 172.21.15.26:0/1096763438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 msgr2=0x564c0e9223e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.447+0000 7f3060e00640 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 0x564c0e9223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.447+0000 7f3060e00640 1 -- 172.21.15.26:0/1096763438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c0e9931f0 con 0x564c0e9145f0 2024-06-13T07:20:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.449+0000 7f3060e00640 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e922920 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3050005c40 tx=0x7f3050003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a140593c8a467395 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.449+0000 7f3043400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:20:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.449+0000 7f3043400640 1 -- 172.21.15.26:0/1096763438 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050035040 con 0x564c0e9145f0 2024-06-13T07:20:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.450+0000 7f3066ff8740 1 -- 172.21.15.26:0/1096763438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3050040070 con 0x564c0e9145f0 2024-06-13T07:20:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.451+0000 7f3043400640 1 -- 172.21.15.26:0/1096763438 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f30500048c0 con 0x564c0e9145f0 2024-06-13T07:20:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.451+0000 7f3043400640 1 -- 172.21.15.26:0/1096763438 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3050066050 con 0x564c0e9145f0 2024-06-13T07:20:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.451+0000 7f3066ff8740 10 client.26903 fetch_fsmap learned FSMap version 47 2024-06-13T07:20:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.451+0000 7f3066ff8740 1 -- 172.21.15.26:0/1096763438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564c0e9944e0 con 0x564c0e9145f0 2024-06-13T07:20:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.451+0000 7f3043400640 1 -- 172.21.15.26:0/1096763438 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3050040070 con 0x564c0e9145f0 2024-06-13T07:20:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.453+0000 7f3043400640 1 -- 172.21.15.26:0/1096763438 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f3050004c20 con 0x564c0e9145f0 2024-06-13T07:20:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.453+0000 7f3066ff8740 10 client.26903 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:20:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.453+0000 7f3066ff8740 10 client.26903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:20:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.453+0000 7f3066ff8740 20 client.26903 populate_metadata read hostname 'smithi026' 2024-06-13T07:20:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.453+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564c0e994d70 0x564c0e997190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.453+0000 7f3066ff8740 4 client.26903 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:20:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.453+0000 7f3066ff8740 1 -- 172.21.15.26:0/1096763438 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c0e87eea0 con 0x564c0e994d70 2024-06-13T07:20:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.454+0000 7f3062200640 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564c0e994d70 0x564c0e997190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.455+0000 7f3062200640 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564c0e994d70 0x564c0e997190 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a2e17b247aedad server_cookie=43bbdde4b6dc22a in_seq=0 out_seq=0 2024-06-13T07:20:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:10.455+0000 7f3043400640 10 client.26903 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:10 smithi135 ceph-mon[36276]: pgmap v1099: 129 pgs: 129 active+clean; 672 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-06-13T07:20:10.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:10 smithi136 ceph-mon[37229]: pgmap v1099: 129 pgs: 129 active+clean; 672 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-06-13T07:20:10.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:10 smithi026 ceph-mon[32041]: pgmap v1099: 129 pgs: 129 active+clean; 672 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-06-13T07:20:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:20:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:20:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:20:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3043400640 1 -- 172.21.15.26:0/1096763438 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564c0e87eea0 con 0x564c0e994d70 2024-06-13T07:20:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3043400640 10 client.26903 handle_command_reply: tid=0 2024-06-13T07:20:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 client.26903 shutdown 2024-06-13T07:20:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 -- 172.21.15.26:0/1096763438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 msgr2=0x564c0e922920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e922920 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3050005c40 tx=0x7f3050003e10 comp rx=0 tx=0).stop 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 -- 172.21.15.26:0/1096763438 shutdown_connections 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x564c0e994d70 0x564c0e997190 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c0e9145f0 0x564c0e922920 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c0e919760 0x564c0e992c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 --2- 172.21.15.26:0/1096763438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c0e913c00 0x564c0e9223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3066ff8740 1 -- 172.21.15.26:0/1096763438 >> 172.21.15.26:0/1096763438 conn(0x564c0e8dbfb0 msgr2=0x564c0e90f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.010+0000 7f3043400640 0 client.26903 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.011+0000 7f3066ff8740 1 -- 172.21.15.26:0/1096763438 shutdown_connections 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.011+0000 7f3066ff8740 1 -- 172.21.15.26:0/1096763438 wait complete. 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.011+0000 7f3066ff8740 20 client.26903 trim_cache size 0 max 16384 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.012+0000 7f3064200640 1 -- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30440a2d30 msgr2=0x7f30440a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.012+0000 7f3064200640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30440a2d30 0x7f30440a51f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3048004770 tx=0x7f30480d3330 comp rx=0 tx=0).stop 2024-06-13T07:20:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.012+0000 7f3064200640 1 -- 172.21.15.26:0/2558519910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c108020 msgr2=0x7f305c19e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.012+0000 7f3064200640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c108020 0x7f305c19e2a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f304800e9c0 tx=0x7f304800ee80 comp rx=0 tx=0).stop 2024-06-13T07:20:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.013+0000 7f3064200640 1 -- 172.21.15.26:0/2558519910 shutdown_connections 2024-06-13T07:20:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.013+0000 7f3064200640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30440a2d30 0x7f30440a51f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.013+0000 7f3064200640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f305c109bd0 0x7f305c1a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.013+0000 7f3064200640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f305c109210 0x7f305c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.013+0000 7f3064200640 1 --2- 172.21.15.26:0/2558519910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f305c108020 0x7f305c19e2a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.013+0000 7f3064200640 1 -- 172.21.15.26:0/2558519910 >> 172.21.15.26:0/2558519910 conn(0x7f305c076fc0 msgr2=0x7f305c10c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.013+0000 7f3064200640 1 -- 172.21.15.26:0/2558519910 shutdown_connections 2024-06-13T07:20:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:11.013+0000 7f3064200640 1 -- 172.21.15.26:0/2558519910 wait complete. 2024-06-13T07:20:11.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 196910 27.15 MB/sec execute 264 sec latency 411.118 ms 2024-06-13T07:20:12.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 197563 27.18 MB/sec execute 265 sec latency 73.669 ms 2024-06-13T07:20:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:12 smithi135 ceph-mon[36276]: pgmap v1100: 129 pgs: 129 active+clean; 676 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 669 op/s 2024-06-13T07:20:12.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:12 smithi136 ceph-mon[37229]: pgmap v1100: 129 pgs: 129 active+clean; 676 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 669 op/s 2024-06-13T07:20:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:12 smithi026 ceph-mon[32041]: pgmap v1100: 129 pgs: 129 active+clean; 676 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 669 op/s 2024-06-13T07:20:13.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 198362 27.22 MB/sec execute 266 sec latency 205.775 ms 2024-06-13T07:20:14.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 198990 27.28 MB/sec execute 267 sec latency 55.618 ms 2024-06-13T07:20:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:14 smithi135 ceph-mon[36276]: pgmap v1101: 129 pgs: 129 active+clean; 676 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 638 op/s 2024-06-13T07:20:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:14 smithi136 ceph-mon[37229]: pgmap v1101: 129 pgs: 129 active+clean; 676 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 638 op/s 2024-06-13T07:20:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:14 smithi026 ceph-mon[32041]: pgmap v1101: 129 pgs: 129 active+clean; 676 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 638 op/s 2024-06-13T07:20:15.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 199360 27.28 MB/sec execute 268 sec latency 244.400 ms 2024-06-13T07:20:16.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 199668 27.23 MB/sec execute 269 sec latency 384.033 ms 2024-06-13T07:20:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:16 smithi135 ceph-mon[36276]: pgmap v1102: 129 pgs: 129 active+clean; 684 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.4 MiB/s wr, 616 op/s 2024-06-13T07:20:16.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:16 smithi136 ceph-mon[37229]: pgmap v1102: 129 pgs: 129 active+clean; 684 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.4 MiB/s wr, 616 op/s 2024-06-13T07:20:16.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:16 smithi026 ceph-mon[32041]: pgmap v1102: 129 pgs: 129 active+clean; 684 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.4 MiB/s wr, 616 op/s 2024-06-13T07:20:17.193 INFO:tasks.workunit.client.0.smithi026.stdout: 10 200061 27.19 MB/sec execute 270 sec latency 119.088 ms 2024-06-13T07:20:18.194 INFO:tasks.workunit.client.0.smithi026.stdout: 10 200604 27.19 MB/sec execute 271 sec latency 95.903 ms 2024-06-13T07:20:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:18 smithi136 ceph-mon[37229]: pgmap v1103: 129 pgs: 129 active+clean; 690 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.5 MiB/s wr, 566 op/s 2024-06-13T07:20:18.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:18 smithi026 ceph-mon[32041]: pgmap v1103: 129 pgs: 129 active+clean; 690 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.5 MiB/s wr, 566 op/s 2024-06-13T07:20:18.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:18 smithi135 ceph-mon[36276]: pgmap v1103: 129 pgs: 129 active+clean; 690 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.5 MiB/s wr, 566 op/s 2024-06-13T07:20:19.194 INFO:tasks.workunit.client.0.smithi026.stdout: 10 201264 27.20 MB/sec execute 272 sec latency 56.285 ms 2024-06-13T07:20:20.194 INFO:tasks.workunit.client.0.smithi026.stdout: 10 201948 27.27 MB/sec execute 273 sec latency 94.071 ms 2024-06-13T07:20:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:20 smithi136 ceph-mon[37229]: pgmap v1104: 129 pgs: 129 active+clean; 697 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 621 op/s 2024-06-13T07:20:20.749 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:20 smithi026 ceph-mon[32041]: pgmap v1104: 129 pgs: 129 active+clean; 697 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 621 op/s 2024-06-13T07:20:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:20 smithi135 ceph-mon[36276]: pgmap v1104: 129 pgs: 129 active+clean; 697 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 621 op/s 2024-06-13T07:20:21.194 INFO:tasks.workunit.client.0.smithi026.stdout: 10 201986 27.18 MB/sec execute 274 sec latency 1031.361 ms 2024-06-13T07:20:22.194 INFO:tasks.workunit.client.0.smithi026.stdout: 10 202149 27.10 MB/sec execute 275 sec latency 1118.405 ms 2024-06-13T07:20:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:22 smithi136 ceph-mon[37229]: pgmap v1105: 129 pgs: 129 active+clean; 698 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.6 MiB/s wr, 502 op/s 2024-06-13T07:20:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:22 smithi026 ceph-mon[32041]: pgmap v1105: 129 pgs: 129 active+clean; 698 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.6 MiB/s wr, 502 op/s 2024-06-13T07:20:22.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:22 smithi135 ceph-mon[36276]: pgmap v1105: 129 pgs: 129 active+clean; 698 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.6 MiB/s wr, 502 op/s 2024-06-13T07:20:23.194 INFO:tasks.workunit.client.0.smithi026.stdout: 10 202476 27.06 MB/sec execute 276 sec latency 143.919 ms 2024-06-13T07:20:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.189+0000 7f9786e00640 1 -- 172.21.15.26:0/703684456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9780108a30 msgr2=0x7f978010ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.189+0000 7f9786e00640 1 --2- 172.21.15.26:0/703684456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9780108a30 0x7f978010ae20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f977400fb10 tx=0x7f977402a160 comp rx=0 tx=0).stop 2024-06-13T07:20:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.189+0000 7f9786e00640 1 -- 172.21.15.26:0/703684456 shutdown_connections 2024-06-13T07:20:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.189+0000 7f9786e00640 1 --2- 172.21.15.26:0/703684456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f978010ded0 0x7f9780110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.189+0000 7f9786e00640 1 --2- 172.21.15.26:0/703684456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f978010b3f0 0x7f978010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.189+0000 7f9786e00640 1 --2- 172.21.15.26:0/703684456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9780108a30 0x7f978010ae20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.189+0000 7f9786e00640 1 -- 172.21.15.26:0/703684456 >> 172.21.15.26:0/703684456 conn(0x7f978006e840 msgr2=0x7f9780070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.190+0000 7f9786e00640 1 -- 172.21.15.26:0/703684456 shutdown_connections 2024-06-13T07:20:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.190+0000 7f9786e00640 1 -- 172.21.15.26:0/703684456 wait complete. 2024-06-13T07:20:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.191+0000 7f9786e00640 1 Processor -- start 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786e00640 1 -- start start 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9780108a30 0x7f978019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f978010b3f0 0x7f978019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f978010ded0 0x7f97801a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9780113af0 con 0x7f9780108a30 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9780113970 con 0x7f978010b3f0 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9780113c70 con 0x7f978010ded0 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f978010ded0 0x7f97801a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f978010ded0 0x7f97801a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55646/0 (socket says 172.21.15.26:55646) 2024-06-13T07:20:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.192+0000 7f9786400640 1 -- 172.21.15.26:0/374228710 learned_addr learned my addr 172.21.15.26:0/374228710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9785000640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f978010b3f0 0x7f978019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9785a00640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9780108a30 0x7f978019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9786400640 1 -- 172.21.15.26:0/374228710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f978010b3f0 msgr2=0x7f978019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9786400640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f978010b3f0 0x7f978019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9786400640 1 -- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9780108a30 msgr2=0x7f978019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9786400640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9780108a30 0x7f978019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9786400640 1 -- 172.21.15.26:0/374228710 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97801a5d10 con 0x7f978010ded0 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9785a00640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9780108a30 0x7f978019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f9786400640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f978010ded0 0x7f97801a56a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f977800eb10 tx=0x7f977800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.193+0000 7f977e000640 1 -- 172.21.15.26:0/374228710 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778004b20 con 0x7f978010ded0 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.194+0000 7f977e000640 1 -- 172.21.15.26:0/374228710 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9778004cc0 con 0x7f978010ded0 2024-06-13T07:20:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.194+0000 7f977e000640 1 -- 172.21.15.26:0/374228710 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778010760 con 0x7f978010ded0 2024-06-13T07:20:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.194+0000 7f9786e00640 1 -- 172.21.15.26:0/374228710 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97801a5fa0 con 0x7f978010ded0 2024-06-13T07:20:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.194+0000 7f9786e00640 1 -- 172.21.15.26:0/374228710 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97801a6410 con 0x7f978010ded0 2024-06-13T07:20:24.195 INFO:tasks.workunit.client.0.smithi026.stdout: 10 202758 27.03 MB/sec execute 277 sec latency 205.315 ms 2024-06-13T07:20:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.199+0000 7f977e000640 1 -- 172.21.15.26:0/374228710 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9778005020 con 0x7f978010ded0 2024-06-13T07:20:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.199+0000 7f978b209740 1 Processor -- start 2024-06-13T07:20:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.199+0000 7f978b209740 1 -- start start 2024-06-13T07:20:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f978b209740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f5698111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f978b209740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f5698162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f978b209740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f569818db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f978b209740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f569774420 con 0x55f569810dc0 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f977e000640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97680a2c40 0x7f97680a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f978b209740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f56971bc50 con 0x55f569816920 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f978b209740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f5697703e0 con 0x55f5698117b0 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f9785000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f5698111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f9785a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f5698162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f9785000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f569818db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f9785a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f5698162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55652/0 (socket says 172.21.15.26:55652) 2024-06-13T07:20:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f9785a00640 1 -- 172.21.15.26:0/3417151837 learned_addr learned my addr 172.21.15.26:0/3417151837 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.200+0000 7f977e000640 1 -- 172.21.15.26:0/374228710 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f97780cfa40 con 0x7f978010ded0 2024-06-13T07:20:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.201+0000 7f9786400640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97680a2c40 0x7f97680a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.201+0000 7f9785000640 1 -- 172.21.15.26:0/3417151837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 msgr2=0x55f5698162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.201+0000 7f9785000640 1 --2- 172.21.15.26:0/3417151837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f5698162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.201+0000 7f9785000640 1 -- 172.21.15.26:0/3417151837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 msgr2=0x55f5698111e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.201+0000 7f9785000640 1 --2- 172.21.15.26:0/3417151837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f5698111e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.201+0000 7f9785000640 1 -- 172.21.15.26:0/3417151837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f569488210 con 0x55f569816920 2024-06-13T07:20:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.201+0000 7f9785000640 1 --2- 172.21.15.26:0/3417151837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f569818db0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f978019f600 tx=0x7f976c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc2c8cde50f9f5bd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.201+0000 7f9772a00640 1 -- 172.21.15.26:0/3417151837 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c01c040 con 0x55f569816920 2024-06-13T07:20:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.202+0000 7f9772a00640 1 -- 172.21.15.26:0/3417151837 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c009d20 con 0x55f569816920 2024-06-13T07:20:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.202+0000 7f9772a00640 1 -- 172.21.15.26:0/3417151837 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c023d40 con 0x55f569816920 2024-06-13T07:20:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.202+0000 7f978b209740 1 -- 172.21.15.26:0/3417151837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 msgr2=0x55f569818db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.202+0000 7f9786400640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97680a2c40 0x7f97680a5100 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f977809bbb0 tx=0x7f97780058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.202+0000 7f978b209740 1 --2- 172.21.15.26:0/3417151837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f569818db0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f978019f600 tx=0x7f976c004730 comp rx=0 tx=0).stop 2024-06-13T07:20:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.203+0000 7f978b209740 1 -- 172.21.15.26:0/3417151837 shutdown_connections 2024-06-13T07:20:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.203+0000 7f978b209740 1 --2- 172.21.15.26:0/3417151837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f569818db0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.203+0000 7f978b209740 1 --2- 172.21.15.26:0/3417151837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f5698162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.203+0000 7f978b209740 1 --2- 172.21.15.26:0/3417151837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f5698111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.203+0000 7f978b209740 1 -- 172.21.15.26:0/3417151837 >> 172.21.15.26:0/3417151837 conn(0x55f5697d8fb0 msgr2=0x55f56980c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.203+0000 7f978b209740 1 -- 172.21.15.26:0/3417151837 shutdown_connections 2024-06-13T07:20:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.203+0000 7f978b209740 1 -- 172.21.15.26:0/3417151837 wait complete. 2024-06-13T07:20:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.203+0000 7f978b209740 1 -- start start 2024-06-13T07:20:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.204+0000 7f978b209740 1 Processor -- start 2024-06-13T07:20:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.204+0000 7f978b209740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.204+0000 7f978b209740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f56981b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.204+0000 7f978b209740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f5698226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.204+0000 7f978b209740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5698198d0 con 0x55f569810dc0 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.204+0000 7f978b209740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f56972f0b0 con 0x55f569816920 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.204+0000 7f978b209740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f569819a50 con 0x55f5698117b0 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.204+0000 7f978b209740 10 client.0 _ll_register_callbacks cb 0x55f56971af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54404/0 (socket says 172.21.15.26:54404) 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785000640 1 -- 172.21.15.26:0/3261853077 learned_addr learned my addr 172.21.15.26:0/3261853077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785a00640 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f56981b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785000640 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f5698226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785a00640 1 -- 172.21.15.26:0/3261853077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 msgr2=0x55f5698226d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785a00640 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f5698226d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785a00640 1 -- 172.21.15.26:0/3261853077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 msgr2=0x55f56981b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785a00640 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.205+0000 7f9785a00640 1 -- 172.21.15.26:0/3261853077 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f569554620 con 0x55f5698117b0 2024-06-13T07:20:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.206+0000 7f9785a00640 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f56981b690 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f977400fa60 tx=0x7f977400f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785207af41a6cae5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.206+0000 7f9767400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:20:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.206+0000 7f9767400640 1 -- 172.21.15.26:0/3261853077 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774040070 con 0x55f5698117b0 2024-06-13T07:20:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.206+0000 7f978b209740 1 -- 172.21.15.26:0/3261853077 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976c009d20 con 0x55f5698117b0 2024-06-13T07:20:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.206+0000 7f9767400640 1 -- 172.21.15.26:0/3261853077 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f977400f600 con 0x55f5698117b0 2024-06-13T07:20:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.206+0000 7f9767400640 1 -- 172.21.15.26:0/3261853077 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f977404c050 con 0x55f5698117b0 2024-06-13T07:20:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.206+0000 7f978b209740 10 client.26968 fetch_fsmap learned FSMap version 47 2024-06-13T07:20:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.206+0000 7f978b209740 1 -- 172.21.15.26:0/3261853077 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f569825ab0 con 0x55f5698117b0 2024-06-13T07:20:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.207+0000 7f9767400640 1 -- 172.21.15.26:0/3261853077 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c009d20 con 0x55f5698117b0 2024-06-13T07:20:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.207+0000 7f9767400640 1 -- 172.21.15.26:0/3261853077 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f977403c7e0 con 0x55f5698117b0 2024-06-13T07:20:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.207+0000 7f978b209740 10 client.26968 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:20:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.207+0000 7f978b209740 10 client.26968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:20:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.207+0000 7f978b209740 20 client.26968 populate_metadata read hostname 'smithi026' 2024-06-13T07:20:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.207+0000 7f978b209740 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f5698271a0 0x55f569829590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.207+0000 7f978b209740 4 client.26968 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:20:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.207+0000 7f978b209740 1 -- 172.21.15.26:0/3261853077 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f569781b50 con 0x55f5698271a0 2024-06-13T07:20:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.208+0000 7f9785000640 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f5698271a0 0x55f569829590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.209+0000 7f9785000640 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f5698271a0 0x55f569829590 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912afaa674b20814 server_cookie=a775d11d32130258 in_seq=0 out_seq=0 2024-06-13T07:20:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.209+0000 7f9767400640 10 client.26968 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.215+0000 7f9767400640 1 -- 172.21.15.26:0/3261853077 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f569781b50 con 0x55f5698271a0 2024-06-13T07:20:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.215+0000 7f9767400640 10 client.26968 handle_command_reply: tid=0 2024-06-13T07:20:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.215+0000 7f978b209740 1 client.26968 shutdown 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f978b209740 1 -- 172.21.15.26:0/3261853077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 msgr2=0x55f56981b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f978b209740 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f56981b690 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f977400fa60 tx=0x7f977400f280 comp rx=0 tx=0).stop 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f978b209740 1 -- 172.21.15.26:0/3261853077 shutdown_connections 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f978b209740 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f5698271a0 0x55f569829590 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f978b209740 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569816920 0x55f5698226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f978b209740 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f5698117b0 0x55f56981b690 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f977400fa60 tx=0x7f977400f280 comp rx=0 tx=0).stop 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f978b209740 1 --2- 172.21.15.26:0/3261853077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f978b209740 1 -- 172.21.15.26:0/3261853077 >> 172.21.15.26:0/3261853077 conn(0x55f5697d8fb0 msgr2=0x55f56980b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.216+0000 7f9767400640 0 client.26968 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.217+0000 7f978b209740 1 -- 172.21.15.26:0/3261853077 shutdown_connections 2024-06-13T07:20:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.218+0000 7f978b209740 1 -- 172.21.15.26:0/3261853077 wait complete. 2024-06-13T07:20:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.218+0000 7f978b209740 20 client.26968 trim_cache size 0 max 16384 2024-06-13T07:20:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.245+0000 7f978b209740 1 Processor -- start 2024-06-13T07:20:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.245+0000 7f978b209740 1 -- start start 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f978b209740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f978b209740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569816920 0x55f569822690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f978b209740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f56981b690 0x55f569819df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f978b209740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b3460 con 0x55f569816920 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f978b209740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b0d30 con 0x55f569810dc0 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f978b209740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f97801aeb80 con 0x55f56981b690 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f9785000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f9785a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569816920 0x55f569822690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f9785000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38064/0 (socket says 172.21.15.26:38064) 2024-06-13T07:20:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f9785000640 1 -- 172.21.15.26:0/3416582639 learned_addr learned my addr 172.21.15.26:0/3416582639 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.246+0000 7f9785000640 1 --2- 172.21.15.26:0/3416582639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f56981b690 0x55f569819df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.247+0000 7f9785a00640 1 -- 172.21.15.26:0/3416582639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f56981b690 msgr2=0x55f569819df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.247+0000 7f9785a00640 1 --2- 172.21.15.26:0/3416582639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f56981b690 0x55f569819df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.247+0000 7f9785a00640 1 -- 172.21.15.26:0/3416582639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569810dc0 msgr2=0x55f56981b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.247+0000 7f9785a00640 1 --2- 172.21.15.26:0/3416582639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.247+0000 7f9785a00640 1 -- 172.21.15.26:0/3416582639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5698081f0 con 0x55f569816920 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.247+0000 7f9785a00640 1 --2- 172.21.15.26:0/3416582639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569816920 0x55f569822690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f977402c000 tx=0x7f977402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83914dea62e19623 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.248+0000 7f9772000640 1 -- 172.21.15.26:0/3416582639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774040070 con 0x55f569816920 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.248+0000 7f9772000640 1 -- 172.21.15.26:0/3416582639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977402e040 con 0x55f569816920 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.248+0000 7f9772000640 1 -- 172.21.15.26:0/3416582639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977403c6d0 con 0x55f569816920 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.249+0000 7f978b209740 1 -- 172.21.15.26:0/3416582639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569816920 msgr2=0x55f569822690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.249+0000 7f978b209740 1 --2- 172.21.15.26:0/3416582639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569816920 0x55f569822690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f977402c000 tx=0x7f977402a160 comp rx=0 tx=0).stop 2024-06-13T07:20:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.249+0000 7f978b209740 1 -- 172.21.15.26:0/3416582639 shutdown_connections 2024-06-13T07:20:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.249+0000 7f978b209740 1 --2- 172.21.15.26:0/3416582639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f56981b690 0x55f569819df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.249+0000 7f978b209740 1 --2- 172.21.15.26:0/3416582639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569816920 0x55f569822690 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.249+0000 7f978b209740 1 --2- 172.21.15.26:0/3416582639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f569810dc0 0x55f56981b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.249+0000 7f978b209740 1 -- 172.21.15.26:0/3416582639 >> 172.21.15.26:0/3416582639 conn(0x55f5697d8fb0 msgr2=0x55f56980c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.249+0000 7f978b209740 1 -- 172.21.15.26:0/3416582639 shutdown_connections 2024-06-13T07:20:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.250+0000 7f978b209740 1 -- 172.21.15.26:0/3416582639 wait complete. 2024-06-13T07:20:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.250+0000 7f978b209740 1 -- start start 2024-06-13T07:20:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.250+0000 7f978b209740 1 Processor -- start 2024-06-13T07:20:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f978b209740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f569810dc0 0x55f56981f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f978b209740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f56981b690 0x55f56981fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f978b209740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569820c70 0x55f5698210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f978b209740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97801af1f0 con 0x55f569820c70 2024-06-13T07:20:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f978b209740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b0660 con 0x55f56981b690 2024-06-13T07:20:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f978b209740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f97801af8c0 con 0x55f569810dc0 2024-06-13T07:20:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f978b209740 10 client.0 _ll_register_callbacks cb 0x7f976c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f9785a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f56981b690 0x55f56981fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f9785a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f56981b690 0x55f56981fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:38074/0 (socket says 172.21.15.26:38074) 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f9785a00640 1 -- 172.21.15.26:0/3131706858 learned_addr learned my addr 172.21.15.26:0/3131706858 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f9785000640 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f569810dc0 0x55f56981f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f9785a00640 1 -- 172.21.15.26:0/3131706858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f569810dc0 msgr2=0x55f56981f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f9785a00640 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f569810dc0 0x55f56981f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f9785000640 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569820c70 0x55f5698210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.251+0000 7f9785a00640 1 -- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569820c70 msgr2=0x55f5698210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.252+0000 7f9785a00640 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569820c70 0x55f5698210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.252+0000 7f9785a00640 1 -- 172.21.15.26:0/3131706858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f569821630 con 0x55f56981b690 2024-06-13T07:20:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.252+0000 7f9785000640 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569820c70 0x55f5698210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:20:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.252+0000 7f9785a00640 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f56981b690 0x55f56981fc60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f977403e040 tx=0x7f9774005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6b21ad55359c88 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.252+0000 7f9767400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:20:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.252+0000 7f9767400640 1 -- 172.21.15.26:0/3131706858 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977405f060 con 0x55f56981b690 2024-06-13T07:20:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.252+0000 7f9767400640 1 -- 172.21.15.26:0/3131706858 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9774037040 con 0x55f56981b690 2024-06-13T07:20:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.252+0000 7f978b209740 1 -- 172.21.15.26:0/3131706858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f977402e040 con 0x55f56981b690 2024-06-13T07:20:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.253+0000 7f9767400640 1 -- 172.21.15.26:0/3131706858 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f977405f060 con 0x55f56981b690 2024-06-13T07:20:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.253+0000 7f978b209740 10 client.26921 fetch_fsmap learned FSMap version 47 2024-06-13T07:20:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.253+0000 7f978b209740 1 -- 172.21.15.26:0/3131706858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f569822940 con 0x55f56981b690 2024-06-13T07:20:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.253+0000 7f9767400640 1 -- 172.21.15.26:0/3131706858 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977402e040 con 0x55f56981b690 2024-06-13T07:20:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.254+0000 7f9767400640 1 -- 172.21.15.26:0/3131706858 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f977400ebb0 con 0x55f56981b690 2024-06-13T07:20:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.254+0000 7f978b209740 10 client.26921 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:20:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.254+0000 7f978b209740 10 client.26921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:20:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.254+0000 7f978b209740 20 client.26921 populate_metadata read hostname 'smithi026' 2024-06-13T07:20:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.254+0000 7f978b209740 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f569892410 0x55f569894800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.254+0000 7f978b209740 4 client.26921 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:20:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.254+0000 7f978b209740 1 -- 172.21.15.26:0/3131706858 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f56977bea0 con 0x55f569892410 2024-06-13T07:20:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.255+0000 7f9785000640 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f569892410 0x55f569894800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.256+0000 7f9785000640 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f569892410 0x55f569894800 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103f7e1a2fe213d5 server_cookie=78801a626289cbd2 in_seq=0 out_seq=0 2024-06-13T07:20:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.256+0000 7f9767400640 10 client.26921 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:24.630 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:24 smithi136 ceph-mon[37229]: pgmap v1106: 129 pgs: 129 active+clean; 701 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-06-13T07:20:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.637+0000 7f9767400640 1 -- 172.21.15.26:0/3131706858 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f56977bea0 con 0x55f569892410 2024-06-13T07:20:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.637+0000 7f9767400640 10 client.26921 handle_command_reply: tid=0 2024-06-13T07:20:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.637+0000 7f978b209740 1 client.26921 shutdown 2024-06-13T07:20:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f978b209740 1 -- 172.21.15.26:0/3131706858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f56981b690 msgr2=0x55f56981fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f978b209740 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f56981b690 0x55f56981fc60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f977403e040 tx=0x7f9774005d50 comp rx=0 tx=0).stop 2024-06-13T07:20:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f978b209740 1 -- 172.21.15.26:0/3131706858 shutdown_connections 2024-06-13T07:20:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f978b209740 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55f569892410 0x55f569894800 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f978b209740 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f56981b690 0x55f56981fc60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f978b209740 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f569820c70 0x55f5698210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f978b209740 1 --2- 172.21.15.26:0/3131706858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f569810dc0 0x55f56981f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f978b209740 1 -- 172.21.15.26:0/3131706858 >> 172.21.15.26:0/3131706858 conn(0x55f5697d8fb0 msgr2=0x55f569819df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.638+0000 7f9767400640 0 client.26921 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.639+0000 7f978b209740 1 -- 172.21.15.26:0/3131706858 shutdown_connections 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.639+0000 7f978b209740 1 -- 172.21.15.26:0/3131706858 wait complete. 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.639+0000 7f978b209740 20 client.26921 trim_cache size 0 max 16384 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:20:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:20:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:20:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.641+0000 7f9786e00640 1 -- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97680a2c40 msgr2=0x7f97680a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.641+0000 7f9786e00640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97680a2c40 0x7f97680a5100 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f977809bbb0 tx=0x7f97780058d0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.641+0000 7f9786e00640 1 -- 172.21.15.26:0/374228710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f978010ded0 msgr2=0x7f97801a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.641+0000 7f9786e00640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f978010ded0 0x7f97801a56a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f977800eb10 tx=0x7f977800c2d0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.642+0000 7f9786e00640 1 -- 172.21.15.26:0/374228710 shutdown_connections 2024-06-13T07:20:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.642+0000 7f9786e00640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f97680a2c40 0x7f97680a5100 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.642+0000 7f9786e00640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f978010b3f0 0x7f978019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.642+0000 7f9786e00640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f978010ded0 0x7f97801a56a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.642+0000 7f9786e00640 1 --2- 172.21.15.26:0/374228710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9780108a30 0x7f978019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.642+0000 7f9786e00640 1 -- 172.21.15.26:0/374228710 >> 172.21.15.26:0/374228710 conn(0x7f978006e840 msgr2=0x7f9780108e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.643+0000 7f9786e00640 1 -- 172.21.15.26:0/374228710 shutdown_connections 2024-06-13T07:20:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:24.643+0000 7f9786e00640 1 -- 172.21.15.26:0/374228710 wait complete. 2024-06-13T07:20:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:24 smithi026 ceph-mon[32041]: pgmap v1106: 129 pgs: 129 active+clean; 701 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-06-13T07:20:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:24 smithi135 ceph-mon[36276]: pgmap v1106: 129 pgs: 129 active+clean; 701 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-06-13T07:20:25.194 INFO:tasks.workunit.client.0.smithi026.stdout: 10 203130 27.01 MB/sec execute 278 sec latency 317.373 ms 2024-06-13T07:20:26.194 INFO:tasks.workunit.client.0.smithi026.stdout: 10 203661 27.01 MB/sec execute 279 sec latency 77.716 ms 2024-06-13T07:20:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:26 smithi136 ceph-mon[37229]: pgmap v1107: 129 pgs: 129 active+clean; 695 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.6 MiB/s wr, 609 op/s 2024-06-13T07:20:26.663 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:26 smithi135 ceph-mon[36276]: pgmap v1107: 129 pgs: 129 active+clean; 695 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.6 MiB/s wr, 609 op/s 2024-06-13T07:20:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:26 smithi026 ceph-mon[32041]: pgmap v1107: 129 pgs: 129 active+clean; 695 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.6 MiB/s wr, 609 op/s 2024-06-13T07:20:27.195 INFO:tasks.workunit.client.0.smithi026.stdout: 10 204364 27.04 MB/sec execute 280 sec latency 55.541 ms 2024-06-13T07:20:28.195 INFO:tasks.workunit.client.0.smithi026.stdout: 10 204845 27.04 MB/sec execute 281 sec latency 80.760 ms 2024-06-13T07:20:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:28 smithi136 ceph-mon[37229]: pgmap v1108: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.3 MiB/s wr, 540 op/s 2024-06-13T07:20:28.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:28 smithi026 ceph-mon[32041]: pgmap v1108: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.3 MiB/s wr, 540 op/s 2024-06-13T07:20:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:28 smithi135 ceph-mon[36276]: pgmap v1108: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.3 MiB/s wr, 540 op/s 2024-06-13T07:20:29.195 INFO:tasks.workunit.client.0.smithi026.stdout: 10 205485 27.09 MB/sec execute 282 sec latency 47.504 ms 2024-06-13T07:20:30.195 INFO:tasks.workunit.client.0.smithi026.stdout: 10 206045 27.08 MB/sec execute 283 sec latency 83.851 ms 2024-06-13T07:20:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:30 smithi136 ceph-mon[37229]: pgmap v1109: 129 pgs: 129 active+clean; 654 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.5 MiB/s wr, 662 op/s 2024-06-13T07:20:30.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:30 smithi026 ceph-mon[32041]: pgmap v1109: 129 pgs: 129 active+clean; 654 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.5 MiB/s wr, 662 op/s 2024-06-13T07:20:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:30 smithi135 ceph-mon[36276]: pgmap v1109: 129 pgs: 129 active+clean; 654 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.5 MiB/s wr, 662 op/s 2024-06-13T07:20:31.195 INFO:tasks.workunit.client.0.smithi026.stdout: 10 206519 27.05 MB/sec execute 284 sec latency 77.163 ms 2024-06-13T07:20:32.198 INFO:tasks.workunit.client.0.smithi026.stdout: 10 207227 27.09 MB/sec execute 285 sec latency 96.780 ms 2024-06-13T07:20:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:32 smithi136 ceph-mon[37229]: pgmap v1110: 129 pgs: 129 active+clean; 657 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.3 MiB/s wr, 544 op/s 2024-06-13T07:20:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:32 smithi026 ceph-mon[32041]: pgmap v1110: 129 pgs: 129 active+clean; 657 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.3 MiB/s wr, 544 op/s 2024-06-13T07:20:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:32 smithi135 ceph-mon[36276]: pgmap v1110: 129 pgs: 129 active+clean; 657 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.3 MiB/s wr, 544 op/s 2024-06-13T07:20:33.195 INFO:tasks.workunit.client.0.smithi026.stdout: 10 207880 27.13 MB/sec execute 286 sec latency 63.304 ms 2024-06-13T07:20:34.196 INFO:tasks.workunit.client.0.smithi026.stdout: 10 208447 27.16 MB/sec execute 287 sec latency 92.476 ms 2024-06-13T07:20:34.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:34 smithi026 ceph-mon[32041]: pgmap v1111: 129 pgs: 129 active+clean; 662 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.1 MiB/s wr, 563 op/s 2024-06-13T07:20:34.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:34 smithi135 ceph-mon[36276]: pgmap v1111: 129 pgs: 129 active+clean; 662 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.1 MiB/s wr, 563 op/s 2024-06-13T07:20:34.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:34 smithi136 ceph-mon[37229]: pgmap v1111: 129 pgs: 129 active+clean; 662 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.1 MiB/s wr, 563 op/s 2024-06-13T07:20:35.197 INFO:tasks.workunit.client.0.smithi026.stdout: 10 208941 27.15 MB/sec execute 288 sec latency 60.406 ms 2024-06-13T07:20:36.201 INFO:tasks.workunit.client.0.smithi026.stdout: 10 209440 27.15 MB/sec execute 289 sec latency 83.811 ms 2024-06-13T07:20:36.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:36 smithi026 ceph-mon[32041]: pgmap v1112: 129 pgs: 129 active+clean; 669 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-06-13T07:20:36.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:36 smithi135 ceph-mon[36276]: pgmap v1112: 129 pgs: 129 active+clean; 669 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-06-13T07:20:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:36 smithi136 ceph-mon[37229]: pgmap v1112: 129 pgs: 129 active+clean; 669 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-06-13T07:20:37.201 INFO:tasks.workunit.client.0.smithi026.stdout: 10 210057 27.16 MB/sec execute 290 sec latency 58.351 ms 2024-06-13T07:20:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.845+0000 7f30f2e00640 1 -- 172.21.15.26:0/3390308140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec1089a0 msgr2=0x7f30ec108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.845+0000 7f30f2e00640 1 --2- 172.21.15.26:0/3390308140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec1089a0 0x7f30ec108dc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009900 tx=0x7f30dc02a400 comp rx=0 tx=0).stop 2024-06-13T07:20:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.845+0000 7f30f2e00640 1 -- 172.21.15.26:0/3390308140 shutdown_connections 2024-06-13T07:20:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.845+0000 7f30f2e00640 1 --2- 172.21.15.26:0/3390308140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ec110220 0x7f30ec112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.845+0000 7f30f2e00640 1 --2- 172.21.15.26:0/3390308140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ec109300 0x7f30ec10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.845+0000 7f30f2e00640 1 --2- 172.21.15.26:0/3390308140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec1089a0 0x7f30ec108dc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.845+0000 7f30f2e00640 1 -- 172.21.15.26:0/3390308140 >> 172.21.15.26:0/3390308140 conn(0x7f30ec06ea10 msgr2=0x7f30ec070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.847+0000 7f30f2e00640 1 -- 172.21.15.26:0/3390308140 shutdown_connections 2024-06-13T07:20:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.848+0000 7f30f2e00640 1 -- 172.21.15.26:0/3390308140 wait complete. 2024-06-13T07:20:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.849+0000 7f30f2e00640 1 Processor -- start 2024-06-13T07:20:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f2e00640 1 -- start start 2024-06-13T07:20:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f2e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ec1089a0 0x7f30ec1a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ec109300 0x7f30ec1a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f2e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec110220 0x7f30ec1a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec115ee0 con 0x7f30ec109300 2024-06-13T07:20:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f2e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec115d60 con 0x7f30ec1089a0 2024-06-13T07:20:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f2e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec116060 con 0x7f30ec110220 2024-06-13T07:20:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec110220 0x7f30ec1a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30ebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ec1089a0 0x7f30ec1a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30ebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ec1089a0 0x7f30ec1a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56474/0 (socket says 172.21.15.26:56474) 2024-06-13T07:20:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec110220 0x7f30ec1a9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40554/0 (socket says 172.21.15.26:40554) 2024-06-13T07:20:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.850+0000 7f30ebe00640 1 -- 172.21.15.26:0/4001514166 learned_addr learned my addr 172.21.15.26:0/4001514166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.851+0000 7f30eb400640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ec109300 0x7f30ec1a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.851+0000 7f30ebe00640 1 -- 172.21.15.26:0/4001514166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec110220 msgr2=0x7f30ec1a9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.851+0000 7f30ebe00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec110220 0x7f30ec1a9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.851+0000 7f30ebe00640 1 -- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ec109300 msgr2=0x7f30ec1a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.853+0000 7f30ebe00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ec109300 0x7f30ec1a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.853+0000 7f30ebe00640 1 -- 172.21.15.26:0/4001514166 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec1aa1e0 con 0x7f30ec1089a0 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.854+0000 7f30ebe00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ec1089a0 0x7f30ec1a25c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f30dc031610 tx=0x7f30dc0328f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.854+0000 7f30e8c00640 1 -- 172.21.15.26:0/4001514166 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc04c070 con 0x7f30ec1089a0 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.854+0000 7f30eb400640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ec109300 0x7f30ec1a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.854+0000 7f30e8c00640 1 -- 172.21.15.26:0/4001514166 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30dc032bf0 con 0x7f30ec1089a0 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.854+0000 7f30e8c00640 1 -- 172.21.15.26:0/4001514166 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc035050 con 0x7f30ec1089a0 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.854+0000 7f30f2e00640 1 -- 172.21.15.26:0/4001514166 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec10bcc0 con 0x7f30ec1089a0 2024-06-13T07:20:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.854+0000 7f30f2e00640 1 -- 172.21.15.26:0/4001514166 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec10c1b0 con 0x7f30ec1089a0 2024-06-13T07:20:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.859+0000 7f30f5b98740 1 Processor -- start 2024-06-13T07:20:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.859+0000 7f30e8c00640 1 -- 172.21.15.26:0/4001514166 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f30dc04f070 con 0x7f30ec1089a0 2024-06-13T07:20:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.859+0000 7f30f5b98740 1 -- start start 2024-06-13T07:20:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f568020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f5685f0 0x56493f56d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 0x56493f56fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f5b98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56493f4cb420 con 0x56493f56d760 2024-06-13T07:20:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f5b98740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56493f472c50 con 0x56493f5685f0 2024-06-13T07:20:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f5b98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56493f4c73e0 con 0x56493f567c00 2024-06-13T07:20:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f568020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f568020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40566/0 (socket says 172.21.15.26:40566) 2024-06-13T07:20:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30f0e00640 1 -- 172.21.15.26:0/1533985852 learned_addr learned my addr 172.21.15.26:0/1533985852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.860+0000 7f30ebe00640 1 --2- 172.21.15.26:0/1533985852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f5685f0 0x56493f56d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.861+0000 7f30f0e00640 1 -- 172.21.15.26:0/1533985852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f5685f0 msgr2=0x56493f56d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.861+0000 7f30f0e00640 1 --2- 172.21.15.26:0/1533985852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f5685f0 0x56493f56d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.861+0000 7f30f0e00640 1 -- 172.21.15.26:0/1533985852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 msgr2=0x56493f56fbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:20:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.861+0000 7f30f0e00640 1 --2- 172.21.15.26:0/1533985852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 0x56493f56fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.861+0000 7f30f0e00640 1 -- 172.21.15.26:0/1533985852 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56493f1df210 con 0x56493f567c00 2024-06-13T07:20:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.861+0000 7f30f0e00640 1 --2- 172.21.15.26:0/1533985852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f568020 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f30e4010370 tx=0x7f30e4010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2caf5398a2a13b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.862+0000 7f30e1600640 1 -- 172.21.15.26:0/1533985852 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4016070 con 0x56493f567c00 2024-06-13T07:20:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.862+0000 7f30e1600640 1 -- 172.21.15.26:0/1533985852 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e4004d20 con 0x56493f567c00 2024-06-13T07:20:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.862+0000 7f30e1600640 1 -- 172.21.15.26:0/1533985852 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4005400 con 0x56493f567c00 2024-06-13T07:20:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.863+0000 7f30f5b98740 1 -- 172.21.15.26:0/1533985852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 msgr2=0x56493f568020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.863+0000 7f30f5b98740 1 --2- 172.21.15.26:0/1533985852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f568020 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f30e4010370 tx=0x7f30e4010830 comp rx=0 tx=0).stop 2024-06-13T07:20:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.863+0000 7f30f5b98740 1 -- 172.21.15.26:0/1533985852 shutdown_connections 2024-06-13T07:20:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.863+0000 7f30f5b98740 1 --2- 172.21.15.26:0/1533985852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f5685f0 0x56493f56d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.863+0000 7f30f5b98740 1 --2- 172.21.15.26:0/1533985852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 0x56493f56fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.863+0000 7f30f5b98740 1 --2- 172.21.15.26:0/1533985852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f568020 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.863+0000 7f30f5b98740 1 -- 172.21.15.26:0/1533985852 >> 172.21.15.26:0/1533985852 conn(0x56493f52ffb0 msgr2=0x56493f563680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.863+0000 7f30f5b98740 1 -- 172.21.15.26:0/1533985852 shutdown_connections 2024-06-13T07:20:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.864+0000 7f30e8c00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30d00a2d00 0x7f30d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.864+0000 7f30e8c00640 1 -- 172.21.15.26:0/4001514166 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f30dc0f75f0 con 0x7f30ec1089a0 2024-06-13T07:20:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.865+0000 7f30eb400640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30d00a2d00 0x7f30d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.866+0000 7f30f5b98740 1 -- 172.21.15.26:0/1533985852 wait complete. 2024-06-13T07:20:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.866+0000 7f30f5b98740 1 -- start start 2024-06-13T07:20:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.867+0000 7f30f5b98740 1 Processor -- start 2024-06-13T07:20:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.867+0000 7f30eb400640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30d00a2d00 0x7f30d00a51c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x56493f564530 tx=0x7f30d8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.867+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f571f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.867+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f5685f0 0x56493f572470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.868+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 0x56493f5794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.868+0000 7f30ebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f571f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.868+0000 7f30ebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f571f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56498/0 (socket says 172.21.15.26:56498) 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.868+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 0x56493f5794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.872+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 0x56493f5794b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47004/0 (socket says 172.21.15.26:47004) 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.872+0000 7f30eb400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f5685f0 0x56493f572470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.872+0000 7f30eb400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f5685f0 0x56493f572470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:40582/0 (socket says 172.21.15.26:40582) 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.872+0000 7f30f5b98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56493f570710 con 0x56493f56d760 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.872+0000 7f30f5b98740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56493f4860b0 con 0x56493f567c00 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.872+0000 7f30f5b98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56493f570890 con 0x56493f5685f0 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.872+0000 7f30f5b98740 10 client.0 _ll_register_callbacks cb 0x56493f471f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:20:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.872+0000 7f30ebe00640 1 -- 172.21.15.26:0/4015841705 learned_addr learned my addr 172.21.15.26:0/4015841705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.873+0000 7f30ebe00640 1 -- 172.21.15.26:0/4015841705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f5685f0 msgr2=0x56493f572470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.873+0000 7f30ebe00640 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f5685f0 0x56493f572470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.873+0000 7f30ebe00640 1 -- 172.21.15.26:0/4015841705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 msgr2=0x56493f5794b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.873+0000 7f30ebe00640 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 0x56493f5794b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.873+0000 7f30ebe00640 1 -- 172.21.15.26:0/4015841705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56493f2ab620 con 0x56493f567c00 2024-06-13T07:20:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.873+0000 7f30ebe00640 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f571f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f30dc03efd0 tx=0x7f30dc003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bbbd1c120a20b21 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.874+0000 7f30d6000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:20:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.874+0000 7f30d6000640 1 -- 172.21.15.26:0/4015841705 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0c38e0 con 0x56493f567c00 2024-06-13T07:20:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.874+0000 7f30d6000640 1 -- 172.21.15.26:0/4015841705 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f30dc0f72c0 con 0x56493f567c00 2024-06-13T07:20:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.874+0000 7f30f5b98740 1 -- 172.21.15.26:0/4015841705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e4004d20 con 0x56493f567c00 2024-06-13T07:20:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.875+0000 7f30d6000640 1 -- 172.21.15.26:0/4015841705 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f30dc0c1c50 con 0x56493f567c00 2024-06-13T07:20:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.875+0000 7f30f5b98740 10 client.26933 fetch_fsmap learned FSMap version 47 2024-06-13T07:20:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.875+0000 7f30f5b98740 1 -- 172.21.15.26:0/4015841705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56493f57c820 con 0x56493f567c00 2024-06-13T07:20:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.875+0000 7f30d6000640 1 -- 172.21.15.26:0/4015841705 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e4004d20 con 0x56493f567c00 2024-06-13T07:20:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.876+0000 7f30d6000640 1 -- 172.21.15.26:0/4015841705 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f30dc0e22a0 con 0x56493f567c00 2024-06-13T07:20:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.876+0000 7f30f5b98740 10 client.26933 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:20:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.876+0000 7f30f5b98740 10 client.26933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:20:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.876+0000 7f30f5b98740 20 client.26933 populate_metadata read hostname 'smithi026' 2024-06-13T07:20:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.876+0000 7f30f5b98740 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56493f57df30 0x56493f5803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.876+0000 7f30f5b98740 4 client.26933 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:20:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.876+0000 7f30f5b98740 1 -- 172.21.15.26:0/4015841705 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56493f4d8b50 con 0x56493f57df30 2024-06-13T07:20:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.877+0000 7f30eb400640 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56493f57df30 0x56493f5803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.878+0000 7f30eb400640 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56493f57df30 0x56493f5803f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7934d6f41346c31a server_cookie=cf923940d3943532 in_seq=0 out_seq=0 2024-06-13T07:20:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.878+0000 7f30d6000640 10 client.26933 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.886+0000 7f30d6000640 1 -- 172.21.15.26:0/4015841705 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56493f4d8b50 con 0x56493f57df30 2024-06-13T07:20:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.886+0000 7f30d6000640 10 client.26933 handle_command_reply: tid=0 2024-06-13T07:20:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.886+0000 7f30f5b98740 1 client.26933 shutdown 2024-06-13T07:20:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.886+0000 7f30f5b98740 1 -- 172.21.15.26:0/4015841705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 msgr2=0x56493f571f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.886+0000 7f30f5b98740 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f571f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f30dc03efd0 tx=0x7f30dc003e70 comp rx=0 tx=0).stop 2024-06-13T07:20:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.887+0000 7f30f5b98740 1 -- 172.21.15.26:0/4015841705 shutdown_connections 2024-06-13T07:20:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.887+0000 7f30f5b98740 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56493f57df30 0x56493f5803f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.887+0000 7f30f5b98740 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f56d760 0x56493f5794b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.887+0000 7f30f5b98740 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f5685f0 0x56493f572470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.887+0000 7f30f5b98740 1 --2- 172.21.15.26:0/4015841705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f571f30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.887+0000 7f30f5b98740 1 -- 172.21.15.26:0/4015841705 >> 172.21.15.26:0/4015841705 conn(0x56493f52ffb0 msgr2=0x56493f562840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.887+0000 7f30d6000640 0 client.26933 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.887+0000 7f30f5b98740 1 -- 172.21.15.26:0/4015841705 shutdown_connections 2024-06-13T07:20:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.888+0000 7f30f5b98740 1 -- 172.21.15.26:0/4015841705 wait complete. 2024-06-13T07:20:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.889+0000 7f30f5b98740 20 client.26933 trim_cache size 0 max 16384 2024-06-13T07:20:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.920+0000 7f30f5b98740 1 Processor -- start 2024-06-13T07:20:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.920+0000 7f30f5b98740 1 -- start start 2024-06-13T07:20:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f52ffb0 0x56493f566570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f579440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f566ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30f5b98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b3490 con 0x56493f5685f0 2024-06-13T07:20:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30f5b98740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b0d60 con 0x56493f567c00 2024-06-13T07:20:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30f5b98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1aebb0 con 0x56493f52ffb0 2024-06-13T07:20:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30eb400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f579440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30eb400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f579440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56510/0 (socket says 172.21.15.26:56510) 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30eb400640 1 -- 172.21.15.26:0/887975220 learned_addr learned my addr 172.21.15.26:0/887975220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30ebe00640 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f52ffb0 0x56493f566570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.921+0000 7f30f0e00640 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f566ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.922+0000 7f30eb400640 1 -- 172.21.15.26:0/887975220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f52ffb0 msgr2=0x56493f566570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.922+0000 7f30eb400640 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f52ffb0 0x56493f566570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.922+0000 7f30eb400640 1 -- 172.21.15.26:0/887975220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 msgr2=0x56493f566ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.922+0000 7f30eb400640 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f566ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.922+0000 7f30eb400640 1 -- 172.21.15.26:0/887975220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56493f564330 con 0x56493f567c00 2024-06-13T07:20:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.922+0000 7f30ebe00640 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f52ffb0 0x56493f566570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:20:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.922+0000 7f30eb400640 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f579440 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f30d800bfd0 tx=0x7f30d8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c4d57d8c26410b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.923+0000 7f30e0c00640 1 -- 172.21.15.26:0/887975220 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8005220 con 0x56493f567c00 2024-06-13T07:20:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.923+0000 7f30e0c00640 1 -- 172.21.15.26:0/887975220 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d8004c80 con 0x56493f567c00 2024-06-13T07:20:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.923+0000 7f30e0c00640 1 -- 172.21.15.26:0/887975220 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80246b0 con 0x56493f567c00 2024-06-13T07:20:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.924+0000 7f30f5b98740 1 -- 172.21.15.26:0/887975220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 msgr2=0x56493f579440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.924+0000 7f30f5b98740 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f579440 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f30d800bfd0 tx=0x7f30d8014000 comp rx=0 tx=0).stop 2024-06-13T07:20:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.924+0000 7f30f5b98740 1 -- 172.21.15.26:0/887975220 shutdown_connections 2024-06-13T07:20:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.924+0000 7f30f5b98740 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f567c00 0x56493f579440 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.924+0000 7f30f5b98740 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f566ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.924+0000 7f30f5b98740 1 --2- 172.21.15.26:0/887975220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f52ffb0 0x56493f566570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.924+0000 7f30f5b98740 1 -- 172.21.15.26:0/887975220 >> 172.21.15.26:0/887975220 conn(0x56493f56d2a0 msgr2=0x56493f563100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.925+0000 7f30f5b98740 1 -- 172.21.15.26:0/887975220 shutdown_connections 2024-06-13T07:20:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.925+0000 7f30f5b98740 1 -- 172.21.15.26:0/887975220 wait complete. 2024-06-13T07:20:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.925+0000 7f30f5b98740 1 -- start start 2024-06-13T07:20:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.926+0000 7f30f5b98740 1 Processor -- start 2024-06-13T07:20:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.926+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f52ffb0 0x56493f576680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f576bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f5b98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f577b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f5b98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1af220 con 0x56493f5685f0 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f5b98740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b0690 con 0x56493f52ffb0 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f5b98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1af8f0 con 0x56493f567c00 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f5b98740 10 client.0 _ll_register_callbacks cb 0x7f30dc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f577b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f577b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47012/0 (socket says 172.21.15.26:47012) 2024-06-13T07:20:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.927+0000 7f30f0e00640 1 -- 172.21.15.26:0/878015044 learned_addr learned my addr 172.21.15.26:0/878015044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.928+0000 7f30ebe00640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f52ffb0 0x56493f576680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.928+0000 7f30ebe00640 1 -- 172.21.15.26:0/878015044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 msgr2=0x56493f576bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:20:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.928+0000 7f30eb400640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f576bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.928+0000 7f30ebe00640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f576bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.928+0000 7f30ebe00640 1 -- 172.21.15.26:0/878015044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 msgr2=0x56493f577b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.929+0000 7f30ebe00640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f577b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.929+0000 7f30ebe00640 1 -- 172.21.15.26:0/878015044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56493f578100 con 0x56493f52ffb0 2024-06-13T07:20:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.929+0000 7f30f0e00640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f577b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:20:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.929+0000 7f30ebe00640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f52ffb0 0x56493f576680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0bfee0 tx=0x7f30dc0f79a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc075b321dce0229 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.929+0000 7f30eb400640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f576bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:20:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.929+0000 7f30d6000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:20:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.929+0000 7f30d6000640 1 -- 172.21.15.26:0/878015044 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0c3470 con 0x56493f52ffb0 2024-06-13T07:20:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.930+0000 7f30d6000640 1 -- 172.21.15.26:0/878015044 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f30dc0bf7d0 con 0x56493f52ffb0 2024-06-13T07:20:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.930+0000 7f30f5b98740 1 -- 172.21.15.26:0/878015044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d8004c80 con 0x56493f52ffb0 2024-06-13T07:20:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.930+0000 7f30d6000640 1 -- 172.21.15.26:0/878015044 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f30dc0c3650 con 0x56493f52ffb0 2024-06-13T07:20:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.930+0000 7f30f5b98740 10 client.26939 fetch_fsmap learned FSMap version 47 2024-06-13T07:20:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.930+0000 7f30f5b98740 1 -- 172.21.15.26:0/878015044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56493f579410 con 0x56493f52ffb0 2024-06-13T07:20:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.931+0000 7f30d6000640 1 -- 172.21.15.26:0/878015044 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d8004c80 con 0x56493f52ffb0 2024-06-13T07:20:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.931+0000 7f30d6000640 1 -- 172.21.15.26:0/878015044 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f30dc0f7a50 con 0x56493f52ffb0 2024-06-13T07:20:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.931+0000 7f30f5b98740 10 client.26939 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:20:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.931+0000 7f30f5b98740 10 client.26939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:20:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.931+0000 7f30f5b98740 20 client.26939 populate_metadata read hostname 'smithi026' 2024-06-13T07:20:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.931+0000 7f30f5b98740 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56493f56d760 0x56493f5eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.931+0000 7f30f5b98740 4 client.26939 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:20:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.931+0000 7f30f5b98740 1 -- 172.21.15.26:0/878015044 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56493f4d2ea0 con 0x56493f56d760 2024-06-13T07:20:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.934+0000 7f30eb400640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56493f56d760 0x56493f5eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.936+0000 7f30d6000640 10 client.26939 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:37.936+0000 7f30eb400640 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56493f56d760 0x56493f5eae60 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d2c9e38e3dd991d server_cookie=bf4f1aa04780a552 in_seq=0 out_seq=0 2024-06-13T07:20:38.201 INFO:tasks.workunit.client.0.smithi026.stdout: 10 210372 27.12 MB/sec execute 291 sec latency 187.701 ms 2024-06-13T07:20:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30d6000640 1 -- 172.21.15.26:0/878015044 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56493f4d2ea0 con 0x56493f56d760 2024-06-13T07:20:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30d6000640 10 client.26939 handle_command_reply: tid=0 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 client.26939 shutdown 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 -- 172.21.15.26:0/878015044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f52ffb0 msgr2=0x56493f576680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f52ffb0 0x56493f576680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0bfee0 tx=0x7f30dc0f79a0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 -- 172.21.15.26:0/878015044 shutdown_connections 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56493f56d760 0x56493f5eae60 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56493f5685f0 0x56493f577b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56493f567c00 0x56493f576bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 --2- 172.21.15.26:0/878015044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56493f52ffb0 0x56493f576680 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30f5b98740 1 -- 172.21.15.26:0/878015044 >> 172.21.15.26:0/878015044 conn(0x56493f56d2a0 msgr2=0x56493f563000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.669+0000 7f30d6000640 0 client.26939 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.670+0000 7f30f5b98740 1 -- 172.21.15.26:0/878015044 shutdown_connections 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.670+0000 7f30f5b98740 1 -- 172.21.15.26:0/878015044 wait complete. 2024-06-13T07:20:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.670+0000 7f30f5b98740 20 client.26939 trim_cache size 0 max 16384 2024-06-13T07:20:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:20:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:20:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:20:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 -- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30d00a2d00 msgr2=0x7f30d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30d00a2d00 0x7f30d00a51c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x56493f564530 tx=0x7f30d8005be0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 -- 172.21.15.26:0/4001514166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ec1089a0 msgr2=0x7f30ec1a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ec1089a0 0x7f30ec1a25c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f30dc031610 tx=0x7f30dc0328f0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 -- 172.21.15.26:0/4001514166 shutdown_connections 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f30d00a2d00 0x7f30d00a51c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30ec110220 0x7f30ec1a9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ec109300 0x7f30ec1a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 --2- 172.21.15.26:0/4001514166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ec1089a0 0x7f30ec1a25c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.671+0000 7f30f2e00640 1 -- 172.21.15.26:0/4001514166 >> 172.21.15.26:0/4001514166 conn(0x7f30ec06ea10 msgr2=0x7f30ec109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.672+0000 7f30f2e00640 1 -- 172.21.15.26:0/4001514166 shutdown_connections 2024-06-13T07:20:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:38.672+0000 7f30f2e00640 1 -- 172.21.15.26:0/4001514166 wait complete. 2024-06-13T07:20:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:38 smithi026 ceph-mon[32041]: pgmap v1113: 129 pgs: 129 active+clean; 672 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-06-13T07:20:38.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:38 smithi135 ceph-mon[36276]: pgmap v1113: 129 pgs: 129 active+clean; 672 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-06-13T07:20:38.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:38 smithi136 ceph-mon[37229]: pgmap v1113: 129 pgs: 129 active+clean; 672 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-06-13T07:20:39.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 210728 27.10 MB/sec execute 292 sec latency 508.231 ms 2024-06-13T07:20:40.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 211253 27.09 MB/sec execute 293 sec latency 85.224 ms 2024-06-13T07:20:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:40 smithi026 ceph-mon[32041]: pgmap v1114: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-06-13T07:20:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:40 smithi135 ceph-mon[36276]: pgmap v1114: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-06-13T07:20:40.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:40 smithi136 ceph-mon[37229]: pgmap v1114: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-06-13T07:20:41.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 211717 27.10 MB/sec execute 294 sec latency 102.332 ms 2024-06-13T07:20:42.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 211827 27.04 MB/sec execute 295 sec latency 554.148 ms 2024-06-13T07:20:42.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:42 smithi026 ceph-mon[32041]: pgmap v1115: 129 pgs: 129 active+clean; 681 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 440 op/s 2024-06-13T07:20:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:42 smithi135 ceph-mon[36276]: pgmap v1115: 129 pgs: 129 active+clean; 681 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 440 op/s 2024-06-13T07:20:42.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:42 smithi136 ceph-mon[37229]: pgmap v1115: 129 pgs: 129 active+clean; 681 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 440 op/s 2024-06-13T07:20:43.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 212217 27.00 MB/sec execute 296 sec latency 136.755 ms 2024-06-13T07:20:43.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:43 smithi026 ceph-mon[32041]: pgmap v1116: 129 pgs: 129 active+clean; 692 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.5 MiB/s wr, 492 op/s 2024-06-13T07:20:43.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:43 smithi135 ceph-mon[36276]: pgmap v1116: 129 pgs: 129 active+clean; 692 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.5 MiB/s wr, 492 op/s 2024-06-13T07:20:43.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:43 smithi136 ceph-mon[37229]: pgmap v1116: 129 pgs: 129 active+clean; 692 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.5 MiB/s wr, 492 op/s 2024-06-13T07:20:44.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 212520 26.96 MB/sec execute 297 sec latency 143.447 ms 2024-06-13T07:20:45.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 213022 27.00 MB/sec execute 298 sec latency 305.037 ms 2024-06-13T07:20:46.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 213590 26.99 MB/sec execute 299 sec latency 38.193 ms 2024-06-13T07:20:46.484 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:46 smithi135 ceph-mon[36276]: pgmap v1117: 129 pgs: 129 active+clean; 695 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.5 MiB/s wr, 519 op/s 2024-06-13T07:20:46.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:46 smithi026 ceph-mon[32041]: pgmap v1117: 129 pgs: 129 active+clean; 695 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.5 MiB/s wr, 519 op/s 2024-06-13T07:20:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:46 smithi136 ceph-mon[37229]: pgmap v1117: 129 pgs: 129 active+clean; 695 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.5 MiB/s wr, 519 op/s 2024-06-13T07:20:47.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 213917 26.94 MB/sec execute 300 sec latency 432.201 ms 2024-06-13T07:20:48.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 214066 26.89 MB/sec execute 301 sec latency 1186.176 ms 2024-06-13T07:20:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:48 smithi026 ceph-mon[32041]: pgmap v1118: 129 pgs: 129 active+clean; 691 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 523 op/s 2024-06-13T07:20:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:48 smithi135 ceph-mon[36276]: pgmap v1118: 129 pgs: 129 active+clean; 691 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 523 op/s 2024-06-13T07:20:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:48 smithi136 ceph-mon[37229]: pgmap v1118: 129 pgs: 129 active+clean; 691 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 523 op/s 2024-06-13T07:20:49.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 214349 26.85 MB/sec execute 302 sec latency 499.092 ms 2024-06-13T07:20:50.202 INFO:tasks.workunit.client.0.smithi026.stdout: 10 214661 26.81 MB/sec execute 303 sec latency 151.675 ms 2024-06-13T07:20:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:50 smithi026 ceph-mon[32041]: pgmap v1119: 129 pgs: 129 active+clean; 713 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.6 MiB/s wr, 591 op/s 2024-06-13T07:20:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:50 smithi135 ceph-mon[36276]: pgmap v1119: 129 pgs: 129 active+clean; 713 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.6 MiB/s wr, 591 op/s 2024-06-13T07:20:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:50 smithi136 ceph-mon[37229]: pgmap v1119: 129 pgs: 129 active+clean; 713 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.6 MiB/s wr, 591 op/s 2024-06-13T07:20:51.203 INFO:tasks.workunit.client.0.smithi026.stdout: 10 215022 26.80 MB/sec execute 304 sec latency 133.274 ms 2024-06-13T07:20:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.880+0000 7ff570c00640 1 -- 172.21.15.26:0/1178972834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c103410 msgr2=0x7ff56c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.880+0000 7ff570c00640 1 --2- 172.21.15.26:0/1178972834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c103410 0x7ff56c109ca0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff564009be0 tx=0x7ff56402a6e0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.880+0000 7ff570c00640 1 -- 172.21.15.26:0/1178972834 shutdown_connections 2024-06-13T07:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.880+0000 7ff570c00640 1 --2- 172.21.15.26:0/1178972834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c103410 0x7ff56c109ca0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.880+0000 7ff570c00640 1 --2- 172.21.15.26:0/1178972834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff56c102a50 0x7ff56c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.880+0000 7ff570c00640 1 --2- 172.21.15.26:0/1178972834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff56c101860 0x7ff56c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.880+0000 7ff570c00640 1 -- 172.21.15.26:0/1178972834 >> 172.21.15.26:0/1178972834 conn(0x7ff56c0fcdd0 msgr2=0x7ff56c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.881+0000 7ff570c00640 1 -- 172.21.15.26:0/1178972834 shutdown_connections 2024-06-13T07:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.882+0000 7ff570c00640 1 -- 172.21.15.26:0/1178972834 wait complete. 2024-06-13T07:20:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.883+0000 7ff570c00640 1 Processor -- start 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.884+0000 7ff570c00640 1 -- start start 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.884+0000 7ff570c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c101860 0x7ff56c19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.884+0000 7ff570c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff56c102a50 0x7ff56c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.884+0000 7ff570c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff56c103410 0x7ff56c1a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.884+0000 7ff570c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c075810 con 0x7ff56c102a50 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.884+0000 7ff570c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c075690 con 0x7ff56c101860 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.884+0000 7ff570c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c075990 con 0x7ff56c103410 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.884+0000 7ff56b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c101860 0x7ff56c19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.885+0000 7ff56b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c101860 0x7ff56c19e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:60744/0 (socket says 172.21.15.26:60744) 2024-06-13T07:20:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.885+0000 7ff56b400640 1 -- 172.21.15.26:0/3398501932 learned_addr learned my addr 172.21.15.26:0/3398501932 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.887+0000 7ff56b400640 1 -- 172.21.15.26:0/3398501932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff56c103410 msgr2=0x7ff56c1a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.887+0000 7ff56b400640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff56c103410 0x7ff56c1a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.887+0000 7ff56b400640 1 -- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff56c102a50 msgr2=0x7ff56c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.887+0000 7ff56b400640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff56c102a50 0x7ff56c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.887+0000 7ff56b400640 1 -- 172.21.15.26:0/3398501932 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56c10fdc0 con 0x7ff56c101860 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.887+0000 7ff56b400640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c101860 0x7ff56c19e0f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00e9c0 tx=0x7ff55c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.887+0000 7ff563e00640 1 -- 172.21.15.26:0/3398501932 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00cd80 con 0x7ff56c101860 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.888+0000 7ff563e00640 1 -- 172.21.15.26:0/3398501932 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff55c0035f0 con 0x7ff56c101860 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.888+0000 7ff563e00640 1 -- 172.21.15.26:0/3398501932 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c010690 con 0x7ff56c101860 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.888+0000 7ff570c00640 1 -- 172.21.15.26:0/3398501932 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56c1100b0 con 0x7ff56c101860 2024-06-13T07:20:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.888+0000 7ff570c00640 1 -- 172.21.15.26:0/3398501932 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56c1105b0 con 0x7ff56c101860 2024-06-13T07:20:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.892+0000 7ff574fb2740 1 Processor -- start 2024-06-13T07:20:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.892+0000 7ff563e00640 1 -- 172.21.15.26:0/3398501932 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff55c0037a0 con 0x7ff56c101860 2024-06-13T07:20:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.892+0000 7ff574fb2740 1 -- start start 2024-06-13T07:20:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.892+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557077234dc0 0x5570772351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.892+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 0x55707723a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.893+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x55707723cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.893+0000 7ff574fb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557077198420 con 0x55707723a920 2024-06-13T07:20:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.893+0000 7ff574fb2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55707713fc50 con 0x5570772357b0 2024-06-13T07:20:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.893+0000 7ff574fb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5570771943e0 con 0x557077234dc0 2024-06-13T07:20:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.893+0000 7ff56be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557077234dc0 0x5570772351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.893+0000 7ff56be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557077234dc0 0x5570772351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49956/0 (socket says 172.21.15.26:49956) 2024-06-13T07:20:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.893+0000 7ff56be00640 1 -- 172.21.15.26:0/1394686870 learned_addr learned my addr 172.21.15.26:0/1394686870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.893+0000 7ff56be00640 1 -- 172.21.15.26:0/1394686870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 msgr2=0x55707723a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.894+0000 7ff56b400640 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 0x55707723a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.894+0000 7ff56be00640 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 0x55707723a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.894+0000 7ff56be00640 1 -- 172.21.15.26:0/1394686870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 msgr2=0x55707723cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.894+0000 7ff56be00640 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x55707723cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.894+0000 7ff56be00640 1 -- 172.21.15.26:0/1394686870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557076eac210 con 0x557077234dc0 2024-06-13T07:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.894+0000 7ff56b400640 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 0x55707723a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.894+0000 7ff56be00640 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557077234dc0 0x5570772351e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff564004af0 tx=0x7ff564030b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c188c699ef2d4be0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.895+0000 7ff557e00640 1 -- 172.21.15.26:0/1394686870 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564040020 con 0x557077234dc0 2024-06-13T07:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.895+0000 7ff557e00640 1 -- 172.21.15.26:0/1394686870 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff564003e50 con 0x557077234dc0 2024-06-13T07:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.895+0000 7ff557e00640 1 -- 172.21.15.26:0/1394686870 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564036040 con 0x557077234dc0 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff574fb2740 1 -- 172.21.15.26:0/1394686870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557077234dc0 msgr2=0x5570772351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff574fb2740 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557077234dc0 0x5570772351e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff564004af0 tx=0x7ff564030b60 comp rx=0 tx=0).stop 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff563e00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff5500a2c80 0x7ff5500a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff574fb2740 1 -- 172.21.15.26:0/1394686870 shutdown_connections 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff574fb2740 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 0x55707723a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff574fb2740 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x55707723cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff574fb2740 1 --2- 172.21.15.26:0/1394686870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557077234dc0 0x5570772351e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff574fb2740 1 -- 172.21.15.26:0/1394686870 >> 172.21.15.26:0/1394686870 conn(0x5570771fcfb0 msgr2=0x557077230840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff563e00640 1 -- 172.21.15.26:0/3398501932 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff55c0cf950 con 0x7ff56c101860 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.896+0000 7ff56be00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff5500a2c80 0x7ff5500a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.897+0000 7ff574fb2740 1 -- 172.21.15.26:0/1394686870 shutdown_connections 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.897+0000 7ff574fb2740 1 -- 172.21.15.26:0/1394686870 wait complete. 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.897+0000 7ff574fb2740 1 -- start start 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.897+0000 7ff56be00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff5500a2c80 0x7ff5500a5140 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff564009560 tx=0x7ff564005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff574fb2740 1 Processor -- start 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 0x557077250620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x557077257660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff574fb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55707723d870 con 0x55707723a920 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff574fb2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5570771530b0 con 0x557077234dc0 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff574fb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55707723d9f0 con 0x5570772357b0 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff574fb2740 10 client.0 _ll_register_callbacks cb 0x55707713ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff56b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff56aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 0x557077250620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff56b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:60758/0 (socket says 172.21.15.26:60758) 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.898+0000 7ff56b400640 1 -- 172.21.15.26:0/3174614709 learned_addr learned my addr 172.21.15.26:0/3174614709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.899+0000 7ff56be00640 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x557077257660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.899+0000 7ff56b400640 1 -- 172.21.15.26:0/3174614709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 msgr2=0x557077250620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.899+0000 7ff56b400640 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 0x557077250620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.899+0000 7ff56b400640 1 -- 172.21.15.26:0/3174614709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 msgr2=0x557077257660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.899+0000 7ff56b400640 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x557077257660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.899+0000 7ff56b400640 1 -- 172.21.15.26:0/3174614709 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557076f78620 con 0x557077234dc0 2024-06-13T07:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.899+0000 7ff56b400640 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772500e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff55c09ba60 tx=0x7ff55c09b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6041cd8a7f34f484 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.899+0000 7ff554c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.900+0000 7ff554c00640 1 -- 172.21.15.26:0/3174614709 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0a0110 con 0x557077234dc0 2024-06-13T07:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.900+0000 7ff554c00640 1 -- 172.21.15.26:0/3174614709 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff55c003b30 con 0x557077234dc0 2024-06-13T07:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.900+0000 7ff574fb2740 1 -- 172.21.15.26:0/3174614709 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff564003e50 con 0x557077234dc0 2024-06-13T07:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.900+0000 7ff554c00640 1 -- 172.21.15.26:0/3174614709 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff55c0a49d0 con 0x557077234dc0 2024-06-13T07:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.900+0000 7ff574fb2740 10 client.26948 fetch_fsmap learned FSMap version 47 2024-06-13T07:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.900+0000 7ff574fb2740 1 -- 172.21.15.26:0/3174614709 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557077240a90 con 0x557077234dc0 2024-06-13T07:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.900+0000 7ff554c00640 1 -- 172.21.15.26:0/3174614709 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff564003e50 con 0x557077234dc0 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.902+0000 7ff554c00640 1 -- 172.21.15.26:0/3174614709 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7ff55c086040 con 0x557077234dc0 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.902+0000 7ff574fb2740 10 client.26948 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.902+0000 7ff574fb2740 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.902+0000 7ff574fb2740 20 client.26948 populate_metadata read hostname 'smithi026' 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.902+0000 7ff574fb2740 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557077242100 0x5570772445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.902+0000 7ff574fb2740 4 client.26948 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.902+0000 7ff574fb2740 1 -- 172.21.15.26:0/3174614709 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570771a5b50 con 0x557077242100 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.903+0000 7ff56aa00640 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557077242100 0x5570772445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.904+0000 7ff56aa00640 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557077242100 0x5570772445a0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397c445434e3bc31 server_cookie=dc1974d4a62836d4 in_seq=0 out_seq=0 2024-06-13T07:20:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.904+0000 7ff554c00640 10 client.26948 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.911+0000 7ff554c00640 1 -- 172.21.15.26:0/3174614709 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5570771a5b50 con 0x557077242100 2024-06-13T07:20:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.911+0000 7ff554c00640 10 client.26948 handle_command_reply: tid=0 2024-06-13T07:20:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.912+0000 7ff574fb2740 1 client.26948 shutdown 2024-06-13T07:20:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.912+0000 7ff574fb2740 1 -- 172.21.15.26:0/3174614709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 msgr2=0x5570772500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.912+0000 7ff574fb2740 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772500e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff55c09ba60 tx=0x7ff55c09b780 comp rx=0 tx=0).stop 2024-06-13T07:20:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.913+0000 7ff574fb2740 1 -- 172.21.15.26:0/3174614709 shutdown_connections 2024-06-13T07:20:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.913+0000 7ff574fb2740 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557077242100 0x5570772445a0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.913+0000 7ff554c00640 0 client.26948 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.913+0000 7ff574fb2740 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x557077257660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.914+0000 7ff574fb2740 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 0x557077250620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.914+0000 7ff574fb2740 1 --2- 172.21.15.26:0/3174614709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772500e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.914+0000 7ff574fb2740 1 -- 172.21.15.26:0/3174614709 >> 172.21.15.26:0/3174614709 conn(0x5570771fcfb0 msgr2=0x55707722f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.914+0000 7ff574fb2740 1 -- 172.21.15.26:0/3174614709 shutdown_connections 2024-06-13T07:20:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.914+0000 7ff574fb2740 1 -- 172.21.15.26:0/3174614709 wait complete. 2024-06-13T07:20:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.915+0000 7ff574fb2740 20 client.26948 trim_cache size 0 max 16384 2024-06-13T07:20:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.940+0000 7ff574fb2740 1 Processor -- start 2024-06-13T07:20:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.940+0000 7ff574fb2740 1 -- start start 2024-06-13T07:20:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.940+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557077234dc0 0x557077233f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.940+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 0x5570772575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.941+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707723a920 0x55707724ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.941+0000 7ff574fb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c070550 con 0x557077234dc0 2024-06-13T07:20:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.941+0000 7ff574fb2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c06de60 con 0x5570772357b0 2024-06-13T07:20:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.941+0000 7ff574fb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c06bcd0 con 0x55707723a920 2024-06-13T07:20:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.941+0000 7ff56b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557077234dc0 0x557077233f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.941+0000 7ff56b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557077234dc0 0x557077233f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37252/0 (socket says 172.21.15.26:37252) 2024-06-13T07:20:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.941+0000 7ff56b400640 1 -- 172.21.15.26:0/1513883243 learned_addr learned my addr 172.21.15.26:0/1513883243 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.942+0000 7ff56be00640 1 --2- 172.21.15.26:0/1513883243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707723a920 0x55707724ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.942+0000 7ff56be00640 1 -- 172.21.15.26:0/1513883243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 msgr2=0x5570772575f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:20:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.942+0000 7ff56be00640 1 --2- 172.21.15.26:0/1513883243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 0x5570772575f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.942+0000 7ff56be00640 1 -- 172.21.15.26:0/1513883243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557077234dc0 msgr2=0x557077233f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.942+0000 7ff56be00640 1 --2- 172.21.15.26:0/1513883243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557077234dc0 0x557077233f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.942+0000 7ff56be00640 1 -- 172.21.15.26:0/1513883243 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557077239d80 con 0x55707723a920 2024-06-13T07:20:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.943+0000 7ff56be00640 1 --2- 172.21.15.26:0/1513883243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707723a920 0x55707724ee90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff56402ac00 tx=0x7ff564048070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db570510eefb07 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.943+0000 7ff557400640 1 -- 172.21.15.26:0/1513883243 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564036040 con 0x55707723a920 2024-06-13T07:20:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.945+0000 7ff557400640 1 -- 172.21.15.26:0/1513883243 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56403cbc0 con 0x55707723a920 2024-06-13T07:20:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.947+0000 7ff557400640 1 -- 172.21.15.26:0/1513883243 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564046430 con 0x55707723a920 2024-06-13T07:20:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.948+0000 7ff574fb2740 1 -- 172.21.15.26:0/1513883243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707723a920 msgr2=0x55707724ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.948+0000 7ff574fb2740 1 --2- 172.21.15.26:0/1513883243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707723a920 0x55707724ee90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff56402ac00 tx=0x7ff564048070 comp rx=0 tx=0).stop 2024-06-13T07:20:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.949+0000 7ff574fb2740 1 -- 172.21.15.26:0/1513883243 shutdown_connections 2024-06-13T07:20:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.949+0000 7ff574fb2740 1 --2- 172.21.15.26:0/1513883243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570772357b0 0x5570772575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.949+0000 7ff574fb2740 1 --2- 172.21.15.26:0/1513883243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55707723a920 0x55707724ee90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.949+0000 7ff574fb2740 1 --2- 172.21.15.26:0/1513883243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557077234dc0 0x557077233f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.949+0000 7ff574fb2740 1 -- 172.21.15.26:0/1513883243 >> 172.21.15.26:0/1513883243 conn(0x5570771fcfb0 msgr2=0x5570772337f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.949+0000 7ff574fb2740 1 -- 172.21.15.26:0/1513883243 shutdown_connections 2024-06-13T07:20:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.949+0000 7ff574fb2740 1 -- 172.21.15.26:0/1513883243 wait complete. 2024-06-13T07:20:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.950+0000 7ff574fb2740 1 -- start start 2024-06-13T07:20:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.950+0000 7ff574fb2740 1 Processor -- start 2024-06-13T07:20:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 0x5570772556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff574fb2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x557077255c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff574fb2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c06c320 con 0x55707723a920 2024-06-13T07:20:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff574fb2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c06d790 con 0x557077234dc0 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff574fb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c06c9f0 con 0x5570772357b0 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff574fb2740 10 client.0 _ll_register_callbacks cb 0x7ff55c09b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff56b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff56b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:60782/0 (socket says 172.21.15.26:60782) 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.951+0000 7ff56b400640 1 -- 172.21.15.26:0/876849803 learned_addr learned my addr 172.21.15.26:0/876849803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff56aa00640 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 0x5570772556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff56be00640 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x557077255c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff56b400640 1 -- 172.21.15.26:0/876849803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 msgr2=0x5570772556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff56b400640 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 0x5570772556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff56b400640 1 -- 172.21.15.26:0/876849803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 msgr2=0x557077255c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff56b400640 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x557077255c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff56b400640 1 -- 172.21.15.26:0/876849803 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557077256160 con 0x557077234dc0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff56b400640 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772570d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009250 tx=0x7ff55c09b1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4525f275e80b83fd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.952+0000 7ff554c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.953+0000 7ff554c00640 1 -- 172.21.15.26:0/876849803 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c07f080 con 0x557077234dc0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.953+0000 7ff554c00640 1 -- 172.21.15.26:0/876849803 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff55c07abd0 con 0x557077234dc0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.953+0000 7ff574fb2740 1 -- 172.21.15.26:0/876849803 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff56403cbc0 con 0x557077234dc0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff554c00640 1 -- 172.21.15.26:0/876849803 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff55c0850a0 con 0x557077234dc0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff574fb2740 10 client.26951 fetch_fsmap learned FSMap version 47 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff574fb2740 1 -- 172.21.15.26:0/876849803 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5570772d7a20 con 0x557077234dc0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff554c00640 1 -- 172.21.15.26:0/876849803 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56403cbc0 con 0x557077234dc0 2024-06-13T07:20:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff554c00640 1 -- 172.21.15.26:0/876849803 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7ff55c086040 con 0x557077234dc0 2024-06-13T07:20:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff574fb2740 10 client.26951 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:20:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff574fb2740 10 client.26951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:20:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff574fb2740 20 client.26951 populate_metadata read hostname 'smithi026' 2024-06-13T07:20:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff574fb2740 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557077242100 0x5570772da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:20:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff574fb2740 4 client.26951 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:20:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.954+0000 7ff574fb2740 1 -- 172.21.15.26:0/876849803 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55707719fea0 con 0x557077242100 2024-06-13T07:20:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.956+0000 7ff56aa00640 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557077242100 0x5570772da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:20:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.957+0000 7ff56aa00640 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557077242100 0x5570772da370 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbdd74cba0cbfb75 server_cookie=9a3b7206808ab320 in_seq=0 out_seq=0 2024-06-13T07:20:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:51.957+0000 7ff554c00640 10 client.26951 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:52.203 INFO:tasks.workunit.client.0.smithi026.stdout: 10 215503 26.76 MB/sec execute 305 sec latency 244.055 ms 2024-06-13T07:20:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.379+0000 7ff554c00640 1 -- 172.21.15.26:0/876849803 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55707719fea0 con 0x557077242100 2024-06-13T07:20:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.379+0000 7ff554c00640 10 client.26951 handle_command_reply: tid=0 2024-06-13T07:20:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.379+0000 7ff574fb2740 1 client.26951 shutdown 2024-06-13T07:20:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff574fb2740 1 -- 172.21.15.26:0/876849803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 msgr2=0x5570772570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff574fb2740 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772570d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009250 tx=0x7ff55c09b1b0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff574fb2740 1 -- 172.21.15.26:0/876849803 shutdown_connections 2024-06-13T07:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff574fb2740 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557077242100 0x5570772da370 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff574fb2740 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55707723a920 0x557077255c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff574fb2740 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570772357b0 0x5570772556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff574fb2740 1 --2- 172.21.15.26:0/876849803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557077234dc0 0x5570772570d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff574fb2740 1 -- 172.21.15.26:0/876849803 >> 172.21.15.26:0/876849803 conn(0x5570771fcfb0 msgr2=0x55707722f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.380+0000 7ff554c00640 0 client.26951 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.381+0000 7ff574fb2740 1 -- 172.21.15.26:0/876849803 shutdown_connections 2024-06-13T07:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.381+0000 7ff574fb2740 1 -- 172.21.15.26:0/876849803 wait complete. 2024-06-13T07:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.381+0000 7ff574fb2740 20 client.26951 trim_cache size 0 max 16384 2024-06-13T07:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 -- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff5500a2c80 msgr2=0x7ff5500a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff5500a2c80 0x7ff5500a5140 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff564009560 tx=0x7ff564005ec0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 -- 172.21.15.26:0/3398501932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c101860 msgr2=0x7ff56c19e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c101860 0x7ff56c19e0f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00e9c0 tx=0x7ff55c00ee80 comp rx=0 tx=0).stop 2024-06-13T07:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 -- 172.21.15.26:0/3398501932 shutdown_connections 2024-06-13T07:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff5500a2c80 0x7ff5500a5140 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff56c103410 0x7ff56c1a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff56c102a50 0x7ff56c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 --2- 172.21.15.26:0/3398501932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56c101860 0x7ff56c19e0f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.383+0000 7ff570c00640 1 -- 172.21.15.26:0/3398501932 >> 172.21.15.26:0/3398501932 conn(0x7ff56c0fcdd0 msgr2=0x7ff56c10dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.384+0000 7ff570c00640 1 -- 172.21.15.26:0/3398501932 shutdown_connections 2024-06-13T07:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:20:52.384+0000 7ff570c00640 1 -- 172.21.15.26:0/3398501932 wait complete. 2024-06-13T07:20:52.427 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:52 smithi136 ceph-mon[37229]: pgmap v1120: 129 pgs: 129 active+clean; 716 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.8 MiB/s wr, 600 op/s 2024-06-13T07:20:52.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:52 smithi026 ceph-mon[32041]: pgmap v1120: 129 pgs: 129 active+clean; 716 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.8 MiB/s wr, 600 op/s 2024-06-13T07:20:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:52 smithi135 ceph-mon[36276]: pgmap v1120: 129 pgs: 129 active+clean; 716 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.8 MiB/s wr, 600 op/s 2024-06-13T07:20:53.203 INFO:tasks.workunit.client.0.smithi026.stdout: 10 216203 26.84 MB/sec execute 306 sec latency 263.366 ms 2024-06-13T07:20:54.203 INFO:tasks.workunit.client.0.smithi026.stdout: 10 216864 26.86 MB/sec execute 307 sec latency 54.115 ms 2024-06-13T07:20:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:54 smithi026 ceph-mon[32041]: pgmap v1121: 129 pgs: 129 active+clean; 701 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-06-13T07:20:54.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:54 smithi135 ceph-mon[36276]: pgmap v1121: 129 pgs: 129 active+clean; 701 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-06-13T07:20:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:54 smithi136 ceph-mon[37229]: pgmap v1121: 129 pgs: 129 active+clean; 701 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-06-13T07:20:55.204 INFO:tasks.workunit.client.0.smithi026.stdout: 10 217290 26.85 MB/sec execute 308 sec latency 130.178 ms 2024-06-13T07:20:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:55 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:20:55.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:55 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:20:55.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:55 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:20:56.204 INFO:tasks.workunit.client.0.smithi026.stdout: 10 217767 26.85 MB/sec execute 309 sec latency 110.519 ms 2024-06-13T07:20:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:56 smithi026 ceph-mon[32041]: pgmap v1122: 129 pgs: 129 active+clean; 681 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.5 MiB/s wr, 605 op/s 2024-06-13T07:20:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:20:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:20:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:56 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:20:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:56 smithi135 ceph-mon[36276]: pgmap v1122: 129 pgs: 129 active+clean; 681 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.5 MiB/s wr, 605 op/s 2024-06-13T07:20:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:20:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:20:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:56 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:20:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:56 smithi136 ceph-mon[37229]: pgmap v1122: 129 pgs: 129 active+clean; 681 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.5 MiB/s wr, 605 op/s 2024-06-13T07:20:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:20:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:20:56.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:56 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:20:57.204 INFO:tasks.workunit.client.0.smithi026.stdout: 10 218213 26.85 MB/sec execute 310 sec latency 82.433 ms 2024-06-13T07:20:58.205 INFO:tasks.workunit.client.0.smithi026.stdout: 10 218732 26.82 MB/sec execute 311 sec latency 71.432 ms 2024-06-13T07:20:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:20:58 smithi026 ceph-mon[32041]: pgmap v1123: 129 pgs: 129 active+clean; 685 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.6 MiB/s wr, 634 op/s 2024-06-13T07:20:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:20:58 smithi135 ceph-mon[36276]: pgmap v1123: 129 pgs: 129 active+clean; 685 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.6 MiB/s wr, 634 op/s 2024-06-13T07:20:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:20:58 smithi136 ceph-mon[37229]: pgmap v1123: 129 pgs: 129 active+clean; 685 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.6 MiB/s wr, 634 op/s 2024-06-13T07:20:59.205 INFO:tasks.workunit.client.0.smithi026.stdout: 10 219334 26.86 MB/sec execute 312 sec latency 134.119 ms 2024-06-13T07:21:00.205 INFO:tasks.workunit.client.0.smithi026.stdout: 10 219983 26.86 MB/sec execute 313 sec latency 66.959 ms 2024-06-13T07:21:00.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:00 smithi026 ceph-mon[32041]: pgmap v1124: 129 pgs: 129 active+clean; 690 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 618 op/s 2024-06-13T07:21:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:00 smithi135 ceph-mon[36276]: pgmap v1124: 129 pgs: 129 active+clean; 690 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 618 op/s 2024-06-13T07:21:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:00 smithi136 ceph-mon[37229]: pgmap v1124: 129 pgs: 129 active+clean; 690 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 618 op/s 2024-06-13T07:21:01.205 INFO:tasks.workunit.client.0.smithi026.stdout: 10 220622 26.90 MB/sec execute 314 sec latency 84.292 ms 2024-06-13T07:21:02.205 INFO:tasks.workunit.client.0.smithi026.stdout: 10 221290 26.96 MB/sec execute 315 sec latency 92.778 ms 2024-06-13T07:21:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:02 smithi026 ceph-mon[32041]: pgmap v1125: 129 pgs: 129 active+clean; 694 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.4 MiB/s wr, 547 op/s 2024-06-13T07:21:02.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:02 smithi135 ceph-mon[36276]: pgmap v1125: 129 pgs: 129 active+clean; 694 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.4 MiB/s wr, 547 op/s 2024-06-13T07:21:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:02 smithi136 ceph-mon[37229]: pgmap v1125: 129 pgs: 129 active+clean; 694 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.4 MiB/s wr, 547 op/s 2024-06-13T07:21:03.205 INFO:tasks.workunit.client.0.smithi026.stdout: 10 221759 26.93 MB/sec execute 316 sec latency 93.478 ms 2024-06-13T07:21:04.206 INFO:tasks.workunit.client.0.smithi026.stdout: 10 222250 26.94 MB/sec execute 317 sec latency 57.267 ms 2024-06-13T07:21:04.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:04 smithi026 ceph-mon[32041]: pgmap v1126: 129 pgs: 129 active+clean; 702 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.4 MiB/s wr, 554 op/s 2024-06-13T07:21:04.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:04 smithi135 ceph-mon[36276]: pgmap v1126: 129 pgs: 129 active+clean; 702 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.4 MiB/s wr, 554 op/s 2024-06-13T07:21:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:04 smithi136 ceph-mon[37229]: pgmap v1126: 129 pgs: 129 active+clean; 702 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.4 MiB/s wr, 554 op/s 2024-06-13T07:21:05.205 INFO:tasks.workunit.client.0.smithi026.stdout: 10 222781 26.94 MB/sec execute 318 sec latency 230.416 ms 2024-06-13T07:21:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.585+0000 7f685fe00640 1 -- 172.21.15.26:0/125298576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 msgr2=0x7f685810c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.585+0000 7f685fe00640 1 --2- 172.21.15.26:0/125298576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 0x7f685810c060 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6850009ca0 tx=0x7f685002a8b0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.586+0000 7f685fe00640 1 -- 172.21.15.26:0/125298576 shutdown_connections 2024-06-13T07:21:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.586+0000 7f685fe00640 1 --2- 172.21.15.26:0/125298576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 0x7f685810c060 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.586+0000 7f685fe00640 1 --2- 172.21.15.26:0/125298576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6858104e10 0x7f6858105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.586+0000 7f685fe00640 1 --2- 172.21.15.26:0/125298576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6858103c20 0x7f6858104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.586+0000 7f685fe00640 1 -- 172.21.15.26:0/125298576 >> 172.21.15.26:0/125298576 conn(0x7f68580ff190 msgr2=0x7f68581015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.586+0000 7f685fe00640 1 -- 172.21.15.26:0/125298576 shutdown_connections 2024-06-13T07:21:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.586+0000 7f685fe00640 1 -- 172.21.15.26:0/125298576 wait complete. 2024-06-13T07:21:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.587+0000 7f685fe00640 1 Processor -- start 2024-06-13T07:21:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.587+0000 7f685fe00640 1 -- start start 2024-06-13T07:21:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.588+0000 7f685fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6858103c20 0x7f6858079750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.588+0000 7f685fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6858104e10 0x7f6858079c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.588+0000 7f685fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 0x7f68580762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.588+0000 7f685fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f685810f8f0 con 0x7f6858103c20 2024-06-13T07:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.588+0000 7f685fe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f685810f770 con 0x7f6858104e10 2024-06-13T07:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.588+0000 7f685fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f685810fa70 con 0x7f68581057d0 2024-06-13T07:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 0x7f68580762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685ca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6858104e10 0x7f6858079c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 0x7f68580762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55430/0 (socket says 172.21.15.26:55430) 2024-06-13T07:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685de00640 1 -- 172.21.15.26:0/763273513 learned_addr learned my addr 172.21.15.26:0/763273513 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685d400640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6858103c20 0x7f6858079750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685de00640 1 -- 172.21.15.26:0/763273513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6858104e10 msgr2=0x7f6858079c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685de00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6858104e10 0x7f6858079c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685de00640 1 -- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6858103c20 msgr2=0x7f6858079750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685de00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6858103c20 0x7f6858079750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.589+0000 7f685de00640 1 -- 172.21.15.26:0/763273513 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6858076b80 con 0x7f68581057d0 2024-06-13T07:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.591+0000 7f685de00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 0x7f68580762c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6850009430 tx=0x7f6850003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.594+0000 7f6856000640 1 -- 172.21.15.26:0/763273513 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850004040 con 0x7f68581057d0 2024-06-13T07:21:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.594+0000 7f6856000640 1 -- 172.21.15.26:0/763273513 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6850040070 con 0x7f68581057d0 2024-06-13T07:21:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.594+0000 7f6856000640 1 -- 172.21.15.26:0/763273513 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850045d00 con 0x7f68581057d0 2024-06-13T07:21:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.594+0000 7f685fe00640 1 -- 172.21.15.26:0/763273513 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68581a6260 con 0x7f68581057d0 2024-06-13T07:21:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.594+0000 7f685fe00640 1 -- 172.21.15.26:0/763273513 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68581a6610 con 0x7f68581057d0 2024-06-13T07:21:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.600+0000 7f6862de6740 1 Processor -- start 2024-06-13T07:21:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.601+0000 7f6856000640 1 -- 172.21.15.26:0/763273513 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6850045470 con 0x7f68581057d0 2024-06-13T07:21:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.601+0000 7f6862de6740 1 -- start start 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f6862de6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d425f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f6862de6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d42640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f6862de6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4266bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f6862de6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5599d41c2420 con 0x5599d425ec00 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f6862de6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5599d4169c50 con 0x5599d4264760 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f6862de6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5599d41be3e0 con 0x5599d425f5f0 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f685d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d42640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f685ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d425f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f685d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d42640f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55438/0 (socket says 172.21.15.26:55438) 2024-06-13T07:21:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f685d400640 1 -- 172.21.15.26:0/4136057070 learned_addr learned my addr 172.21.15.26:0/4136057070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.602+0000 7f685ca00640 1 --2- 172.21.15.26:0/4136057070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4266bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f685d400640 1 -- 172.21.15.26:0/4136057070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 msgr2=0x5599d4266bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f685d400640 1 --2- 172.21.15.26:0/4136057070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4266bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f685d400640 1 -- 172.21.15.26:0/4136057070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 msgr2=0x5599d425f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f685d400640 1 --2- 172.21.15.26:0/4136057070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d425f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f685d400640 1 -- 172.21.15.26:0/4136057070 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d3ed6210 con 0x5599d425f5f0 2024-06-13T07:21:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f685d400640 1 --2- 172.21.15.26:0/4136057070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d42640f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f684c009ca0 tx=0x7f684c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e3af022462a78 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f684a600640 1 -- 172.21.15.26:0/4136057070 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c009e50 con 0x5599d425f5f0 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f6856000640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f68400a2cb0 0x7f68400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f684a600640 1 -- 172.21.15.26:0/4136057070 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c0035f0 con 0x5599d425f5f0 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.603+0000 7f684a600640 1 -- 172.21.15.26:0/4136057070 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c019670 con 0x5599d425f5f0 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.604+0000 7f6856000640 1 -- 172.21.15.26:0/763273513 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f68500f07b0 con 0x7f68581057d0 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.604+0000 7f685ca00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f68400a2cb0 0x7f68400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.604+0000 7f6862de6740 1 -- 172.21.15.26:0/4136057070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 msgr2=0x5599d42640f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.604+0000 7f6862de6740 1 --2- 172.21.15.26:0/4136057070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d42640f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f684c009ca0 tx=0x7f684c011810 comp rx=0 tx=0).stop 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.604+0000 7f6862de6740 1 -- 172.21.15.26:0/4136057070 shutdown_connections 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.604+0000 7f6862de6740 1 --2- 172.21.15.26:0/4136057070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4266bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.605+0000 7f6862de6740 1 --2- 172.21.15.26:0/4136057070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d42640f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.605+0000 7f6862de6740 1 --2- 172.21.15.26:0/4136057070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d425f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.605+0000 7f6862de6740 1 -- 172.21.15.26:0/4136057070 >> 172.21.15.26:0/4136057070 conn(0x5599d4226fb0 msgr2=0x5599d425a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.605+0000 7f685ca00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f68400a2cb0 0x7f68400a5170 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x5599d425fe50 tx=0x7f6844005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.606+0000 7f6862de6740 1 -- 172.21.15.26:0/4136057070 shutdown_connections 2024-06-13T07:21:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.606+0000 7f6862de6740 1 -- 172.21.15.26:0/4136057070 wait complete. 2024-06-13T07:21:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.606+0000 7f6862de6740 1 -- start start 2024-06-13T07:21:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.607+0000 7f6862de6740 1 Processor -- start 2024-06-13T07:21:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.607+0000 7f6862de6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.607+0000 7f6862de6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d4269440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.607+0000 7f6862de6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.607+0000 7f6862de6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5599d4267710 con 0x5599d425ec00 2024-06-13T07:21:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.607+0000 7f6862de6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5599d417d0b0 con 0x5599d4264760 2024-06-13T07:21:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.608+0000 7f685ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d4269440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.608+0000 7f685d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.608+0000 7f685ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d4269440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55450/0 (socket says 172.21.15.26:55450) 2024-06-13T07:21:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.608+0000 7f685d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40298/0 (socket says 172.21.15.26:40298) 2024-06-13T07:21:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.608+0000 7f685ca00640 1 -- 172.21.15.26:0/3613649247 learned_addr learned my addr 172.21.15.26:0/3613649247 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.607+0000 7f6862de6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5599d4267890 con 0x5599d425f5f0 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.608+0000 7f6862de6740 10 client.0 _ll_register_callbacks cb 0x5599d4168f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f685de00640 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f685ca00640 1 -- 172.21.15.26:0/3613649247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 msgr2=0x5599d4270480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f685ca00640 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f685ca00640 1 -- 172.21.15.26:0/3613649247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 msgr2=0x5599d4268f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f685ca00640 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f685ca00640 1 -- 172.21.15.26:0/3613649247 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d3fa2620 con 0x5599d425f5f0 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f685d400640 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f685ca00640 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d4269440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6844004540 tx=0x7f684400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61ebdf748188e1dd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.609+0000 7f683ea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.610+0000 7f683ea00640 1 -- 172.21.15.26:0/3613649247 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684402e060 con 0x5599d425f5f0 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.610+0000 7f6862de6740 1 -- 172.21.15.26:0/3613649247 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f684c0035f0 con 0x5599d425f5f0 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.610+0000 7f683ea00640 1 -- 172.21.15.26:0/3613649247 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f684401b030 con 0x5599d425f5f0 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.611+0000 7f683ea00640 1 -- 172.21.15.26:0/3613649247 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6844035050 con 0x5599d425f5f0 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.611+0000 7f6862de6740 10 client.27016 fetch_fsmap learned FSMap version 47 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.611+0000 7f6862de6740 1 -- 172.21.15.26:0/3613649247 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5599d42738b0 con 0x5599d425f5f0 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.611+0000 7f683ea00640 1 -- 172.21.15.26:0/3613649247 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c0035f0 con 0x5599d425f5f0 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.611+0000 7f683ea00640 1 -- 172.21.15.26:0/3613649247 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f6844010650 con 0x5599d425f5f0 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.612+0000 7f6862de6740 10 client.27016 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.612+0000 7f6862de6740 10 client.27016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:21:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.612+0000 7f6862de6740 20 client.27016 populate_metadata read hostname 'smithi026' 2024-06-13T07:21:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.612+0000 7f6862de6740 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5599d4274fa0 0x5599d4277430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.612+0000 7f6862de6740 4 client.27016 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:21:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.612+0000 7f6862de6740 1 -- 172.21.15.26:0/3613649247 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599d41cfb50 con 0x5599d4274fa0 2024-06-13T07:21:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.613+0000 7f685d400640 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5599d4274fa0 0x5599d4277430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.614+0000 7f685d400640 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5599d4274fa0 0x5599d4277430 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4369c1f69ad48080 server_cookie=e6b2493d6b07630d in_seq=0 out_seq=0 2024-06-13T07:21:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.614+0000 7f683ea00640 10 client.27016 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.617+0000 7f683ea00640 1 -- 172.21.15.26:0/3613649247 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5599d41cfb50 con 0x5599d4274fa0 2024-06-13T07:21:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.617+0000 7f683ea00640 10 client.27016 handle_command_reply: tid=0 2024-06-13T07:21:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.617+0000 7f6862de6740 1 client.27016 shutdown 2024-06-13T07:21:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.617+0000 7f6862de6740 1 -- 172.21.15.26:0/3613649247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 msgr2=0x5599d4269440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.617+0000 7f6862de6740 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d4269440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6844004540 tx=0x7f684400b390 comp rx=0 tx=0).stop 2024-06-13T07:21:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.618+0000 7f6862de6740 1 -- 172.21.15.26:0/3613649247 shutdown_connections 2024-06-13T07:21:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.618+0000 7f6862de6740 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5599d4274fa0 0x5599d4277430 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.618+0000 7f6862de6740 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.618+0000 7f6862de6740 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d4269440 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.618+0000 7f6862de6740 1 --2- 172.21.15.26:0/3613649247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.618+0000 7f6862de6740 1 -- 172.21.15.26:0/3613649247 >> 172.21.15.26:0/3613649247 conn(0x5599d4226fb0 msgr2=0x5599d4259840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.618+0000 7f683ea00640 0 client.27016 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.619+0000 7f6862de6740 1 -- 172.21.15.26:0/3613649247 shutdown_connections 2024-06-13T07:21:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.619+0000 7f6862de6740 1 -- 172.21.15.26:0/3613649247 wait complete. 2024-06-13T07:21:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.620+0000 7f6862de6740 20 client.27016 trim_cache size 0 max 16384 2024-06-13T07:21:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f6862de6740 1 Processor -- start 2024-06-13T07:21:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f6862de6740 1 -- start start 2024-06-13T07:21:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f6862de6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f6862de6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425f5f0 0x5599d4269440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f6862de6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f6862de6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68581b34f0 con 0x5599d425f5f0 2024-06-13T07:21:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f6862de6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68581b0dc0 con 0x5599d4264760 2024-06-13T07:21:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f6862de6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f68581aec10 con 0x5599d425ec00 2024-06-13T07:21:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f685d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f685de00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f685d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55462/0 (socket says 172.21.15.26:55462) 2024-06-13T07:21:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f685d400640 1 -- 172.21.15.26:0/526550913 learned_addr learned my addr 172.21.15.26:0/526550913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.643+0000 7f685ca00640 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425f5f0 0x5599d4269440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.644+0000 7f685d400640 1 -- 172.21.15.26:0/526550913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 msgr2=0x5599d4270440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.644+0000 7f685d400640 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.644+0000 7f685d400640 1 -- 172.21.15.26:0/526550913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425f5f0 msgr2=0x5599d4269440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.644+0000 7f685d400640 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425f5f0 0x5599d4269440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.644+0000 7f685d400640 1 -- 172.21.15.26:0/526550913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d425b020 con 0x5599d425ec00 2024-06-13T07:21:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.644+0000 7f685de00640 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:21:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.644+0000 7f685d400640 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425ec00 0x5599d4268f00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f684c012fd0 tx=0x7f684c005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8262e5bccc282f46 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.645+0000 7f6849c00640 1 -- 172.21.15.26:0/526550913 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c02b070 con 0x5599d425ec00 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.645+0000 7f6849c00640 1 -- 172.21.15.26:0/526550913 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c005520 con 0x5599d425ec00 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.645+0000 7f6849c00640 1 -- 172.21.15.26:0/526550913 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c025970 con 0x5599d425ec00 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.645+0000 7f6862de6740 1 -- 172.21.15.26:0/526550913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425ec00 msgr2=0x5599d4268f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.645+0000 7f6862de6740 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425ec00 0x5599d4268f00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f684c012fd0 tx=0x7f684c005d30 comp rx=0 tx=0).stop 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 -- 172.21.15.26:0/526550913 shutdown_connections 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d4270440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425f5f0 0x5599d4269440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 --2- 172.21.15.26:0/526550913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425ec00 0x5599d4268f00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 -- 172.21.15.26:0/526550913 >> 172.21.15.26:0/526550913 conn(0x5599d4226fb0 msgr2=0x5599d4276fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 -- 172.21.15.26:0/526550913 shutdown_connections 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 -- 172.21.15.26:0/526550913 wait complete. 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 -- start start 2024-06-13T07:21:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.646+0000 7f6862de6740 1 Processor -- start 2024-06-13T07:21:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f6862de6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4270200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f6862de6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d426cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f6862de6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d426d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f6862de6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68581af280 con 0x5599d425ec00 2024-06-13T07:21:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f6862de6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68581b06f0 con 0x5599d4264760 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f6862de6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f68581af950 con 0x5599d425f5f0 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f6862de6740 10 client.0 _ll_register_callbacks cb 0x7f68440122a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685de00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d426d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d426cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4270200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d426cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:55478/0 (socket says 172.21.15.26:55478) 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685ca00640 1 -- 172.21.15.26:0/1103481441 learned_addr learned my addr 172.21.15.26:0/1103481441 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685de00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d426d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:53556/0 (socket says 172.21.15.26:53556) 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685de00640 1 -- 172.21.15.26:0/1103481441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 msgr2=0x5599d426cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685de00640 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d426cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685de00640 1 -- 172.21.15.26:0/1103481441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 msgr2=0x5599d4270200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685de00640 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4270200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.647+0000 7f685de00640 1 -- 172.21.15.26:0/1103481441 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d42ddd00 con 0x5599d4264760 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f685de00640 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d426d1e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6850006010 tx=0x7f68500bd2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1445bc598bb4a33 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f683ea00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f683ea00640 1 -- 172.21.15.26:0/1103481441 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500c5830 con 0x5599d4264760 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f6862de6740 1 -- 172.21.15.26:0/1103481441 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f684c005520 con 0x5599d4264760 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f683ea00640 1 -- 172.21.15.26:0/1103481441 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f68500bce20 con 0x5599d4264760 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f683ea00640 1 -- 172.21.15.26:0/1103481441 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f68500db5e0 con 0x5599d4264760 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f6862de6740 10 client.26969 fetch_fsmap learned FSMap version 47 2024-06-13T07:21:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f6862de6740 1 -- 172.21.15.26:0/1103481441 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5599d42def40 con 0x5599d4264760 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.648+0000 7f683ea00640 1 -- 172.21.15.26:0/1103481441 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c005520 con 0x5599d4264760 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.649+0000 7f683ea00640 1 -- 172.21.15.26:0/1103481441 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f68500bc010 con 0x5599d4264760 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.649+0000 7f6862de6740 10 client.26969 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.649+0000 7f6862de6740 10 client.26969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.649+0000 7f6862de6740 20 client.26969 populate_metadata read hostname 'smithi026' 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.649+0000 7f6862de6740 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5599d42df8a0 0x5599d42e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.649+0000 7f6862de6740 4 client.26969 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.649+0000 7f6862de6740 1 -- 172.21.15.26:0/1103481441 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599d4264460 con 0x5599d42df8a0 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.649+0000 7f685d400640 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5599d42df8a0 0x5599d42e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.650+0000 7f685d400640 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5599d42df8a0 0x5599d42e1cc0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f59bba73b2dd51 server_cookie=838edf25a37f03d4 in_seq=0 out_seq=0 2024-06-13T07:21:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:05.650+0000 7f683ea00640 10 client.26969 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:06.207 INFO:tasks.workunit.client.0.smithi026.stdout: 10 223089 26.89 MB/sec execute 319 sec latency 291.033 ms 2024-06-13T07:21:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f683ea00640 1 -- 172.21.15.26:0/1103481441 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5599d4264460 con 0x5599d42df8a0 2024-06-13T07:21:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f683ea00640 10 client.26969 handle_command_reply: tid=0 2024-06-13T07:21:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 client.26969 shutdown 2024-06-13T07:21:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 -- 172.21.15.26:0/1103481441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 msgr2=0x5599d426d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d426d1e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6850006010 tx=0x7f68500bd2c0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 -- 172.21.15.26:0/1103481441 shutdown_connections 2024-06-13T07:21:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5599d42df8a0 0x5599d42e1cc0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599d4264760 0x5599d426d1e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5599d425f5f0 0x5599d426cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 --2- 172.21.15.26:0/1103481441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d425ec00 0x5599d4270200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 -- 172.21.15.26:0/1103481441 >> 172.21.15.26:0/1103481441 conn(0x5599d4226fb0 msgr2=0x5599d4260870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f683ea00640 0 client.26969 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.371+0000 7f6862de6740 1 -- 172.21.15.26:0/1103481441 shutdown_connections 2024-06-13T07:21:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.372+0000 7f6862de6740 1 -- 172.21.15.26:0/1103481441 wait complete. 2024-06-13T07:21:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.372+0000 7f6862de6740 20 client.26969 trim_cache size 0 max 16384 2024-06-13T07:21:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:21:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:21:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:21:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.373+0000 7f685fe00640 1 -- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f68400a2cb0 msgr2=0x7f68400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.373+0000 7f685fe00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f68400a2cb0 0x7f68400a5170 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x5599d425fe50 tx=0x7f6844005c90 comp rx=0 tx=0).stop 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.373+0000 7f685fe00640 1 -- 172.21.15.26:0/763273513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 msgr2=0x7f68580762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.373+0000 7f685fe00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 0x7f68580762c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6850009430 tx=0x7f6850003970 comp rx=0 tx=0).stop 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.374+0000 7f685fe00640 1 -- 172.21.15.26:0/763273513 shutdown_connections 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.374+0000 7f685fe00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f68400a2cb0 0x7f68400a5170 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.374+0000 7f685fe00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6858104e10 0x7f6858079c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.374+0000 7f685fe00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68581057d0 0x7f68580762c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.374+0000 7f685fe00640 1 --2- 172.21.15.26:0/763273513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6858103c20 0x7f6858079750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.374+0000 7f685fe00640 1 -- 172.21.15.26:0/763273513 >> 172.21.15.26:0/763273513 conn(0x7f68580ff190 msgr2=0x7f6858108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.374+0000 7f685fe00640 1 -- 172.21.15.26:0/763273513 shutdown_connections 2024-06-13T07:21:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:06.374+0000 7f685fe00640 1 -- 172.21.15.26:0/763273513 wait complete. 2024-06-13T07:21:06.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:06 smithi026 ceph-mon[32041]: pgmap v1127: 129 pgs: 129 active+clean; 703 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-06-13T07:21:06.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:06 smithi135 ceph-mon[36276]: pgmap v1127: 129 pgs: 129 active+clean; 703 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-06-13T07:21:06.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:06 smithi136 ceph-mon[37229]: pgmap v1127: 129 pgs: 129 active+clean; 703 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-06-13T07:21:07.206 INFO:tasks.workunit.client.0.smithi026.stdout: 10 223608 26.89 MB/sec execute 320 sec latency 257.755 ms 2024-06-13T07:21:08.206 INFO:tasks.workunit.client.0.smithi026.stdout: 10 223941 26.86 MB/sec execute 321 sec latency 348.358 ms 2024-06-13T07:21:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:08 smithi026 ceph-mon[32041]: pgmap v1128: 129 pgs: 129 active+clean; 703 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 471 op/s 2024-06-13T07:21:08.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:08 smithi135 ceph-mon[36276]: pgmap v1128: 129 pgs: 129 active+clean; 703 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 471 op/s 2024-06-13T07:21:08.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:08 smithi136 ceph-mon[37229]: pgmap v1128: 129 pgs: 129 active+clean; 703 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 471 op/s 2024-06-13T07:21:09.207 INFO:tasks.workunit.client.0.smithi026.stdout: 10 224348 26.83 MB/sec execute 322 sec latency 699.167 ms 2024-06-13T07:21:10.207 INFO:tasks.workunit.client.0.smithi026.stdout: 10 224711 26.82 MB/sec execute 323 sec latency 111.685 ms 2024-06-13T07:21:10.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:10 smithi026 ceph-mon[32041]: pgmap v1129: 129 pgs: 129 active+clean; 704 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-06-13T07:21:10.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:10 smithi135 ceph-mon[36276]: pgmap v1129: 129 pgs: 129 active+clean; 704 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-06-13T07:21:10.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:10 smithi136 ceph-mon[37229]: pgmap v1129: 129 pgs: 129 active+clean; 704 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-06-13T07:21:11.207 INFO:tasks.workunit.client.0.smithi026.stdout: 10 225350 26.88 MB/sec execute 324 sec latency 70.686 ms 2024-06-13T07:21:12.207 INFO:tasks.workunit.client.0.smithi026.stdout: 10 225895 26.88 MB/sec execute 325 sec latency 89.841 ms 2024-06-13T07:21:12.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:12 smithi026 ceph-mon[32041]: pgmap v1130: 129 pgs: 129 active+clean; 705 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-06-13T07:21:12.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:12 smithi135 ceph-mon[36276]: pgmap v1130: 129 pgs: 129 active+clean; 705 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-06-13T07:21:12.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:12 smithi136 ceph-mon[37229]: pgmap v1130: 129 pgs: 129 active+clean; 705 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-06-13T07:21:13.208 INFO:tasks.workunit.client.0.smithi026.stdout: 10 226347 26.88 MB/sec execute 326 sec latency 89.610 ms 2024-06-13T07:21:14.210 INFO:tasks.workunit.client.0.smithi026.stdout: 10 226442 26.81 MB/sec execute 327 sec latency 820.969 ms 2024-06-13T07:21:14.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:14 smithi026 ceph-mon[32041]: pgmap v1131: 129 pgs: 129 active+clean; 695 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-06-13T07:21:14.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:14 smithi135 ceph-mon[36276]: pgmap v1131: 129 pgs: 129 active+clean; 695 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-06-13T07:21:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:14 smithi136 ceph-mon[37229]: pgmap v1131: 129 pgs: 129 active+clean; 695 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-06-13T07:21:15.210 INFO:tasks.workunit.client.0.smithi026.stdout: 10 226563 26.75 MB/sec execute 328 sec latency 1195.106 ms 2024-06-13T07:21:16.210 INFO:tasks.workunit.client.0.smithi026.stdout: 10 226726 26.68 MB/sec execute 329 sec latency 1195.527 ms 2024-06-13T07:21:16.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:16 smithi026 ceph-mon[32041]: pgmap v1132: 129 pgs: 129 active+clean; 689 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 518 op/s 2024-06-13T07:21:16.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:16 smithi135 ceph-mon[36276]: pgmap v1132: 129 pgs: 129 active+clean; 689 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 518 op/s 2024-06-13T07:21:16.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:16 smithi136 ceph-mon[37229]: pgmap v1132: 129 pgs: 129 active+clean; 689 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 518 op/s 2024-06-13T07:21:17.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 226881 26.63 MB/sec execute 330 sec latency 225.792 ms 2024-06-13T07:21:18.210 INFO:tasks.workunit.client.0.smithi026.stdout: 10 227247 26.60 MB/sec execute 331 sec latency 244.342 ms 2024-06-13T07:21:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:18 smithi135 ceph-mon[36276]: pgmap v1133: 129 pgs: 129 active+clean; 698 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.5 MiB/s wr, 601 op/s 2024-06-13T07:21:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:18 smithi136 ceph-mon[37229]: pgmap v1133: 129 pgs: 129 active+clean; 698 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.5 MiB/s wr, 601 op/s 2024-06-13T07:21:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:18 smithi026 ceph-mon[32041]: pgmap v1133: 129 pgs: 129 active+clean; 698 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.5 MiB/s wr, 601 op/s 2024-06-13T07:21:19.210 INFO:tasks.workunit.client.0.smithi026.stdout: 10 227826 26.61 MB/sec execute 332 sec latency 94.762 ms 2024-06-13T07:21:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.585+0000 7f2edfa00640 1 -- 172.21.15.26:0/4263814499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 msgr2=0x7f2ed8108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.585+0000 7f2edfa00640 1 --2- 172.21.15.26:0/4263814499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 0x7f2ed8108dc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009900 tx=0x7f2ecc02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.586+0000 7f2edfa00640 1 -- 172.21.15.26:0/4263814499 shutdown_connections 2024-06-13T07:21:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.586+0000 7f2edfa00640 1 --2- 172.21.15.26:0/4263814499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed8110220 0x7f2ed8112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.586+0000 7f2edfa00640 1 --2- 172.21.15.26:0/4263814499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ed8109300 0x7f2ed810fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.586+0000 7f2edfa00640 1 --2- 172.21.15.26:0/4263814499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 0x7f2ed8108dc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.586+0000 7f2edfa00640 1 -- 172.21.15.26:0/4263814499 >> 172.21.15.26:0/4263814499 conn(0x7f2ed806ea10 msgr2=0x7f2ed8070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.587+0000 7f2edfa00640 1 -- 172.21.15.26:0/4263814499 shutdown_connections 2024-06-13T07:21:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.587+0000 7f2edfa00640 1 -- 172.21.15.26:0/4263814499 wait complete. 2024-06-13T07:21:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.589+0000 7f2edfa00640 1 Processor -- start 2024-06-13T07:21:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.589+0000 7f2edfa00640 1 -- start start 2024-06-13T07:21:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.589+0000 7f2edfa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 0x7f2ed819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.589+0000 7f2edfa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ed8109300 0x7f2ed819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.589+0000 7f2edfa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed8110220 0x7f2ed81a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.589+0000 7f2edfa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81180c0 con 0x7f2ed8109300 2024-06-13T07:21:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.589+0000 7f2edfa00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8117f40 con 0x7f2ed8110220 2024-06-13T07:21:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.589+0000 7f2edfa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8118240 con 0x7f2ed81089a0 2024-06-13T07:21:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 0x7f2ed819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 0x7f2ed819e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58850/0 (socket says 172.21.15.26:58850) 2024-06-13T07:21:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 -- 172.21.15.26:0/2067316695 learned_addr learned my addr 172.21.15.26:0/2067316695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edda00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed8110220 0x7f2ed81a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 -- 172.21.15.26:0/2067316695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed8110220 msgr2=0x7f2ed81a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed8110220 0x7f2ed81a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 -- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ed8109300 msgr2=0x7f2ed819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ed8109300 0x7f2ed819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 -- 172.21.15.26:0/2067316695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed81b6fa0 con 0x7f2ed81089a0 2024-06-13T07:21:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.590+0000 7f2edd000640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 0x7f2ed819e210 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0058c0 tx=0x7f2ecc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.591+0000 7f2ed5600640 1 -- 172.21.15.26:0/2067316695 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc03f070 con 0x7f2ed81089a0 2024-06-13T07:21:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.591+0000 7f2ed5600640 1 -- 172.21.15.26:0/2067316695 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc031cc0 con 0x7f2ed81089a0 2024-06-13T07:21:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.591+0000 7f2ed5600640 1 -- 172.21.15.26:0/2067316695 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc035030 con 0x7f2ed81089a0 2024-06-13T07:21:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.591+0000 7f2edfa00640 1 -- 172.21.15.26:0/2067316695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed81b7170 con 0x7f2ed81089a0 2024-06-13T07:21:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.592+0000 7f2edfa00640 1 -- 172.21.15.26:0/2067316695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81b7580 con 0x7f2ed81089a0 2024-06-13T07:21:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.596+0000 7f2ed5600640 1 -- 172.21.15.26:0/2067316695 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ecc042070 con 0x7f2ed81089a0 2024-06-13T07:21:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.597+0000 7f2ed5600640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ec00a2da0 0x7f2ec00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.597+0000 7f2ed5600640 1 -- 172.21.15.26:0/2067316695 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2ecc03b4f0 con 0x7f2ed81089a0 2024-06-13T07:21:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.598+0000 7f2ee280c740 1 Processor -- start 2024-06-13T07:21:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.598+0000 7f2ed7e00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ec00a2da0 0x7f2ec00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.599+0000 7f2ee280c740 1 -- start start 2024-06-13T07:21:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727a9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727a95f0 0x5652727ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727ae760 0x5652727b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2ee280c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56527270c420 con 0x5652727a95f0 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2ee280c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5652726b3c50 con 0x5652727ae760 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2ee280c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5652727083e0 con 0x5652727a8c00 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2edd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2edd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58866/0 (socket says 172.21.15.26:58866) 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2edd000640 1 -- 172.21.15.26:0/1558587495 learned_addr learned my addr 172.21.15.26:0/1558587495 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.600+0000 7f2edda00640 1 --2- 172.21.15.26:0/1558587495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727ae760 0x5652727b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2edd000640 1 -- 172.21.15.26:0/1558587495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727ae760 msgr2=0x5652727b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2edd000640 1 --2- 172.21.15.26:0/1558587495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727ae760 0x5652727b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2edd000640 1 -- 172.21.15.26:0/1558587495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727a95f0 msgr2=0x5652727ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2edd000640 1 --2- 172.21.15.26:0/1558587495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727a95f0 0x5652727ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2edd000640 1 -- 172.21.15.26:0/1558587495 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565272420210 con 0x5652727a8c00 2024-06-13T07:21:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2edd000640 1 --2- 172.21.15.26:0/1558587495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727a9020 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0bd000 tx=0x7f2ecc03b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7594bcf999b40d79 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2eca000640 1 -- 172.21.15.26:0/1558587495 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc049040 con 0x5652727a8c00 2024-06-13T07:21:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2eca000640 1 -- 172.21.15.26:0/1558587495 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc0bd070 con 0x5652727a8c00 2024-06-13T07:21:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.601+0000 7f2eca000640 1 -- 172.21.15.26:0/1558587495 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0b96e0 con 0x5652727a8c00 2024-06-13T07:21:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 -- 172.21.15.26:0/1558587495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 msgr2=0x5652727a9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 --2- 172.21.15.26:0/1558587495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727a9020 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0bd000 tx=0x7f2ecc03b460 comp rx=0 tx=0).stop 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ed7e00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ec00a2da0 0x7f2ec00a5260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4005fd0 tx=0x7f2ec400be80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 -- 172.21.15.26:0/1558587495 shutdown_connections 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 --2- 172.21.15.26:0/1558587495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727ae760 0x5652727b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 --2- 172.21.15.26:0/1558587495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727a95f0 0x5652727ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 --2- 172.21.15.26:0/1558587495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727a9020 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 -- 172.21.15.26:0/1558587495 >> 172.21.15.26:0/1558587495 conn(0x565272770fb0 msgr2=0x5652727a4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 -- 172.21.15.26:0/1558587495 shutdown_connections 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.603+0000 7f2ee280c740 1 -- 172.21.15.26:0/1558587495 wait complete. 2024-06-13T07:21:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.604+0000 7f2ee280c740 1 -- start start 2024-06-13T07:21:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.604+0000 7f2ee280c740 1 Processor -- start 2024-06-13T07:21:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2ee280c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5652727b14d0 con 0x5652727ae760 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2ee280c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5652726c70b0 con 0x5652727a95f0 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2ee280c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5652727b1650 con 0x5652727a8c00 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2ee280c740 10 client.0 _ll_register_callbacks cb 0x5652726b2f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2edd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2edd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58872/0 (socket says 172.21.15.26:58872) 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2edd000640 1 -- 172.21.15.26:0/3378033477 learned_addr learned my addr 172.21.15.26:0/3378033477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.605+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.606+0000 7f2edd000640 1 -- 172.21.15.26:0/3378033477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 msgr2=0x5652727b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.606+0000 7f2edd000640 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.606+0000 7f2edd000640 1 -- 172.21.15.26:0/3378033477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 msgr2=0x5652727ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.606+0000 7f2edd000640 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.606+0000 7f2edd000640 1 -- 172.21.15.26:0/3378033477 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652724ec620 con 0x5652727a8c00 2024-06-13T07:21:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.606+0000 7f2edd000640 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b2d10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc006010 tx=0x7f2ecc031f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc91224366a586b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.606+0000 7f2ebea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:21:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.607+0000 7f2ebea00640 1 -- 172.21.15.26:0/3378033477 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0d5070 con 0x5652727a8c00 2024-06-13T07:21:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.607+0000 7f2ebea00640 1 -- 172.21.15.26:0/3378033477 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2ecc0b96e0 con 0x5652727a8c00 2024-06-13T07:21:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.607+0000 7f2ee280c740 1 -- 172.21.15.26:0/3378033477 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ecc0bd070 con 0x5652727a8c00 2024-06-13T07:21:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.607+0000 7f2ebea00640 1 -- 172.21.15.26:0/3378033477 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2ecc049040 con 0x5652727a8c00 2024-06-13T07:21:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.607+0000 7f2ee280c740 10 client.27034 fetch_fsmap learned FSMap version 47 2024-06-13T07:21:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.607+0000 7f2ee280c740 1 -- 172.21.15.26:0/3378033477 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5652727b14d0 con 0x5652727a8c00 2024-06-13T07:21:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.607+0000 7f2ebea00640 1 -- 172.21.15.26:0/3378033477 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc0bd070 con 0x5652727a8c00 2024-06-13T07:21:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.608+0000 7f2ebea00640 1 -- 172.21.15.26:0/3378033477 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f2ecc0c5850 con 0x5652727a8c00 2024-06-13T07:21:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.609+0000 7f2ee280c740 10 client.27034 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:21:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.609+0000 7f2ee280c740 10 client.27034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:21:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.609+0000 7f2ee280c740 20 client.27034 populate_metadata read hostname 'smithi026' 2024-06-13T07:21:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.609+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5652727beac0 0x5652727c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.609+0000 7f2ee280c740 4 client.27034 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:21:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.609+0000 7f2ee280c740 1 -- 172.21.15.26:0/3378033477 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565272719b50 con 0x5652727beac0 2024-06-13T07:21:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.611+0000 7f2ed7e00640 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5652727beac0 0x5652727c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.612+0000 7f2ed7e00640 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5652727beac0 0x5652727c0eb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515762edbae3bcd5 server_cookie=8bc1a505fd6e20a2 in_seq=0 out_seq=0 2024-06-13T07:21:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.613+0000 7f2ebea00640 10 client.27034 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.616+0000 7f2ebea00640 1 -- 172.21.15.26:0/3378033477 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565272719b50 con 0x5652727beac0 2024-06-13T07:21:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ebea00640 10 client.27034 handle_command_reply: tid=0 2024-06-13T07:21:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 client.27034 shutdown 2024-06-13T07:21:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 -- 172.21.15.26:0/3378033477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 msgr2=0x5652727b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b2d10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc006010 tx=0x7f2ecc031f20 comp rx=0 tx=0).stop 2024-06-13T07:21:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 -- 172.21.15.26:0/3378033477 shutdown_connections 2024-06-13T07:21:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5652727beac0 0x5652727c0eb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3378033477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b2d10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.617+0000 7f2ee280c740 1 -- 172.21.15.26:0/3378033477 >> 172.21.15.26:0/3378033477 conn(0x565272770fb0 msgr2=0x5652727a3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.618+0000 7f2ebea00640 0 client.27034 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.618+0000 7f2ee280c740 1 -- 172.21.15.26:0/3378033477 shutdown_connections 2024-06-13T07:21:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.618+0000 7f2ee280c740 1 -- 172.21.15.26:0/3378033477 wait complete. 2024-06-13T07:21:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.618+0000 7f2ee280c740 20 client.27034 trim_cache size 0 max 16384 2024-06-13T07:21:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.640+0000 7f2ee280c740 1 Processor -- start 2024-06-13T07:21:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.640+0000 7f2ee280c740 1 -- start start 2024-06-13T07:21:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.640+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a8c00 0x5652727b1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.640+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a95f0 0x5652727ba220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.640+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.640+0000 7f2ee280c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81c45e0 con 0x5652727ae760 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.640+0000 7f2ee280c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81c1eb0 con 0x5652727a8c00 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.640+0000 7f2ee280c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81bfd00 con 0x5652727a95f0 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edd000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a8c00 0x5652727b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edd000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a8c00 0x5652727b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59656/0 (socket says 172.21.15.26:59656) 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edd000640 1 -- 172.21.15.26:0/1089584679 learned_addr learned my addr 172.21.15.26:0/1089584679 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edda00640 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2ed7e00640 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a95f0 0x5652727ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edd000640 1 -- 172.21.15.26:0/1089584679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a95f0 msgr2=0x5652727ba220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edd000640 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a95f0 0x5652727ba220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edd000640 1 -- 172.21.15.26:0/1089584679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 msgr2=0x5652727c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edd000640 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.641+0000 7f2edd000640 1 -- 172.21.15.26:0/1089584679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652727a5330 con 0x5652727a8c00 2024-06-13T07:21:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.642+0000 7f2edd000640 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a8c00 0x5652727b1940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0bcd70 tx=0x7f2ecc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d0a0a03080f856 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.642+0000 7f2ec9600640 1 -- 172.21.15.26:0/1089584679 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0d5070 con 0x5652727a8c00 2024-06-13T07:21:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.642+0000 7f2ec9600640 1 -- 172.21.15.26:0/1089584679 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc0bd070 con 0x5652727a8c00 2024-06-13T07:21:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.642+0000 7f2ec9600640 1 -- 172.21.15.26:0/1089584679 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0c5850 con 0x5652727a8c00 2024-06-13T07:21:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.643+0000 7f2ee280c740 1 -- 172.21.15.26:0/1089584679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a8c00 msgr2=0x5652727b1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.643+0000 7f2ee280c740 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a8c00 0x5652727b1940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0bcd70 tx=0x7f2ecc009260 comp rx=0 tx=0).stop 2024-06-13T07:21:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.643+0000 7f2ee280c740 1 -- 172.21.15.26:0/1089584679 shutdown_connections 2024-06-13T07:21:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.643+0000 7f2ee280c740 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.643+0000 7f2ee280c740 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a95f0 0x5652727ba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.643+0000 7f2ee280c740 1 --2- 172.21.15.26:0/1089584679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a8c00 0x5652727b1940 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.643+0000 7f2ee280c740 1 -- 172.21.15.26:0/1089584679 >> 172.21.15.26:0/1089584679 conn(0x565272770fb0 msgr2=0x5652727aa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.643+0000 7f2ee280c740 1 -- 172.21.15.26:0/1089584679 shutdown_connections 2024-06-13T07:21:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.644+0000 7f2ee280c740 1 -- 172.21.15.26:0/1089584679 wait complete. 2024-06-13T07:21:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.644+0000 7f2ee280c740 1 -- start start 2024-06-13T07:21:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.644+0000 7f2ee280c740 1 Processor -- start 2024-06-13T07:21:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2ee280c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2ee280c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81c0370 con 0x5652727ae760 2024-06-13T07:21:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2ee280c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81c17e0 con 0x5652727a95f0 2024-06-13T07:21:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2ee280c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81c0a40 con 0x5652727a8c00 2024-06-13T07:21:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2ee280c740 10 client.0 _ll_register_callbacks cb 0x7f2ecc0bce40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:21:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2edd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2edd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58894/0 (socket says 172.21.15.26:58894) 2024-06-13T07:21:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2edd000640 1 -- 172.21.15.26:0/3985927963 learned_addr learned my addr 172.21.15.26:0/3985927963 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.645+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.646+0000 7f2edd000640 1 -- 172.21.15.26:0/3985927963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 msgr2=0x5652727b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.646+0000 7f2edd000640 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.646+0000 7f2edd000640 1 -- 172.21.15.26:0/3985927963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 msgr2=0x5652727b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.646+0000 7f2edd000640 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.646+0000 7f2edd000640 1 -- 172.21.15.26:0/3985927963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652727b75f0 con 0x5652727a8c00 2024-06-13T07:21:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.646+0000 7f2ed7e00640 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.648+0000 7f2edd000640 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b9ef0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc031400 tx=0x7f2ecc0dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a55dedbcdddbebb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.648+0000 7f2ebea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.648+0000 7f2ebea00640 1 -- 172.21.15.26:0/3985927963 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0d5070 con 0x5652727a8c00 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.648+0000 7f2ee280c740 1 -- 172.21.15.26:0/3985927963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ecc0bd070 con 0x5652727a8c00 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.648+0000 7f2ebea00640 1 -- 172.21.15.26:0/3985927963 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2ecc0bd420 con 0x5652727a8c00 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.648+0000 7f2ebea00640 1 -- 172.21.15.26:0/3985927963 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2ecc0b96e0 con 0x5652727a8c00 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.648+0000 7f2ee280c740 10 client.27037 fetch_fsmap learned FSMap version 47 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.648+0000 7f2ee280c740 1 -- 172.21.15.26:0/3985927963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5652727b75f0 con 0x5652727a8c00 2024-06-13T07:21:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.649+0000 7f2ebea00640 1 -- 172.21.15.26:0/3985927963 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc0bd070 con 0x5652727a8c00 2024-06-13T07:21:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.649+0000 7f2ebea00640 1 -- 172.21.15.26:0/3985927963 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f2ecc0bc5e0 con 0x5652727a8c00 2024-06-13T07:21:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.650+0000 7f2ee280c740 10 client.27037 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:21:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.650+0000 7f2ee280c740 10 client.27037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:21:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.650+0000 7f2ee280c740 20 client.27037 populate_metadata read hostname 'smithi026' 2024-06-13T07:21:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.650+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x565272828c40 0x56527282b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.650+0000 7f2ee280c740 4 client.27037 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:21:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.650+0000 7f2ee280c740 1 -- 172.21.15.26:0/3985927963 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565272713ea0 con 0x565272828c40 2024-06-13T07:21:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.650+0000 7f2ed7e00640 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x565272828c40 0x56527282b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.652+0000 7f2ed7e00640 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x565272828c40 0x56527282b060 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f52440726be59c server_cookie=456b8e661af6a94a in_seq=0 out_seq=0 2024-06-13T07:21:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:19.653+0000 7f2ebea00640 10 client.27037 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:20.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 228168 26.58 MB/sec execute 333 sec latency 518.296 ms 2024-06-13T07:21:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.439+0000 7f2ebea00640 1 -- 172.21.15.26:0/3985927963 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565272713ea0 con 0x565272828c40 2024-06-13T07:21:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.439+0000 7f2ebea00640 10 client.27037 handle_command_reply: tid=0 2024-06-13T07:21:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.439+0000 7f2ee280c740 1 client.27037 shutdown 2024-06-13T07:21:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.439+0000 7f2ee280c740 1 -- 172.21.15.26:0/3985927963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 msgr2=0x5652727b9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.439+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b9ef0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc031400 tx=0x7f2ecc0dd040 comp rx=0 tx=0).stop 2024-06-13T07:21:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.440+0000 7f2ee280c740 1 -- 172.21.15.26:0/3985927963 shutdown_connections 2024-06-13T07:21:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.440+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x565272828c40 0x56527282b060 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.440+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652727a95f0 0x5652727b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.440+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652727ae760 0x5652727b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.440+0000 7f2ee280c740 1 --2- 172.21.15.26:0/3985927963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652727a8c00 0x5652727b9ef0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.440+0000 7f2ee280c740 1 -- 172.21.15.26:0/3985927963 >> 172.21.15.26:0/3985927963 conn(0x565272770fb0 msgr2=0x5652727a7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.440+0000 7f2ebea00640 0 client.27037 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.443+0000 7f2ee280c740 1 -- 172.21.15.26:0/3985927963 shutdown_connections 2024-06-13T07:21:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.443+0000 7f2ee280c740 1 -- 172.21.15.26:0/3985927963 wait complete. 2024-06-13T07:21:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.443+0000 7f2ee280c740 20 client.27037 trim_cache size 0 max 16384 2024-06-13T07:21:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:21:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:21:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:21:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 -- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ec00a2da0 msgr2=0x7f2ec00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ec00a2da0 0x7f2ec00a5260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4005fd0 tx=0x7f2ec400be80 comp rx=0 tx=0).stop 2024-06-13T07:21:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 -- 172.21.15.26:0/2067316695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 msgr2=0x7f2ed819e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 0x7f2ed819e210 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0058c0 tx=0x7f2ecc003c10 comp rx=0 tx=0).stop 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 -- 172.21.15.26:0/2067316695 shutdown_connections 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2ec00a2da0 0x7f2ec00a5260 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed8110220 0x7f2ed81a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ed8109300 0x7f2ed819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 --2- 172.21.15.26:0/2067316695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ed81089a0 0x7f2ed819e210 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.445+0000 7f2edfa00640 1 -- 172.21.15.26:0/2067316695 >> 172.21.15.26:0/2067316695 conn(0x7f2ed806ea10 msgr2=0x7f2ed8070600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.446+0000 7f2edfa00640 1 -- 172.21.15.26:0/2067316695 shutdown_connections 2024-06-13T07:21:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:20.446+0000 7f2edfa00640 1 -- 172.21.15.26:0/2067316695 wait complete. 2024-06-13T07:21:20.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:20 smithi135 ceph-mon[36276]: pgmap v1134: 129 pgs: 129 active+clean; 685 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 666 op/s 2024-06-13T07:21:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:20 smithi136 ceph-mon[37229]: pgmap v1134: 129 pgs: 129 active+clean; 685 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 666 op/s 2024-06-13T07:21:20.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:20 smithi026 ceph-mon[32041]: pgmap v1134: 129 pgs: 129 active+clean; 685 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 666 op/s 2024-06-13T07:21:21.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 228744 26.58 MB/sec execute 334 sec latency 675.001 ms 2024-06-13T07:21:22.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 229498 26.64 MB/sec execute 335 sec latency 45.634 ms 2024-06-13T07:21:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:22 smithi135 ceph-mon[36276]: pgmap v1135: 129 pgs: 129 active+clean; 663 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-06-13T07:21:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:22 smithi136 ceph-mon[37229]: pgmap v1135: 129 pgs: 129 active+clean; 663 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-06-13T07:21:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:22 smithi026 ceph-mon[32041]: pgmap v1135: 129 pgs: 129 active+clean; 663 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-06-13T07:21:23.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 230118 26.69 MB/sec execute 336 sec latency 42.480 ms 2024-06-13T07:21:24.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 230637 26.68 MB/sec execute 337 sec latency 73.173 ms 2024-06-13T07:21:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:24 smithi135 ceph-mon[36276]: pgmap v1136: 129 pgs: 129 active+clean; 659 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 609 op/s 2024-06-13T07:21:24.550 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:24 smithi136 ceph-mon[37229]: pgmap v1136: 129 pgs: 129 active+clean; 659 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 609 op/s 2024-06-13T07:21:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:24 smithi026 ceph-mon[32041]: pgmap v1136: 129 pgs: 129 active+clean; 659 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 609 op/s 2024-06-13T07:21:25.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 231003 26.65 MB/sec execute 338 sec latency 85.719 ms 2024-06-13T07:21:26.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 231510 26.64 MB/sec execute 339 sec latency 106.331 ms 2024-06-13T07:21:26.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:26 smithi136 ceph-mon[37229]: pgmap v1137: 129 pgs: 129 active+clean; 669 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.4 MiB/s wr, 632 op/s 2024-06-13T07:21:26.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:26 smithi026 ceph-mon[32041]: pgmap v1137: 129 pgs: 129 active+clean; 669 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.4 MiB/s wr, 632 op/s 2024-06-13T07:21:26.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:26 smithi135 ceph-mon[36276]: pgmap v1137: 129 pgs: 129 active+clean; 669 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.4 MiB/s wr, 632 op/s 2024-06-13T07:21:27.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 232331 26.67 MB/sec execute 340 sec latency 60.751 ms 2024-06-13T07:21:28.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 233047 26.74 MB/sec execute 341 sec latency 121.482 ms 2024-06-13T07:21:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:28 smithi136 ceph-mon[37229]: pgmap v1138: 129 pgs: 129 active+clean; 667 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 586 op/s 2024-06-13T07:21:28.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:28 smithi026 ceph-mon[32041]: pgmap v1138: 129 pgs: 129 active+clean; 667 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 586 op/s 2024-06-13T07:21:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:28 smithi135 ceph-mon[36276]: pgmap v1138: 129 pgs: 129 active+clean; 667 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 586 op/s 2024-06-13T07:21:29.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 233575 26.76 MB/sec execute 342 sec latency 67.064 ms 2024-06-13T07:21:30.211 INFO:tasks.workunit.client.0.smithi026.stdout: 10 233807 26.72 MB/sec execute 343 sec latency 289.850 ms 2024-06-13T07:21:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:30 smithi136 ceph-mon[37229]: pgmap v1139: 129 pgs: 129 active+clean; 672 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.0 MiB/s wr, 559 op/s 2024-06-13T07:21:30.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:30 smithi026 ceph-mon[32041]: pgmap v1139: 129 pgs: 129 active+clean; 672 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.0 MiB/s wr, 559 op/s 2024-06-13T07:21:30.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:30 smithi135 ceph-mon[36276]: pgmap v1139: 129 pgs: 129 active+clean; 672 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.0 MiB/s wr, 559 op/s 2024-06-13T07:21:31.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 234131 26.69 MB/sec execute 344 sec latency 114.998 ms 2024-06-13T07:21:32.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 234676 26.69 MB/sec execute 345 sec latency 124.308 ms 2024-06-13T07:21:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:32 smithi136 ceph-mon[37229]: pgmap v1140: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 455 op/s 2024-06-13T07:21:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:32 smithi026 ceph-mon[32041]: pgmap v1140: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 455 op/s 2024-06-13T07:21:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:32 smithi135 ceph-mon[36276]: pgmap v1140: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 455 op/s 2024-06-13T07:21:33.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 235188 26.67 MB/sec execute 346 sec latency 131.043 ms 2024-06-13T07:21:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.641+0000 7f5f0ce00640 1 -- 172.21.15.26:0/1190631654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f08104e10 msgr2=0x7f5f08105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.641+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/1190631654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f08104e10 0x7f5f08105290 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009900 tx=0x7f5ef402a3f0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.641+0000 7f5f0ce00640 1 -- 172.21.15.26:0/1190631654 shutdown_connections 2024-06-13T07:21:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.641+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/1190631654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f081057d0 0x7f5f0810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.641+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/1190631654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f08104e10 0x7f5f08105290 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.641+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/1190631654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f08103c20 0x7f5f08104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.641+0000 7f5f0ce00640 1 -- 172.21.15.26:0/1190631654 >> 172.21.15.26:0/1190631654 conn(0x7f5f080ff190 msgr2=0x7f5f081015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.642+0000 7f5f0ce00640 1 -- 172.21.15.26:0/1190631654 shutdown_connections 2024-06-13T07:21:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.642+0000 7f5f0ce00640 1 -- 172.21.15.26:0/1190631654 wait complete. 2024-06-13T07:21:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.643+0000 7f5f0ce00640 1 Processor -- start 2024-06-13T07:21:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.644+0000 7f5f0ce00640 1 -- start start 2024-06-13T07:21:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.644+0000 7f5f0ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f08103c20 0x7f5f08199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.644+0000 7f5f0ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f08104e10 0x7f5f0819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.644+0000 7f5f0ce00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f081057d0 0x7f5f081a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f0ce00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0810f8f0 con 0x7f5f08104e10 2024-06-13T07:21:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f0ce00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0810f770 con 0x7f5f081057d0 2024-06-13T07:21:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f0ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0810fa70 con 0x7f5f08103c20 2024-06-13T07:21:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f05e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f08103c20 0x7f5f08199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f05e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f08103c20 0x7f5f08199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49084/0 (socket says 172.21.15.26:49084) 2024-06-13T07:21:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f05e00640 1 -- 172.21.15.26:0/2677993870 learned_addr learned my addr 172.21.15.26:0/2677993870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f06800640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f081057d0 0x7f5f081a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f05e00640 1 -- 172.21.15.26:0/2677993870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f081057d0 msgr2=0x7f5f081a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f05e00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f081057d0 0x7f5f081a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f05e00640 1 -- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f08104e10 msgr2=0x7f5f0819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f05e00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f08104e10 0x7f5f0819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f05e00640 1 -- 172.21.15.26:0/2677993870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f081a1ac0 con 0x7f5f08103c20 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.645+0000 7f5f06800640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f081057d0 0x7f5f081a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.646+0000 7f5f05e00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f08103c20 0x7f5f08199ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800e9c0 tx=0x7f5ef800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.646+0000 7f5efea00640 1 -- 172.21.15.26:0/2677993870 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800ccc0 con 0x7f5f08103c20 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.646+0000 7f5f0ce00640 1 -- 172.21.15.26:0/2677993870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f081a1db0 con 0x7f5f08103c20 2024-06-13T07:21:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.646+0000 7f5f0ce00640 1 -- 172.21.15.26:0/2677993870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f081a2270 con 0x7f5f08103c20 2024-06-13T07:21:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.646+0000 7f5efea00640 1 -- 172.21.15.26:0/2677993870 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef80035f0 con 0x7f5f08103c20 2024-06-13T07:21:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.646+0000 7f5efea00640 1 -- 172.21.15.26:0/2677993870 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800ce60 con 0x7f5f08103c20 2024-06-13T07:21:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.652+0000 7f5efea00640 1 -- 172.21.15.26:0/2677993870 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ef8004000 con 0x7f5f08103c20 2024-06-13T07:21:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.652+0000 7f5f0fc34740 1 Processor -- start 2024-06-13T07:21:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.653+0000 7f5f0fc34740 1 -- start start 2024-06-13T07:21:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.653+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0410 0x55af2cea0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.654+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0e00 0x55af2cea5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.654+0000 7f5efea00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5eec0a2cb0 0x7f5eec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f06800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0410 0x55af2cea0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f06800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0410 0x55af2cea0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54740/0 (socket says 172.21.15.26:54740) 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.654+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2cea8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5efea00640 1 -- 172.21.15.26:0/2677993870 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5ef80cf710 con 0x7f5f08103c20 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f05e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0e00 0x55af2cea5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f05e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0e00 0x55af2cea5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:33024/0 (socket says 172.21.15.26:33024) 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f05e00640 1 -- 172.21.15.26:0/1143183789 learned_addr learned my addr 172.21.15.26:0/1143183789 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1143183789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55af2cdfb420 con 0x55af2cea0410 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1143183789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55af2cda2c50 con 0x55af2cea0e00 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1143183789 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55af2cdf73e0 con 0x55af2cea5f70 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.655+0000 7f5f05400640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5eec0a2cb0 0x7f5eec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.656+0000 7f5f06800640 1 -- 172.21.15.26:0/1143183789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 msgr2=0x55af2cea8400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:21:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.656+0000 7f5f06800640 1 --2- 172.21.15.26:0/1143183789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2cea8400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.656+0000 7f5f06800640 1 -- 172.21.15.26:0/1143183789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0e00 msgr2=0x55af2cea5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.656+0000 7f5f06800640 1 --2- 172.21.15.26:0/1143183789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0e00 0x55af2cea5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.656+0000 7f5f06800640 1 -- 172.21.15.26:0/1143183789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af2cb0f210 con 0x55af2cea0410 2024-06-13T07:21:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.657+0000 7f5f05e00640 1 --2- 172.21.15.26:0/1143183789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0e00 0x55af2cea5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:21:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.657+0000 7f5f06800640 1 --2- 172.21.15.26:0/1143183789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0410 0x55af2cea0830 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5f00011760 tx=0x7f5f00011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6e00c56c330215 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.657+0000 7f5f05400640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5eec0a2cb0 0x7f5eec0a5170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40057c0 tx=0x7f5ef4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.658+0000 7f5ef3400640 1 -- 172.21.15.26:0/1143183789 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00004d20 con 0x55af2cea0410 2024-06-13T07:21:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.658+0000 7f5ef3400640 1 -- 172.21.15.26:0/1143183789 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f00003970 con 0x55af2cea0410 2024-06-13T07:21:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5ef3400640 1 -- 172.21.15.26:0/1143183789 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00004610 con 0x55af2cea0410 2024-06-13T07:21:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1143183789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0410 msgr2=0x55af2cea0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1143183789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0410 0x55af2cea0830 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5f00011760 tx=0x7f5f00011c20 comp rx=0 tx=0).stop 2024-06-13T07:21:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1143183789 shutdown_connections 2024-06-13T07:21:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1143183789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0e00 0x55af2cea5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1143183789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2cea8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1143183789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0410 0x55af2cea0830 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1143183789 >> 172.21.15.26:0/1143183789 conn(0x55af2ce5ffb0 msgr2=0x55af2ce9be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1143183789 shutdown_connections 2024-06-13T07:21:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.659+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1143183789 wait complete. 2024-06-13T07:21:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.660+0000 7f5f0fc34740 1 -- start start 2024-06-13T07:21:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.660+0000 7f5f0fc34740 1 Processor -- start 2024-06-13T07:21:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.661+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.661+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceaac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.661+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceb1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.661+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55af2cea8f70 con 0x55af2cea0e00 2024-06-13T07:21:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.661+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55af2cdb60b0 con 0x55af2cea0410 2024-06-13T07:21:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.661+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55af2cea90f0 con 0x55af2cea5f70 2024-06-13T07:21:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.661+0000 7f5f0fc34740 10 client.0 _ll_register_callbacks cb 0x55af2cda1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:21:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.661+0000 7f5f05400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceaac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.662+0000 7f5f05400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceaac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54752/0 (socket says 172.21.15.26:54752) 2024-06-13T07:21:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.662+0000 7f5f05400640 1 -- 172.21.15.26:0/1020040223 learned_addr learned my addr 172.21.15.26:0/1020040223 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.662+0000 7f5f05e00640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.662+0000 7f5f06800640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceb1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.663+0000 7f5f05400640 1 -- 172.21.15.26:0/1020040223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 msgr2=0x55af2ceb1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.663+0000 7f5f05400640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceb1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.663+0000 7f5f05400640 1 -- 172.21.15.26:0/1020040223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 msgr2=0x55af2ceaa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.663+0000 7f5f05400640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.663+0000 7f5f05400640 1 -- 172.21.15.26:0/1020040223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af2cbdb620 con 0x55af2cea0e00 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.663+0000 7f5f05e00640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.663+0000 7f5f06800640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceb1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:21:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.664+0000 7f5f05400640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceaac00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009900 tx=0x7f5ef4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d951757f80b797e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.664+0000 7f5ee7e00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.665+0000 7f5ee7e00640 1 -- 172.21.15.26:0/1020040223 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4040070 con 0x55af2cea0e00 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.665+0000 7f5ee7e00640 1 -- 172.21.15.26:0/1020040223 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5ef40035f0 con 0x55af2cea0e00 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.665+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1020040223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f00003970 con 0x55af2cea0e00 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.666+0000 7f5ee7e00640 1 -- 172.21.15.26:0/1020040223 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5ef4035040 con 0x55af2cea0e00 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.666+0000 7f5f0fc34740 10 client.16752 fetch_fsmap learned FSMap version 47 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.666+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1020040223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55af2cec6030 con 0x55af2cea0e00 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.666+0000 7f5ee7e00640 1 -- 172.21.15.26:0/1020040223 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f00003970 con 0x55af2cea0e00 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.667+0000 7f5ee7e00640 1 -- 172.21.15.26:0/1020040223 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f5ef40478a0 con 0x55af2cea0e00 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.667+0000 7f5f0fc34740 10 client.16752 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:21:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.667+0000 7f5f0fc34740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:21:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.667+0000 7f5f0fc34740 20 client.16752 populate_metadata read hostname 'smithi026' 2024-06-13T07:21:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.667+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55af2cec7880 0x55af2cec9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.667+0000 7f5f0fc34740 4 client.16752 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:21:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.667+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1020040223 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af2ce08b50 con 0x55af2cec7880 2024-06-13T07:21:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.667+0000 7f5f05e00640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55af2cec7880 0x55af2cec9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.668+0000 7f5f05e00640 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55af2cec7880 0x55af2cec9ce0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d730138fcdf3765 server_cookie=1ca5de7928cadaa9 in_seq=0 out_seq=0 2024-06-13T07:21:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.669+0000 7f5ee7e00640 10 client.16752 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.672+0000 7f5ee7e00640 1 -- 172.21.15.26:0/1020040223 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55af2ce08b50 con 0x55af2cec7880 2024-06-13T07:21:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.672+0000 7f5ee7e00640 10 client.16752 handle_command_reply: tid=0 2024-06-13T07:21:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.672+0000 7f5f0fc34740 1 client.16752 shutdown 2024-06-13T07:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1020040223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 msgr2=0x55af2ceaac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceaac00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009900 tx=0x7f5ef4009260 comp rx=0 tx=0).stop 2024-06-13T07:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1020040223 shutdown_connections 2024-06-13T07:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55af2cec7880 0x55af2cec9ce0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceb1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceaac00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1020040223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1020040223 >> 172.21.15.26:0/1020040223 conn(0x55af2ce5ffb0 msgr2=0x55af2ce92840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.673+0000 7f5ee7e00640 0 client.16752 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.674+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1020040223 shutdown_connections 2024-06-13T07:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.674+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1020040223 wait complete. 2024-06-13T07:21:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.674+0000 7f5f0fc34740 20 client.16752 trim_cache size 0 max 16384 2024-06-13T07:21:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.702+0000 7f5f0fc34740 1 Processor -- start 2024-06-13T07:21:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.702+0000 7f5f0fc34740 1 -- start start 2024-06-13T07:21:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.702+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.702+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceb1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ce8e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8012610 con 0x55af2cea0e00 2024-06-13T07:21:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee800fee0 con 0x55af2cea0410 2024-06-13T07:21:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee800dd30 con 0x55af2cea5f70 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f05e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f05e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:33050/0 (socket says 172.21.15.26:33050) 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f05e00640 1 -- 172.21.15.26:0/1135888627 learned_addr learned my addr 172.21.15.26:0/1135888627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f06800640 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ce8e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f05400640 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceb1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f05e00640 1 -- 172.21.15.26:0/1135888627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 msgr2=0x55af2ce8e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f05e00640 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ce8e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f05e00640 1 -- 172.21.15.26:0/1135888627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 msgr2=0x55af2ceb1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.703+0000 7f5f05e00640 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceb1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.704+0000 7f5f05e00640 1 -- 172.21.15.26:0/1135888627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af2cea5cb0 con 0x55af2cea0410 2024-06-13T07:21:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.704+0000 7f5f06800640 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ce8e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:21:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.704+0000 7f5f05e00640 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80cce70 tx=0x7f5ef809ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a33f0a6a72961ce server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.704+0000 7f5ef2a00640 1 -- 172.21.15.26:0/1135888627 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8085110 con 0x55af2cea0410 2024-06-13T07:21:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.704+0000 7f5ef2a00640 1 -- 172.21.15.26:0/1135888627 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef80042f0 con 0x55af2cea0410 2024-06-13T07:21:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.704+0000 7f5ef2a00640 1 -- 172.21.15.26:0/1135888627 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8086570 con 0x55af2cea0410 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1135888627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 msgr2=0x55af2ceaa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80cce70 tx=0x7f5ef809ba80 comp rx=0 tx=0).stop 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1135888627 shutdown_connections 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ce8e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceb1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/1135888627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceaa0a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1135888627 >> 172.21.15.26:0/1135888627 conn(0x55af2ce5ffb0 msgr2=0x55af2cea9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1135888627 shutdown_connections 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.705+0000 7f5f0fc34740 1 -- 172.21.15.26:0/1135888627 wait complete. 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.706+0000 7f5f0fc34740 1 -- start start 2024-06-13T07:21:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.706+0000 7f5f0fc34740 1 Processor -- start 2024-06-13T07:21:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceb1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f0fc34740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceaecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee800e3a0 con 0x55af2cea0e00 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee800f810 con 0x55af2cea0410 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f0fc34740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee800ea70 con 0x55af2cea5f70 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f0fc34740 10 client.0 _ll_register_callbacks cb 0x7f5f000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f06800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceaecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f06800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceaecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:49104/0 (socket says 172.21.15.26:49104) 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f06800640 1 -- 172.21.15.26:0/866583543 learned_addr learned my addr 172.21.15.26:0/866583543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f05e00640 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceb1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f06800640 1 -- 172.21.15.26:0/866583543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 msgr2=0x55af2ceb1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f06800640 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceb1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f06800640 1 -- 172.21.15.26:0/866583543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 msgr2=0x55af2ceae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f06800640 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.707+0000 7f5f06800640 1 -- 172.21.15.26:0/866583543 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af2cf30630 con 0x55af2cea5f70 2024-06-13T07:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.708+0000 7f5f06800640 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceaecc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5f0001a000 tx=0x7f5f00011270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df049613d028b75 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.708+0000 7f5ee7e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.708+0000 7f5ee7e00640 1 -- 172.21.15.26:0/866583543 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00029070 con 0x55af2cea5f70 2024-06-13T07:21:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.708+0000 7f5f0fc34740 1 -- 172.21.15.26:0/866583543 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef80042f0 con 0x55af2cea5f70 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.708+0000 7f5ee7e00640 1 -- 172.21.15.26:0/866583543 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f5f00005040 con 0x55af2cea5f70 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.709+0000 7f5ee7e00640 1 -- 172.21.15.26:0/866583543 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5f00035050 con 0x55af2cea5f70 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.709+0000 7f5f0fc34740 10 client.27052 fetch_fsmap learned FSMap version 47 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.709+0000 7f5f0fc34740 1 -- 172.21.15.26:0/866583543 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55af2cf31990 con 0x55af2cea5f70 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.709+0000 7f5ee7e00640 1 -- 172.21.15.26:0/866583543 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef80042f0 con 0x55af2cea5f70 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.710+0000 7f5ee7e00640 1 -- 172.21.15.26:0/866583543 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f5f00023040 con 0x55af2cea5f70 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.710+0000 7f5f0fc34740 10 client.27052 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.710+0000 7f5f0fc34740 10 client.27052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.710+0000 7f5f0fc34740 20 client.27052 populate_metadata read hostname 'smithi026' 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.710+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55af2cf321a0 0x55af2cf345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.710+0000 7f5f0fc34740 4 client.27052 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:21:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.710+0000 7f5f0fc34740 1 -- 172.21.15.26:0/866583543 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af2ce8edb0 con 0x55af2cf321a0 2024-06-13T07:21:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.711+0000 7f5f05e00640 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55af2cf321a0 0x55af2cf345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.712+0000 7f5f05e00640 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55af2cf321a0 0x55af2cf345c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2c69b7dcf06ae0 server_cookie=d8ef10128a578bdb in_seq=0 out_seq=0 2024-06-13T07:21:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:33.712+0000 7f5ee7e00640 10 client.27052 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:34.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 235567 26.67 MB/sec execute 347 sec latency 403.577 ms 2024-06-13T07:21:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.445+0000 7f5ee7e00640 1 -- 172.21.15.26:0/866583543 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55af2ce8edb0 con 0x55af2cf321a0 2024-06-13T07:21:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.445+0000 7f5ee7e00640 10 client.27052 handle_command_reply: tid=0 2024-06-13T07:21:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.445+0000 7f5f0fc34740 1 client.27052 shutdown 2024-06-13T07:21:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.445+0000 7f5f0fc34740 1 -- 172.21.15.26:0/866583543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 msgr2=0x55af2ceaecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.448+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceaecc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5f0001a000 tx=0x7f5f00011270 comp rx=0 tx=0).stop 2024-06-13T07:21:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.448+0000 7f5f0fc34740 1 -- 172.21.15.26:0/866583543 shutdown_connections 2024-06-13T07:21:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.448+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55af2cf321a0 0x55af2cf345c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.448+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af2cea5f70 0x55af2ceaecc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.448+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af2cea0e00 0x55af2ceae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.448+0000 7f5f0fc34740 1 --2- 172.21.15.26:0/866583543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af2cea0410 0x55af2ceb1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.448+0000 7f5f0fc34740 1 -- 172.21.15.26:0/866583543 >> 172.21.15.26:0/866583543 conn(0x55af2ce5ffb0 msgr2=0x55af2ce9b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.448+0000 7f5ee7e00640 0 client.27052 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.449+0000 7f5f0fc34740 1 -- 172.21.15.26:0/866583543 shutdown_connections 2024-06-13T07:21:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.449+0000 7f5f0fc34740 1 -- 172.21.15.26:0/866583543 wait complete. 2024-06-13T07:21:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.449+0000 7f5f0fc34740 20 client.27052 trim_cache size 0 max 16384 2024-06-13T07:21:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:21:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:21:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:21:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 -- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5eec0a2cb0 msgr2=0x7f5eec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5eec0a2cb0 0x7f5eec0a5170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40057c0 tx=0x7f5ef4031400 comp rx=0 tx=0).stop 2024-06-13T07:21:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 -- 172.21.15.26:0/2677993870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f08103c20 msgr2=0x7f5f08199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f08103c20 0x7f5f08199ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800e9c0 tx=0x7f5ef800edc0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 -- 172.21.15.26:0/2677993870 shutdown_connections 2024-06-13T07:21:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f5eec0a2cb0 0x7f5eec0a5170 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f081057d0 0x7f5f081a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f08104e10 0x7f5f0819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 --2- 172.21.15.26:0/2677993870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f08103c20 0x7f5f08199ea0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 -- 172.21.15.26:0/2677993870 >> 172.21.15.26:0/2677993870 conn(0x7f5f080ff190 msgr2=0x7f5f081080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 -- 172.21.15.26:0/2677993870 shutdown_connections 2024-06-13T07:21:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:34.452+0000 7f5f0ce00640 1 -- 172.21.15.26:0/2677993870 wait complete. 2024-06-13T07:21:34.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:34 smithi136 ceph-mon[37229]: pgmap v1141: 129 pgs: 129 active+clean; 683 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-06-13T07:21:34.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:34 smithi026 ceph-mon[32041]: pgmap v1141: 129 pgs: 129 active+clean; 683 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-06-13T07:21:34.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:34 smithi135 ceph-mon[36276]: pgmap v1141: 129 pgs: 129 active+clean; 683 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-06-13T07:21:35.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 235664 26.61 MB/sec execute 348 sec latency 782.315 ms 2024-06-13T07:21:35.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:35 smithi136 ceph-mon[37229]: pgmap v1142: 129 pgs: 129 active+clean; 680 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-06-13T07:21:36.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:35 smithi026 ceph-mon[32041]: pgmap v1142: 129 pgs: 129 active+clean; 680 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-06-13T07:21:36.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:35 smithi135 ceph-mon[36276]: pgmap v1142: 129 pgs: 129 active+clean; 680 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-06-13T07:21:36.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 235808 26.55 MB/sec execute 349 sec latency 1506.122 ms 2024-06-13T07:21:37.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 236139 26.51 MB/sec execute 350 sec latency 135.790 ms 2024-06-13T07:21:38.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 236502 26.49 MB/sec execute 351 sec latency 127.016 ms 2024-06-13T07:21:38.505 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:38 smithi026 ceph-mon[32041]: pgmap v1143: 129 pgs: 129 active+clean; 697 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 440 op/s 2024-06-13T07:21:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:38 smithi135 ceph-mon[36276]: pgmap v1143: 129 pgs: 129 active+clean; 697 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 440 op/s 2024-06-13T07:21:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:38 smithi136 ceph-mon[37229]: pgmap v1143: 129 pgs: 129 active+clean; 697 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 440 op/s 2024-06-13T07:21:39.212 INFO:tasks.workunit.client.0.smithi026.stdout: 10 236890 26.49 MB/sec execute 352 sec latency 112.270 ms 2024-06-13T07:21:40.213 INFO:tasks.workunit.client.0.smithi026.stdout: 10 237570 26.53 MB/sec execute 353 sec latency 95.909 ms 2024-06-13T07:21:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:40 smithi026 ceph-mon[32041]: pgmap v1144: 129 pgs: 129 active+clean; 709 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 649 op/s 2024-06-13T07:21:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:40 smithi135 ceph-mon[36276]: pgmap v1144: 129 pgs: 129 active+clean; 709 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 649 op/s 2024-06-13T07:21:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:40 smithi136 ceph-mon[37229]: pgmap v1144: 129 pgs: 129 active+clean; 709 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 649 op/s 2024-06-13T07:21:41.214 INFO:tasks.workunit.client.0.smithi026.stdout: 10 238086 26.54 MB/sec execute 354 sec latency 65.973 ms 2024-06-13T07:21:42.214 INFO:tasks.workunit.client.0.smithi026.stdout: 10 238544 26.53 MB/sec execute 355 sec latency 120.136 ms 2024-06-13T07:21:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:42 smithi026 ceph-mon[32041]: pgmap v1145: 129 pgs: 129 active+clean; 700 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-06-13T07:21:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:42 smithi135 ceph-mon[36276]: pgmap v1145: 129 pgs: 129 active+clean; 700 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-06-13T07:21:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:42 smithi136 ceph-mon[37229]: pgmap v1145: 129 pgs: 129 active+clean; 700 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-06-13T07:21:43.214 INFO:tasks.workunit.client.0.smithi026.stdout: 10 239017 26.52 MB/sec execute 356 sec latency 127.305 ms 2024-06-13T07:21:44.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 239573 26.52 MB/sec execute 357 sec latency 173.168 ms 2024-06-13T07:21:44.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:44 smithi026 ceph-mon[32041]: pgmap v1146: 129 pgs: 129 active+clean; 678 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.6 MiB/s wr, 642 op/s 2024-06-13T07:21:44.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:44 smithi135 ceph-mon[36276]: pgmap v1146: 129 pgs: 129 active+clean; 678 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.6 MiB/s wr, 642 op/s 2024-06-13T07:21:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:44 smithi136 ceph-mon[37229]: pgmap v1146: 129 pgs: 129 active+clean; 678 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.6 MiB/s wr, 642 op/s 2024-06-13T07:21:45.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 240242 26.53 MB/sec execute 358 sec latency 67.012 ms 2024-06-13T07:21:46.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 240915 26.58 MB/sec execute 359 sec latency 70.058 ms 2024-06-13T07:21:46.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:46 smithi026 ceph-mon[32041]: pgmap v1147: 129 pgs: 129 active+clean; 681 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.2 MiB/s wr, 622 op/s 2024-06-13T07:21:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:46 smithi135 ceph-mon[36276]: pgmap v1147: 129 pgs: 129 active+clean; 681 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.2 MiB/s wr, 622 op/s 2024-06-13T07:21:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:46 smithi136 ceph-mon[37229]: pgmap v1147: 129 pgs: 129 active+clean; 681 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.2 MiB/s wr, 622 op/s 2024-06-13T07:21:47.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 241510 26.61 MB/sec execute 360 sec latency 67.910 ms 2024-06-13T07:21:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 -- 172.21.15.26:0/3619893911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff518100900 msgr2=0x7ff518100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 --2- 172.21.15.26:0/3619893911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff518100900 0x7ff518100d20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff508009860 tx=0x7ff50802a340 comp rx=0 tx=0).stop 2024-06-13T07:21:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 -- 172.21.15.26:0/3619893911 shutdown_connections 2024-06-13T07:21:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 --2- 172.21.15.26:0/3619893911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff518101af0 0x7ff518101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 --2- 172.21.15.26:0/3619893911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5181024b0 0x7ff518108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 --2- 172.21.15.26:0/3619893911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff518100900 0x7ff518100d20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 -- 172.21.15.26:0/3619893911 >> 172.21.15.26:0/3619893911 conn(0x7ff5180fbe90 msgr2=0x7ff5180fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 -- 172.21.15.26:0/3619893911 shutdown_connections 2024-06-13T07:21:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.662+0000 7ff51f200640 1 -- 172.21.15.26:0/3619893911 wait complete. 2024-06-13T07:21:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.663+0000 7ff51f200640 1 Processor -- start 2024-06-13T07:21:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.663+0000 7ff51f200640 1 -- start start 2024-06-13T07:21:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff51f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff518100900 0x7ff51819e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff51f200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff518101af0 0x7ff51819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff51f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5181024b0 0x7ff5181a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff517400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff518101af0 0x7ff51819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff517400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff518101af0 0x7ff51819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39746/0 (socket says 172.21.15.26:39746) 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff51f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff51810b490 con 0x7ff5181024b0 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff51f200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff51810b310 con 0x7ff518101af0 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff51f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff51810b610 con 0x7ff518100900 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff517400640 1 -- 172.21.15.26:0/1286158252 learned_addr learned my addr 172.21.15.26:0/1286158252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.664+0000 7ff51d200640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5181024b0 0x7ff5181a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.665+0000 7ff517e00640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff518100900 0x7ff51819e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.665+0000 7ff517400640 1 -- 172.21.15.26:0/1286158252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff518100900 msgr2=0x7ff51819e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.665+0000 7ff517400640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff518100900 0x7ff51819e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.665+0000 7ff517400640 1 -- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5181024b0 msgr2=0x7ff5181a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.665+0000 7ff517400640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5181024b0 0x7ff5181a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.665+0000 7ff517400640 1 -- 172.21.15.26:0/1286158252 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5181a5e60 con 0x7ff518101af0 2024-06-13T07:21:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.665+0000 7ff517400640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff518101af0 0x7ff51819e780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff504009ad0 tx=0x7ff504009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.666+0000 7ff514c00640 1 -- 172.21.15.26:0/1286158252 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400cde0 con 0x7ff518101af0 2024-06-13T07:21:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.666+0000 7ff514c00640 1 -- 172.21.15.26:0/1286158252 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5040035f0 con 0x7ff518101af0 2024-06-13T07:21:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.666+0000 7ff514c00640 1 -- 172.21.15.26:0/1286158252 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504017660 con 0x7ff518101af0 2024-06-13T07:21:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.666+0000 7ff51f200640 1 -- 172.21.15.26:0/1286158252 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5181a6150 con 0x7ff518101af0 2024-06-13T07:21:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.667+0000 7ff51f200640 1 -- 172.21.15.26:0/1286158252 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5181a6620 con 0x7ff518101af0 2024-06-13T07:21:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.672+0000 7ff521ffb740 1 Processor -- start 2024-06-13T07:21:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff521ffb740 1 -- start start 2024-06-13T07:21:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff514c00640 1 -- 172.21.15.26:0/1286158252 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff504003b10 con 0x7ff518101af0 2024-06-13T07:21:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a923c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923c5f0 0x5649a92410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a9241760 0x5649a9243bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff521ffb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5649a919f420 con 0x5649a9241760 2024-06-13T07:21:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff521ffb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5649a9146c50 con 0x5649a923c5f0 2024-06-13T07:21:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff521ffb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5649a919b3e0 con 0x5649a923bc00 2024-06-13T07:21:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff51d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a923c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff51d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a923c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59442/0 (socket says 172.21.15.26:59442) 2024-06-13T07:21:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.673+0000 7ff51d200640 1 -- 172.21.15.26:0/3475229375 learned_addr learned my addr 172.21.15.26:0/3475229375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff51d200640 1 -- 172.21.15.26:0/3475229375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923c5f0 msgr2=0x5649a92410f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:21:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff517e00640 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923c5f0 0x5649a92410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff51d200640 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923c5f0 0x5649a92410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff517400640 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a9241760 0x5649a9243bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff51d200640 1 -- 172.21.15.26:0/3475229375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a9241760 msgr2=0x5649a9243bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff51d200640 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a9241760 0x5649a9243bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff51d200640 1 -- 172.21.15.26:0/3475229375 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a8eb3210 con 0x5649a923bc00 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff517e00640 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923c5f0 0x5649a92410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.674+0000 7ff517400640 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a9241760 0x5649a9243bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.675+0000 7ff51d200640 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a923c020 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff510009fd0 tx=0x7ff5100118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740c256542a77f1f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.675+0000 7ff514c00640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4fc0a2d30 0x7ff4fc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.675+0000 7ff514c00640 1 -- 172.21.15.26:0/1286158252 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff5040cf1b0 con 0x7ff518101af0 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.676+0000 7ff50d600640 1 -- 172.21.15.26:0/3475229375 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510004d20 con 0x5649a923bc00 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.676+0000 7ff50d600640 1 -- 172.21.15.26:0/3475229375 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510004050 con 0x5649a923bc00 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.676+0000 7ff50d600640 1 -- 172.21.15.26:0/3475229375 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510005260 con 0x5649a923bc00 2024-06-13T07:21:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.676+0000 7ff517e00640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4fc0a2d30 0x7ff4fc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 -- 172.21.15.26:0/3475229375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 msgr2=0x5649a923c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff517e00640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4fc0a2d30 0x7ff4fc0a51f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff508030410 tx=0x7ff508030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a923c020 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff510009fd0 tx=0x7ff5100118d0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 -- 172.21.15.26:0/3475229375 shutdown_connections 2024-06-13T07:21:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923c5f0 0x5649a92410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a9241760 0x5649a9243bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3475229375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a923c020 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 -- 172.21.15.26:0/3475229375 >> 172.21.15.26:0/3475229375 conn(0x5649a9203fb0 msgr2=0x5649a9237680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 -- 172.21.15.26:0/3475229375 shutdown_connections 2024-06-13T07:21:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 -- 172.21.15.26:0/3475229375 wait complete. 2024-06-13T07:21:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.677+0000 7ff521ffb740 1 -- start start 2024-06-13T07:21:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.678+0000 7ff521ffb740 1 Processor -- start 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.678+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923bc00 0x5649a9245fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.678+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 0x5649a92464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.678+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a9241760 0x5649a924d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.678+0000 7ff521ffb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5649a9244710 con 0x5649a923c5f0 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.678+0000 7ff521ffb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5649a915a0b0 con 0x5649a923bc00 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.678+0000 7ff521ffb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5649a9244890 con 0x5649a9241760 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.678+0000 7ff521ffb740 10 client.0 _ll_register_callbacks cb 0x5649a9145f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923bc00 0x5649a9245fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923bc00 0x5649a9245fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39760/0 (socket says 172.21.15.26:39760) 2024-06-13T07:21:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517e00640 1 -- 172.21.15.26:0/3384643970 learned_addr learned my addr 172.21.15.26:0/3384643970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff51d200640 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a9241760 0x5649a924d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517400640 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 0x5649a92464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517e00640 1 -- 172.21.15.26:0/3384643970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a9241760 msgr2=0x5649a924d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517e00640 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a9241760 0x5649a924d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517e00640 1 -- 172.21.15.26:0/3384643970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 msgr2=0x5649a92464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517e00640 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 0x5649a92464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.679+0000 7ff517e00640 1 -- 172.21.15.26:0/3384643970 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a8f7f620 con 0x5649a923bc00 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.680+0000 7ff517e00640 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923bc00 0x5649a9245fa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff508009830 tx=0x7ff50802a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8d63078e8dd46f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.680+0000 7ff502000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.680+0000 7ff502000640 1 -- 172.21.15.26:0/3384643970 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508037040 con 0x5649a923bc00 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.680+0000 7ff502000640 1 -- 172.21.15.26:0/3384643970 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff508005040 con 0x5649a923bc00 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.680+0000 7ff521ffb740 1 -- 172.21.15.26:0/3384643970 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff510004050 con 0x5649a923bc00 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff502000640 1 -- 172.21.15.26:0/3384643970 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff508005240 con 0x5649a923bc00 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff521ffb740 10 client.27014 fetch_fsmap learned FSMap version 47 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff521ffb740 1 -- 172.21.15.26:0/3384643970 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5649a9250800 con 0x5649a923bc00 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff502000640 1 -- 172.21.15.26:0/3384643970 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510004050 con 0x5649a923bc00 2024-06-13T07:21:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff502000640 1 -- 172.21.15.26:0/3384643970 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7ff508003710 con 0x5649a923bc00 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff521ffb740 10 client.27014 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff521ffb740 10 client.27014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff521ffb740 20 client.27014 populate_metadata read hostname 'smithi026' 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649a9251ee0 0x5649a92542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff521ffb740 4 client.27014 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.681+0000 7ff521ffb740 1 -- 172.21.15.26:0/3384643970 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649a91acb50 con 0x5649a9251ee0 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.682+0000 7ff517400640 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649a9251ee0 0x5649a92542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.682+0000 7ff517400640 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649a9251ee0 0x5649a92542f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c18532b9197ed31 server_cookie=f62b84b0ed7ebccf in_seq=0 out_seq=0 2024-06-13T07:21:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.682+0000 7ff502000640 10 client.27014 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.685+0000 7ff502000640 1 -- 172.21.15.26:0/3384643970 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5649a91acb50 con 0x5649a9251ee0 2024-06-13T07:21:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.685+0000 7ff502000640 10 client.27014 handle_command_reply: tid=0 2024-06-13T07:21:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.685+0000 7ff521ffb740 1 client.27014 shutdown 2024-06-13T07:21:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.685+0000 7ff521ffb740 1 -- 172.21.15.26:0/3384643970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923bc00 msgr2=0x5649a9245fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.685+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923bc00 0x5649a9245fa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff508009830 tx=0x7ff50802a340 comp rx=0 tx=0).stop 2024-06-13T07:21:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.685+0000 7ff521ffb740 1 -- 172.21.15.26:0/3384643970 shutdown_connections 2024-06-13T07:21:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649a9251ee0 0x5649a92542f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a9241760 0x5649a924d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 0x5649a92464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff521ffb740 1 --2- 172.21.15.26:0/3384643970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a923bc00 0x5649a9245fa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff521ffb740 1 -- 172.21.15.26:0/3384643970 >> 172.21.15.26:0/3384643970 conn(0x5649a9203fb0 msgr2=0x5649a9236840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff502000640 0 client.27014 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff521ffb740 1 -- 172.21.15.26:0/3384643970 shutdown_connections 2024-06-13T07:21:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff521ffb740 1 -- 172.21.15.26:0/3384643970 wait complete. 2024-06-13T07:21:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.686+0000 7ff521ffb740 20 client.27014 trim_cache size 0 max 16384 2024-06-13T07:21:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff521ffb740 1 Processor -- start 2024-06-13T07:21:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff521ffb740 1 -- start start 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a92366c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 0x5649a924d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a92455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff521ffb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff5181b3640 con 0x5649a923c5f0 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff521ffb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5181b0f10 con 0x5649a9241760 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff521ffb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5181aed60 con 0x5649a923bc00 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff517e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a92366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff517e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a92366c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59452/0 (socket says 172.21.15.26:59452) 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.722+0000 7ff517e00640 1 -- 172.21.15.26:0/810978357 learned_addr learned my addr 172.21.15.26:0/810978357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff51d200640 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a92455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff517400640 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 0x5649a924d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff517e00640 1 -- 172.21.15.26:0/810978357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 msgr2=0x5649a92455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff517e00640 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a92455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff517e00640 1 -- 172.21.15.26:0/810978357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 msgr2=0x5649a924d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff517e00640 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 0x5649a924d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff517e00640 1 -- 172.21.15.26:0/810978357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a9254120 con 0x5649a923bc00 2024-06-13T07:21:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff51d200640 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a92455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:21:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.723+0000 7ff517e00640 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a92366c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff508003b50 tx=0x7ff508000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa454e85af39beef server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.724+0000 7ff50cc00640 1 -- 172.21.15.26:0/810978357 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508037040 con 0x5649a923bc00 2024-06-13T07:21:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.724+0000 7ff50cc00640 1 -- 172.21.15.26:0/810978357 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50805f070 con 0x5649a923bc00 2024-06-13T07:21:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.724+0000 7ff50cc00640 1 -- 172.21.15.26:0/810978357 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508056030 con 0x5649a923bc00 2024-06-13T07:21:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.725+0000 7ff521ffb740 1 -- 172.21.15.26:0/810978357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 msgr2=0x5649a92366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.725+0000 7ff521ffb740 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a92366c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff508003b50 tx=0x7ff508000bc0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.725+0000 7ff521ffb740 1 -- 172.21.15.26:0/810978357 shutdown_connections 2024-06-13T07:21:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.725+0000 7ff521ffb740 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a92455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.725+0000 7ff521ffb740 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923c5f0 0x5649a924d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.725+0000 7ff521ffb740 1 --2- 172.21.15.26:0/810978357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923bc00 0x5649a92366c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.725+0000 7ff521ffb740 1 -- 172.21.15.26:0/810978357 >> 172.21.15.26:0/810978357 conn(0x5649a9203fb0 msgr2=0x5649a9235fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.725+0000 7ff521ffb740 1 -- 172.21.15.26:0/810978357 shutdown_connections 2024-06-13T07:21:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.726+0000 7ff521ffb740 1 -- 172.21.15.26:0/810978357 wait complete. 2024-06-13T07:21:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.726+0000 7ff521ffb740 1 -- start start 2024-06-13T07:21:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff521ffb740 1 Processor -- start 2024-06-13T07:21:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923bc00 0x5649a9248c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923c5f0 0x5649a9249150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff521ffb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a924c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff521ffb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff5181af3d0 con 0x5649a923bc00 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff521ffb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5181b0840 con 0x5649a9241760 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff521ffb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5181afaa0 con 0x5649a923c5f0 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff521ffb740 10 client.0 _ll_register_callbacks cb 0x7ff504005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff517400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923c5f0 0x5649a9249150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff517400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923c5f0 0x5649a9249150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:59462/0 (socket says 172.21.15.26:59462) 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.727+0000 7ff517400640 1 -- 172.21.15.26:0/2530015907 learned_addr learned my addr 172.21.15.26:0/2530015907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff51d200640 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a924c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff517400640 1 -- 172.21.15.26:0/2530015907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 msgr2=0x5649a924c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff517400640 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a924c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff517400640 1 -- 172.21.15.26:0/2530015907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923bc00 msgr2=0x5649a9248c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:21:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff517400640 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923bc00 0x5649a9248c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff517400640 1 -- 172.21.15.26:0/2530015907 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a924c710 con 0x5649a923c5f0 2024-06-13T07:21:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff517400640 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923c5f0 0x5649a9249150 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff504097ad0 tx=0x7ff5040d1580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f03439f62b8038 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:21:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff502000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.728+0000 7ff502000640 1 -- 172.21.15.26:0/2530015907 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040b3070 con 0x5649a923c5f0 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff502000640 1 -- 172.21.15.26:0/2530015907 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff504099970 con 0x5649a923c5f0 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 1 -- 172.21.15.26:0/2530015907 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff50805f070 con 0x5649a923c5f0 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff502000640 1 -- 172.21.15.26:0/2530015907 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff50409f970 con 0x5649a923c5f0 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 10 client.27070 fetch_fsmap learned FSMap version 47 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 1 -- 172.21.15.26:0/2530015907 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5649a92cdcb0 con 0x5649a923c5f0 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff502000640 1 -- 172.21.15.26:0/2530015907 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50805f070 con 0x5649a923c5f0 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff502000640 1 -- 172.21.15.26:0/2530015907 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7ff5040d10b0 con 0x5649a923c5f0 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 10 client.27070 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 10 client.27070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 20 client.27070 populate_metadata read hostname 'smithi026' 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649a92ce6b0 0x5649a92d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:21:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 4 client.27070 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:21:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.729+0000 7ff521ffb740 1 -- 172.21.15.26:0/2530015907 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649a91a6ea0 con 0x5649a92ce6b0 2024-06-13T07:21:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.730+0000 7ff517e00640 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649a92ce6b0 0x5649a92d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:21:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.730+0000 7ff517e00640 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649a92ce6b0 0x5649a92d0ad0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a76c476ecdf12ca server_cookie=70857edcd6837e8c in_seq=0 out_seq=0 2024-06-13T07:21:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:47.730+0000 7ff502000640 10 client.27070 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:48.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 241815 26.58 MB/sec execute 361 sec latency 380.912 ms 2024-06-13T07:21:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.399+0000 7ff502000640 1 -- 172.21.15.26:0/2530015907 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5649a91a6ea0 con 0x5649a92ce6b0 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.399+0000 7ff502000640 10 client.27070 handle_command_reply: tid=0 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.399+0000 7ff521ffb740 1 client.27070 shutdown 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 -- 172.21.15.26:0/2530015907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923c5f0 msgr2=0x5649a9249150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923c5f0 0x5649a9249150 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff504097ad0 tx=0x7ff5040d1580 comp rx=0 tx=0).stop 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 -- 172.21.15.26:0/2530015907 shutdown_connections 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5649a92ce6b0 0x5649a92d0ad0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649a9241760 0x5649a924c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5649a923c5f0 0x5649a9249150 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff502000640 0 client.27070 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 --2- 172.21.15.26:0/2530015907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5649a923bc00 0x5649a9248c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 -- 172.21.15.26:0/2530015907 >> 172.21.15.26:0/2530015907 conn(0x5649a9203fb0 msgr2=0x5649a923abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 -- 172.21.15.26:0/2530015907 shutdown_connections 2024-06-13T07:21:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 1 -- 172.21.15.26:0/2530015907 wait complete. 2024-06-13T07:21:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.400+0000 7ff521ffb740 20 client.27070 trim_cache size 0 max 16384 2024-06-13T07:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 -- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4fc0a2d30 msgr2=0x7ff4fc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4fc0a2d30 0x7ff4fc0a51f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff508030410 tx=0x7ff508030550 comp rx=0 tx=0).stop 2024-06-13T07:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 -- 172.21.15.26:0/1286158252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff518101af0 msgr2=0x7ff51819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff518101af0 0x7ff51819e780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff504009ad0 tx=0x7ff504009f90 comp rx=0 tx=0).stop 2024-06-13T07:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 -- 172.21.15.26:0/1286158252 shutdown_connections 2024-06-13T07:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7ff4fc0a2d30 0x7ff4fc0a51f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff518101af0 0x7ff51819e780 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5181024b0 0x7ff5181a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.401+0000 7ff51f200640 1 --2- 172.21.15.26:0/1286158252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff518100900 0x7ff51819e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.402+0000 7ff51f200640 1 -- 172.21.15.26:0/1286158252 >> 172.21.15.26:0/1286158252 conn(0x7ff5180fbe90 msgr2=0x7ff518104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.402+0000 7ff51f200640 1 -- 172.21.15.26:0/1286158252 shutdown_connections 2024-06-13T07:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:21:48.402+0000 7ff51f200640 1 -- 172.21.15.26:0/1286158252 wait complete. 2024-06-13T07:21:48.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:48 smithi026 ceph-mon[32041]: pgmap v1148: 129 pgs: 129 active+clean; 677 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.9 MiB/s wr, 629 op/s 2024-06-13T07:21:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:48 smithi135 ceph-mon[36276]: pgmap v1148: 129 pgs: 129 active+clean; 677 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.9 MiB/s wr, 629 op/s 2024-06-13T07:21:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:48 smithi136 ceph-mon[37229]: pgmap v1148: 129 pgs: 129 active+clean; 677 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.9 MiB/s wr, 629 op/s 2024-06-13T07:21:49.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 242285 26.57 MB/sec execute 362 sec latency 510.463 ms 2024-06-13T07:21:50.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 242717 26.56 MB/sec execute 363 sec latency 193.871 ms 2024-06-13T07:21:50.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:50 smithi026 ceph-mon[32041]: pgmap v1149: 129 pgs: 129 active+clean; 680 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 623 op/s 2024-06-13T07:21:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:50 smithi135 ceph-mon[36276]: pgmap v1149: 129 pgs: 129 active+clean; 680 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 623 op/s 2024-06-13T07:21:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:50 smithi136 ceph-mon[37229]: pgmap v1149: 129 pgs: 129 active+clean; 680 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 623 op/s 2024-06-13T07:21:51.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 243112 26.53 MB/sec execute 364 sec latency 421.019 ms 2024-06-13T07:21:52.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 243748 26.57 MB/sec execute 365 sec latency 63.396 ms 2024-06-13T07:21:52.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:52 smithi026 ceph-mon[32041]: pgmap v1150: 129 pgs: 129 active+clean; 665 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.7 MiB/s wr, 428 op/s 2024-06-13T07:21:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:52 smithi135 ceph-mon[36276]: pgmap v1150: 129 pgs: 129 active+clean; 665 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.7 MiB/s wr, 428 op/s 2024-06-13T07:21:52.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:52 smithi136 ceph-mon[37229]: pgmap v1150: 129 pgs: 129 active+clean; 665 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.7 MiB/s wr, 428 op/s 2024-06-13T07:21:53.215 INFO:tasks.workunit.client.0.smithi026.stdout: 10 244362 26.56 MB/sec execute 366 sec latency 75.603 ms 2024-06-13T07:21:54.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 244953 26.59 MB/sec execute 367 sec latency 87.369 ms 2024-06-13T07:21:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:54 smithi026 ceph-mon[32041]: pgmap v1151: 129 pgs: 129 active+clean; 669 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.3 MiB/s wr, 538 op/s 2024-06-13T07:21:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:54 smithi135 ceph-mon[36276]: pgmap v1151: 129 pgs: 129 active+clean; 669 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.3 MiB/s wr, 538 op/s 2024-06-13T07:21:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:54 smithi136 ceph-mon[37229]: pgmap v1151: 129 pgs: 129 active+clean; 669 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.3 MiB/s wr, 538 op/s 2024-06-13T07:21:55.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 245593 26.62 MB/sec execute 368 sec latency 66.643 ms 2024-06-13T07:21:56.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 245718 26.57 MB/sec execute 369 sec latency 547.725 ms 2024-06-13T07:21:56.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:56 smithi026 ceph-mon[32041]: pgmap v1152: 129 pgs: 129 active+clean; 670 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-06-13T07:21:56.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:56 smithi135 ceph-mon[36276]: pgmap v1152: 129 pgs: 129 active+clean; 670 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-06-13T07:21:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:56 smithi136 ceph-mon[37229]: pgmap v1152: 129 pgs: 129 active+clean; 670 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-06-13T07:21:57.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 245899 26.52 MB/sec execute 370 sec latency 281.052 ms 2024-06-13T07:21:57.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:57 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:21:57.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:57 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:21:57.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:57 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:21:58.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 246166 26.49 MB/sec execute 371 sec latency 243.169 ms 2024-06-13T07:21:58.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:58 smithi026 ceph-mon[32041]: pgmap v1153: 129 pgs: 129 active+clean; 687 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.1 MiB/s wr, 497 op/s 2024-06-13T07:21:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:58 smithi135 ceph-mon[36276]: pgmap v1153: 129 pgs: 129 active+clean; 687 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.1 MiB/s wr, 497 op/s 2024-06-13T07:21:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:58 smithi136 ceph-mon[37229]: pgmap v1153: 129 pgs: 129 active+clean; 687 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.1 MiB/s wr, 497 op/s 2024-06-13T07:21:59.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 246677 26.49 MB/sec execute 372 sec latency 83.771 ms 2024-06-13T07:21:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:21:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:21:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:21:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:21:59.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:21:59.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:21:59.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:21:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:21:59.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:21:59.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:21:59.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:21:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:22:00.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 247186 26.48 MB/sec execute 373 sec latency 193.877 ms 2024-06-13T07:22:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:00 smithi136 ceph-mon[37229]: pgmap v1154: 129 pgs: 129 active+clean; 692 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.1 MiB/s wr, 548 op/s 2024-06-13T07:22:00.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:00 smithi026 ceph-mon[32041]: pgmap v1154: 129 pgs: 129 active+clean; 692 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.1 MiB/s wr, 548 op/s 2024-06-13T07:22:00.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:00 smithi135 ceph-mon[36276]: pgmap v1154: 129 pgs: 129 active+clean; 692 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.1 MiB/s wr, 548 op/s 2024-06-13T07:22:01.216 INFO:tasks.workunit.client.0.smithi026.stdout: 10 247689 26.49 MB/sec execute 374 sec latency 243.956 ms 2024-06-13T07:22:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.613+0000 7f5524e00640 1 -- 172.21.15.26:0/2857426310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55201098b0 msgr2=0x7f552010bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.613+0000 7f5524e00640 1 --2- 172.21.15.26:0/2857426310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55201098b0 0x7f552010bd40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5518009be0 tx=0x7f551802a750 comp rx=0 tx=0).stop 2024-06-13T07:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.614+0000 7f5524e00640 1 -- 172.21.15.26:0/2857426310 shutdown_connections 2024-06-13T07:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.614+0000 7f5524e00640 1 --2- 172.21.15.26:0/2857426310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55201098b0 0x7f552010bd40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.614+0000 7f5524e00640 1 --2- 172.21.15.26:0/2857426310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520074380 0x7f55200747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.614+0000 7f5524e00640 1 --2- 172.21.15.26:0/2857426310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5520075960 0x7f5520073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.614+0000 7f5524e00640 1 -- 172.21.15.26:0/2857426310 >> 172.21.15.26:0/2857426310 conn(0x7f55200fcf90 msgr2=0x7f55200ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.614+0000 7f5524e00640 1 -- 172.21.15.26:0/2857426310 shutdown_connections 2024-06-13T07:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.614+0000 7f5524e00640 1 -- 172.21.15.26:0/2857426310 wait complete. 2024-06-13T07:22:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.616+0000 7f5524e00640 1 Processor -- start 2024-06-13T07:22:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.618+0000 7f5524e00640 1 -- start start 2024-06-13T07:22:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.618+0000 7f5524e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520074380 0x7f5520199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.618+0000 7f5524e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5520075960 0x7f552019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.618+0000 7f5524e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55201098b0 0x7f55201a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.618+0000 7f5524e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f552010f870 con 0x7f5520074380 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.618+0000 7f5524e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f552010f6f0 con 0x7f55201098b0 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.618+0000 7f5524e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f552010f9f0 con 0x7f5520075960 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5520075960 0x7f552019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551e800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55201098b0 0x7f55201a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520074380 0x7f5520199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5520075960 0x7f552019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60490/0 (socket says 172.21.15.26:60490) 2024-06-13T07:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551e800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55201098b0 0x7f55201a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39798/0 (socket says 172.21.15.26:39798) 2024-06-13T07:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551d400640 1 -- 172.21.15.26:0/1480146518 learned_addr learned my addr 172.21.15.26:0/1480146518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551d400640 1 -- 172.21.15.26:0/1480146518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55201098b0 msgr2=0x7f55201a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551d400640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55201098b0 0x7f55201a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.619+0000 7f551d400640 1 -- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520074380 msgr2=0x7f5520199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.620+0000 7f551d400640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520074380 0x7f5520199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.620+0000 7f551d400640 1 -- 172.21.15.26:0/1480146518 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55201a1a40 con 0x7f5520075960 2024-06-13T07:22:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.620+0000 7f551de00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520074380 0x7f5520199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:22:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.623+0000 7f551d400640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5520075960 0x7f552019a360 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f550c00e9e0 tx=0x7f550c00ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.623+0000 7f5516a00640 1 -- 172.21.15.26:0/1480146518 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c00cd20 con 0x7f5520075960 2024-06-13T07:22:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.623+0000 7f5516a00640 1 -- 172.21.15.26:0/1480146518 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550c0035f0 con 0x7f5520075960 2024-06-13T07:22:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.623+0000 7f5516a00640 1 -- 172.21.15.26:0/1480146518 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c010690 con 0x7f5520075960 2024-06-13T07:22:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.623+0000 7f5524e00640 1 -- 172.21.15.26:0/1480146518 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55201a1d30 con 0x7f5520075960 2024-06-13T07:22:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.623+0000 7f5524e00640 1 -- 172.21.15.26:0/1480146518 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55201a2210 con 0x7f5520075960 2024-06-13T07:22:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.630+0000 7f5527c03740 1 Processor -- start 2024-06-13T07:22:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.630+0000 7f5527c03740 1 -- start start 2024-06-13T07:22:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f5527c03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc698020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f5527c03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 0x5608fc69d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f5527c03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc69fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f5527c03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5608fc5fb420 con 0x5608fc6985f0 2024-06-13T07:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f5527c03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5608fc5a2c50 con 0x5608fc69d760 2024-06-13T07:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f5527c03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5608fc5f73e0 con 0x5608fc697c00 2024-06-13T07:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f551e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc698020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f551e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc698020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60492/0 (socket says 172.21.15.26:60492) 2024-06-13T07:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.631+0000 7f551e800640 1 -- 172.21.15.26:0/1188770701 learned_addr learned my addr 172.21.15.26:0/1188770701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.632+0000 7f551e800640 1 -- 172.21.15.26:0/1188770701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 msgr2=0x5608fc69fbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.632+0000 7f551e800640 1 --2- 172.21.15.26:0/1188770701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc69fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.632+0000 7f551e800640 1 -- 172.21.15.26:0/1188770701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 msgr2=0x5608fc69d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.632+0000 7f551e800640 1 --2- 172.21.15.26:0/1188770701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 0x5608fc69d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.632+0000 7f551e800640 1 -- 172.21.15.26:0/1188770701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608fc30f210 con 0x5608fc697c00 2024-06-13T07:22:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.633+0000 7f551e800640 1 --2- 172.21.15.26:0/1188770701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc698020 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5518009be0 tx=0x7f5518005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf8c4d752a345623 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.633+0000 7f550b400640 1 -- 172.21.15.26:0/1188770701 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518041030 con 0x5608fc697c00 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.634+0000 7f5516a00640 1 -- 172.21.15.26:0/1480146518 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f550c0037a0 con 0x7f5520075960 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.635+0000 7f550b400640 1 -- 172.21.15.26:0/1188770701 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5518030c70 con 0x5608fc697c00 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.635+0000 7f550b400640 1 -- 172.21.15.26:0/1188770701 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518036040 con 0x5608fc697c00 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.636+0000 7f5527c03740 1 -- 172.21.15.26:0/1188770701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 msgr2=0x5608fc698020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.636+0000 7f5527c03740 1 --2- 172.21.15.26:0/1188770701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc698020 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5518009be0 tx=0x7f5518005ec0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.637+0000 7f5527c03740 1 -- 172.21.15.26:0/1188770701 shutdown_connections 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.637+0000 7f5527c03740 1 --2- 172.21.15.26:0/1188770701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc69fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.637+0000 7f5527c03740 1 --2- 172.21.15.26:0/1188770701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 0x5608fc69d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.637+0000 7f5527c03740 1 --2- 172.21.15.26:0/1188770701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc698020 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.637+0000 7f5527c03740 1 -- 172.21.15.26:0/1188770701 >> 172.21.15.26:0/1188770701 conn(0x5608fc65ffb0 msgr2=0x5608fc693680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.637+0000 7f5527c03740 1 -- 172.21.15.26:0/1188770701 shutdown_connections 2024-06-13T07:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.637+0000 7f5516a00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f55040a2c10 0x7f55040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.638+0000 7f5527c03740 1 -- 172.21.15.26:0/1188770701 wait complete. 2024-06-13T07:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.638+0000 7f5527c03740 1 -- start start 2024-06-13T07:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.639+0000 7f5527c03740 1 Processor -- start 2024-06-13T07:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.639+0000 7f551de00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f55040a2c10 0x7f55040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.641+0000 7f5516a00640 1 -- 172.21.15.26:0/1480146518 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f550c0cf730 con 0x7f5520075960 2024-06-13T07:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.642+0000 7f5527c03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.642+0000 7f5527c03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6aac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.642+0000 7f5527c03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 0x5608fc6b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.642+0000 7f5527c03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5608fc6a06b0 con 0x5608fc69d760 2024-06-13T07:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.642+0000 7f5527c03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5608fc5b60b0 con 0x5608fc6985f0 2024-06-13T07:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.642+0000 7f5527c03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5608fc6a0830 con 0x5608fc697c00 2024-06-13T07:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.642+0000 7f5527c03740 10 client.0 _ll_register_callbacks cb 0x5608fc5a1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.643+0000 7f551d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.643+0000 7f551d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6aac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39800/0 (socket says 172.21.15.26:39800) 2024-06-13T07:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.643+0000 7f551d400640 1 -- 172.21.15.26:0/3730063166 learned_addr learned my addr 172.21.15.26:0/3730063166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.643+0000 7f551de00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f55040a2c10 0x7f55040a50d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5608fc694530 tx=0x7f55100046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.643+0000 7f551de00640 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.644+0000 7f551e800640 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 0x5608fc6b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.644+0000 7f551d400640 1 -- 172.21.15.26:0/3730063166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 msgr2=0x5608fc6aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.644+0000 7f551d400640 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.644+0000 7f551d400640 1 -- 172.21.15.26:0/3730063166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 msgr2=0x5608fc6b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.644+0000 7f551d400640 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 0x5608fc6b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.644+0000 7f551d400640 1 -- 172.21.15.26:0/3730063166 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608fc3db620 con 0x5608fc6985f0 2024-06-13T07:22:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.645+0000 7f551d400640 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6aac10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f550c09bad0 tx=0x7f550c09b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f598e6e43fd6cf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.645+0000 7f54ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.645+0000 7f54ffe00640 1 -- 172.21.15.26:0/3730063166 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c09fef0 con 0x5608fc6985f0 2024-06-13T07:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.646+0000 7f54ffe00640 1 -- 172.21.15.26:0/3730063166 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f550c0cf3b0 con 0x5608fc6985f0 2024-06-13T07:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.646+0000 7f5527c03740 1 -- 172.21.15.26:0/3730063166 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5518030c70 con 0x5608fc6985f0 2024-06-13T07:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.647+0000 7f54ffe00640 1 -- 172.21.15.26:0/3730063166 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f550c0a47b0 con 0x5608fc6985f0 2024-06-13T07:22:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.647+0000 7f5527c03740 10 client.27029 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.647+0000 7f5527c03740 1 -- 172.21.15.26:0/3730063166 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5608fc6b5050 con 0x5608fc6985f0 2024-06-13T07:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.648+0000 7f54ffe00640 1 -- 172.21.15.26:0/3730063166 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5518030c70 con 0x5608fc6985f0 2024-06-13T07:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.648+0000 7f54ffe00640 1 -- 172.21.15.26:0/3730063166 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f550c09af90 con 0x5608fc6985f0 2024-06-13T07:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.648+0000 7f5527c03740 10 client.27029 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.648+0000 7f5527c03740 10 client.27029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.648+0000 7f5527c03740 20 client.27029 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.648+0000 7f5527c03740 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc6b6700 0x5608fc6b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.648+0000 7f5527c03740 4 client.27029 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.648+0000 7f5527c03740 1 -- 172.21.15.26:0/3730063166 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608fc608b50 con 0x5608fc6b6700 2024-06-13T07:22:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.649+0000 7f551de00640 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc6b6700 0x5608fc6b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.651+0000 7f551de00640 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc6b6700 0x5608fc6b8af0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703ee10b1f57698f server_cookie=2930cb67e031b19d in_seq=0 out_seq=0 2024-06-13T07:22:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.651+0000 7f54ffe00640 10 client.27029 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.660+0000 7f54ffe00640 1 -- 172.21.15.26:0/3730063166 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5608fc608b50 con 0x5608fc6b6700 2024-06-13T07:22:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.660+0000 7f54ffe00640 10 client.27029 handle_command_reply: tid=0 2024-06-13T07:22:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.661+0000 7f5527c03740 1 client.27029 shutdown 2024-06-13T07:22:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.662+0000 7f5527c03740 1 -- 172.21.15.26:0/3730063166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 msgr2=0x5608fc6aac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.662+0000 7f5527c03740 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6aac10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f550c09bad0 tx=0x7f550c09b740 comp rx=0 tx=0).stop 2024-06-13T07:22:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f5527c03740 1 -- 172.21.15.26:0/3730063166 shutdown_connections 2024-06-13T07:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f5527c03740 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc6b6700 0x5608fc6b8af0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f5527c03740 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6aac10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f54ffe00640 0 client.27029 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f5527c03740 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 0x5608fc6b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f5527c03740 1 --2- 172.21.15.26:0/3730063166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f5527c03740 1 -- 172.21.15.26:0/3730063166 >> 172.21.15.26:0/3730063166 conn(0x5608fc65ffb0 msgr2=0x5608fc692840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f5527c03740 1 -- 172.21.15.26:0/3730063166 shutdown_connections 2024-06-13T07:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.663+0000 7f5527c03740 1 -- 172.21.15.26:0/3730063166 wait complete. 2024-06-13T07:22:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.664+0000 7f5527c03740 20 client.27029 trim_cache size 0 max 16384 2024-06-13T07:22:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.697+0000 7f5527c03740 1 Processor -- start 2024-06-13T07:22:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.698+0000 7f5527c03740 1 -- start start 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f5527c03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc696e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f5527c03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 0x5608fc6b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f5527c03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc6936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f5527c03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55201aad60 con 0x5608fc6985f0 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f5527c03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55201a8630 con 0x5608fc69d760 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f5527c03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55201a6480 con 0x5608fc697c00 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f551e800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc6936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f551e800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc6936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:39802/0 (socket says 172.21.15.26:39802) 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f551e800640 1 -- 172.21.15.26:0/89329237 learned_addr learned my addr 172.21.15.26:0/89329237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f551de00640 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc696e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f551d400640 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 0x5608fc6b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f551e800640 1 -- 172.21.15.26:0/89329237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 msgr2=0x5608fc696e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.699+0000 7f551e800640 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc696e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.700+0000 7f551e800640 1 -- 172.21.15.26:0/89329237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 msgr2=0x5608fc6b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.700+0000 7f551e800640 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 0x5608fc6b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.700+0000 7f551e800640 1 -- 172.21.15.26:0/89329237 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608fc6aa290 con 0x5608fc69d760 2024-06-13T07:22:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.700+0000 7f551e800640 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc6936d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f551802ad40 tx=0x7f551802a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=645ff0d59bfa8d6b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.703+0000 7f550aa00640 1 -- 172.21.15.26:0/89329237 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551804a070 con 0x5608fc69d760 2024-06-13T07:22:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.703+0000 7f550aa00640 1 -- 172.21.15.26:0/89329237 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5518003e50 con 0x5608fc69d760 2024-06-13T07:22:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.703+0000 7f550aa00640 1 -- 172.21.15.26:0/89329237 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518041040 con 0x5608fc69d760 2024-06-13T07:22:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 -- 172.21.15.26:0/89329237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 msgr2=0x5608fc6936d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc6936d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f551802ad40 tx=0x7f551802a750 comp rx=0 tx=0).stop 2024-06-13T07:22:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 -- 172.21.15.26:0/89329237 shutdown_connections 2024-06-13T07:22:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc69d760 0x5608fc6936d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc6985f0 0x5608fc6b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 --2- 172.21.15.26:0/89329237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc696e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 -- 172.21.15.26:0/89329237 >> 172.21.15.26:0/89329237 conn(0x5608fc65ffb0 msgr2=0x5608fc691fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 -- 172.21.15.26:0/89329237 shutdown_connections 2024-06-13T07:22:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.704+0000 7f5527c03740 1 -- 172.21.15.26:0/89329237 wait complete. 2024-06-13T07:22:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.705+0000 7f5527c03740 1 -- start start 2024-06-13T07:22:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.705+0000 7f5527c03740 1 Processor -- start 2024-06-13T07:22:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f5527c03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f5527c03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f5527c03740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 0x5608fc6b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f5527c03740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55201a6af0 con 0x5608fc69d760 2024-06-13T07:22:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f5527c03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55201a7f60 con 0x5608fc6985f0 2024-06-13T07:22:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f5527c03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55201a71c0 con 0x5608fc697c00 2024-06-13T07:22:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f5527c03740 10 client.0 _ll_register_callbacks cb 0x7f550c003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f551de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f551de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:60522/0 (socket says 172.21.15.26:60522) 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.706+0000 7f551de00640 1 -- 172.21.15.26:0/797123504 learned_addr learned my addr 172.21.15.26:0/797123504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.707+0000 7f551d400640 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.707+0000 7f551e800640 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 0x5608fc6b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.707+0000 7f551de00640 1 -- 172.21.15.26:0/797123504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 msgr2=0x5608fc6af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.707+0000 7f551de00640 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.707+0000 7f551de00640 1 -- 172.21.15.26:0/797123504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 msgr2=0x5608fc6b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.707+0000 7f551de00640 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 0x5608fc6b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.707+0000 7f551de00640 1 -- 172.21.15.26:0/797123504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608fc6b09b0 con 0x5608fc697c00 2024-06-13T07:22:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.709+0000 7f551de00640 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aee70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5608fc6989f0 tx=0x7f5510009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92aa86c5a8446793 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.709+0000 7f54ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:22:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.710+0000 7f54ffe00640 1 -- 172.21.15.26:0/797123504 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551001f840 con 0x5608fc697c00 2024-06-13T07:22:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.710+0000 7f54ffe00640 1 -- 172.21.15.26:0/797123504 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f551001f9e0 con 0x5608fc697c00 2024-06-13T07:22:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.710+0000 7f5527c03740 1 -- 172.21.15.26:0/797123504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5518003e50 con 0x5608fc697c00 2024-06-13T07:22:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.712+0000 7f54ffe00640 1 -- 172.21.15.26:0/797123504 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f551001fbe0 con 0x5608fc697c00 2024-06-13T07:22:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.712+0000 7f5527c03740 10 client.27088 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.712+0000 7f5527c03740 1 -- 172.21.15.26:0/797123504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5608fc6b1d90 con 0x5608fc697c00 2024-06-13T07:22:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.716+0000 7f54ffe00640 1 -- 172.21.15.26:0/797123504 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5518003e50 con 0x5608fc697c00 2024-06-13T07:22:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.716+0000 7f54ffe00640 1 -- 172.21.15.26:0/797123504 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f551003b420 con 0x5608fc697c00 2024-06-13T07:22:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.716+0000 7f5527c03740 10 client.27088 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.716+0000 7f5527c03740 10 client.27088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.716+0000 7f5527c03740 20 client.27088 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.716+0000 7f5527c03740 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc6a1340 0x5608fc6a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.716+0000 7f5527c03740 4 client.27088 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:22:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.716+0000 7f5527c03740 1 -- 172.21.15.26:0/797123504 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608fc602ea0 con 0x5608fc6a1340 2024-06-13T07:22:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.717+0000 7f551d400640 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc6a1340 0x5608fc6a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.718+0000 7f54ffe00640 10 client.27088 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:01.718+0000 7f551d400640 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc6a1340 0x5608fc6a3760 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f90b6d5ad65c47c server_cookie=2fcf2004cf438b7 in_seq=0 out_seq=0 2024-06-13T07:22:02.217 INFO:tasks.workunit.client.0.smithi026.stdout: 10 247791 26.44 MB/sec execute 375 sec latency 1039.509 ms 2024-06-13T07:22:02.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:02 smithi026 ceph-mon[32041]: pgmap v1155: 129 pgs: 129 active+clean; 693 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 474 op/s 2024-06-13T07:22:02.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:02 smithi135 ceph-mon[36276]: pgmap v1155: 129 pgs: 129 active+clean; 693 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 474 op/s 2024-06-13T07:22:02.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:02 smithi136 ceph-mon[37229]: pgmap v1155: 129 pgs: 129 active+clean; 693 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 474 op/s 2024-06-13T07:22:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.194+0000 7f54ffe00640 1 -- 172.21.15.26:0/797123504 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5608fc602ea0 con 0x5608fc6a1340 2024-06-13T07:22:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.194+0000 7f54ffe00640 10 client.27088 handle_command_reply: tid=0 2024-06-13T07:22:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.194+0000 7f5527c03740 1 client.27088 shutdown 2024-06-13T07:22:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f5527c03740 1 -- 172.21.15.26:0/797123504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 msgr2=0x5608fc6aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f5527c03740 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aee70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5608fc6989f0 tx=0x7f5510009260 comp rx=0 tx=0).stop 2024-06-13T07:22:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f5527c03740 1 -- 172.21.15.26:0/797123504 shutdown_connections 2024-06-13T07:22:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f5527c03740 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5608fc6a1340 0x5608fc6a3760 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f5527c03740 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5608fc6985f0 0x5608fc6af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f5527c03740 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5608fc69d760 0x5608fc6b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f5527c03740 1 --2- 172.21.15.26:0/797123504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608fc697c00 0x5608fc6aee70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f5527c03740 1 -- 172.21.15.26:0/797123504 >> 172.21.15.26:0/797123504 conn(0x5608fc65ffb0 msgr2=0x5608fc696c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.195+0000 7f54ffe00640 0 client.27088 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.196+0000 7f5527c03740 1 -- 172.21.15.26:0/797123504 shutdown_connections 2024-06-13T07:22:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.196+0000 7f5527c03740 1 -- 172.21.15.26:0/797123504 wait complete. 2024-06-13T07:22:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.196+0000 7f5527c03740 20 client.27088 trim_cache size 0 max 16384 2024-06-13T07:22:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:22:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:22:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:22:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 -- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f55040a2c10 msgr2=0x7f55040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f55040a2c10 0x7f55040a50d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5608fc694530 tx=0x7f55100046c0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 -- 172.21.15.26:0/1480146518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5520075960 msgr2=0x7f552019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5520075960 0x7f552019a360 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f550c00e9e0 tx=0x7f550c00ede0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 -- 172.21.15.26:0/1480146518 shutdown_connections 2024-06-13T07:22:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f55040a2c10 0x7f55040a50d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55201098b0 0x7f55201a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5520075960 0x7f552019a360 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 --2- 172.21.15.26:0/1480146518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520074380 0x7f5520199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.198+0000 7f5524e00640 1 -- 172.21.15.26:0/1480146518 >> 172.21.15.26:0/1480146518 conn(0x7f55200fcf90 msgr2=0x7f5520105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.199+0000 7f5524e00640 1 -- 172.21.15.26:0/1480146518 shutdown_connections 2024-06-13T07:22:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:03.199+0000 7f5524e00640 1 -- 172.21.15.26:0/1480146518 wait complete. 2024-06-13T07:22:03.218 INFO:tasks.workunit.client.0.smithi026.stdout: 10 247943 26.39 MB/sec execute 376 sec latency 1115.494 ms 2024-06-13T07:22:04.218 INFO:tasks.workunit.client.0.smithi026.stdout: 10 248225 26.36 MB/sec execute 377 sec latency 138.070 ms 2024-06-13T07:22:04.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:04 smithi026 ceph-mon[32041]: pgmap v1156: 129 pgs: 129 active+clean; 713 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.8 MiB/s wr, 589 op/s 2024-06-13T07:22:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:04 smithi135 ceph-mon[36276]: pgmap v1156: 129 pgs: 129 active+clean; 713 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.8 MiB/s wr, 589 op/s 2024-06-13T07:22:04.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:04 smithi136 ceph-mon[37229]: pgmap v1156: 129 pgs: 129 active+clean; 713 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.8 MiB/s wr, 589 op/s 2024-06-13T07:22:05.218 INFO:tasks.workunit.client.0.smithi026.stdout: 10 248496 26.31 MB/sec execute 378 sec latency 183.053 ms 2024-06-13T07:22:06.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 248786 26.27 MB/sec execute 379 sec latency 337.909 ms 2024-06-13T07:22:06.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:06 smithi026 ceph-mon[32041]: pgmap v1157: 129 pgs: 129 active+clean; 725 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 9.8 MiB/s wr, 501 op/s 2024-06-13T07:22:06.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:06 smithi135 ceph-mon[36276]: pgmap v1157: 129 pgs: 129 active+clean; 725 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 9.8 MiB/s wr, 501 op/s 2024-06-13T07:22:06.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:06 smithi136 ceph-mon[37229]: pgmap v1157: 129 pgs: 129 active+clean; 725 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 9.8 MiB/s wr, 501 op/s 2024-06-13T07:22:07.218 INFO:tasks.workunit.client.0.smithi026.stdout: 10 249365 26.32 MB/sec execute 380 sec latency 103.482 ms 2024-06-13T07:22:08.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 249785 26.31 MB/sec execute 381 sec latency 155.934 ms 2024-06-13T07:22:08.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:08 smithi026 ceph-mon[32041]: pgmap v1158: 129 pgs: 129 active+clean; 715 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 624 op/s 2024-06-13T07:22:08.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:08 smithi135 ceph-mon[36276]: pgmap v1158: 129 pgs: 129 active+clean; 715 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 624 op/s 2024-06-13T07:22:08.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:08 smithi136 ceph-mon[37229]: pgmap v1158: 129 pgs: 129 active+clean; 715 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 624 op/s 2024-06-13T07:22:09.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 250036 26.28 MB/sec execute 382 sec latency 350.929 ms 2024-06-13T07:22:10.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 250422 26.24 MB/sec execute 383 sec latency 118.003 ms 2024-06-13T07:22:10.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:10 smithi026 ceph-mon[32041]: pgmap v1159: 129 pgs: 129 active+clean; 663 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.8 MiB/s wr, 615 op/s 2024-06-13T07:22:10.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:10 smithi135 ceph-mon[36276]: pgmap v1159: 129 pgs: 129 active+clean; 663 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.8 MiB/s wr, 615 op/s 2024-06-13T07:22:10.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:10 smithi136 ceph-mon[37229]: pgmap v1159: 129 pgs: 129 active+clean; 663 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.8 MiB/s wr, 615 op/s 2024-06-13T07:22:11.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 250873 26.23 MB/sec execute 384 sec latency 88.823 ms 2024-06-13T07:22:12.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 251409 26.25 MB/sec execute 385 sec latency 112.801 ms 2024-06-13T07:22:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:12 smithi026 ceph-mon[32041]: pgmap v1160: 129 pgs: 129 active+clean; 655 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.6 MiB/s wr, 524 op/s 2024-06-13T07:22:12.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:12 smithi135 ceph-mon[36276]: pgmap v1160: 129 pgs: 129 active+clean; 655 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.6 MiB/s wr, 524 op/s 2024-06-13T07:22:12.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:12 smithi136 ceph-mon[37229]: pgmap v1160: 129 pgs: 129 active+clean; 655 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.6 MiB/s wr, 524 op/s 2024-06-13T07:22:13.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 252106 26.27 MB/sec execute 386 sec latency 76.663 ms 2024-06-13T07:22:14.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 252655 26.29 MB/sec execute 387 sec latency 59.822 ms 2024-06-13T07:22:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:14 smithi026 ceph-mon[32041]: pgmap v1161: 129 pgs: 129 active+clean; 664 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.8 MiB/s wr, 593 op/s 2024-06-13T07:22:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:14 smithi135 ceph-mon[36276]: pgmap v1161: 129 pgs: 129 active+clean; 664 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.8 MiB/s wr, 593 op/s 2024-06-13T07:22:14.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:14 smithi136 ceph-mon[37229]: pgmap v1161: 129 pgs: 129 active+clean; 664 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.8 MiB/s wr, 593 op/s 2024-06-13T07:22:15.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 253226 26.31 MB/sec execute 388 sec latency 78.104 ms 2024-06-13T07:22:16.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 253686 26.31 MB/sec execute 389 sec latency 87.686 ms 2024-06-13T07:22:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.399+0000 7f418a400640 1 -- 172.21.15.26:0/1606045918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4184101860 msgr2=0x7f4184101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.399+0000 7f418a400640 1 --2- 172.21.15.26:0/1606045918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4184101860 0x7f4184101c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f417800fb10 tx=0x7f417802e570 comp rx=0 tx=0).stop 2024-06-13T07:22:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.399+0000 7f418a400640 1 -- 172.21.15.26:0/1606045918 shutdown_connections 2024-06-13T07:22:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.399+0000 7f418a400640 1 --2- 172.21.15.26:0/1606045918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4184103410 0x7f4184109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.399+0000 7f418a400640 1 --2- 172.21.15.26:0/1606045918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4184102a50 0x7f4184102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.400+0000 7f418a400640 1 --2- 172.21.15.26:0/1606045918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4184101860 0x7f4184101c80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.400+0000 7f418a400640 1 -- 172.21.15.26:0/1606045918 >> 172.21.15.26:0/1606045918 conn(0x7f41840fcdd0 msgr2=0x7f41840ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.400+0000 7f418a400640 1 -- 172.21.15.26:0/1606045918 shutdown_connections 2024-06-13T07:22:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.400+0000 7f418a400640 1 -- 172.21.15.26:0/1606045918 wait complete. 2024-06-13T07:22:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.401+0000 7f418a400640 1 Processor -- start 2024-06-13T07:22:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f418a400640 1 -- start start 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f418a400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4184101860 0x7f4184195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f418a400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4184102a50 0x7f4184195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f418a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4184103410 0x7f418419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f418a400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f418410b300 con 0x7f4184102a50 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f418a400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f418410b180 con 0x7f4184101860 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f418a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f418410b480 con 0x7f4184103410 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f4189a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4184103410 0x7f418419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f4183e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4184102a50 0x7f4184195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f4189a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4184103410 0x7f418419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56526/0 (socket says 172.21.15.26:56526) 2024-06-13T07:22:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f4189a00640 1 -- 172.21.15.26:0/1614764694 learned_addr learned my addr 172.21.15.26:0/1614764694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.402+0000 7f4189000640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4184101860 0x7f4184195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.403+0000 7f4189a00640 1 -- 172.21.15.26:0/1614764694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4184101860 msgr2=0x7f4184195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.403+0000 7f4189a00640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4184101860 0x7f4184195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.403+0000 7f4189a00640 1 -- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4184102a50 msgr2=0x7f4184195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.403+0000 7f4189a00640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4184102a50 0x7f4184195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.403+0000 7f4189a00640 1 -- 172.21.15.26:0/1614764694 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418419d560 con 0x7f4184103410 2024-06-13T07:22:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.403+0000 7f4183e00640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4184102a50 0x7f4184195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:22:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.404+0000 7f4189a00640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4184103410 0x7f418419cef0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f417c00eaa0 tx=0x7f417c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.405+0000 7f4181600640 1 -- 172.21.15.26:0/1614764694 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c00cdb0 con 0x7f4184103410 2024-06-13T07:22:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.405+0000 7f4181600640 1 -- 172.21.15.26:0/1614764694 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417c005020 con 0x7f4184103410 2024-06-13T07:22:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.405+0000 7f4181600640 1 -- 172.21.15.26:0/1614764694 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0116d0 con 0x7f4184103410 2024-06-13T07:22:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.405+0000 7f418a400640 1 -- 172.21.15.26:0/1614764694 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418419d850 con 0x7f4184103410 2024-06-13T07:22:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.405+0000 7f418a400640 1 -- 172.21.15.26:0/1614764694 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418419dda0 con 0x7f4184103410 2024-06-13T07:22:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.412+0000 7f418e800740 1 Processor -- start 2024-06-13T07:22:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.414+0000 7f4181600640 1 -- 172.21.15.26:0/1614764694 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f417c005540 con 0x7f4184103410 2024-06-13T07:22:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.416+0000 7f4181600640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f416c0a2c80 0x7f416c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.417+0000 7f4181600640 1 -- 172.21.15.26:0/1614764694 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f417c0cf5e0 con 0x7f4184103410 2024-06-13T07:22:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.417+0000 7f418e800740 1 -- start start 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.417+0000 7f4183e00640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f416c0a2c80 0x7f416c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f418e800740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629f5d0 0x563ff629f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f418e800740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f418e800740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f418e800740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563ff61fa420 con 0x563ff629ffc0 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f418e800740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563ff61a1c50 con 0x563ff62a5130 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f418e800740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ff61f63e0 con 0x563ff629f5d0 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f4189a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f4189000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629f5d0 0x563ff629f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f4189000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629f5d0 0x563ff629f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56536/0 (socket says 172.21.15.26:56536) 2024-06-13T07:22:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f4189000640 1 -- 172.21.15.26:0/2576458766 learned_addr learned my addr 172.21.15.26:0/2576458766 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.418+0000 7f4189a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51558/0 (socket says 172.21.15.26:51558) 2024-06-13T07:22:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.419+0000 7f4189000640 1 -- 172.21.15.26:0/2576458766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 msgr2=0x563ff62a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.419+0000 7f4189000640 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.419+0000 7f4189000640 1 -- 172.21.15.26:0/2576458766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 msgr2=0x563ff62a4ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:22:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.419+0000 7f4183e00640 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.420+0000 7f4189000640 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.420+0000 7f4189000640 1 -- 172.21.15.26:0/2576458766 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff5f0e210 con 0x563ff629f5d0 2024-06-13T07:22:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.420+0000 7f4183e00640 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:22:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.420+0000 7f4183e00640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f416c0a2c80 0x7f416c0a5140 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4184196e50 tx=0x7f4170005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.422+0000 7f4189000640 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629f5d0 0x563ff629f9f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4178005ab0 tx=0x7f4178004880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee47bffa17fe32d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.422+0000 7f4176000640 1 -- 172.21.15.26:0/2576458766 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178044020 con 0x563ff629f5d0 2024-06-13T07:22:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.423+0000 7f4176000640 1 -- 172.21.15.26:0/2576458766 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417800e5b0 con 0x563ff629f5d0 2024-06-13T07:22:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.423+0000 7f4176000640 1 -- 172.21.15.26:0/2576458766 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417800f400 con 0x563ff629f5d0 2024-06-13T07:22:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.424+0000 7f418e800740 1 -- 172.21.15.26:0/2576458766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629f5d0 msgr2=0x563ff629f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.424+0000 7f418e800740 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629f5d0 0x563ff629f9f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4178005ab0 tx=0x7f4178004880 comp rx=0 tx=0).stop 2024-06-13T07:22:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.424+0000 7f418e800740 1 -- 172.21.15.26:0/2576458766 shutdown_connections 2024-06-13T07:22:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.424+0000 7f418e800740 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.424+0000 7f418e800740 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.424+0000 7f418e800740 1 --2- 172.21.15.26:0/2576458766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629f5d0 0x563ff629f9f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.424+0000 7f418e800740 1 -- 172.21.15.26:0/2576458766 >> 172.21.15.26:0/2576458766 conn(0x563ff625efb0 msgr2=0x563ff6292840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.425+0000 7f418e800740 1 -- 172.21.15.26:0/2576458766 shutdown_connections 2024-06-13T07:22:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.426+0000 7f418e800740 1 -- 172.21.15.26:0/2576458766 wait complete. 2024-06-13T07:22:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.426+0000 7f418e800740 1 -- start start 2024-06-13T07:22:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.428+0000 7f418e800740 1 Processor -- start 2024-06-13T07:22:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.430+0000 7f418e800740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.430+0000 7f418e800740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.430+0000 7f418e800740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff62a5130 0x563ff62b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.431+0000 7f418e800740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563ff62a8080 con 0x563ff629ffc0 2024-06-13T07:22:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.431+0000 7f418e800740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563ff61b50b0 con 0x563ff629f5d0 2024-06-13T07:22:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.431+0000 7f418e800740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ff62a8200 con 0x563ff62a5130 2024-06-13T07:22:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.431+0000 7f418e800740 10 client.0 _ll_register_callbacks cb 0x563ff61a0f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.431+0000 7f4189a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff62a5130 0x563ff62b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.431+0000 7f4189a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff62a5130 0x563ff62b0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:56538/0 (socket says 172.21.15.26:56538) 2024-06-13T07:22:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.431+0000 7f4189a00640 1 -- 172.21.15.26:0/3589664717 learned_addr learned my addr 172.21.15.26:0/3589664717 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.431+0000 7f4189000640 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f4183e00640 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f4189a00640 1 -- 172.21.15.26:0/3589664717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 msgr2=0x563ff62a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f4189a00640 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f4189a00640 1 -- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 msgr2=0x563ff62a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f4189a00640 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f4189a00640 1 -- 172.21.15.26:0/3589664717 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff5fda620 con 0x563ff62a5130 2024-06-13T07:22:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f4183e00640 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:22:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f4189a00640 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff62a5130 0x563ff62b0eb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f417c0ccc30 tx=0x7f417c09fed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ecac8bdeb590b39 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.432+0000 7f416aa00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:22:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.433+0000 7f416aa00640 1 -- 172.21.15.26:0/3589664717 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c01f030 con 0x563ff62a5130 2024-06-13T07:22:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.433+0000 7f416aa00640 1 -- 172.21.15.26:0/3589664717 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f417c0cf240 con 0x563ff62a5130 2024-06-13T07:22:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.434+0000 7f418e800740 1 -- 172.21.15.26:0/3589664717 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f417800e5b0 con 0x563ff62a5130 2024-06-13T07:22:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.434+0000 7f416aa00640 1 -- 172.21.15.26:0/3589664717 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f417c09aef0 con 0x563ff62a5130 2024-06-13T07:22:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.434+0000 7f418e800740 10 client.27100 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.434+0000 7f418e800740 1 -- 172.21.15.26:0/3589664717 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563ff62b4200 con 0x563ff62a5130 2024-06-13T07:22:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.435+0000 7f416aa00640 1 -- 172.21.15.26:0/3589664717 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417800e5b0 con 0x563ff62a5130 2024-06-13T07:22:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.436+0000 7f416aa00640 1 -- 172.21.15.26:0/3589664717 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f417c00f040 con 0x563ff62a5130 2024-06-13T07:22:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.436+0000 7f418e800740 10 client.27100 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.437+0000 7f418e800740 10 client.27100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.437+0000 7f418e800740 20 client.27100 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.437+0000 7f418e800740 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ff62b58f0 0x563ff62b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.437+0000 7f418e800740 4 client.27100 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:22:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.437+0000 7f418e800740 1 -- 172.21.15.26:0/3589664717 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ff6207b50 con 0x563ff62b58f0 2024-06-13T07:22:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.437+0000 7f4189000640 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ff62b58f0 0x563ff62b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.440+0000 7f4189000640 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ff62b58f0 0x563ff62b7ce0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f381a5e5bb627266 server_cookie=4b21496c4032d0e1 in_seq=0 out_seq=0 2024-06-13T07:22:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.442+0000 7f416aa00640 10 client.27100 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.445+0000 7f416aa00640 1 -- 172.21.15.26:0/3589664717 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ff6207b50 con 0x563ff62b58f0 2024-06-13T07:22:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.445+0000 7f416aa00640 10 client.27100 handle_command_reply: tid=0 2024-06-13T07:22:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.445+0000 7f418e800740 1 client.27100 shutdown 2024-06-13T07:22:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f418e800740 1 -- 172.21.15.26:0/3589664717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff62a5130 msgr2=0x563ff62b0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f418e800740 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff62a5130 0x563ff62b0eb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f417c0ccc30 tx=0x7f417c09fed0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f418e800740 1 -- 172.21.15.26:0/3589664717 shutdown_connections 2024-06-13T07:22:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f418e800740 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ff62b58f0 0x563ff62b7ce0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f418e800740 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff62a5130 0x563ff62b0eb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f418e800740 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629ffc0 0x563ff62a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f418e800740 1 --2- 172.21.15.26:0/3589664717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f416aa00640 0 client.27100 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.446+0000 7f418e800740 1 -- 172.21.15.26:0/3589664717 >> 172.21.15.26:0/3589664717 conn(0x563ff625efb0 msgr2=0x563ff62919a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.448+0000 7f418e800740 1 -- 172.21.15.26:0/3589664717 shutdown_connections 2024-06-13T07:22:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.448+0000 7f418e800740 1 -- 172.21.15.26:0/3589664717 wait complete. 2024-06-13T07:22:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.448+0000 7f418e800740 20 client.27100 trim_cache size 0 max 16384 2024-06-13T07:22:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.506+0000 7f418e800740 1 Processor -- start 2024-06-13T07:22:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.506+0000 7f418e800740 1 -- start start 2024-06-13T07:22:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.507+0000 7f418e800740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629f5d0 0x563ff6295e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.507+0000 7f418e800740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f418e800740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f418e800740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41841aacb0 con 0x563ff629f5d0 2024-06-13T07:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f418e800740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a8580 con 0x563ff62a5130 2024-06-13T07:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f418e800740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a6390 con 0x563ff629ffc0 2024-06-13T07:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f4189000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629f5d0 0x563ff6295e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f4189000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629f5d0 0x563ff6295e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44780/0 (socket says 172.21.15.26:44780) 2024-06-13T07:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f4189000640 1 -- 172.21.15.26:0/1117803205 learned_addr learned my addr 172.21.15.26:0/1117803205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f4183e00640 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.508+0000 7f4189a00640 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.509+0000 7f4183e00640 1 -- 172.21.15.26:0/1117803205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 msgr2=0x563ff62bc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.509+0000 7f4183e00640 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62bc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.509+0000 7f4183e00640 1 -- 172.21.15.26:0/1117803205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629f5d0 msgr2=0x563ff6295e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.510+0000 7f4183e00640 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629f5d0 0x563ff6295e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.510+0000 7f4183e00640 1 -- 172.21.15.26:0/1117803205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff62b8050 con 0x563ff629ffc0 2024-06-13T07:22:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.510+0000 7f4189a00640 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62bc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:22:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.511+0000 7f4183e00640 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62b0e70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x563ff62a03c0 tx=0x7f4170016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8b5a72ef92535b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.512+0000 7f4175600640 1 -- 172.21.15.26:0/1117803205 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417002a070 con 0x563ff629ffc0 2024-06-13T07:22:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.512+0000 7f4175600640 1 -- 172.21.15.26:0/1117803205 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41700043e0 con 0x563ff629ffc0 2024-06-13T07:22:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.512+0000 7f4175600640 1 -- 172.21.15.26:0/1117803205 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170025810 con 0x563ff629ffc0 2024-06-13T07:22:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.513+0000 7f418e800740 1 -- 172.21.15.26:0/1117803205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 msgr2=0x563ff62b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.513+0000 7f418e800740 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62b0e70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x563ff62a03c0 tx=0x7f4170016000 comp rx=0 tx=0).stop 2024-06-13T07:22:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.513+0000 7f418e800740 1 -- 172.21.15.26:0/1117803205 shutdown_connections 2024-06-13T07:22:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.513+0000 7f418e800740 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff62a5130 0x563ff62bc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.513+0000 7f418e800740 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62b0e70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.513+0000 7f418e800740 1 --2- 172.21.15.26:0/1117803205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff629f5d0 0x563ff6295e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.513+0000 7f418e800740 1 -- 172.21.15.26:0/1117803205 >> 172.21.15.26:0/1117803205 conn(0x563ff625efb0 msgr2=0x563ff6295730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.513+0000 7f418e800740 1 -- 172.21.15.26:0/1117803205 shutdown_connections 2024-06-13T07:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.516+0000 7f418e800740 1 -- 172.21.15.26:0/1117803205 wait complete. 2024-06-13T07:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.517+0000 7f418e800740 1 -- start start 2024-06-13T07:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.517+0000 7f418e800740 1 Processor -- start 2024-06-13T07:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.518+0000 7f418e800740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.519+0000 7f418e800740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.519+0000 7f4189000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.519+0000 7f4189000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:51582/0 (socket says 172.21.15.26:51582) 2024-06-13T07:22:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.520+0000 7f4189000640 1 -- 172.21.15.26:0/3383207001 learned_addr learned my addr 172.21.15.26:0/3383207001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.520+0000 7f418e800740 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff62a5130 0x563ff62b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.520+0000 7f4189a00640 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff62a5130 0x563ff62b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.521+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a6a40 con 0x563ff62a5130 2024-06-13T07:22:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.521+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a7eb0 con 0x563ff629f5d0 2024-06-13T07:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.521+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a7110 con 0x563ff629ffc0 2024-06-13T07:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.521+0000 7f418e800740 10 client.0 _ll_register_callbacks cb 0x7f417c09b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.520+0000 7f4183e00640 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.522+0000 7f4189a00640 1 -- 172.21.15.26:0/3383207001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 msgr2=0x563ff62af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.522+0000 7f4189a00640 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.522+0000 7f4189a00640 1 -- 172.21.15.26:0/3383207001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 msgr2=0x563ff62aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.522+0000 7f4189a00640 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.522+0000 7f4189a00640 1 -- 172.21.15.26:0/3383207001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff62b0620 con 0x563ff62a5130 2024-06-13T07:22:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.526+0000 7f4189a00640 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff62a5130 0x563ff62b0030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f417c098470 tx=0x7f417c09b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94481221be4c6b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.526+0000 7f416aa00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:22:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.526+0000 7f416aa00640 1 -- 172.21.15.26:0/3383207001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c09aef0 con 0x563ff62a5130 2024-06-13T07:22:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.526+0000 7f416aa00640 1 -- 172.21.15.26:0/3383207001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f417c0a0fb0 con 0x563ff62a5130 2024-06-13T07:22:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.527+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41700043e0 con 0x563ff62a5130 2024-06-13T07:22:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.527+0000 7f416aa00640 1 -- 172.21.15.26:0/3383207001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f417c0ccc60 con 0x563ff62a5130 2024-06-13T07:22:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.527+0000 7f418e800740 10 client.16785 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.527+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563ff6296b50 con 0x563ff62a5130 2024-06-13T07:22:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.528+0000 7f416aa00640 1 -- 172.21.15.26:0/3383207001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41700043e0 con 0x563ff62a5130 2024-06-13T07:22:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.528+0000 7f416aa00640 1 -- 172.21.15.26:0/3383207001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f417c004cf0 con 0x563ff62a5130 2024-06-13T07:22:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.528+0000 7f418e800740 10 client.16785 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.528+0000 7f418e800740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.528+0000 7f418e800740 20 client.16785 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.528+0000 7f418e800740 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ff62973c0 0x563ff62997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.528+0000 7f418e800740 4 client.16785 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:22:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.528+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ff6201ea0 con 0x563ff62973c0 2024-06-13T07:22:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.529+0000 7f4189000640 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ff62973c0 0x563ff62997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.532+0000 7f4189000640 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ff62973c0 0x563ff62997e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4958e6a5b1baae2c server_cookie=42f3ec59ac7569a5 in_seq=0 out_seq=0 2024-06-13T07:22:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:16.532+0000 7f416aa00640 10 client.16785 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:16 smithi026 ceph-mon[32041]: pgmap v1162: 129 pgs: 129 active+clean; 665 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-06-13T07:22:16.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:16 smithi135 ceph-mon[36276]: pgmap v1162: 129 pgs: 129 active+clean; 665 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-06-13T07:22:16.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:16 smithi136 ceph-mon[37229]: pgmap v1162: 129 pgs: 129 active+clean; 665 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-06-13T07:22:17.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 253877 26.26 MB/sec execute 390 sec latency 515.343 ms 2024-06-13T07:22:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.191+0000 7f416aa00640 1 -- 172.21.15.26:0/3383207001 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563ff6201ea0 con 0x563ff62973c0 2024-06-13T07:22:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.191+0000 7f416aa00640 10 client.16785 handle_command_reply: tid=0 2024-06-13T07:22:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.191+0000 7f418e800740 1 client.16785 shutdown 2024-06-13T07:22:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.193+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff62a5130 msgr2=0x563ff62b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.193+0000 7f418e800740 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff62a5130 0x563ff62b0030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f417c098470 tx=0x7f417c09b2c0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.194+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 shutdown_connections 2024-06-13T07:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.194+0000 7f418e800740 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x563ff62973c0 0x563ff62997e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.194+0000 7f418e800740 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ff62a5130 0x563ff62b0030 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.194+0000 7f418e800740 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ff629ffc0 0x563ff62af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.194+0000 7f418e800740 1 --2- 172.21.15.26:0/3383207001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff629f5d0 0x563ff62aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.194+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 >> 172.21.15.26:0/3383207001 conn(0x563ff625efb0 msgr2=0x563ff6291110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.194+0000 7f416aa00640 0 client.16785 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.195+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 shutdown_connections 2024-06-13T07:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.196+0000 7f418e800740 1 -- 172.21.15.26:0/3383207001 wait complete. 2024-06-13T07:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.196+0000 7f418e800740 20 client.16785 trim_cache size 0 max 16384 2024-06-13T07:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.198+0000 7f418a400640 1 -- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f416c0a2c80 msgr2=0x7f416c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.198+0000 7f418a400640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f416c0a2c80 0x7f416c0a5140 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4184196e50 tx=0x7f4170005f90 comp rx=0 tx=0).stop 2024-06-13T07:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.198+0000 7f418a400640 1 -- 172.21.15.26:0/1614764694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4184103410 msgr2=0x7f418419cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.198+0000 7f418a400640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4184103410 0x7f418419cef0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f417c00eaa0 tx=0x7f417c00ef60 comp rx=0 tx=0).stop 2024-06-13T07:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.198+0000 7f418a400640 1 -- 172.21.15.26:0/1614764694 shutdown_connections 2024-06-13T07:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.198+0000 7f418a400640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f416c0a2c80 0x7f416c0a5140 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.198+0000 7f418a400640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4184103410 0x7f418419cef0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.198+0000 7f418a400640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4184102a50 0x7f4184195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.199+0000 7f418a400640 1 --2- 172.21.15.26:0/1614764694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4184101860 0x7f4184195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.199+0000 7f418a400640 1 -- 172.21.15.26:0/1614764694 >> 172.21.15.26:0/1614764694 conn(0x7f41840fcdd0 msgr2=0x7f4184105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.199+0000 7f418a400640 1 -- 172.21.15.26:0/1614764694 shutdown_connections 2024-06-13T07:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:18.199+0000 7f418a400640 1 -- 172.21.15.26:0/1614764694 wait complete. 2024-06-13T07:22:18.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 253982 26.21 MB/sec execute 391 sec latency 1126.745 ms 2024-06-13T07:22:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:18 smithi026 ceph-mon[32041]: pgmap v1163: 129 pgs: 129 active+clean; 672 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.0 MiB/s wr, 480 op/s 2024-06-13T07:22:18.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:18 smithi135 ceph-mon[36276]: pgmap v1163: 129 pgs: 129 active+clean; 672 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.0 MiB/s wr, 480 op/s 2024-06-13T07:22:18.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:18 smithi136 ceph-mon[37229]: pgmap v1163: 129 pgs: 129 active+clean; 672 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.0 MiB/s wr, 480 op/s 2024-06-13T07:22:19.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 254235 26.16 MB/sec execute 392 sec latency 1182.946 ms 2024-06-13T07:22:20.219 INFO:tasks.workunit.client.0.smithi026.stdout: 10 254519 26.13 MB/sec execute 393 sec latency 165.490 ms 2024-06-13T07:22:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:20 smithi026 ceph-mon[32041]: pgmap v1164: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.9 MiB/s wr, 506 op/s 2024-06-13T07:22:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:20 smithi135 ceph-mon[36276]: pgmap v1164: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.9 MiB/s wr, 506 op/s 2024-06-13T07:22:20.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:20 smithi136 ceph-mon[37229]: pgmap v1164: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.9 MiB/s wr, 506 op/s 2024-06-13T07:22:21.220 INFO:tasks.workunit.client.0.smithi026.stdout: 10 255224 26.17 MB/sec execute 394 sec latency 81.002 ms 2024-06-13T07:22:22.221 INFO:tasks.workunit.client.0.smithi026.stdout: 10 255890 26.20 MB/sec execute 395 sec latency 51.807 ms 2024-06-13T07:22:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:22 smithi026 ceph-mon[32041]: pgmap v1165: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 453 op/s 2024-06-13T07:22:22.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:22 smithi135 ceph-mon[36276]: pgmap v1165: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 453 op/s 2024-06-13T07:22:22.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:22 smithi136 ceph-mon[37229]: pgmap v1165: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 453 op/s 2024-06-13T07:22:23.221 INFO:tasks.workunit.client.0.smithi026.stdout: 10 256103 26.16 MB/sec execute 396 sec latency 377.002 ms 2024-06-13T07:22:24.221 INFO:tasks.workunit.client.0.smithi026.stdout: 10 256370 26.15 MB/sec execute 397 sec latency 196.323 ms 2024-06-13T07:22:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:24 smithi026 ceph-mon[32041]: pgmap v1166: 129 pgs: 129 active+clean; 684 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 547 op/s 2024-06-13T07:22:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:24 smithi135 ceph-mon[36276]: pgmap v1166: 129 pgs: 129 active+clean; 684 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 547 op/s 2024-06-13T07:22:24.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:24 smithi136 ceph-mon[37229]: pgmap v1166: 129 pgs: 129 active+clean; 684 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 547 op/s 2024-06-13T07:22:25.221 INFO:tasks.workunit.client.0.smithi026.stdout: 10 256702 26.12 MB/sec execute 398 sec latency 115.412 ms 2024-06-13T07:22:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:25 smithi026 ceph-mon[32041]: pgmap v1167: 129 pgs: 129 active+clean; 686 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.6 MiB/s wr, 573 op/s 2024-06-13T07:22:25.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:25 smithi135 ceph-mon[36276]: pgmap v1167: 129 pgs: 129 active+clean; 686 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.6 MiB/s wr, 573 op/s 2024-06-13T07:22:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:25 smithi136 ceph-mon[37229]: pgmap v1167: 129 pgs: 129 active+clean; 686 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.6 MiB/s wr, 573 op/s 2024-06-13T07:22:26.221 INFO:tasks.workunit.client.0.smithi026.stdout: 10 257193 26.11 MB/sec execute 399 sec latency 95.590 ms 2024-06-13T07:22:27.222 INFO:tasks.workunit.client.0.smithi026.stdout: 10 257658 26.11 MB/sec execute 400 sec latency 76.223 ms 2024-06-13T07:22:28.222 INFO:tasks.workunit.client.0.smithi026.stdout: 10 258278 26.12 MB/sec execute 401 sec latency 95.059 ms 2024-06-13T07:22:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:28 smithi026 ceph-mon[32041]: pgmap v1168: 129 pgs: 129 active+clean; 686 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 617 op/s 2024-06-13T07:22:28.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:28 smithi135 ceph-mon[36276]: pgmap v1168: 129 pgs: 129 active+clean; 686 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 617 op/s 2024-06-13T07:22:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:28 smithi136 ceph-mon[37229]: pgmap v1168: 129 pgs: 129 active+clean; 686 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 617 op/s 2024-06-13T07:22:29.222 INFO:tasks.workunit.client.0.smithi026.stdout: 10 258785 26.14 MB/sec execute 402 sec latency 112.512 ms 2024-06-13T07:22:30.222 INFO:tasks.workunit.client.0.smithi026.stdout: 10 259369 26.14 MB/sec execute 403 sec latency 115.104 ms 2024-06-13T07:22:30.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:30 smithi026 ceph-mon[32041]: pgmap v1169: 129 pgs: 129 active+clean; 664 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.8 MiB/s wr, 653 op/s 2024-06-13T07:22:30.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:30 smithi135 ceph-mon[36276]: pgmap v1169: 129 pgs: 129 active+clean; 664 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.8 MiB/s wr, 653 op/s 2024-06-13T07:22:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:30 smithi136 ceph-mon[37229]: pgmap v1169: 129 pgs: 129 active+clean; 664 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.8 MiB/s wr, 653 op/s 2024-06-13T07:22:31.222 INFO:tasks.workunit.client.0.smithi026.stdout: 10 259871 26.15 MB/sec execute 404 sec latency 77.082 ms 2024-06-13T07:22:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.399+0000 7f4af5800640 1 -- 172.21.15.26:0/1240976343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 msgr2=0x7f4af0101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.399+0000 7f4af5800640 1 --2- 172.21.15.26:0/1240976343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 0x7f4af0101e30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009900 tx=0x7f4ae002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.399+0000 7f4af5800640 1 -- 172.21.15.26:0/1240976343 shutdown_connections 2024-06-13T07:22:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.399+0000 7f4af5800640 1 --2- 172.21.15.26:0/1240976343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4af01035c0 0x7f4af0109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.399+0000 7f4af5800640 1 --2- 172.21.15.26:0/1240976343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af0102c00 0x7f4af0103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.399+0000 7f4af5800640 1 --2- 172.21.15.26:0/1240976343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 0x7f4af0101e30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.399+0000 7f4af5800640 1 -- 172.21.15.26:0/1240976343 >> 172.21.15.26:0/1240976343 conn(0x7f4af00fcf60 msgr2=0x7f4af00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.400+0000 7f4af5800640 1 -- 172.21.15.26:0/1240976343 shutdown_connections 2024-06-13T07:22:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.400+0000 7f4af5800640 1 -- 172.21.15.26:0/1240976343 wait complete. 2024-06-13T07:22:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.401+0000 7f4af5800640 1 Processor -- start 2024-06-13T07:22:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.401+0000 7f4af5800640 1 -- start start 2024-06-13T07:22:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.401+0000 7f4af5800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 0x7f4af01105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.401+0000 7f4af5800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4af0102c00 0x7f4af0110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.401+0000 7f4af5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af01035c0 0x7f4af010d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.401+0000 7f4af5800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4af0074620 con 0x7f4af01035c0 2024-06-13T07:22:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.401+0000 7f4af5800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4af00744a0 con 0x7f4af0101a10 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.401+0000 7f4af5800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4af00747a0 con 0x7f4af0102c00 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.402+0000 7f4aeea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 0x7f4af01105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.402+0000 7f4aee000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4af0102c00 0x7f4af0110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.402+0000 7f4aeea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 0x7f4af01105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43140/0 (socket says 172.21.15.26:43140) 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.402+0000 7f4aeea00640 1 -- 172.21.15.26:0/984361800 learned_addr learned my addr 172.21.15.26:0/984361800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.403+0000 7f4aeea00640 1 -- 172.21.15.26:0/984361800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4af0102c00 msgr2=0x7f4af0110b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.403+0000 7f4aeea00640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4af0102c00 0x7f4af0110b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.403+0000 7f4aeea00640 1 -- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af01035c0 msgr2=0x7f4af010d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.403+0000 7f4aeea00640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af01035c0 0x7f4af010d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.403+0000 7f4aeea00640 1 -- 172.21.15.26:0/984361800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af010da00 con 0x7f4af0101a10 2024-06-13T07:22:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.404+0000 7f4aeea00640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 0x7f4af01105d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0031440 tx=0x7f4ae0032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.404+0000 7f4ae7400640 1 -- 172.21.15.26:0/984361800 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae003f030 con 0x7f4af0101a10 2024-06-13T07:22:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.404+0000 7f4ae7400640 1 -- 172.21.15.26:0/984361800 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae0003ce0 con 0x7f4af0101a10 2024-06-13T07:22:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.404+0000 7f4ae7400640 1 -- 172.21.15.26:0/984361800 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0035070 con 0x7f4af0101a10 2024-06-13T07:22:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.404+0000 7f4af5800640 1 -- 172.21.15.26:0/984361800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af01a6290 con 0x7f4af0101a10 2024-06-13T07:22:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.404+0000 7f4af5800640 1 -- 172.21.15.26:0/984361800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af01a6640 con 0x7f4af0101a10 2024-06-13T07:22:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.410+0000 7f4af87a0740 1 Processor -- start 2024-06-13T07:22:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.411+0000 7f4af87a0740 1 -- start start 2024-06-13T07:22:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.411+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394ab020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.411+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.411+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.411+0000 7f4af87a0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562d3940e420 con 0x562d394b0760 2024-06-13T07:22:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.411+0000 7f4af87a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562d393b5c50 con 0x562d394ab5f0 2024-06-13T07:22:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.411+0000 7f4af87a0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562d3940a3e0 con 0x562d394aac00 2024-06-13T07:22:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.412+0000 7f4aef400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.412+0000 7f4aee000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.412+0000 7f4aee000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42252/0 (socket says 172.21.15.26:42252) 2024-06-13T07:22:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.412+0000 7f4aee000640 1 -- 172.21.15.26:0/96619274 learned_addr learned my addr 172.21.15.26:0/96619274 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.413+0000 7f4aee000640 1 -- 172.21.15.26:0/96619274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 msgr2=0x562d394ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.413+0000 7f4aee000640 1 --2- 172.21.15.26:0/96619274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394ab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.413+0000 7f4aee000640 1 -- 172.21.15.26:0/96619274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 msgr2=0x562d394b00f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:22:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.413+0000 7f4aee000640 1 --2- 172.21.15.26:0/96619274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394b00f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.413+0000 7f4aee000640 1 -- 172.21.15.26:0/96619274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d39122210 con 0x562d394b0760 2024-06-13T07:22:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.413+0000 7f4aee000640 1 --2- 172.21.15.26:0/96619274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394b2bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00bcb0 tx=0x7f4adc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b688a08cadfae5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.415+0000 7f4adbe00640 1 -- 172.21.15.26:0/96619274 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc004030 con 0x562d394b0760 2024-06-13T07:22:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.415+0000 7f4adbe00640 1 -- 172.21.15.26:0/96619274 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4adc0041d0 con 0x562d394b0760 2024-06-13T07:22:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.415+0000 7f4adbe00640 1 -- 172.21.15.26:0/96619274 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc011770 con 0x562d394b0760 2024-06-13T07:22:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.416+0000 7f4ae7400640 1 -- 172.21.15.26:0/984361800 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ae0042030 con 0x7f4af0101a10 2024-06-13T07:22:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.416+0000 7f4af87a0740 1 -- 172.21.15.26:0/96619274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 msgr2=0x562d394b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.416+0000 7f4af87a0740 1 --2- 172.21.15.26:0/96619274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394b2bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00bcb0 tx=0x7f4adc0107a0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.416+0000 7f4af87a0740 1 -- 172.21.15.26:0/96619274 shutdown_connections 2024-06-13T07:22:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.416+0000 7f4af87a0740 1 --2- 172.21.15.26:0/96619274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.416+0000 7f4af87a0740 1 --2- 172.21.15.26:0/96619274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394b2bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.416+0000 7f4af87a0740 1 --2- 172.21.15.26:0/96619274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.416+0000 7f4af87a0740 1 -- 172.21.15.26:0/96619274 >> 172.21.15.26:0/96619274 conn(0x562d39472fb0 msgr2=0x562d394a6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.419+0000 7f4af87a0740 1 -- 172.21.15.26:0/96619274 shutdown_connections 2024-06-13T07:22:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.419+0000 7f4af87a0740 1 -- 172.21.15.26:0/96619274 wait complete. 2024-06-13T07:22:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.419+0000 7f4ae7400640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ad40a2c40 0x7f4ad40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.419+0000 7f4ae7400640 1 -- 172.21.15.26:0/984361800 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4ae00efa50 con 0x7f4af0101a10 2024-06-13T07:22:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.420+0000 7f4aee000640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ad40a2c40 0x7f4ad40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.421+0000 7f4af87a0740 1 -- start start 2024-06-13T07:22:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.422+0000 7f4aee000640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ad40a2c40 0x7f4ad40a5100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000c00 tx=0x7f4adc00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.422+0000 7f4af87a0740 1 Processor -- start 2024-06-13T07:22:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.423+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.423+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.423+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.423+0000 7f4af87a0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562d394c4720 con 0x562d394b0760 2024-06-13T07:22:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.423+0000 7f4af87a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562d393c90b0 con 0x562d394ab5f0 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.423+0000 7f4af87a0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562d394c48a0 con 0x562d394aac00 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.423+0000 7f4af87a0740 10 client.0 _ll_register_callbacks cb 0x562d393b4f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.424+0000 7f4aee000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.424+0000 7f4aeea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.424+0000 7f4aee000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43150/0 (socket says 172.21.15.26:43150) 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.424+0000 7f4aee000640 1 -- 172.21.15.26:0/1848064942 learned_addr learned my addr 172.21.15.26:0/1848064942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.424+0000 7f4aeea00640 1 -- 172.21.15.26:0/1848064942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 msgr2=0x562d394c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.424+0000 7f4aeea00640 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.424+0000 7f4aeea00640 1 -- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 msgr2=0x562d394cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4aef400640 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4aeea00640 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4aeea00640 1 -- 172.21.15.26:0/1848064942 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d391ee620 con 0x562d394aac00 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4aef400640 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4aeea00640 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394c5fc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x562d394ac060 tx=0x7f4ae00bb9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56da143633631af1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4ad8c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1848064942 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00bbe60 con 0x562d394aac00 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1848064942 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4ae00b84d0 con 0x562d394aac00 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.425+0000 7f4af87a0740 1 -- 172.21.15.26:0/1848064942 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4adc0041d0 con 0x562d394aac00 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1848064942 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4ae00c0210 con 0x562d394aac00 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4af87a0740 10 client.27118 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4af87a0740 1 -- 172.21.15.26:0/1848064942 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562d394c47f0 con 0x562d394aac00 2024-06-13T07:22:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1848064942 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4adc0041d0 con 0x562d394aac00 2024-06-13T07:22:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1848064942 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae00bb2b0 con 0x562d394aac00 2024-06-13T07:22:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4af87a0740 10 client.27118 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4af87a0740 10 client.27118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4af87a0740 20 client.27118 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562d394d2010 0x562d394d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4af87a0740 4 client.27118 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:22:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.426+0000 7f4af87a0740 1 -- 172.21.15.26:0/1848064942 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d3941bb50 con 0x562d394d2010 2024-06-13T07:22:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.427+0000 7f4aee000640 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562d394d2010 0x562d394d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.429+0000 7f4aee000640 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562d394d2010 0x562d394d4460 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f449594125e3dc80 server_cookie=9b9057f4d4be36ef in_seq=0 out_seq=0 2024-06-13T07:22:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.429+0000 7f4ad8c00640 10 client.27118 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.431+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1848064942 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562d3941bb50 con 0x562d394d2010 2024-06-13T07:22:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.431+0000 7f4ad8c00640 10 client.27118 handle_command_reply: tid=0 2024-06-13T07:22:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.431+0000 7f4af87a0740 1 client.27118 shutdown 2024-06-13T07:22:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.431+0000 7f4af87a0740 1 -- 172.21.15.26:0/1848064942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 msgr2=0x562d394c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.431+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394c5fc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x562d394ac060 tx=0x7f4ae00bb9a0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 1 -- 172.21.15.26:0/1848064942 shutdown_connections 2024-06-13T07:22:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562d394d2010 0x562d394d4460 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394b0760 0x562d394cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1848064942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394c5fc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 1 -- 172.21.15.26:0/1848064942 >> 172.21.15.26:0/1848064942 conn(0x562d39472fb0 msgr2=0x562d394a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4ad8c00640 0 client.27118 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 1 -- 172.21.15.26:0/1848064942 shutdown_connections 2024-06-13T07:22:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 1 -- 172.21.15.26:0/1848064942 wait complete. 2024-06-13T07:22:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.432+0000 7f4af87a0740 20 client.27118 trim_cache size 0 max 16384 2024-06-13T07:22:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.478+0000 7f4af87a0740 1 Processor -- start 2024-06-13T07:22:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.478+0000 7f4af87a0740 1 -- start start 2024-06-13T07:22:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394aac00 0x562d394cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394b0760 0x562d394a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4af87a0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4af006f800 con 0x562d394aac00 2024-06-13T07:22:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4af87a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4af006d110 con 0x562d394ab5f0 2024-06-13T07:22:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4af87a0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4af01a1e00 con 0x562d394b0760 2024-06-13T07:22:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aee000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aef400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394b0760 0x562d394a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aee000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43164/0 (socket says 172.21.15.26:43164) 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aee000640 1 -- 172.21.15.26:0/3539092009 learned_addr learned my addr 172.21.15.26:0/3539092009 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aef400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394b0760 0x562d394a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:43410/0 (socket says 172.21.15.26:43410) 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aee000640 1 -- 172.21.15.26:0/3539092009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394b0760 msgr2=0x562d394a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aee000640 1 --2- 172.21.15.26:0/3539092009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394b0760 0x562d394a9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aee000640 1 -- 172.21.15.26:0/3539092009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394aac00 msgr2=0x562d394cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aee000640 1 --2- 172.21.15.26:0/3539092009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394aac00 0x562d394cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.479+0000 7f4aee000640 1 -- 172.21.15.26:0/3539092009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d394a7330 con 0x562d394ab5f0 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.480+0000 7f4aef400640 1 --2- 172.21.15.26:0/3539092009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394b0760 0x562d394a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.480+0000 7f4aee000640 1 --2- 172.21.15.26:0/3539092009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394cd500 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011010 tx=0x7f4adc022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5277e89432ae51f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.480+0000 7f4adb400640 1 -- 172.21.15.26:0/3539092009 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc00bda0 con 0x562d394ab5f0 2024-06-13T07:22:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.480+0000 7f4adb400640 1 -- 172.21.15.26:0/3539092009 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4adc003de0 con 0x562d394ab5f0 2024-06-13T07:22:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.480+0000 7f4adb400640 1 -- 172.21.15.26:0/3539092009 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0049b0 con 0x562d394ab5f0 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.483+0000 7f4af87a0740 1 -- 172.21.15.26:0/3539092009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 msgr2=0x562d394cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.483+0000 7f4af87a0740 1 --2- 172.21.15.26:0/3539092009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394cd500 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011010 tx=0x7f4adc022000 comp rx=0 tx=0).stop 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.483+0000 7f4af87a0740 1 -- 172.21.15.26:0/3539092009 shutdown_connections 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.483+0000 7f4af87a0740 1 --2- 172.21.15.26:0/3539092009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394ab5f0 0x562d394cd500 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011010 tx=0x7f4adc022000 comp rx=0 tx=0).stop 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.483+0000 7f4af87a0740 1 --2- 172.21.15.26:0/3539092009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394b0760 0x562d394a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.483+0000 7f4af87a0740 1 --2- 172.21.15.26:0/3539092009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394aac00 0x562d394cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.483+0000 7f4af87a0740 1 -- 172.21.15.26:0/3539092009 >> 172.21.15.26:0/3539092009 conn(0x562d39472fb0 msgr2=0x562d394a9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.483+0000 7f4af87a0740 1 -- 172.21.15.26:0/3539092009 shutdown_connections 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.484+0000 7f4af87a0740 1 -- 172.21.15.26:0/3539092009 wait complete. 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.484+0000 7f4af87a0740 1 -- start start 2024-06-13T07:22:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.484+0000 7f4af87a0740 1 Processor -- start 2024-06-13T07:22:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394b0760 0x562d394cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4af87a0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394cbdf0 0x562d394cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4af87a0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4af006b5d0 con 0x562d394cbdf0 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4af87a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4af006ca40 con 0x562d394b0760 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4af87a0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4af006bca0 con 0x562d394aac00 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4af87a0740 10 client.0 _ll_register_callbacks cb 0x7f4ae0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4aef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394cbdf0 0x562d394cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4aef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394cbdf0 0x562d394cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42294/0 (socket says 172.21.15.26:42294) 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.486+0000 7f4aef400640 1 -- 172.21.15.26:0/1831896467 learned_addr learned my addr 172.21.15.26:0/1831896467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.487+0000 7f4aef400640 1 -- 172.21.15.26:0/1831896467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 msgr2=0x562d394ca910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.487+0000 7f4aef400640 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394ca910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.487+0000 7f4aef400640 1 -- 172.21.15.26:0/1831896467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394b0760 msgr2=0x562d394cae50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:22:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.487+0000 7f4aef400640 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394b0760 0x562d394cae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.487+0000 7f4aef400640 1 -- 172.21.15.26:0/1831896467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d394cc7b0 con 0x562d394cbdf0 2024-06-13T07:22:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.491+0000 7f4ad8c00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:22:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.491+0000 7f4aef400640 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394cbdf0 0x562d394cc270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80126a0 tx=0x7f4ae8012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76b9f7bf7bb46c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.492+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1831896467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8003c40 con 0x562d394cbdf0 2024-06-13T07:22:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.492+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1831896467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4ae8018070 con 0x562d394cbdf0 2024-06-13T07:22:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.493+0000 7f4af87a0740 1 -- 172.21.15.26:0/1831896467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4adc003de0 con 0x562d394cbdf0 2024-06-13T07:22:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.493+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1831896467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4ae8003c40 con 0x562d394cbdf0 2024-06-13T07:22:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.493+0000 7f4af87a0740 10 client.16794 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.493+0000 7f4af87a0740 1 -- 172.21.15.26:0/1831896467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562d3953ce70 con 0x562d394cbdf0 2024-06-13T07:22:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.494+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1831896467 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae8003de0 con 0x562d394cbdf0 2024-06-13T07:22:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.495+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1831896467 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae800f040 con 0x562d394cbdf0 2024-06-13T07:22:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.495+0000 7f4af87a0740 10 client.16794 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.495+0000 7f4af87a0740 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.495+0000 7f4af87a0740 20 client.16794 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.495+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562d3953d8b0 0x562d3953fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.495+0000 7f4af87a0740 4 client.16794 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:22:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.495+0000 7f4af87a0740 1 -- 172.21.15.26:0/1831896467 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4adc003de0 con 0x562d3953d8b0 2024-06-13T07:22:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.496+0000 7f4aeea00640 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562d3953d8b0 0x562d3953fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.497+0000 7f4aeea00640 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562d3953d8b0 0x562d3953fcd0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca74ca1a493119ac server_cookie=60a4840c1f21fad8 in_seq=0 out_seq=0 2024-06-13T07:22:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:31.497+0000 7f4ad8c00640 10 client.16794 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:32.224 INFO:tasks.workunit.client.0.smithi026.stdout: 10 260114 26.11 MB/sec execute 405 sec latency 461.263 ms 2024-06-13T07:22:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.265+0000 7f4ad8c00640 1 -- 172.21.15.26:0/1831896467 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f4adc003de0 con 0x562d3953d8b0 2024-06-13T07:22:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.265+0000 7f4ad8c00640 10 client.16794 handle_command_reply: tid=0 2024-06-13T07:22:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.265+0000 7f4af87a0740 1 client.16794 shutdown 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.265+0000 7f4af87a0740 1 -- 172.21.15.26:0/1831896467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394cbdf0 msgr2=0x562d394cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.266+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394cbdf0 0x562d394cc270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80126a0 tx=0x7f4ae8012b60 comp rx=0 tx=0).stop 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.266+0000 7f4af87a0740 1 -- 172.21.15.26:0/1831896467 shutdown_connections 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.266+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562d3953d8b0 0x562d3953fcd0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.266+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562d394b0760 0x562d394cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.266+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d394cbdf0 0x562d394cc270 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.266+0000 7f4af87a0740 1 --2- 172.21.15.26:0/1831896467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562d394aac00 0x562d394ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.266+0000 7f4af87a0740 1 -- 172.21.15.26:0/1831896467 >> 172.21.15.26:0/1831896467 conn(0x562d39472fb0 msgr2=0x562d394c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.266+0000 7f4ad8c00640 0 client.16794 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.267+0000 7f4af87a0740 1 -- 172.21.15.26:0/1831896467 shutdown_connections 2024-06-13T07:22:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.267+0000 7f4af87a0740 1 -- 172.21.15.26:0/1831896467 wait complete. 2024-06-13T07:22:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.267+0000 7f4af87a0740 20 client.16794 trim_cache size 0 max 16384 2024-06-13T07:22:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:22:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:22:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 -- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ad40a2c40 msgr2=0x7f4ad40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ad40a2c40 0x7f4ad40a5100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000c00 tx=0x7f4adc00b590 comp rx=0 tx=0).stop 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 -- 172.21.15.26:0/984361800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 msgr2=0x7f4af01105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 0x7f4af01105d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0031440 tx=0x7f4ae0032e20 comp rx=0 tx=0).stop 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 -- 172.21.15.26:0/984361800 shutdown_connections 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f4ad40a2c40 0x7f4ad40a5100 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af01035c0 0x7f4af010d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4af0102c00 0x7f4af0110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 --2- 172.21.15.26:0/984361800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af0101a10 0x7f4af01105d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.272+0000 7f4af5800640 1 -- 172.21.15.26:0/984361800 >> 172.21.15.26:0/984361800 conn(0x7f4af00fcf60 msgr2=0x7f4af00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.274+0000 7f4af5800640 1 -- 172.21.15.26:0/984361800 shutdown_connections 2024-06-13T07:22:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:32.274+0000 7f4af5800640 1 -- 172.21.15.26:0/984361800 wait complete. 2024-06-13T07:22:32.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:32 smithi026 ceph-mon[32041]: pgmap v1170: 129 pgs: 129 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 494 op/s 2024-06-13T07:22:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:32 smithi135 ceph-mon[36276]: pgmap v1170: 129 pgs: 129 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 494 op/s 2024-06-13T07:22:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:32 smithi136 ceph-mon[37229]: pgmap v1170: 129 pgs: 129 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 494 op/s 2024-06-13T07:22:33.224 INFO:tasks.workunit.client.0.smithi026.stdout: 10 260490 26.10 MB/sec execute 406 sec latency 91.882 ms 2024-06-13T07:22:34.224 INFO:tasks.workunit.client.0.smithi026.stdout: 10 260979 26.11 MB/sec execute 407 sec latency 116.932 ms 2024-06-13T07:22:34.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:34 smithi026 ceph-mon[32041]: pgmap v1171: 129 pgs: 129 active+clean; 662 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-06-13T07:22:34.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:34 smithi135 ceph-mon[36276]: pgmap v1171: 129 pgs: 129 active+clean; 662 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-06-13T07:22:34.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:34 smithi136 ceph-mon[37229]: pgmap v1171: 129 pgs: 129 active+clean; 662 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-06-13T07:22:35.224 INFO:tasks.workunit.client.0.smithi026.stdout: 10 261532 26.10 MB/sec execute 408 sec latency 74.046 ms 2024-06-13T07:22:36.224 INFO:tasks.workunit.client.0.smithi026.stdout: 10 262066 26.12 MB/sec execute 409 sec latency 82.986 ms 2024-06-13T07:22:36.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:36 smithi026 ceph-mon[32041]: pgmap v1172: 129 pgs: 129 active+clean; 660 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.2 MiB/s wr, 560 op/s 2024-06-13T07:22:36.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:36 smithi135 ceph-mon[36276]: pgmap v1172: 129 pgs: 129 active+clean; 660 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.2 MiB/s wr, 560 op/s 2024-06-13T07:22:36.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:36 smithi136 ceph-mon[37229]: pgmap v1172: 129 pgs: 129 active+clean; 660 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.2 MiB/s wr, 560 op/s 2024-06-13T07:22:37.224 INFO:tasks.workunit.client.0.smithi026.stdout: 10 262558 26.13 MB/sec execute 410 sec latency 132.212 ms 2024-06-13T07:22:38.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 263124 26.12 MB/sec execute 411 sec latency 80.548 ms 2024-06-13T07:22:38.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:38 smithi026 ceph-mon[32041]: pgmap v1173: 129 pgs: 129 active+clean; 662 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.6 MiB/s wr, 456 op/s 2024-06-13T07:22:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:38 smithi135 ceph-mon[36276]: pgmap v1173: 129 pgs: 129 active+clean; 662 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.6 MiB/s wr, 456 op/s 2024-06-13T07:22:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:38 smithi136 ceph-mon[37229]: pgmap v1173: 129 pgs: 129 active+clean; 662 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.6 MiB/s wr, 456 op/s 2024-06-13T07:22:39.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 263531 26.10 MB/sec execute 412 sec latency 311.332 ms 2024-06-13T07:22:40.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 264133 26.13 MB/sec execute 413 sec latency 78.872 ms 2024-06-13T07:22:40.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:40 smithi026 ceph-mon[32041]: pgmap v1174: 129 pgs: 129 active+clean; 671 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-06-13T07:22:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:40 smithi135 ceph-mon[36276]: pgmap v1174: 129 pgs: 129 active+clean; 671 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-06-13T07:22:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:40 smithi136 ceph-mon[37229]: pgmap v1174: 129 pgs: 129 active+clean; 671 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-06-13T07:22:41.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 264941 26.16 MB/sec execute 414 sec latency 47.277 ms 2024-06-13T07:22:42.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 265584 26.21 MB/sec execute 415 sec latency 74.061 ms 2024-06-13T07:22:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:42 smithi026 ceph-mon[32041]: pgmap v1175: 129 pgs: 129 active+clean; 672 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 394 op/s 2024-06-13T07:22:42.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:42 smithi135 ceph-mon[36276]: pgmap v1175: 129 pgs: 129 active+clean; 672 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 394 op/s 2024-06-13T07:22:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:42 smithi136 ceph-mon[37229]: pgmap v1175: 129 pgs: 129 active+clean; 672 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.5 MiB/s wr, 394 op/s 2024-06-13T07:22:43.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 266056 26.21 MB/sec execute 416 sec latency 67.501 ms 2024-06-13T07:22:44.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 266173 26.16 MB/sec execute 417 sec latency 398.940 ms 2024-06-13T07:22:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:44 smithi135 ceph-mon[36276]: pgmap v1176: 129 pgs: 129 active+clean; 679 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.2 MiB/s wr, 463 op/s 2024-06-13T07:22:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:44 smithi136 ceph-mon[37229]: pgmap v1176: 129 pgs: 129 active+clean; 679 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.2 MiB/s wr, 463 op/s 2024-06-13T07:22:44.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:44 smithi026 ceph-mon[32041]: pgmap v1176: 129 pgs: 129 active+clean; 679 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.2 MiB/s wr, 463 op/s 2024-06-13T07:22:45.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 266336 26.11 MB/sec execute 418 sec latency 402.260 ms 2024-06-13T07:22:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.512+0000 7f6a76600640 1 -- 172.21.15.26:0/3295310817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 msgr2=0x7f6a70105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.512+0000 7f6a76600640 1 --2- 172.21.15.26:0/3295310817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 0x7f6a70105290 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009860 tx=0x7f6a5c02a340 comp rx=0 tx=0).stop 2024-06-13T07:22:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.515+0000 7f6a76600640 1 -- 172.21.15.26:0/3295310817 shutdown_connections 2024-06-13T07:22:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.515+0000 7f6a76600640 1 --2- 172.21.15.26:0/3295310817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 0x7f6a70105290 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.515+0000 7f6a76600640 1 --2- 172.21.15.26:0/3295310817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a701057d0 0x7f6a7010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.515+0000 7f6a76600640 1 --2- 172.21.15.26:0/3295310817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70103c20 0x7f6a70104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.515+0000 7f6a76600640 1 -- 172.21.15.26:0/3295310817 >> 172.21.15.26:0/3295310817 conn(0x7f6a700ff190 msgr2=0x7f6a701015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.516+0000 7f6a76600640 1 -- 172.21.15.26:0/3295310817 shutdown_connections 2024-06-13T07:22:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.516+0000 7f6a76600640 1 -- 172.21.15.26:0/3295310817 wait complete. 2024-06-13T07:22:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.517+0000 7f6a76600640 1 Processor -- start 2024-06-13T07:22:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.517+0000 7f6a76600640 1 -- start start 2024-06-13T07:22:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a76600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70103c20 0x7f6a70199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a76600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 0x7f6a7019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a76600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a701057d0 0x7f6a701a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a76600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010f880 con 0x7f6a701057d0 2024-06-13T07:22:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a76600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010f700 con 0x7f6a70104e10 2024-06-13T07:22:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a76600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010fa00 con 0x7f6a70103c20 2024-06-13T07:22:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a6ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 0x7f6a7019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.519+0000 7f6a6ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 0x7f6a7019a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37910/0 (socket says 172.21.15.26:37910) 2024-06-13T07:22:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.519+0000 7f6a6ea00640 1 -- 172.21.15.26:0/2888896398 learned_addr learned my addr 172.21.15.26:0/2888896398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a6f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70103c20 0x7f6a70199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.518+0000 7f6a6fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a701057d0 0x7f6a701a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.520+0000 7f6a6ea00640 1 -- 172.21.15.26:0/2888896398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70103c20 msgr2=0x7f6a70199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.520+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70103c20 0x7f6a70199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.520+0000 7f6a6ea00640 1 -- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a701057d0 msgr2=0x7f6a701a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.520+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a701057d0 0x7f6a701a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.520+0000 7f6a6ea00640 1 -- 172.21.15.26:0/2888896398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a701a18c0 con 0x7f6a70104e10 2024-06-13T07:22:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.520+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a701057d0 0x7f6a701a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:22:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.521+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 0x7f6a7019a1c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c02a930 tx=0x7f6a5c030870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.522+0000 7f6a67e00640 1 -- 172.21.15.26:0/2888896398 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c037040 con 0x7f6a70104e10 2024-06-13T07:22:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.522+0000 7f6a67e00640 1 -- 172.21.15.26:0/2888896398 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a5c033e50 con 0x7f6a70104e10 2024-06-13T07:22:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.522+0000 7f6a67e00640 1 -- 172.21.15.26:0/2888896398 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0049b0 con 0x7f6a70104e10 2024-06-13T07:22:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.522+0000 7f6a76600640 1 -- 172.21.15.26:0/2888896398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a701a1b50 con 0x7f6a70104e10 2024-06-13T07:22:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.523+0000 7f6a76600640 1 -- 172.21.15.26:0/2888896398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a701a1ff0 con 0x7f6a70104e10 2024-06-13T07:22:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.529+0000 7f6a79565740 1 Processor -- start 2024-06-13T07:22:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.529+0000 7f6a79565740 1 -- start start 2024-06-13T07:22:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.529+0000 7f6a79565740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d49830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a6fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d49830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a6fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d49830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37926/0 (socket says 172.21.15.26:37926) 2024-06-13T07:22:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a79565740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49e00 0x55c9d1d4e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a79565740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d51400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a79565740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c9d1ca4420 con 0x55c9d1d4ef70 2024-06-13T07:22:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a79565740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9d1c4bc50 con 0x55c9d1d49410 2024-06-13T07:22:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a79565740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c9d1ca03e0 con 0x55c9d1d49e00 2024-06-13T07:22:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a67e00640 1 -- 172.21.15.26:0/2888896398 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a5c004b50 con 0x7f6a70104e10 2024-06-13T07:22:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.530+0000 7f6a6fe00640 1 -- 172.21.15.26:0/735260322 learned_addr learned my addr 172.21.15.26:0/735260322 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.531+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d51400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.531+0000 7f6a6fe00640 1 -- 172.21.15.26:0/735260322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49e00 msgr2=0x55c9d1d4e900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:22:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.531+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49e00 0x55c9d1d4e900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.531+0000 7f6a6fe00640 1 -- 172.21.15.26:0/735260322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d4ef70 msgr2=0x55c9d1d51400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.531+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d51400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.531+0000 7f6a6fe00640 1 -- 172.21.15.26:0/735260322 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d19b8210 con 0x55c9d1d49410 2024-06-13T07:22:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.531+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:22:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.532+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d49830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6a60010480 tx=0x7f6a60010940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7395762064a885b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.532+0000 7f6a64c00640 1 -- 172.21.15.26:0/735260322 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60011760 con 0x55c9d1d49410 2024-06-13T07:22:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.532+0000 7f6a64c00640 1 -- 172.21.15.26:0/735260322 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a60011de0 con 0x55c9d1d49410 2024-06-13T07:22:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.532+0000 7f6a64c00640 1 -- 172.21.15.26:0/735260322 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6000e430 con 0x55c9d1d49410 2024-06-13T07:22:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.533+0000 7f6a67e00640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a540a2da0 0x7f6a540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.533+0000 7f6a67e00640 1 -- 172.21.15.26:0/2888896398 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6a5c0ef230 con 0x7f6a70104e10 2024-06-13T07:22:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.536+0000 7f6a79565740 1 -- 172.21.15.26:0/735260322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 msgr2=0x55c9d1d49830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.536+0000 7f6a79565740 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d49830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6a60010480 tx=0x7f6a60010940 comp rx=0 tx=0).stop 2024-06-13T07:22:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.536+0000 7f6a79565740 1 -- 172.21.15.26:0/735260322 shutdown_connections 2024-06-13T07:22:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.536+0000 7f6a79565740 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.536+0000 7f6a79565740 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49e00 0x55c9d1d4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.536+0000 7f6a79565740 1 --2- 172.21.15.26:0/735260322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d49830 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.536+0000 7f6a79565740 1 -- 172.21.15.26:0/735260322 >> 172.21.15.26:0/735260322 conn(0x55c9d1d08fb0 msgr2=0x55c9d1d3c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.537+0000 7f6a79565740 1 -- 172.21.15.26:0/735260322 shutdown_connections 2024-06-13T07:22:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.537+0000 7f6a6f400640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a540a2da0 0x7f6a540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.538+0000 7f6a79565740 1 -- 172.21.15.26:0/735260322 wait complete. 2024-06-13T07:22:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.538+0000 7f6a79565740 1 -- start start 2024-06-13T07:22:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.538+0000 7f6a6f400640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a540a2da0 0x7f6a540a5260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c9d1d4a290 tx=0x7f6a680096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.539+0000 7f6a79565740 1 Processor -- start 2024-06-13T07:22:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.541+0000 7f6a79565740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49410 0x55c9d1d538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.541+0000 7f6a79565740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d53de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.541+0000 7f6a6f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49410 0x55c9d1d538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.541+0000 7f6a6f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49410 0x55c9d1d538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52382/0 (socket says 172.21.15.26:52382) 2024-06-13T07:22:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.541+0000 7f6a6f400640 1 -- 172.21.15.26:0/2031477995 learned_addr learned my addr 172.21.15.26:0/2031477995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.541+0000 7f6a79565740 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 0x55c9d1d5b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d53de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c9d1d51f20 con 0x55c9d1d58e10 2024-06-13T07:22:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9d1c5f0b0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c9d1d520a0 con 0x55c9d1d49410 2024-06-13T07:22:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a79565740 10 client.0 _ll_register_callbacks cb 0x55c9d1c4af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a6ea00640 1 -- 172.21.15.26:0/2031477995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49410 msgr2=0x55c9d1d538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49410 0x55c9d1d538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a6ea00640 1 -- 172.21.15.26:0/2031477995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 msgr2=0x55c9d1d5b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.542+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 0x55c9d1d5b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.543+0000 7f6a6ea00640 1 -- 172.21.15.26:0/2031477995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d1a84620 con 0x55c9d1d4ef70 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.543+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d53de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0ec990 tx=0x7f6a5c0bb270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765d1907daba15f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.543+0000 7f6a59600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.543+0000 7f6a59600640 1 -- 172.21.15.26:0/2031477995 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c004250 con 0x55c9d1d4ef70 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.543+0000 7f6a59600640 1 -- 172.21.15.26:0/2031477995 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6a5c0b7dc0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.543+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a60011de0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.544+0000 7f6a59600640 1 -- 172.21.15.26:0/2031477995 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6a5c0b7a10 con 0x55c9d1d4ef70 2024-06-13T07:22:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.544+0000 7f6a79565740 10 client.27074 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.544+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c9d1d5e4e0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.544+0000 7f6a59600640 1 -- 172.21.15.26:0/2031477995 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a60011de0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.544+0000 7f6a59600640 1 -- 172.21.15.26:0/2031477995 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f6a5c03a070 con 0x55c9d1d4ef70 2024-06-13T07:22:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.545+0000 7f6a79565740 10 client.27074 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.545+0000 7f6a79565740 10 client.27074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.545+0000 7f6a79565740 20 client.27074 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.545+0000 7f6a79565740 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c9d1d5fc40 0x55c9d1d620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.545+0000 7f6a79565740 4 client.27074 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:22:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.545+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9d1cb1b50 con 0x55c9d1d5fc40 2024-06-13T07:22:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.547+0000 7f6a6f400640 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c9d1d5fc40 0x55c9d1d620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.549+0000 7f6a6f400640 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c9d1d5fc40 0x55c9d1d620a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886de8332d998f3a server_cookie=93913f06fb3f2fef in_seq=0 out_seq=0 2024-06-13T07:22:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.549+0000 7f6a59600640 10 client.27074 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.553+0000 7f6a59600640 1 -- 172.21.15.26:0/2031477995 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c9d1cb1b50 con 0x55c9d1d5fc40 2024-06-13T07:22:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.553+0000 7f6a59600640 10 client.27074 handle_command_reply: tid=0 2024-06-13T07:22:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.553+0000 7f6a79565740 1 client.27074 shutdown 2024-06-13T07:22:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 msgr2=0x55c9d1d53de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d53de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0ec990 tx=0x7f6a5c0bb270 comp rx=0 tx=0).stop 2024-06-13T07:22:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 shutdown_connections 2024-06-13T07:22:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c9d1d5fc40 0x55c9d1d620a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d53de0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 0x55c9d1d5b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 --2- 172.21.15.26:0/2031477995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d49410 0x55c9d1d538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 >> 172.21.15.26:0/2031477995 conn(0x55c9d1d08fb0 msgr2=0x55c9d1d3b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a59600640 0 client.27074 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 shutdown_connections 2024-06-13T07:22:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.554+0000 7f6a79565740 1 -- 172.21.15.26:0/2031477995 wait complete. 2024-06-13T07:22:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.555+0000 7f6a79565740 20 client.27074 trim_cache size 0 max 16384 2024-06-13T07:22:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.585+0000 7f6a79565740 1 Processor -- start 2024-06-13T07:22:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.585+0000 7f6a79565740 1 -- start start 2024-06-13T07:22:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.586+0000 7f6a79565740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d49410 0x55c9d1d524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.586+0000 7f6a79565740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d3bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.586+0000 7f6a79565740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d58e10 0x55c9d1d3c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.586+0000 7f6a79565740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701aabc0 con 0x55c9d1d49410 2024-06-13T07:22:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.586+0000 7f6a79565740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701a8490 con 0x55c9d1d4ef70 2024-06-13T07:22:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.586+0000 7f6a79565740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701a62e0 con 0x55c9d1d58e10 2024-06-13T07:22:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.586+0000 7f6a6ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d3bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.587+0000 7f6a6ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d3bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37946/0 (socket says 172.21.15.26:37946) 2024-06-13T07:22:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.587+0000 7f6a6ea00640 1 -- 172.21.15.26:0/1406296521 learned_addr learned my addr 172.21.15.26:0/1406296521 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.587+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d58e10 0x55c9d1d3c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.587+0000 7f6a6f400640 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d49410 0x55c9d1d524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.587+0000 7f6a6fe00640 1 -- 172.21.15.26:0/1406296521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 msgr2=0x55c9d1d3bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.587+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d3bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.588+0000 7f6a6fe00640 1 -- 172.21.15.26:0/1406296521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d49410 msgr2=0x55c9d1d524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.588+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d49410 0x55c9d1d524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.588+0000 7f6a6fe00640 1 -- 172.21.15.26:0/1406296521 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d1d497f0 con 0x55c9d1d58e10 2024-06-13T07:22:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.589+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d58e10 0x55c9d1d3c4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55c9d1d4a660 tx=0x7f6a60005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c57efbba11884cd6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.590+0000 7f6a5be00640 1 -- 172.21.15.26:0/1406296521 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60005220 con 0x55c9d1d58e10 2024-06-13T07:22:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.591+0000 7f6a5be00640 1 -- 172.21.15.26:0/1406296521 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a600053c0 con 0x55c9d1d58e10 2024-06-13T07:22:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.591+0000 7f6a5be00640 1 -- 172.21.15.26:0/1406296521 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60004c00 con 0x55c9d1d58e10 2024-06-13T07:22:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.592+0000 7f6a79565740 1 -- 172.21.15.26:0/1406296521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d58e10 msgr2=0x55c9d1d3c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.592+0000 7f6a79565740 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d58e10 0x55c9d1d3c4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55c9d1d4a660 tx=0x7f6a60005740 comp rx=0 tx=0).stop 2024-06-13T07:22:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.592+0000 7f6a79565740 1 -- 172.21.15.26:0/1406296521 shutdown_connections 2024-06-13T07:22:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.593+0000 7f6a79565740 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d3bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.593+0000 7f6a79565740 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d58e10 0x55c9d1d3c4b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.593+0000 7f6a79565740 1 --2- 172.21.15.26:0/1406296521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d49410 0x55c9d1d524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.593+0000 7f6a79565740 1 -- 172.21.15.26:0/1406296521 >> 172.21.15.26:0/1406296521 conn(0x55c9d1d08fb0 msgr2=0x55c9d1d4b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.593+0000 7f6a79565740 1 -- 172.21.15.26:0/1406296521 shutdown_connections 2024-06-13T07:22:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.593+0000 7f6a79565740 1 -- 172.21.15.26:0/1406296521 wait complete. 2024-06-13T07:22:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.594+0000 7f6a79565740 1 -- start start 2024-06-13T07:22:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.594+0000 7f6a79565740 1 Processor -- start 2024-06-13T07:22:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.595+0000 7f6a79565740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d70ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.595+0000 7f6a79565740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.595+0000 7f6a79565740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 0x55c9d1dc87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a6f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d70ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a6f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d70ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:37956/0 (socket says 172.21.15.26:37956) 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a79565740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701a6950 con 0x55c9d1d58e10 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a79565740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701a7dc0 con 0x55c9d1d49410 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a79565740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701a7020 con 0x55c9d1d4ef70 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a79565740 10 client.0 _ll_register_callbacks cb 0x7f6a5c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a6ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a6ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52398/0 (socket says 172.21.15.26:52398) 2024-06-13T07:22:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.596+0000 7f6a6ea00640 1 -- 172.21.15.26:0/635453500 learned_addr learned my addr 172.21.15.26:0/635453500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.597+0000 7f6a6fe00640 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 0x55c9d1dc87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.597+0000 7f6a6ea00640 1 -- 172.21.15.26:0/635453500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 msgr2=0x55c9d1d70ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.597+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d70ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.597+0000 7f6a6ea00640 1 -- 172.21.15.26:0/635453500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 msgr2=0x55c9d1dc87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.597+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 0x55c9d1dc87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.597+0000 7f6a6ea00640 1 -- 172.21.15.26:0/635453500 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d1d717f0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.598+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d711e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c02a900 tx=0x7f6a5c004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ff0f898cae92e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.598+0000 7f6a59600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:22:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.598+0000 7f6a59600640 1 -- 172.21.15.26:0/635453500 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0c4080 con 0x55c9d1d4ef70 2024-06-13T07:22:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.598+0000 7f6a79565740 1 -- 172.21.15.26:0/635453500 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a600053c0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.599+0000 7f6a59600640 1 -- 172.21.15.26:0/635453500 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6a5c0cfbc0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.599+0000 7f6a59600640 1 -- 172.21.15.26:0/635453500 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6a5c004250 con 0x55c9d1d4ef70 2024-06-13T07:22:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.599+0000 7f6a79565740 10 client.27136 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.599+0000 7f6a79565740 1 -- 172.21.15.26:0/635453500 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c9d1dc9d90 con 0x55c9d1d4ef70 2024-06-13T07:22:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.599+0000 7f6a59600640 1 -- 172.21.15.26:0/635453500 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a600053c0 con 0x55c9d1d4ef70 2024-06-13T07:22:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.600+0000 7f6a59600640 1 -- 172.21.15.26:0/635453500 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7f6a5c03a070 con 0x55c9d1d4ef70 2024-06-13T07:22:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.600+0000 7f6a79565740 10 client.27136 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.600+0000 7f6a79565740 10 client.27136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.600+0000 7f6a79565740 20 client.27136 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.600+0000 7f6a79565740 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c9d1dca540 0x55c9d1dcc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.600+0000 7f6a79565740 4 client.27136 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:22:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.600+0000 7f6a79565740 1 -- 172.21.15.26:0/635453500 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9d1cabea0 con 0x55c9d1dca540 2024-06-13T07:22:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.601+0000 7f6a6f400640 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c9d1dca540 0x55c9d1dcc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.602+0000 7f6a6f400640 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c9d1dca540 0x55c9d1dcc960 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c22cb40902c5fd server_cookie=1c7bae1c05d906f6 in_seq=0 out_seq=0 2024-06-13T07:22:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:45.602+0000 7f6a59600640 10 client.27136 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:46.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 266466 26.06 MB/sec execute 419 sec latency 395.848 ms 2024-06-13T07:22:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:46 smithi135 ceph-mon[36276]: pgmap v1177: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-06-13T07:22:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.538+0000 7f6a59600640 1 -- 172.21.15.26:0/635453500 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c9d1cabea0 con 0x55c9d1dca540 2024-06-13T07:22:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.538+0000 7f6a59600640 10 client.27136 handle_command_reply: tid=0 2024-06-13T07:22:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.539+0000 7f6a79565740 1 client.27136 shutdown 2024-06-13T07:22:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.540+0000 7f6a79565740 1 -- 172.21.15.26:0/635453500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d4ef70 msgr2=0x55c9d1d711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.540+0000 7f6a79565740 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d711e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c02a900 tx=0x7f6a5c004f70 comp rx=0 tx=0).stop 2024-06-13T07:22:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.540+0000 7f6a79565740 1 -- 172.21.15.26:0/635453500 shutdown_connections 2024-06-13T07:22:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.540+0000 7f6a79565740 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55c9d1dca540 0x55c9d1dcc960 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.540+0000 7f6a79565740 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9d1d58e10 0x55c9d1dc87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.540+0000 7f6a79565740 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9d1d4ef70 0x55c9d1d711e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.541+0000 7f6a79565740 1 --2- 172.21.15.26:0/635453500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d1d49410 0x55c9d1d70ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.541+0000 7f6a79565740 1 -- 172.21.15.26:0/635453500 >> 172.21.15.26:0/635453500 conn(0x55c9d1d08fb0 msgr2=0x55c9d1d3f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.540+0000 7f6a59600640 0 client.27136 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.541+0000 7f6a79565740 1 -- 172.21.15.26:0/635453500 shutdown_connections 2024-06-13T07:22:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.541+0000 7f6a79565740 1 -- 172.21.15.26:0/635453500 wait complete. 2024-06-13T07:22:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.542+0000 7f6a79565740 20 client.27136 trim_cache size 0 max 16384 2024-06-13T07:22:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:22:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:22:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:22:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.543+0000 7f6a76600640 1 -- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a540a2da0 msgr2=0x7f6a540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.543+0000 7f6a76600640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a540a2da0 0x7f6a540a5260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c9d1d4a290 tx=0x7f6a680096a0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.543+0000 7f6a76600640 1 -- 172.21.15.26:0/2888896398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 msgr2=0x7f6a7019a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.543+0000 7f6a76600640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 0x7f6a7019a1c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c02a930 tx=0x7f6a5c030870 comp rx=0 tx=0).stop 2024-06-13T07:22:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.543+0000 7f6a76600640 1 -- 172.21.15.26:0/2888896398 shutdown_connections 2024-06-13T07:22:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.543+0000 7f6a76600640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6a540a2da0 0x7f6a540a5260 secure :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c9d1d4a290 tx=0x7f6a680096a0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.544+0000 7f6a76600640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a70104e10 0x7f6a7019a1c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.544+0000 7f6a76600640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a701057d0 0x7f6a701a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.544+0000 7f6a76600640 1 --2- 172.21.15.26:0/2888896398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70103c20 0x7f6a70199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.544+0000 7f6a76600640 1 -- 172.21.15.26:0/2888896398 >> 172.21.15.26:0/2888896398 conn(0x7f6a700ff190 msgr2=0x7f6a70108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.544+0000 7f6a76600640 1 -- 172.21.15.26:0/2888896398 shutdown_connections 2024-06-13T07:22:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:46.545+0000 7f6a76600640 1 -- 172.21.15.26:0/2888896398 wait complete. 2024-06-13T07:22:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:46 smithi136 ceph-mon[37229]: pgmap v1177: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-06-13T07:22:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:46 smithi026 ceph-mon[32041]: pgmap v1177: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-06-13T07:22:47.227 INFO:tasks.workunit.client.0.smithi026.stdout: 10 266761 26.02 MB/sec execute 420 sec latency 458.274 ms 2024-06-13T07:22:48.225 INFO:tasks.workunit.client.0.smithi026.stdout: 10 267412 26.07 MB/sec execute 421 sec latency 62.034 ms 2024-06-13T07:22:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:48 smithi135 ceph-mon[36276]: pgmap v1178: 129 pgs: 129 active+clean; 693 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 395 op/s 2024-06-13T07:22:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:48 smithi136 ceph-mon[37229]: pgmap v1178: 129 pgs: 129 active+clean; 693 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 395 op/s 2024-06-13T07:22:48.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:48 smithi026 ceph-mon[32041]: pgmap v1178: 129 pgs: 129 active+clean; 693 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 395 op/s 2024-06-13T07:22:49.228 INFO:tasks.workunit.client.0.smithi026.stdout: 10 268196 26.06 MB/sec execute 422 sec latency 127.453 ms 2024-06-13T07:22:50.228 INFO:tasks.workunit.client.0.smithi026.stdout: 10 268216 26.01 MB/sec execute 423 sec latency 1084.329 ms 2024-06-13T07:22:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:50 smithi135 ceph-mon[36276]: pgmap v1179: 129 pgs: 129 active+clean; 685 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 563 op/s 2024-06-13T07:22:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:50 smithi136 ceph-mon[37229]: pgmap v1179: 129 pgs: 129 active+clean; 685 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 563 op/s 2024-06-13T07:22:50.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:50 smithi026 ceph-mon[32041]: pgmap v1179: 129 pgs: 129 active+clean; 685 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 563 op/s 2024-06-13T07:22:51.228 INFO:tasks.workunit.client.0.smithi026.stdout: 10 268413 25.98 MB/sec execute 424 sec latency 1522.350 ms 2024-06-13T07:22:52.228 INFO:tasks.workunit.client.0.smithi026.stdout: 10 268741 25.98 MB/sec execute 425 sec latency 115.899 ms 2024-06-13T07:22:52.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:52 smithi136 ceph-mon[37229]: pgmap v1180: 129 pgs: 129 active+clean; 692 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 548 op/s 2024-06-13T07:22:52.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:52 smithi026 ceph-mon[32041]: pgmap v1180: 129 pgs: 129 active+clean; 692 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 548 op/s 2024-06-13T07:22:52.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:52 smithi135 ceph-mon[36276]: pgmap v1180: 129 pgs: 129 active+clean; 692 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 548 op/s 2024-06-13T07:22:53.229 INFO:tasks.workunit.client.0.smithi026.stdout: 10 269067 25.98 MB/sec execute 426 sec latency 160.476 ms 2024-06-13T07:22:54.229 INFO:tasks.workunit.client.0.smithi026.stdout: 10 269466 25.97 MB/sec execute 427 sec latency 158.628 ms 2024-06-13T07:22:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:54 smithi136 ceph-mon[37229]: pgmap v1181: 129 pgs: 129 active+clean; 700 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 655 op/s 2024-06-13T07:22:54.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:54 smithi135 ceph-mon[36276]: pgmap v1181: 129 pgs: 129 active+clean; 700 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 655 op/s 2024-06-13T07:22:55.229 INFO:tasks.workunit.client.0.smithi026.stdout: 10 269809 25.94 MB/sec execute 428 sec latency 68.964 ms 2024-06-13T07:22:56.229 INFO:tasks.workunit.client.0.smithi026.stdout: 10 270203 25.92 MB/sec execute 429 sec latency 91.038 ms 2024-06-13T07:22:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:56 smithi136 ceph-mon[37229]: pgmap v1182: 129 pgs: 129 active+clean; 695 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 677 op/s 2024-06-13T07:22:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:54 smithi026 ceph-mon[32041]: pgmap v1181: 129 pgs: 129 active+clean; 700 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 655 op/s 2024-06-13T07:22:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:56 smithi026 ceph-mon[32041]: pgmap v1182: 129 pgs: 129 active+clean; 695 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 677 op/s 2024-06-13T07:22:56.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:56 smithi135 ceph-mon[36276]: pgmap v1182: 129 pgs: 129 active+clean; 695 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 677 op/s 2024-06-13T07:22:57.229 INFO:tasks.workunit.client.0.smithi026.stdout: 10 270763 25.91 MB/sec execute 430 sec latency 105.941 ms 2024-06-13T07:22:58.229 INFO:tasks.workunit.client.0.smithi026.stdout: 10 271476 25.94 MB/sec execute 431 sec latency 130.639 ms 2024-06-13T07:22:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:58 smithi136 ceph-mon[37229]: pgmap v1183: 129 pgs: 129 active+clean; 684 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 635 op/s 2024-06-13T07:22:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:58 smithi026 ceph-mon[32041]: pgmap v1183: 129 pgs: 129 active+clean; 684 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 635 op/s 2024-06-13T07:22:58.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:58 smithi135 ceph-mon[36276]: pgmap v1183: 129 pgs: 129 active+clean; 684 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 635 op/s 2024-06-13T07:22:59.229 INFO:tasks.workunit.client.0.smithi026.stdout: 10 272129 25.97 MB/sec execute 432 sec latency 309.320 ms 2024-06-13T07:22:59.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:22:59 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:22:59.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:22:59 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:22:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 -- 172.21.15.26:0/2117839073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd664103c20 msgr2=0x7fd664104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 --2- 172.21.15.26:0/2117839073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd664103c20 0x7fd664104040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd654009860 tx=0x7fd65402a340 comp rx=0 tx=0).stop 2024-06-13T07:22:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 -- 172.21.15.26:0/2117839073 shutdown_connections 2024-06-13T07:22:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 --2- 172.21.15.26:0/2117839073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd664104e10 0x7fd664105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 --2- 172.21.15.26:0/2117839073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6641057d0 0x7fd66410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 --2- 172.21.15.26:0/2117839073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd664103c20 0x7fd664104040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 -- 172.21.15.26:0/2117839073 >> 172.21.15.26:0/2117839073 conn(0x7fd6640ff190 msgr2=0x7fd6641015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 -- 172.21.15.26:0/2117839073 shutdown_connections 2024-06-13T07:22:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.766+0000 7fd66a200640 1 -- 172.21.15.26:0/2117839073 wait complete. 2024-06-13T07:22:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.768+0000 7fd66a200640 1 Processor -- start 2024-06-13T07:22:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.768+0000 7fd66a200640 1 -- start start 2024-06-13T07:22:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.769+0000 7fd66a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd664103c20 0x7fd664070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.769+0000 7fd66a200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd664104e10 0x7fd664070850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.769+0000 7fd66a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6641057d0 0x7fd6641a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.769+0000 7fd662a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd664104e10 0x7fd664070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.769+0000 7fd662a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd664104e10 0x7fd664070850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49446/0 (socket says 172.21.15.26:49446) 2024-06-13T07:22:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.770+0000 7fd66a200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410f870 con 0x7fd664103c20 2024-06-13T07:22:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.770+0000 7fd66a200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410f6f0 con 0x7fd664104e10 2024-06-13T07:22:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.770+0000 7fd66a200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410f9f0 con 0x7fd6641057d0 2024-06-13T07:22:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.770+0000 7fd662a00640 1 -- 172.21.15.26:0/3961204376 learned_addr learned my addr 172.21.15.26:0/3961204376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.770+0000 7fd663e00640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6641057d0 0x7fd6641a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.771+0000 7fd663400640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd664103c20 0x7fd664070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.771+0000 7fd663e00640 1 -- 172.21.15.26:0/3961204376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd664104e10 msgr2=0x7fd664070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.771+0000 7fd663e00640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd664104e10 0x7fd664070850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.771+0000 7fd663e00640 1 -- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd664103c20 msgr2=0x7fd664070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.772+0000 7fd663e00640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd664103c20 0x7fd664070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.772+0000 7fd663e00640 1 -- 172.21.15.26:0/3961204376 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6641aa4c0 con 0x7fd6641057d0 2024-06-13T07:22:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.772+0000 7fd663400640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd664103c20 0x7fd664070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:22:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.773+0000 7fd663e00640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6641057d0 0x7fd6641a9f80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009fd0 tx=0x7fd65c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.773+0000 7fd65be00640 1 -- 172.21.15.26:0/3961204376 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c00ce20 con 0x7fd6641057d0 2024-06-13T07:22:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.773+0000 7fd65be00640 1 -- 172.21.15.26:0/3961204376 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65c005020 con 0x7fd6641057d0 2024-06-13T07:22:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.773+0000 7fd65be00640 1 -- 172.21.15.26:0/3961204376 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c010720 con 0x7fd6641057d0 2024-06-13T07:22:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.774+0000 7fd66a200640 1 -- 172.21.15.26:0/3961204376 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6641aa690 con 0x7fd6641057d0 2024-06-13T07:22:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.774+0000 7fd66a200640 1 -- 172.21.15.26:0/3961204376 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6641aab00 con 0x7fd6641057d0 2024-06-13T07:22:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.778+0000 7fd65be00640 1 -- 172.21.15.26:0/3961204376 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd65c005540 con 0x7fd6641057d0 2024-06-13T07:22:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.779+0000 7fd66d1b8740 1 Processor -- start 2024-06-13T07:22:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.779+0000 7fd66d1b8740 1 -- start start 2024-06-13T07:22:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.779+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009410 0x5587bb009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb00e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd662a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009410 0x5587bb009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb00ef70 0x5587bb011400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5587baf64420 con 0x5587bb00ef70 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5587baf0bc50 con 0x5587bb009410 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5587baf603e0 con 0x5587bb009e00 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd663400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb00e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd663400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb00e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53720/0 (socket says 172.21.15.26:53720) 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd663400640 1 -- 172.21.15.26:0/245914965 learned_addr learned my addr 172.21.15.26:0/245914965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd663400640 1 -- 172.21.15.26:0/245914965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009410 msgr2=0x5587bb009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd663400640 1 --2- 172.21.15.26:0/245914965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009410 0x5587bb009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd663400640 1 -- 172.21.15.26:0/245914965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb00ef70 msgr2=0x5587bb011400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd663400640 1 --2- 172.21.15.26:0/245914965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb00ef70 0x5587bb011400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.780+0000 7fd663400640 1 -- 172.21.15.26:0/245914965 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bac78210 con 0x5587bb009e00 2024-06-13T07:22:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.781+0000 7fd663400640 1 --2- 172.21.15.26:0/245914965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb00e900 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd65402a930 tx=0x7fd654005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f1375a890d6f663 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.781+0000 7fd65be00640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd6480a2cb0 0x7fd6480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.781+0000 7fd64fe00640 1 -- 172.21.15.26:0/245914965 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654037040 con 0x5587bb009e00 2024-06-13T07:22:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.782+0000 7fd64fe00640 1 -- 172.21.15.26:0/245914965 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd654042070 con 0x5587bb009e00 2024-06-13T07:22:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.782+0000 7fd64fe00640 1 -- 172.21.15.26:0/245914965 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6540046c0 con 0x5587bb009e00 2024-06-13T07:22:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.782+0000 7fd65be00640 1 -- 172.21.15.26:0/3961204376 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd65c0cfec0 con 0x7fd6641057d0 2024-06-13T07:22:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd662a00640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd6480a2cb0 0x7fd6480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd66d1b8740 1 -- 172.21.15.26:0/245914965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 msgr2=0x5587bb00e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/245914965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb00e900 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd65402a930 tx=0x7fd654005c90 comp rx=0 tx=0).stop 2024-06-13T07:22:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd66d1b8740 1 -- 172.21.15.26:0/245914965 shutdown_connections 2024-06-13T07:22:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/245914965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb00ef70 0x5587bb011400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/245914965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb00e900 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/245914965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009410 0x5587bb009830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd66d1b8740 1 -- 172.21.15.26:0/245914965 >> 172.21.15.26:0/245914965 conn(0x5587bafc8fb0 msgr2=0x5587bb004e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:59.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:22:59 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:22:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.783+0000 7fd66d1b8740 1 -- 172.21.15.26:0/245914965 shutdown_connections 2024-06-13T07:22:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.784+0000 7fd66d1b8740 1 -- 172.21.15.26:0/245914965 wait complete. 2024-06-13T07:22:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.785+0000 7fd66d1b8740 1 -- start start 2024-06-13T07:22:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.785+0000 7fd66d1b8740 1 Processor -- start 2024-06-13T07:22:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.786+0000 7fd662a00640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd6480a2cb0 0x7fd6480a5170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd6500098b0 tx=0x7fd650009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb0136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb013c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb01ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5587bb011ec0 con 0x5587bb009410 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd662a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb013c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd662a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb013c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53732/0 (socket says 172.21.15.26:53732) 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd663400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb0136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd662a00640 1 -- 172.21.15.26:0/1252796238 learned_addr learned my addr 172.21.15.26:0/1252796238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd663e00640 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb01ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5587baf1f0b0 con 0x5587bb00ef70 2024-06-13T07:22:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.787+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5587bb012040 con 0x5587bb009e00 2024-06-13T07:22:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.788+0000 7fd66d1b8740 10 client.0 _ll_register_callbacks cb 0x5587baf0af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.788+0000 7fd663e00640 1 -- 172.21.15.26:0/1252796238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 msgr2=0x5587bb013c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.788+0000 7fd663e00640 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb013c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.788+0000 7fd663e00640 1 -- 172.21.15.26:0/1252796238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 msgr2=0x5587bb0136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.788+0000 7fd663e00640 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb0136c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.788+0000 7fd663e00640 1 -- 172.21.15.26:0/1252796238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bad44620 con 0x5587bb00ef70 2024-06-13T07:22:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.788+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd654042070 con 0x5587bb00ef70 2024-06-13T07:22:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.789+0000 7fd663e00640 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb01ac40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd65c01f000 tx=0x7fd65c09c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71adadc93c241599 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.789+0000 7fd64cc00640 10 client.27092 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:22:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.789+0000 7fd64cc00640 1 -- 172.21.15.26:0/1252796238 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0051d0 con 0x5587bb00ef70 2024-06-13T07:22:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.789+0000 7fd64cc00640 1 -- 172.21.15.26:0/1252796238 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd65c0a0170 con 0x5587bb00ef70 2024-06-13T07:22:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.790+0000 7fd64cc00640 1 -- 172.21.15.26:0/1252796238 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd65c0a0370 con 0x5587bb00ef70 2024-06-13T07:22:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.790+0000 7fd66d1b8740 10 client.27092 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.790+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=47}) -- 0x5587bb01dfc0 con 0x5587bb00ef70 2024-06-13T07:22:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.790+0000 7fd64cc00640 1 -- 172.21.15.26:0/1252796238 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fd65c020070 con 0x5587bb00ef70 2024-06-13T07:22:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.790+0000 7fd66d1b8740 10 client.27092 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.790+0000 7fd66d1b8740 10 client.27092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.790+0000 7fd66d1b8740 20 client.27092 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.791+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5587bb01f580 0x5587bb021970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.791+0000 7fd66d1b8740 4 client.27092 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:22:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.791+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587baf71b50 con 0x5587bb01f580 2024-06-13T07:22:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.791+0000 7fd663400640 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5587bb01f580 0x5587bb021970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.793+0000 7fd663400640 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5587bb01f580 0x5587bb021970 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d0973297ac3aba server_cookie=f9b6ddffc279fe48 in_seq=0 out_seq=0 2024-06-13T07:22:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.793+0000 7fd64cc00640 10 client.27092 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.796+0000 7fd64cc00640 1 -- 172.21.15.26:0/1252796238 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5587baf71b50 con 0x5587bb01f580 2024-06-13T07:22:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.796+0000 7fd64cc00640 10 client.27092 handle_command_reply: tid=0 2024-06-13T07:22:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.796+0000 7fd66d1b8740 1 client.27092 shutdown 2024-06-13T07:22:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.796+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 msgr2=0x5587bb01ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.796+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb01ac40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd65c01f000 tx=0x7fd65c09c390 comp rx=0 tx=0).stop 2024-06-13T07:22:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.797+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 shutdown_connections 2024-06-13T07:22:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.797+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5587bb01f580 0x5587bb021970 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.797+0000 7fd64cc00640 0 client.27092 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:22:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.797+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb01ac40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.797+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009e00 0x5587bb013c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.797+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/1252796238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb0136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.797+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 >> 172.21.15.26:0/1252796238 conn(0x5587bafc8fb0 msgr2=0x5587baffb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.798+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 shutdown_connections 2024-06-13T07:22:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.800+0000 7fd66d1b8740 1 -- 172.21.15.26:0/1252796238 wait complete. 2024-06-13T07:22:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.800+0000 7fd66d1b8740 20 client.27092 trim_cache size 0 max 16384 2024-06-13T07:22:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.834+0000 7fd66d1b8740 1 Processor -- start 2024-06-13T07:22:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.834+0000 7fd66d1b8740 1 -- start start 2024-06-13T07:22:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.834+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009410 0x5587bb00b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.834+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009e00 0x5587bb01ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.834+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb00b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641b36f0 con 0x5587bb009e00 2024-06-13T07:22:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641b0fc0 con 0x5587bb00ef70 2024-06-13T07:22:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641aee30 con 0x5587bb009410 2024-06-13T07:22:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd663e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb00b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd663e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb00b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:49466/0 (socket says 172.21.15.26:49466) 2024-06-13T07:22:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd663e00640 1 -- 172.21.15.26:0/3638017651 learned_addr learned my addr 172.21.15.26:0/3638017651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd663e00640 1 -- 172.21.15.26:0/3638017651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009410 msgr2=0x5587bb00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:22:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd663e00640 1 --2- 172.21.15.26:0/3638017651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009410 0x5587bb00b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.835+0000 7fd663e00640 1 -- 172.21.15.26:0/3638017651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009e00 msgr2=0x5587bb01ac00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:22:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.836+0000 7fd662a00640 1 --2- 172.21.15.26:0/3638017651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009e00 0x5587bb01ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.836+0000 7fd663e00640 1 --2- 172.21.15.26:0/3638017651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009e00 0x5587bb01ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.837+0000 7fd663e00640 1 -- 172.21.15.26:0/3638017651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bb00e720 con 0x5587bb00ef70 2024-06-13T07:22:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.837+0000 7fd663e00640 1 --2- 172.21.15.26:0/3638017651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb00b620 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd65c09bfd0 tx=0x7fd65c0bc050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce59841174b5cfb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.837+0000 7fd64f400640 1 -- 172.21.15.26:0/3638017651 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0051d0 con 0x5587bb00ef70 2024-06-13T07:22:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.837+0000 7fd64f400640 1 -- 172.21.15.26:0/3638017651 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65c0ba050 con 0x5587bb00ef70 2024-06-13T07:22:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.837+0000 7fd64f400640 1 -- 172.21.15.26:0/3638017651 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c09b720 con 0x5587bb00ef70 2024-06-13T07:22:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.838+0000 7fd66d1b8740 1 -- 172.21.15.26:0/3638017651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 msgr2=0x5587bb00b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:22:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.838+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/3638017651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb00b620 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd65c09bfd0 tx=0x7fd65c0bc050 comp rx=0 tx=0).stop 2024-06-13T07:22:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.839+0000 7fd66d1b8740 1 -- 172.21.15.26:0/3638017651 shutdown_connections 2024-06-13T07:22:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.839+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/3638017651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb00ef70 0x5587bb00b620 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.839+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/3638017651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009e00 0x5587bb01ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.839+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/3638017651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb009410 0x5587bb00b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.839+0000 7fd66d1b8740 1 -- 172.21.15.26:0/3638017651 >> 172.21.15.26:0/3638017651 conn(0x5587bafc8fb0 msgr2=0x5587bb004910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:22:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.840+0000 7fd66d1b8740 1 -- 172.21.15.26:0/3638017651 shutdown_connections 2024-06-13T07:22:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.840+0000 7fd66d1b8740 1 -- 172.21.15.26:0/3638017651 wait complete. 2024-06-13T07:22:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.840+0000 7fd66d1b8740 1 -- start start 2024-06-13T07:22:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.840+0000 7fd66d1b8740 1 Processor -- start 2024-06-13T07:22:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.841+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb017f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.842+0000 7fd663400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb017f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.843+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009e00 0x5587bb018460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.843+0000 7fd663400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb017f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40122/0 (socket says 172.21.15.26:40122) 2024-06-13T07:22:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.843+0000 7fd66d1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb00ef70 0x5587bb019470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.843+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641af480 con 0x5587bb009410 2024-06-13T07:22:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.843+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641b08f0 con 0x5587bb009e00 2024-06-13T07:22:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.843+0000 7fd66d1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641afb50 con 0x5587bb00ef70 2024-06-13T07:22:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.843+0000 7fd663400640 1 -- 172.21.15.26:0/2245915043 learned_addr learned my addr 172.21.15.26:0/2245915043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:22:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.843+0000 7fd66d1b8740 10 client.0 _ll_register_callbacks cb 0x7fd65c098570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:22:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.844+0000 7fd663e00640 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb00ef70 0x5587bb019470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.844+0000 7fd663400640 1 -- 172.21.15.26:0/2245915043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb00ef70 msgr2=0x5587bb019470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:22:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.844+0000 7fd663400640 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb00ef70 0x5587bb019470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.844+0000 7fd663400640 1 -- 172.21.15.26:0/2245915043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009e00 msgr2=0x5587bb018460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:22:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.844+0000 7fd663400640 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009e00 0x5587bb018460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:22:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.845+0000 7fd663400640 1 -- 172.21.15.26:0/2245915043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bb019a60 con 0x5587bb009410 2024-06-13T07:22:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.847+0000 7fd663400640 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb017f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd65402a860 tx=0x7fd65402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265a22203584e7e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:22:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.847+0000 7fd64cc00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:22:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.848+0000 7fd64cc00640 1 -- 172.21.15.26:0/2245915043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654057060 con 0x5587bb009410 2024-06-13T07:22:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.849+0000 7fd64cc00640 1 -- 172.21.15.26:0/2245915043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd654030890 con 0x5587bb009410 2024-06-13T07:22:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.849+0000 7fd66d1b8740 1 -- 172.21.15.26:0/2245915043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd65c0ba050 con 0x5587bb009410 2024-06-13T07:22:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.850+0000 7fd64cc00640 1 -- 172.21.15.26:0/2245915043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd65405e050 con 0x5587bb009410 2024-06-13T07:22:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.850+0000 7fd66d1b8740 10 client.16815 fetch_fsmap learned FSMap version 47 2024-06-13T07:22:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.850+0000 7fd66d1b8740 1 -- 172.21.15.26:0/2245915043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5587bb01ade0 con 0x5587bb009410 2024-06-13T07:22:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.851+0000 7fd64cc00640 1 -- 172.21.15.26:0/2245915043 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65c0ba050 con 0x5587bb009410 2024-06-13T07:22:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.851+0000 7fd64cc00640 1 -- 172.21.15.26:0/2245915043 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8346+0+0 (secure 0 0 0) 0x7fd654050070 con 0x5587bb009410 2024-06-13T07:22:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.851+0000 7fd66d1b8740 10 client.16815 fetch_fsmap finished waiting for FSMap version 47 2024-06-13T07:22:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.851+0000 7fd66d1b8740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:22:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.851+0000 7fd66d1b8740 20 client.16815 populate_metadata read hostname 'smithi026' 2024-06-13T07:22:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.851+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5587bb01f580 0x5587bb08c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:22:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.851+0000 7fd66d1b8740 4 client.16815 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:22:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.851+0000 7fd66d1b8740 1 -- 172.21.15.26:0/2245915043 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587baf6bea0 con 0x5587bb01f580 2024-06-13T07:22:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.852+0000 7fd662a00640 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5587bb01f580 0x5587bb08c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:22:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.852+0000 7fd662a00640 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5587bb01f580 0x5587bb08c420 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d3803997c3e3d0 server_cookie=4bcbdfd33c790056 in_seq=0 out_seq=0 2024-06-13T07:22:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:22:59.852+0000 7fd64cc00640 10 client.16815 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:00.232 INFO:tasks.workunit.client.0.smithi026.stdout: 10 272457 25.96 MB/sec execute 433 sec latency 305.113 ms 2024-06-13T07:23:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:00 smithi136 ceph-mon[37229]: pgmap v1184: 129 pgs: 129 active+clean; 668 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.9 MiB/s wr, 689 op/s 2024-06-13T07:23:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.671+0000 7fd64cc00640 1 -- 172.21.15.26:0/2245915043 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5587baf6bea0 con 0x5587bb01f580 2024-06-13T07:23:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.671+0000 7fd64cc00640 10 client.16815 handle_command_reply: tid=0 2024-06-13T07:23:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.671+0000 7fd66d1b8740 1 client.16815 shutdown 2024-06-13T07:23:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.672+0000 7fd66d1b8740 1 -- 172.21.15.26:0/2245915043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 msgr2=0x5587bb017f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.672+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb017f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd65402a860 tx=0x7fd65402a340 comp rx=0 tx=0).stop 2024-06-13T07:23:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.673+0000 7fd66d1b8740 1 -- 172.21.15.26:0/2245915043 shutdown_connections 2024-06-13T07:23:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.673+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5587bb01f580 0x5587bb08c420 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.673+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587bb009e00 0x5587bb018460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.673+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587bb00ef70 0x5587bb019470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.673+0000 7fd66d1b8740 1 --2- 172.21.15.26:0/2245915043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587bb009410 0x5587bb017f20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.673+0000 7fd66d1b8740 1 -- 172.21.15.26:0/2245915043 >> 172.21.15.26:0/2245915043 conn(0x5587bafc8fb0 msgr2=0x5587bb00b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.673+0000 7fd64cc00640 0 client.16815 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.674+0000 7fd66d1b8740 1 -- 172.21.15.26:0/2245915043 shutdown_connections 2024-06-13T07:23:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.674+0000 7fd66d1b8740 1 -- 172.21.15.26:0/2245915043 wait complete. 2024-06-13T07:23:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.674+0000 7fd66d1b8740 20 client.16815 trim_cache size 0 max 16384 2024-06-13T07:23:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:23:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:23:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:23:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.678+0000 7fd66a200640 1 -- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd6480a2cb0 msgr2=0x7fd6480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.678+0000 7fd66a200640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd6480a2cb0 0x7fd6480a5170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd6500098b0 tx=0x7fd650009260 comp rx=0 tx=0).stop 2024-06-13T07:23:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.678+0000 7fd66a200640 1 -- 172.21.15.26:0/3961204376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6641057d0 msgr2=0x7fd6641a9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.678+0000 7fd66a200640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6641057d0 0x7fd6641a9f80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009fd0 tx=0x7fd65c00ef10 comp rx=0 tx=0).stop 2024-06-13T07:23:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.679+0000 7fd66a200640 1 -- 172.21.15.26:0/3961204376 shutdown_connections 2024-06-13T07:23:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.679+0000 7fd66a200640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd6480a2cb0 0x7fd6480a5170 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.679+0000 7fd66a200640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd664104e10 0x7fd664070850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.679+0000 7fd66a200640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6641057d0 0x7fd6641a9f80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.679+0000 7fd66a200640 1 --2- 172.21.15.26:0/3961204376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd664103c20 0x7fd664070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.679+0000 7fd66a200640 1 -- 172.21.15.26:0/3961204376 >> 172.21.15.26:0/3961204376 conn(0x7fd6640ff190 msgr2=0x7fd664108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.679+0000 7fd66a200640 1 -- 172.21.15.26:0/3961204376 shutdown_connections 2024-06-13T07:23:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:00.679+0000 7fd66a200640 1 -- 172.21.15.26:0/3961204376 wait complete. 2024-06-13T07:23:00.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:00 smithi026 ceph-mon[32041]: pgmap v1184: 129 pgs: 129 active+clean; 668 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.9 MiB/s wr, 689 op/s 2024-06-13T07:23:00.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:00 smithi135 ceph-mon[36276]: pgmap v1184: 129 pgs: 129 active+clean; 668 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.9 MiB/s wr, 689 op/s 2024-06-13T07:23:01.233 INFO:tasks.workunit.client.0.smithi026.stdout: 10 272846 25.97 MB/sec execute 434 sec latency 402.288 ms 2024-06-13T07:23:02.233 INFO:tasks.workunit.client.0.smithi026.stdout: 10 273160 25.95 MB/sec execute 435 sec latency 119.929 ms 2024-06-13T07:23:02.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:02 smithi026 ceph-mon[32041]: pgmap v1185: 129 pgs: 129 active+clean; 665 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 535 op/s 2024-06-13T07:23:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:02 smithi136 ceph-mon[37229]: pgmap v1185: 129 pgs: 129 active+clean; 665 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 535 op/s 2024-06-13T07:23:02.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:02 smithi135 ceph-mon[36276]: pgmap v1185: 129 pgs: 129 active+clean; 665 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 535 op/s 2024-06-13T07:23:03.234 INFO:tasks.workunit.client.0.smithi026.stdout: 10 273550 25.93 MB/sec execute 436 sec latency 110.327 ms 2024-06-13T07:23:04.234 INFO:tasks.workunit.client.0.smithi026.stdout: 10 273986 25.91 MB/sec execute 437 sec latency 158.804 ms 2024-06-13T07:23:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:04 smithi136 ceph-mon[37229]: pgmap v1186: 129 pgs: 129 active+clean; 662 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 546 op/s 2024-06-13T07:23:04.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:04 smithi026 ceph-mon[32041]: pgmap v1186: 129 pgs: 129 active+clean; 662 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 546 op/s 2024-06-13T07:23:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:04 smithi135 ceph-mon[36276]: pgmap v1186: 129 pgs: 129 active+clean; 662 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 546 op/s 2024-06-13T07:23:05.234 INFO:tasks.workunit.client.0.smithi026.stdout: 10 274206 25.87 MB/sec execute 438 sec latency 507.243 ms 2024-06-13T07:23:06.234 INFO:tasks.workunit.client.0.smithi026.stdout: 10 274249 25.81 MB/sec execute 439 sec latency 1214.669 ms 2024-06-13T07:23:06.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:06 smithi136 ceph-mon[37229]: pgmap v1187: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-13T07:23:06.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:06 smithi026 ceph-mon[32041]: pgmap v1187: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-13T07:23:06.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:06 smithi135 ceph-mon[36276]: pgmap v1187: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-13T07:23:07.234 INFO:tasks.workunit.client.0.smithi026.stdout: 10 274404 25.77 MB/sec execute 440 sec latency 1320.355 ms 2024-06-13T07:23:08.234 INFO:tasks.workunit.client.0.smithi026.stdout: 10 274603 25.73 MB/sec execute 441 sec latency 434.887 ms 2024-06-13T07:23:08.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:08 smithi026 ceph-mon[32041]: pgmap v1188: 129 pgs: 129 active+clean; 680 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-06-13T07:23:08.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:08 smithi136 ceph-mon[37229]: pgmap v1188: 129 pgs: 129 active+clean; 680 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-06-13T07:23:08.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:08 smithi135 ceph-mon[36276]: pgmap v1188: 129 pgs: 129 active+clean; 680 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-06-13T07:23:09.234 INFO:tasks.workunit.client.0.smithi026.stdout: 10 274837 25.71 MB/sec execute 442 sec latency 230.418 ms 2024-06-13T07:23:10.236 INFO:tasks.workunit.client.0.smithi026.stdout: 10 275129 25.68 MB/sec execute 443 sec latency 145.907 ms 2024-06-13T07:23:10.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:10 smithi136 ceph-mon[37229]: pgmap v1189: 129 pgs: 129 active+clean; 707 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.3 MiB/s wr, 559 op/s 2024-06-13T07:23:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:10 smithi026 ceph-mon[32041]: pgmap v1189: 129 pgs: 129 active+clean; 707 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.3 MiB/s wr, 559 op/s 2024-06-13T07:23:10.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:10 smithi135 ceph-mon[36276]: pgmap v1189: 129 pgs: 129 active+clean; 707 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.3 MiB/s wr, 559 op/s 2024-06-13T07:23:11.236 INFO:tasks.workunit.client.0.smithi026.stdout: 10 275503 25.69 MB/sec execute 444 sec latency 254.146 ms 2024-06-13T07:23:12.238 INFO:tasks.workunit.client.0.smithi026.stdout: 10 276051 25.70 MB/sec execute 445 sec latency 82.253 ms 2024-06-13T07:23:12.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:12 smithi026 ceph-mon[32041]: pgmap v1190: 129 pgs: 129 active+clean; 698 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 505 op/s 2024-06-13T07:23:12.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:12 smithi135 ceph-mon[36276]: pgmap v1190: 129 pgs: 129 active+clean; 698 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 505 op/s 2024-06-13T07:23:12.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:12 smithi136 ceph-mon[37229]: pgmap v1190: 129 pgs: 129 active+clean; 698 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 505 op/s 2024-06-13T07:23:13.238 INFO:tasks.workunit.client.0.smithi026.stdout: 10 276295 25.67 MB/sec execute 446 sec latency 362.409 ms 2024-06-13T07:23:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.924+0000 7fdadf000640 1 -- 172.21.15.26:0/3008470446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad81035b0 msgr2=0x7fdad8109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.924+0000 7fdadf000640 1 --2- 172.21.15.26:0/3008470446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad81035b0 0x7fdad8109e40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009900 tx=0x7fdac802a410 comp rx=0 tx=0).stop 2024-06-13T07:23:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.925+0000 7fdadf000640 1 -- 172.21.15.26:0/3008470446 shutdown_connections 2024-06-13T07:23:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.925+0000 7fdadf000640 1 --2- 172.21.15.26:0/3008470446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad81035b0 0x7fdad8109e40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.925+0000 7fdadf000640 1 --2- 172.21.15.26:0/3008470446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad8102bf0 0x7fdad8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.925+0000 7fdadf000640 1 --2- 172.21.15.26:0/3008470446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad8101a00 0x7fdad8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.925+0000 7fdadf000640 1 -- 172.21.15.26:0/3008470446 >> 172.21.15.26:0/3008470446 conn(0x7fdad80fcf90 msgr2=0x7fdad80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.928+0000 7fdadf000640 1 -- 172.21.15.26:0/3008470446 shutdown_connections 2024-06-13T07:23:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.928+0000 7fdadf000640 1 -- 172.21.15.26:0/3008470446 wait complete. 2024-06-13T07:23:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.930+0000 7fdadf000640 1 Processor -- start 2024-06-13T07:23:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.930+0000 7fdadf000640 1 -- start start 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdadf000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad8101a00 0x7fdad8199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdadf000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad8102bf0 0x7fdad819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdadf000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad81035b0 0x7fdad81a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdadf000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8075910 con 0x7fdad81035b0 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdadf000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8075790 con 0x7fdad8101a00 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdadf000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8075a90 con 0x7fdad8102bf0 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdad7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad8101a00 0x7fdad8199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdad7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad8101a00 0x7fdad8199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:33674/0 (socket says 172.21.15.26:33674) 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdad7e00640 1 -- 172.21.15.26:0/2904271424 learned_addr learned my addr 172.21.15.26:0/2904271424 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.931+0000 7fdad7400640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad8102bf0 0x7fdad819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.932+0000 7fdadd000640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad81035b0 0x7fdad81a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.932+0000 7fdad7400640 1 -- 172.21.15.26:0/2904271424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad8101a00 msgr2=0x7fdad8199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.932+0000 7fdad7400640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad8101a00 0x7fdad8199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.932+0000 7fdad7400640 1 -- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad81035b0 msgr2=0x7fdad81a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.932+0000 7fdad7400640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad81035b0 0x7fdad81a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.932+0000 7fdad7400640 1 -- 172.21.15.26:0/2904271424 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad810b9f0 con 0x7fdad8102bf0 2024-06-13T07:23:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.933+0000 7fdad7400640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad8102bf0 0x7fdad819a390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdac400b3f0 tx=0x7fdac400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.933+0000 7fdad4c00640 1 -- 172.21.15.26:0/2904271424 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4015020 con 0x7fdad8102bf0 2024-06-13T07:23:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.934+0000 7fdad4c00640 1 -- 172.21.15.26:0/2904271424 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4011d10 con 0x7fdad8102bf0 2024-06-13T07:23:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.934+0000 7fdad4c00640 1 -- 172.21.15.26:0/2904271424 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac400f2e0 con 0x7fdad8102bf0 2024-06-13T07:23:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.934+0000 7fdadf000640 1 -- 172.21.15.26:0/2904271424 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad810bce0 con 0x7fdad8102bf0 2024-06-13T07:23:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.934+0000 7fdadf000640 1 -- 172.21.15.26:0/2904271424 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad810c1e0 con 0x7fdad8102bf0 2024-06-13T07:23:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.940+0000 7fdae1f79740 1 Processor -- start 2024-06-13T07:23:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.944+0000 7fdae1f79740 1 -- start start 2024-06-13T07:23:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.945+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.946+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 0x555a789c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.946+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.946+0000 7fdae1f79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555a78920420 con 0x555a789bd5f0 2024-06-13T07:23:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.946+0000 7fdae1f79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555a788c7c50 con 0x555a789c2760 2024-06-13T07:23:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.946+0000 7fdae1f79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a7891c3e0 con 0x555a789bcc00 2024-06-13T07:23:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.946+0000 7fdadd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.946+0000 7fdadd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54606/0 (socket says 172.21.15.26:54606) 2024-06-13T07:23:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.946+0000 7fdadd000640 1 -- 172.21.15.26:0/3184869477 learned_addr learned my addr 172.21.15.26:0/3184869477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.947+0000 7fdad4c00640 1 -- 172.21.15.26:0/2904271424 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdac4018020 con 0x7fdad8102bf0 2024-06-13T07:23:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.947+0000 7fdad7400640 1 --2- 172.21.15.26:0/3184869477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.948+0000 7fdadd000640 1 -- 172.21.15.26:0/3184869477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 msgr2=0x555a789c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.948+0000 7fdadd000640 1 --2- 172.21.15.26:0/3184869477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.948+0000 7fdadd000640 1 -- 172.21.15.26:0/3184869477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 msgr2=0x555a789c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.948+0000 7fdadd000640 1 --2- 172.21.15.26:0/3184869477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 0x555a789c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.948+0000 7fdadd000640 1 -- 172.21.15.26:0/3184869477 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a78634210 con 0x555a789bcc00 2024-06-13T07:23:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.949+0000 7fdad4c00640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdabc0a2d30 0x7fdabc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.949+0000 7fdad4c00640 1 -- 172.21.15.26:0/2904271424 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdac40ceb20 con 0x7fdad8102bf0 2024-06-13T07:23:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.950+0000 7fdadd000640 1 --2- 172.21.15.26:0/3184869477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bd020 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdac8031530 tx=0x7fdac8004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d541fefe6ab67e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.951+0000 7fdacd600640 1 -- 172.21.15.26:0/3184869477 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8035030 con 0x555a789bcc00 2024-06-13T07:23:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.952+0000 7fdacd600640 1 -- 172.21.15.26:0/3184869477 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac8003cd0 con 0x555a789bcc00 2024-06-13T07:23:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.952+0000 7fdacd600640 1 -- 172.21.15.26:0/3184869477 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8031da0 con 0x555a789bcc00 2024-06-13T07:23:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.953+0000 7fdae1f79740 1 -- 172.21.15.26:0/3184869477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 msgr2=0x555a789bd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.953+0000 7fdae1f79740 1 --2- 172.21.15.26:0/3184869477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bd020 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdac8031530 tx=0x7fdac8004310 comp rx=0 tx=0).stop 2024-06-13T07:23:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.955+0000 7fdad7e00640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdabc0a2d30 0x7fdabc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.955+0000 7fdae1f79740 1 -- 172.21.15.26:0/3184869477 shutdown_connections 2024-06-13T07:23:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.955+0000 7fdae1f79740 1 --2- 172.21.15.26:0/3184869477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.955+0000 7fdae1f79740 1 --2- 172.21.15.26:0/3184869477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 0x555a789c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.955+0000 7fdae1f79740 1 --2- 172.21.15.26:0/3184869477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bd020 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.955+0000 7fdae1f79740 1 -- 172.21.15.26:0/3184869477 >> 172.21.15.26:0/3184869477 conn(0x555a78984fb0 msgr2=0x555a789b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.955+0000 7fdae1f79740 1 -- 172.21.15.26:0/3184869477 shutdown_connections 2024-06-13T07:23:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.955+0000 7fdae1f79740 1 -- 172.21.15.26:0/3184869477 wait complete. 2024-06-13T07:23:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.956+0000 7fdae1f79740 1 -- start start 2024-06-13T07:23:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.956+0000 7fdad7e00640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdabc0a2d30 0x7fdabc0a51f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x555a789b9530 tx=0x7fdad0007880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.956+0000 7fdae1f79740 1 Processor -- start 2024-06-13T07:23:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.958+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 0x555a789c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.958+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789bd5f0 0x555a789c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.958+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789c2760 0x555a789ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.958+0000 7fdae1f79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555a789c56b0 con 0x555a789bcc00 2024-06-13T07:23:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.958+0000 7fdae1f79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555a788db0b0 con 0x555a789bd5f0 2024-06-13T07:23:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.958+0000 7fdae1f79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a789c5830 con 0x555a789c2760 2024-06-13T07:23:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.958+0000 7fdae1f79740 10 client.0 _ll_register_callbacks cb 0x555a788c6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:23:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.959+0000 7fdad7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789bd5f0 0x555a789c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.959+0000 7fdad7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789bd5f0 0x555a789c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:33698/0 (socket says 172.21.15.26:33698) 2024-06-13T07:23:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.959+0000 7fdad7400640 1 -- 172.21.15.26:0/2160483279 learned_addr learned my addr 172.21.15.26:0/2160483279 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.960+0000 7fdad7400640 1 -- 172.21.15.26:0/2160483279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789c2760 msgr2=0x555a789ce490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:23:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.960+0000 7fdad7400640 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789c2760 0x555a789ce490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.960+0000 7fdad7400640 1 -- 172.21.15.26:0/2160483279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 msgr2=0x555a789c6f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:23:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.960+0000 7fdad7400640 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 0x555a789c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.960+0000 7fdad7400640 1 -- 172.21.15.26:0/2160483279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a78700620 con 0x555a789bd5f0 2024-06-13T07:23:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.960+0000 7fdad7400640 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789bd5f0 0x555a789c7450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdac4004810 tx=0x7fdac4097290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683c90c11f964df4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.960+0000 7fdac2000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:23:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.961+0000 7fdac2000640 1 -- 172.21.15.26:0/2160483279 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4009200 con 0x555a789bd5f0 2024-06-13T07:23:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.961+0000 7fdac2000640 1 -- 172.21.15.26:0/2160483279 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdac40975a0 con 0x555a789bd5f0 2024-06-13T07:23:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.961+0000 7fdae1f79740 1 -- 172.21.15.26:0/2160483279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac8003cd0 con 0x555a789bd5f0 2024-06-13T07:23:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.961+0000 7fdac2000640 1 -- 172.21.15.26:0/2160483279 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdac4010eb0 con 0x555a789bd5f0 2024-06-13T07:23:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.962+0000 7fdae1f79740 10 client.27104 fetch_fsmap learned FSMap version 48 2024-06-13T07:23:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.962+0000 7fdae1f79740 1 -- 172.21.15.26:0/2160483279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555a789d1800 con 0x555a789bd5f0 2024-06-13T07:23:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.963+0000 7fdac2000640 1 -- 172.21.15.26:0/2160483279 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac8003cd0 con 0x555a789bd5f0 2024-06-13T07:23:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.965+0000 7fdac2000640 1 -- 172.21.15.26:0/2160483279 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 48) ==== 8350+0+0 (secure 0 0 0) 0x7fdac4022060 con 0x555a789bd5f0 2024-06-13T07:23:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.965+0000 7fdae1f79740 10 client.27104 fetch_fsmap finished waiting for FSMap version 48 2024-06-13T07:23:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.965+0000 7fdae1f79740 10 client.27104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:23:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.965+0000 7fdae1f79740 20 client.27104 populate_metadata read hostname 'smithi026' 2024-06-13T07:23:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.965+0000 7fdae1f79740 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555a789d3260 0x555a789d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.965+0000 7fdae1f79740 4 client.27104 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:23:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.965+0000 7fdae1f79740 1 -- 172.21.15.26:0/2160483279 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a7892db50 con 0x555a789d3260 2024-06-13T07:23:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.966+0000 7fdad7e00640 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555a789d3260 0x555a789d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.968+0000 7fdad7e00640 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555a789d3260 0x555a789d5720 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad469ee4b9bb9a7a server_cookie=14bc8340f5a88ab6 in_seq=0 out_seq=0 2024-06-13T07:23:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.968+0000 7fdac2000640 10 client.27104 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.983+0000 7fdac2000640 1 -- 172.21.15.26:0/2160483279 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a7892db50 con 0x555a789d3260 2024-06-13T07:23:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.983+0000 7fdac2000640 10 client.27104 handle_command_reply: tid=0 2024-06-13T07:23:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.983+0000 7fdae1f79740 1 client.27104 shutdown 2024-06-13T07:23:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdae1f79740 1 -- 172.21.15.26:0/2160483279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789bd5f0 msgr2=0x555a789c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdae1f79740 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789bd5f0 0x555a789c7450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdac4004810 tx=0x7fdac4097290 comp rx=0 tx=0).stop 2024-06-13T07:23:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdae1f79740 1 -- 172.21.15.26:0/2160483279 shutdown_connections 2024-06-13T07:23:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdae1f79740 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555a789d3260 0x555a789d5720 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdae1f79740 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789bd5f0 0x555a789c7450 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdae1f79740 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789c2760 0x555a789ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdae1f79740 1 --2- 172.21.15.26:0/2160483279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 0x555a789c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdae1f79740 1 -- 172.21.15.26:0/2160483279 >> 172.21.15.26:0/2160483279 conn(0x555a78984fb0 msgr2=0x555a789b7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.986+0000 7fdac2000640 0 client.27104 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.987+0000 7fdae1f79740 1 -- 172.21.15.26:0/2160483279 shutdown_connections 2024-06-13T07:23:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.987+0000 7fdae1f79740 1 -- 172.21.15.26:0/2160483279 wait complete. 2024-06-13T07:23:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:13.987+0000 7fdae1f79740 20 client.27104 trim_cache size 0 max 16384 2024-06-13T07:23:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.026+0000 7fdae1f79740 1 Processor -- start 2024-06-13T07:23:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdae1f79740 1 -- start start 2024-06-13T07:23:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 0x555a789ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdae1f79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b37a0 con 0x555a789bd5f0 2024-06-13T07:23:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdae1f79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b1040 con 0x555a789c2760 2024-06-13T07:23:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdae1f79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81aee50 con 0x555a789bcc00 2024-06-13T07:23:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdad7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdad7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54622/0 (socket says 172.21.15.26:54622) 2024-06-13T07:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.027+0000 7fdad7e00640 1 -- 172.21.15.26:0/3943938784 learned_addr learned my addr 172.21.15.26:0/3943938784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.028+0000 7fdadd000640 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.029+0000 7fdad7400640 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 0x555a789ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.031+0000 7fdad7e00640 1 -- 172.21.15.26:0/3943938784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 msgr2=0x555a789d9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.031+0000 7fdad7e00640 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789d9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.031+0000 7fdad7e00640 1 -- 172.21.15.26:0/3943938784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 msgr2=0x555a789ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.031+0000 7fdad7e00640 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 0x555a789ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.031+0000 7fdad7e00640 1 -- 172.21.15.26:0/3943938784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a789bde80 con 0x555a789bcc00 2024-06-13T07:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.031+0000 7fdadd000640 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789d9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.031+0000 7fdad7400640 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 0x555a789ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.031+0000 7fdad7e00640 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bbc80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x555a789be060 tx=0x7fdad0012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1dce12b514dfb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.032+0000 7fdaccc00640 1 -- 172.21.15.26:0/3943938784 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad002b070 con 0x555a789bcc00 2024-06-13T07:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.032+0000 7fdaccc00640 1 -- 172.21.15.26:0/3943938784 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0025870 con 0x555a789bcc00 2024-06-13T07:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.032+0000 7fdaccc00640 1 -- 172.21.15.26:0/3943938784 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad001daf0 con 0x555a789bcc00 2024-06-13T07:23:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.035+0000 7fdae1f79740 1 -- 172.21.15.26:0/3943938784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 msgr2=0x555a789bbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.035+0000 7fdae1f79740 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bbc80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x555a789be060 tx=0x7fdad0012000 comp rx=0 tx=0).stop 2024-06-13T07:23:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.036+0000 7fdae1f79740 1 -- 172.21.15.26:0/3943938784 shutdown_connections 2024-06-13T07:23:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.036+0000 7fdae1f79740 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789d9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.036+0000 7fdae1f79740 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bd5f0 0x555a789ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.036+0000 7fdae1f79740 1 --2- 172.21.15.26:0/3943938784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bcc00 0x555a789bbc80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.036+0000 7fdae1f79740 1 -- 172.21.15.26:0/3943938784 >> 172.21.15.26:0/3943938784 conn(0x555a78984fb0 msgr2=0x555a789bb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.039+0000 7fdae1f79740 1 -- 172.21.15.26:0/3943938784 shutdown_connections 2024-06-13T07:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.039+0000 7fdae1f79740 1 -- 172.21.15.26:0/3943938784 wait complete. 2024-06-13T07:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.039+0000 7fdae1f79740 1 -- start start 2024-06-13T07:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.040+0000 7fdae1f79740 1 Processor -- start 2024-06-13T07:23:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 0x555a789cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bd5f0 0x555a789cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdae1f79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdae1f79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81af500 con 0x555a789bcc00 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdae1f79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b0970 con 0x555a789c2760 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdae1f79740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81afbd0 con 0x555a789bd5f0 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdae1f79740 10 client.0 _ll_register_callbacks cb 0x7fdac4010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdad7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bd5f0 0x555a789cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdad7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bd5f0 0x555a789cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:54636/0 (socket says 172.21.15.26:54636) 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdad7400640 1 -- 172.21.15.26:0/671239613 learned_addr learned my addr 172.21.15.26:0/671239613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.041+0000 7fdadd000640 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.042+0000 7fdad7e00640 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 0x555a789cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.042+0000 7fdad7400640 1 -- 172.21.15.26:0/671239613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 msgr2=0x555a789cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.042+0000 7fdad7400640 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.042+0000 7fdad7400640 1 -- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 msgr2=0x555a789cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.043+0000 7fdad7400640 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 0x555a789cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.044+0000 7fdad7400640 1 -- 172.21.15.26:0/671239613 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a789cd7d0 con 0x555a789bd5f0 2024-06-13T07:23:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.044+0000 7fdad7e00640 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 0x555a789cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:23:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.044+0000 7fdad7400640 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bd5f0 0x555a789cc1d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdac400bdb0 tx=0x7fdac409b030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fff6864f2414543 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.044+0000 7fdac2000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.045+0000 7fdac2000640 1 -- 172.21.15.26:0/671239613 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4010eb0 con 0x555a789bd5f0 2024-06-13T07:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.045+0000 7fdac2000640 1 -- 172.21.15.26:0/671239613 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fdac4099a60 con 0x555a789bd5f0 2024-06-13T07:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.046+0000 7fdae1f79740 1 -- 172.21.15.26:0/671239613 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad0025870 con 0x555a789bd5f0 2024-06-13T07:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.046+0000 7fdac2000640 1 -- 172.21.15.26:0/671239613 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdac40d24f0 con 0x555a789bd5f0 2024-06-13T07:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.046+0000 7fdae1f79740 10 client.27166 fetch_fsmap learned FSMap version 48 2024-06-13T07:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.046+0000 7fdae1f79740 1 -- 172.21.15.26:0/671239613 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555a78a3e090 con 0x555a789bd5f0 2024-06-13T07:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.047+0000 7fdac2000640 1 -- 172.21.15.26:0/671239613 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0025870 con 0x555a789bd5f0 2024-06-13T07:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.047+0000 7fdac2000640 1 -- 172.21.15.26:0/671239613 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 48) ==== 8350+0+0 (secure 0 0 0) 0x7fdac4022060 con 0x555a789bd5f0 2024-06-13T07:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.047+0000 7fdae1f79740 10 client.27166 fetch_fsmap finished waiting for FSMap version 48 2024-06-13T07:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.047+0000 7fdae1f79740 10 client.27166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.047+0000 7fdae1f79740 20 client.27166 populate_metadata read hostname 'smithi026' 2024-06-13T07:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.047+0000 7fdae1f79740 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555a78a3e8b0 0x555a78a40cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.047+0000 7fdae1f79740 4 client.27166 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.047+0000 7fdae1f79740 1 -- 172.21.15.26:0/671239613 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a789c6d00 con 0x555a78a3e8b0 2024-06-13T07:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.048+0000 7fdad7e00640 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555a78a3e8b0 0x555a78a40cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.050+0000 7fdad7e00640 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555a78a3e8b0 0x555a78a40cd0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80e6d2596088753 server_cookie=6f78e87797ea13a1 in_seq=0 out_seq=0 2024-06-13T07:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.050+0000 7fdac2000640 10 client.27166 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:14.240 INFO:tasks.workunit.client.0.smithi026.stdout: 10 276567 25.66 MB/sec execute 447 sec latency 160.638 ms 2024-06-13T07:23:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:14 smithi026 ceph-mon[32041]: pgmap v1191: 129 pgs: 129 active+clean; 685 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.3 MiB/s wr, 553 op/s 2024-06-13T07:23:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:14 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:23:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:23:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:23:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:14 smithi135 ceph-mon[36276]: pgmap v1191: 129 pgs: 129 active+clean; 685 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.3 MiB/s wr, 553 op/s 2024-06-13T07:23:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:14 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:23:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:23:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:23:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.831+0000 7fdac2000640 1 -- 172.21.15.26:0/671239613 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555a789c6d00 con 0x555a78a3e8b0 2024-06-13T07:23:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.831+0000 7fdac2000640 10 client.27166 handle_command_reply: tid=0 2024-06-13T07:23:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.831+0000 7fdae1f79740 1 client.27166 shutdown 2024-06-13T07:23:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.831+0000 7fdae1f79740 1 -- 172.21.15.26:0/671239613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bd5f0 msgr2=0x555a789cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.831+0000 7fdae1f79740 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bd5f0 0x555a789cc1d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdac400bdb0 tx=0x7fdac409b030 comp rx=0 tx=0).stop 2024-06-13T07:23:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.832+0000 7fdae1f79740 1 -- 172.21.15.26:0/671239613 shutdown_connections 2024-06-13T07:23:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.832+0000 7fdae1f79740 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x555a78a3e8b0 0x555a78a40cd0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.832+0000 7fdac2000640 0 client.27166 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.832+0000 7fdae1f79740 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a789c2760 0x555a789cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.832+0000 7fdae1f79740 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a789bd5f0 0x555a789cc1d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.832+0000 7fdae1f79740 1 --2- 172.21.15.26:0/671239613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a789bcc00 0x555a789cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.832+0000 7fdae1f79740 1 -- 172.21.15.26:0/671239613 >> 172.21.15.26:0/671239613 conn(0x555a78984fb0 msgr2=0x555a789b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.833+0000 7fdae1f79740 1 -- 172.21.15.26:0/671239613 shutdown_connections 2024-06-13T07:23:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.833+0000 7fdae1f79740 1 -- 172.21.15.26:0/671239613 wait complete. 2024-06-13T07:23:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.834+0000 7fdae1f79740 20 client.27166 trim_cache size 0 max 16384 2024-06-13T07:23:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:23:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:23:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:23:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.835+0000 7fdadf000640 1 -- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdabc0a2d30 msgr2=0x7fdabc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.835+0000 7fdadf000640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdabc0a2d30 0x7fdabc0a51f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x555a789b9530 tx=0x7fdad0007880 comp rx=0 tx=0).stop 2024-06-13T07:23:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.835+0000 7fdadf000640 1 -- 172.21.15.26:0/2904271424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad8102bf0 msgr2=0x7fdad819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.835+0000 7fdadf000640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad8102bf0 0x7fdad819a390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdac400b3f0 tx=0x7fdac400b8b0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.836+0000 7fdadf000640 1 -- 172.21.15.26:0/2904271424 shutdown_connections 2024-06-13T07:23:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.836+0000 7fdadf000640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fdabc0a2d30 0x7fdabc0a51f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.836+0000 7fdadf000640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdad81035b0 0x7fdad81a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.836+0000 7fdadf000640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad8102bf0 0x7fdad819a390 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.836+0000 7fdadf000640 1 --2- 172.21.15.26:0/2904271424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad8101a00 0x7fdad8199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.836+0000 7fdadf000640 1 -- 172.21.15.26:0/2904271424 >> 172.21.15.26:0/2904271424 conn(0x7fdad80fcf90 msgr2=0x7fdad8105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.836+0000 7fdadf000640 1 -- 172.21.15.26:0/2904271424 shutdown_connections 2024-06-13T07:23:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:14.836+0000 7fdadf000640 1 -- 172.21.15.26:0/2904271424 wait complete. 2024-06-13T07:23:14.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:14 smithi136 ceph-mon[37229]: pgmap v1191: 129 pgs: 129 active+clean; 685 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.3 MiB/s wr, 553 op/s 2024-06-13T07:23:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:14 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:23:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:23:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:23:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:23:15.240 INFO:tasks.workunit.client.0.smithi026.stdout: 10 276850 25.63 MB/sec execute 448 sec latency 617.670 ms 2024-06-13T07:23:16.240 INFO:tasks.workunit.client.0.smithi026.stdout: 10 277216 25.62 MB/sec execute 449 sec latency 147.455 ms 2024-06-13T07:23:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:16 smithi026 ceph-mon[32041]: pgmap v1192: 129 pgs: 129 active+clean; 675 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.8 MiB/s wr, 531 op/s 2024-06-13T07:23:16.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:16 smithi135 ceph-mon[36276]: pgmap v1192: 129 pgs: 129 active+clean; 675 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.8 MiB/s wr, 531 op/s 2024-06-13T07:23:16.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:16 smithi136 ceph-mon[37229]: pgmap v1192: 129 pgs: 129 active+clean; 675 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.8 MiB/s wr, 531 op/s 2024-06-13T07:23:17.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 277628 25.59 MB/sec execute 450 sec latency 94.782 ms 2024-06-13T07:23:18.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 278189 25.61 MB/sec execute 451 sec latency 94.558 ms 2024-06-13T07:23:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:18 smithi026 ceph-mon[32041]: pgmap v1193: 129 pgs: 129 active+clean; 659 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 504 op/s 2024-06-13T07:23:18.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:18 smithi135 ceph-mon[36276]: pgmap v1193: 129 pgs: 129 active+clean; 659 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 504 op/s 2024-06-13T07:23:18.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:18 smithi136 ceph-mon[37229]: pgmap v1193: 129 pgs: 129 active+clean; 659 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 504 op/s 2024-06-13T07:23:19.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 278745 25.61 MB/sec execute 452 sec latency 91.594 ms 2024-06-13T07:23:19.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:19 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:23:19.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:19 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:23:19.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:19 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:23:20.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 279287 25.63 MB/sec execute 453 sec latency 75.474 ms 2024-06-13T07:23:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:20 smithi026 ceph-mon[32041]: pgmap v1194: 129 pgs: 129 active+clean; 650 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-06-13T07:23:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:20 smithi135 ceph-mon[36276]: pgmap v1194: 129 pgs: 129 active+clean; 650 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-06-13T07:23:20.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:20 smithi136 ceph-mon[37229]: pgmap v1194: 129 pgs: 129 active+clean; 650 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-06-13T07:23:21.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 280005 25.66 MB/sec execute 454 sec latency 82.681 ms 2024-06-13T07:23:22.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 280453 25.64 MB/sec execute 455 sec latency 97.265 ms 2024-06-13T07:23:22.677 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:22 smithi136 ceph-mon[37229]: pgmap v1195: 129 pgs: 129 active+clean; 651 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 492 op/s 2024-06-13T07:23:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:22 smithi026 ceph-mon[32041]: pgmap v1195: 129 pgs: 129 active+clean; 651 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 492 op/s 2024-06-13T07:23:22.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:22 smithi135 ceph-mon[36276]: pgmap v1195: 129 pgs: 129 active+clean; 651 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 492 op/s 2024-06-13T07:23:23.245 INFO:tasks.workunit.client.0.smithi026.stdout: 10 281044 25.67 MB/sec execute 456 sec latency 62.189 ms 2024-06-13T07:23:24.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 281628 25.69 MB/sec execute 457 sec latency 92.871 ms 2024-06-13T07:23:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:24 smithi026 ceph-mon[32041]: pgmap v1196: 129 pgs: 129 active+clean; 655 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 534 op/s 2024-06-13T07:23:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:24 smithi135 ceph-mon[36276]: pgmap v1196: 129 pgs: 129 active+clean; 655 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 534 op/s 2024-06-13T07:23:24.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:24 smithi136 ceph-mon[37229]: pgmap v1196: 129 pgs: 129 active+clean; 655 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 534 op/s 2024-06-13T07:23:25.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 282168 25.68 MB/sec execute 458 sec latency 80.808 ms 2024-06-13T07:23:26.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 282640 25.69 MB/sec execute 459 sec latency 126.579 ms 2024-06-13T07:23:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:26 smithi026 ceph-mon[32041]: pgmap v1197: 129 pgs: 129 active+clean; 654 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 464 op/s 2024-06-13T07:23:26.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:26 smithi135 ceph-mon[36276]: pgmap v1197: 129 pgs: 129 active+clean; 654 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 464 op/s 2024-06-13T07:23:26.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:26 smithi136 ceph-mon[37229]: pgmap v1197: 129 pgs: 129 active+clean; 654 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 464 op/s 2024-06-13T07:23:27.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 283280 25.70 MB/sec execute 460 sec latency 81.092 ms 2024-06-13T07:23:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 -- 172.21.15.26:0/1126352521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 msgr2=0x7f90cc1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 --2- 172.21.15.26:0/1126352521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 0x7f90cc1052a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009860 tx=0x7f90b802a340 comp rx=0 tx=0).stop 2024-06-13T07:23:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 -- 172.21.15.26:0/1126352521 shutdown_connections 2024-06-13T07:23:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 --2- 172.21.15.26:0/1126352521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 0x7f90cc1052a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 --2- 172.21.15.26:0/1126352521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90cc1057e0 0x7f90cc10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 --2- 172.21.15.26:0/1126352521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90cc103c30 0x7f90cc104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 -- 172.21.15.26:0/1126352521 >> 172.21.15.26:0/1126352521 conn(0x7f90cc0ff1c0 msgr2=0x7f90cc1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 -- 172.21.15.26:0/1126352521 shutdown_connections 2024-06-13T07:23:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.109+0000 7f90d3800640 1 -- 172.21.15.26:0/1126352521 wait complete. 2024-06-13T07:23:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.110+0000 7f90d3800640 1 Processor -- start 2024-06-13T07:23:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.110+0000 7f90d3800640 1 -- start start 2024-06-13T07:23:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d3800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90cc103c30 0x7f90cc195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d3800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 0x7f90cc195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d3800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90cc1057e0 0x7f90cc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d3800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc0789e0 con 0x7f90cc103c30 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d3800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc078860 con 0x7f90cc104e20 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d3800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc078b60 con 0x7f90cc1057e0 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90cc1057e0 0x7f90cc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90cbe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 0x7f90cc195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90cc1057e0 0x7f90cc19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57504/0 (socket says 172.21.15.26:57504) 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.111+0000 7f90d1800640 1 -- 172.21.15.26:0/1088046205 learned_addr learned my addr 172.21.15.26:0/1088046205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.112+0000 7f90d0e00640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90cc103c30 0x7f90cc195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.112+0000 7f90cbe00640 1 -- 172.21.15.26:0/1088046205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90cc1057e0 msgr2=0x7f90cc19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.112+0000 7f90cbe00640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90cc1057e0 0x7f90cc19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.112+0000 7f90cbe00640 1 -- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90cc103c30 msgr2=0x7f90cc195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.112+0000 7f90cbe00640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90cc103c30 0x7f90cc195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.112+0000 7f90cbe00640 1 -- 172.21.15.26:0/1088046205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc19d620 con 0x7f90cc104e20 2024-06-13T07:23:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.113+0000 7f90cbe00640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 0x7f90cc195f40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f90b8035040 tx=0x7f90b8033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.113+0000 7f90c9600640 1 -- 172.21.15.26:0/1088046205 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8037040 con 0x7f90cc104e20 2024-06-13T07:23:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.113+0000 7f90c9600640 1 -- 172.21.15.26:0/1088046205 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b8030850 con 0x7f90cc104e20 2024-06-13T07:23:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.113+0000 7f90c9600640 1 -- 172.21.15.26:0/1088046205 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8004920 con 0x7f90cc104e20 2024-06-13T07:23:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.114+0000 7f90d3800640 1 -- 172.21.15.26:0/1088046205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc19d8b0 con 0x7f90cc104e20 2024-06-13T07:23:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.114+0000 7f90d3800640 1 -- 172.21.15.26:0/1088046205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc19dd20 con 0x7f90cc104e20 2024-06-13T07:23:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.120+0000 7f90d6620740 1 Processor -- start 2024-06-13T07:23:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.121+0000 7f90d6620740 1 -- start start 2024-06-13T07:23:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.122+0000 7f90d6620740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a57f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.122+0000 7f90d6620740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.122+0000 7f90d6620740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a586bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.122+0000 7f90d6620740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b33a4e2420 con 0x55b33a57ec00 2024-06-13T07:23:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.122+0000 7f90d6620740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b33a489c50 con 0x55b33a57f5f0 2024-06-13T07:23:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.122+0000 7f90d6620740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b33a4de3e0 con 0x55b33a584760 2024-06-13T07:23:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.124+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.124+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35866/0 (socket says 172.21.15.26:35866) 2024-06-13T07:23:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.125+0000 7f90d0e00640 1 -- 172.21.15.26:0/172425843 learned_addr learned my addr 172.21.15.26:0/172425843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.125+0000 7f90d0e00640 1 -- 172.21.15.26:0/172425843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 msgr2=0x55b33a586bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:23:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.125+0000 7f90d0e00640 1 --2- 172.21.15.26:0/172425843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a586bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.125+0000 7f90d0e00640 1 -- 172.21.15.26:0/172425843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 msgr2=0x55b33a57f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.125+0000 7f90d0e00640 1 --2- 172.21.15.26:0/172425843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a57f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.125+0000 7f90d0e00640 1 -- 172.21.15.26:0/172425843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33a1f6210 con 0x55b33a57f5f0 2024-06-13T07:23:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.125+0000 7f90c9600640 1 -- 172.21.15.26:0/1088046205 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90b8004050 con 0x7f90cc104e20 2024-06-13T07:23:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.127+0000 7f90d0e00640 1 --2- 172.21.15.26:0/172425843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5840f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f90c000fd60 tx=0x7f90c0011a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24cc27021af9f59f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.128+0000 7f90be000640 1 -- 172.21.15.26:0/172425843 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0021030 con 0x55b33a57f5f0 2024-06-13T07:23:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.128+0000 7f90be000640 1 -- 172.21.15.26:0/172425843 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c0005500 con 0x55b33a57f5f0 2024-06-13T07:23:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.128+0000 7f90be000640 1 -- 172.21.15.26:0/172425843 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c001d890 con 0x55b33a57f5f0 2024-06-13T07:23:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.129+0000 7f90d6620740 1 -- 172.21.15.26:0/172425843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 msgr2=0x55b33a5840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.129+0000 7f90d6620740 1 --2- 172.21.15.26:0/172425843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5840f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f90c000fd60 tx=0x7f90c0011a70 comp rx=0 tx=0).stop 2024-06-13T07:23:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.130+0000 7f90c9600640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f90b40a2cb0 0x7f90b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.130+0000 7f90c9600640 1 -- 172.21.15.26:0/1088046205 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f90b80eeb70 con 0x7f90cc104e20 2024-06-13T07:23:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.131+0000 7f90d1800640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f90b40a2cb0 0x7f90b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.132+0000 7f90d6620740 1 -- 172.21.15.26:0/172425843 shutdown_connections 2024-06-13T07:23:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.132+0000 7f90d6620740 1 --2- 172.21.15.26:0/172425843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5840f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.132+0000 7f90d6620740 1 --2- 172.21.15.26:0/172425843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a586bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.132+0000 7f90d6620740 1 --2- 172.21.15.26:0/172425843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a57f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.132+0000 7f90d6620740 1 -- 172.21.15.26:0/172425843 >> 172.21.15.26:0/172425843 conn(0x55b33a546fb0 msgr2=0x55b33a57a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.132+0000 7f90d1800640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f90b40a2cb0 0x7f90b40a5170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009aa0 tx=0x7f90c4009a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.133+0000 7f90d6620740 1 -- 172.21.15.26:0/172425843 shutdown_connections 2024-06-13T07:23:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.134+0000 7f90d6620740 1 -- 172.21.15.26:0/172425843 wait complete. 2024-06-13T07:23:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.134+0000 7f90d6620740 1 -- start start 2024-06-13T07:23:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.135+0000 7f90d6620740 1 Processor -- start 2024-06-13T07:23:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90d6620740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a588fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90d6620740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57f5f0 0x55b33a5894e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90d6620740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a590520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90d6620740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b33a587710 con 0x55b33a57ec00 2024-06-13T07:23:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90d6620740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b33a49d0b0 con 0x55b33a584760 2024-06-13T07:23:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90d6620740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b33a587890 con 0x55b33a57f5f0 2024-06-13T07:23:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90d6620740 10 client.0 _ll_register_callbacks cb 0x55b33a488f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:23:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90cbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57f5f0 0x55b33a5894e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90cbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57f5f0 0x55b33a5894e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57514/0 (socket says 172.21.15.26:57514) 2024-06-13T07:23:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.136+0000 7f90cbe00640 1 -- 172.21.15.26:0/512538385 learned_addr learned my addr 172.21.15.26:0/512538385 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.137+0000 7f90d0e00640 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a588fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.137+0000 7f90d1800640 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a590520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.138+0000 7f90d0e00640 1 -- 172.21.15.26:0/512538385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57f5f0 msgr2=0x55b33a5894e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.138+0000 7f90d0e00640 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57f5f0 0x55b33a5894e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.138+0000 7f90d0e00640 1 -- 172.21.15.26:0/512538385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 msgr2=0x55b33a590520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.138+0000 7f90d0e00640 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a590520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.138+0000 7f90d0e00640 1 -- 172.21.15.26:0/512538385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33a2c2620 con 0x55b33a57ec00 2024-06-13T07:23:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.138+0000 7f90cbe00640 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57f5f0 0x55b33a5894e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:23:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.139+0000 7f90d0e00640 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a588fa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b33a57b530 tx=0x7f90c00052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b097d2483c806324 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.139+0000 7f90b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:23:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.142+0000 7f90b2a00640 1 -- 172.21.15.26:0/512538385 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c002b070 con 0x55b33a57ec00 2024-06-13T07:23:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.142+0000 7f90b2a00640 1 -- 172.21.15.26:0/512538385 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f90c001da60 con 0x55b33a57ec00 2024-06-13T07:23:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.142+0000 7f90d6620740 1 -- 172.21.15.26:0/512538385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c0005500 con 0x55b33a57ec00 2024-06-13T07:23:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.143+0000 7f90b2a00640 1 -- 172.21.15.26:0/512538385 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f90c000f9d0 con 0x55b33a57ec00 2024-06-13T07:23:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.143+0000 7f90d6620740 10 client.16833 fetch_fsmap learned FSMap version 49 2024-06-13T07:23:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.143+0000 7f90d6620740 1 -- 172.21.15.26:0/512538385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b33a593950 con 0x55b33a57ec00 2024-06-13T07:23:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.144+0000 7f90b2a00640 1 -- 172.21.15.26:0/512538385 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c0005500 con 0x55b33a57ec00 2024-06-13T07:23:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.144+0000 7f90b2a00640 1 -- 172.21.15.26:0/512538385 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f90c0027ca0 con 0x55b33a57ec00 2024-06-13T07:23:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.145+0000 7f90d6620740 10 client.16833 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:23:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.145+0000 7f90d6620740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:23:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.145+0000 7f90d6620740 20 client.16833 populate_metadata read hostname 'smithi026' 2024-06-13T07:23:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.145+0000 7f90d6620740 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b33a594d60 0x55b33a597150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.145+0000 7f90d6620740 4 client.16833 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:23:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.145+0000 7f90d6620740 1 -- 172.21.15.26:0/512538385 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b33a4efb50 con 0x55b33a594d60 2024-06-13T07:23:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.145+0000 7f90cbe00640 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b33a594d60 0x55b33a597150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.146+0000 7f90cbe00640 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b33a594d60 0x55b33a597150 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5219c7feed180d server_cookie=bd1fab245fbdf53c in_seq=0 out_seq=0 2024-06-13T07:23:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.146+0000 7f90b2a00640 10 client.16833 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.147+0000 7f90b2a00640 1 -- 172.21.15.26:0/512538385 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f90b8099e50 con 0x55b33a594d60 2024-06-13T07:23:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.147+0000 7f90b2a00640 10 client.16833 handle_command_reply: tid=0 2024-06-13T07:23:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.147+0000 7f90d6620740 1 client.16833 shutdown 2024-06-13T07:23:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 -- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 msgr2=0x55b33a588fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a588fa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b33a57b530 tx=0x7f90c00052d0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 -- 172.21.15.26:0/512538385 shutdown_connections 2024-06-13T07:23:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b33a594d60 0x55b33a597150 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a590520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57f5f0 0x55b33a5894e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 --2- 172.21.15.26:0/512538385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a588fa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 -- 172.21.15.26:0/512538385 >> 172.21.15.26:0/512538385 conn(0x55b33a546fb0 msgr2=0x55b33a579840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90b2a00640 0 client.16833 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 -- 172.21.15.26:0/512538385 shutdown_connections 2024-06-13T07:23:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.148+0000 7f90d6620740 1 -- 172.21.15.26:0/512538385 wait complete. 2024-06-13T07:23:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.149+0000 7f90d6620740 20 client.16833 trim_cache size 0 max 16384 2024-06-13T07:23:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.177+0000 7f90d6620740 1 Processor -- start 2024-06-13T07:23:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.177+0000 7f90d6620740 1 -- start start 2024-06-13T07:23:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.177+0000 7f90d6620740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a57dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.177+0000 7f90d6620740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.177+0000 7f90d6620740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a588290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.177+0000 7f90d6620740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1aae70 con 0x55b33a57ec00 2024-06-13T07:23:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.177+0000 7f90d6620740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1a86e0 con 0x55b33a57f5f0 2024-06-13T07:23:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.177+0000 7f90d6620740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1a64f0 con 0x55b33a584760 2024-06-13T07:23:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.178+0000 7f90d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a588290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.178+0000 7f90d1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a588290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:57516/0 (socket says 172.21.15.26:57516) 2024-06-13T07:23:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.178+0000 7f90d1800640 1 -- 172.21.15.26:0/2102802601 learned_addr learned my addr 172.21.15.26:0/2102802601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.178+0000 7f90d0e00640 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a57dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.178+0000 7f90d1800640 1 -- 172.21.15.26:0/2102802601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 msgr2=0x55b33a5904e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.178+0000 7f90d1800640 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5904e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.178+0000 7f90d1800640 1 -- 172.21.15.26:0/2102802601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 msgr2=0x55b33a57dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.179+0000 7f90d1800640 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a57dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.179+0000 7f90d1800640 1 -- 172.21.15.26:0/2102802601 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33a57b330 con 0x55b33a584760 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.179+0000 7f90d0e00640 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a57dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.179+0000 7f90d1800640 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a588290 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f90c401d780 tx=0x7f90c4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ae5c4f0ba68261 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.180+0000 7f90bd600640 1 -- 172.21.15.26:0/2102802601 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c401ddc0 con 0x55b33a584760 2024-06-13T07:23:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.180+0000 7f90bd600640 1 -- 172.21.15.26:0/2102802601 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c401e930 con 0x55b33a584760 2024-06-13T07:23:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.180+0000 7f90bd600640 1 -- 172.21.15.26:0/2102802601 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40278c0 con 0x55b33a584760 2024-06-13T07:23:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 -- 172.21.15.26:0/2102802601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 msgr2=0x55b33a588290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a588290 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f90c401d780 tx=0x7f90c4014000 comp rx=0 tx=0).stop 2024-06-13T07:23:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 -- 172.21.15.26:0/2102802601 shutdown_connections 2024-06-13T07:23:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a57f5f0 0x55b33a5904e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a584760 0x55b33a588290 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 --2- 172.21.15.26:0/2102802601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57ec00 0x55b33a57dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 -- 172.21.15.26:0/2102802601 >> 172.21.15.26:0/2102802601 conn(0x55b33a546fb0 msgr2=0x55b33a57d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 -- 172.21.15.26:0/2102802601 shutdown_connections 2024-06-13T07:23:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.181+0000 7f90d6620740 1 -- 172.21.15.26:0/2102802601 wait complete. 2024-06-13T07:23:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.182+0000 7f90d6620740 1 -- start start 2024-06-13T07:23:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.182+0000 7f90d6620740 1 Processor -- start 2024-06-13T07:23:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.183+0000 7f90d6620740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57ec00 0x55b33a58d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.183+0000 7f90d6620740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57f5f0 0x55b33a58d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.183+0000 7f90d6620740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a5fd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.184+0000 7f90d1800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a5fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.184+0000 7f90d1800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a5fd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35882/0 (socket says 172.21.15.26:35882) 2024-06-13T07:23:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.184+0000 7f90d1800640 1 -- 172.21.15.26:0/3332726367 learned_addr learned my addr 172.21.15.26:0/3332726367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.184+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1a6ba0 con 0x55b33a57f5f0 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.184+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1a8010 con 0x55b33a584760 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.184+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1a7270 con 0x55b33a57ec00 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.184+0000 7f90d6620740 10 client.0 _ll_register_callbacks cb 0x7f90c000fc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.184+0000 7f90d0e00640 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57ec00 0x55b33a58d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.185+0000 7f90d0e00640 1 -- 172.21.15.26:0/3332726367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 msgr2=0x55b33a5fd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.185+0000 7f90d0e00640 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a5fd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.185+0000 7f90d0e00640 1 -- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57f5f0 msgr2=0x55b33a58d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.185+0000 7f90cbe00640 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57f5f0 0x55b33a58d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.185+0000 7f90d0e00640 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57f5f0 0x55b33a58d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.185+0000 7f90d0e00640 1 -- 172.21.15.26:0/3332726367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33a590640 con 0x55b33a57ec00 2024-06-13T07:23:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.185+0000 7f90cbe00640 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57f5f0 0x55b33a58d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:23:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.186+0000 7f90d0e00640 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57ec00 0x55b33a58d0f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f90c000fb70 tx=0x7f90c00118b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2423ae54348608a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.186+0000 7f90b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:23:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.187+0000 7f90b2a00640 1 -- 172.21.15.26:0/3332726367 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0021070 con 0x55b33a57ec00 2024-06-13T07:23:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.189+0000 7f90b2a00640 1 -- 172.21.15.26:0/3332726367 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f90c003ace0 con 0x55b33a57ec00 2024-06-13T07:23:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.189+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c401e930 con 0x55b33a57ec00 2024-06-13T07:23:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90b2a00640 1 -- 172.21.15.26:0/3332726367 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f90c0047050 con 0x55b33a57ec00 2024-06-13T07:23:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90d6620740 10 client.27181 fetch_fsmap learned FSMap version 49 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b33a5fec20 con 0x55b33a57ec00 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90b2a00640 1 -- 172.21.15.26:0/3332726367 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c401e930 con 0x55b33a57ec00 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90b2a00640 1 -- 172.21.15.26:0/3332726367 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f90c001d860 con 0x55b33a57ec00 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90d6620740 10 client.27181 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90d6620740 10 client.27181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90d6620740 20 client.27181 populate_metadata read hostname 'smithi026' 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.193+0000 7f90d6620740 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b33a5ff510 0x55b33a601930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.194+0000 7f90d6620740 4 client.27181 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:23:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.194+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b33a4e9ea0 con 0x55b33a5ff510 2024-06-13T07:23:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.194+0000 7f90cbe00640 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b33a5ff510 0x55b33a601930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.195+0000 7f90cbe00640 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b33a5ff510 0x55b33a601930 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0a7395322e0cee server_cookie=8b6a716144cbc396 in_seq=0 out_seq=0 2024-06-13T07:23:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.195+0000 7f90b2a00640 10 client.27181 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:28.241 INFO:tasks.workunit.client.0.smithi026.stdout: 10 283793 25.70 MB/sec execute 461 sec latency 124.787 ms 2024-06-13T07:23:28.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:28 smithi026 ceph-mon[32041]: pgmap v1198: 129 pgs: 129 active+clean; 660 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.5 MiB/s wr, 452 op/s 2024-06-13T07:23:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:28 smithi135 ceph-mon[36276]: pgmap v1198: 129 pgs: 129 active+clean; 660 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.5 MiB/s wr, 452 op/s 2024-06-13T07:23:28.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:28 smithi136 ceph-mon[37229]: pgmap v1198: 129 pgs: 129 active+clean; 660 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.5 MiB/s wr, 452 op/s 2024-06-13T07:23:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90b2a00640 1 -- 172.21.15.26:0/3332726367 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b33a4e9ea0 con 0x55b33a5ff510 2024-06-13T07:23:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90b2a00640 10 client.27181 handle_command_reply: tid=0 2024-06-13T07:23:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 client.27181 shutdown 2024-06-13T07:23:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57ec00 msgr2=0x55b33a58d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57ec00 0x55b33a58d0f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f90c000fb70 tx=0x7f90c00118b0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 shutdown_connections 2024-06-13T07:23:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55b33a5ff510 0x55b33a601930 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b33a584760 0x55b33a5fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b33a57f5f0 0x55b33a58d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 --2- 172.21.15.26:0/3332726367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b33a57ec00 0x55b33a58d0f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 >> 172.21.15.26:0/3332726367 conn(0x55b33a546fb0 msgr2=0x55b33a578fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.962+0000 7f90b2a00640 0 client.27181 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.963+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 shutdown_connections 2024-06-13T07:23:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.963+0000 7f90d6620740 1 -- 172.21.15.26:0/3332726367 wait complete. 2024-06-13T07:23:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.963+0000 7f90d6620740 20 client.27181 trim_cache size 0 max 16384 2024-06-13T07:23:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:23:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:23:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:23:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 -- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f90b40a2cb0 msgr2=0x7f90b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f90b40a2cb0 0x7f90b40a5170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009aa0 tx=0x7f90c4009a30 comp rx=0 tx=0).stop 2024-06-13T07:23:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 -- 172.21.15.26:0/1088046205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 msgr2=0x7f90cc195f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 0x7f90cc195f40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f90b8035040 tx=0x7f90b8033d90 comp rx=0 tx=0).stop 2024-06-13T07:23:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 -- 172.21.15.26:0/1088046205 shutdown_connections 2024-06-13T07:23:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f90b40a2cb0 0x7f90b40a5170 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90cc104e20 0x7f90cc195f40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90cc1057e0 0x7f90cc19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 --2- 172.21.15.26:0/1088046205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90cc103c30 0x7f90cc195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.967+0000 7f90d3800640 1 -- 172.21.15.26:0/1088046205 >> 172.21.15.26:0/1088046205 conn(0x7f90cc0ff1c0 msgr2=0x7f90cc0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.968+0000 7f90d3800640 1 -- 172.21.15.26:0/1088046205 shutdown_connections 2024-06-13T07:23:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:28.968+0000 7f90d3800640 1 -- 172.21.15.26:0/1088046205 wait complete. 2024-06-13T07:23:29.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 284125 25.69 MB/sec execute 462 sec latency 538.991 ms 2024-06-13T07:23:30.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 284773 25.71 MB/sec execute 463 sec latency 61.486 ms 2024-06-13T07:23:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:30 smithi026 ceph-mon[32041]: pgmap v1199: 129 pgs: 129 active+clean; 665 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-06-13T07:23:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:30 smithi135 ceph-mon[36276]: pgmap v1199: 129 pgs: 129 active+clean; 665 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-06-13T07:23:30.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:30 smithi136 ceph-mon[37229]: pgmap v1199: 129 pgs: 129 active+clean; 665 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-06-13T07:23:31.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 285309 25.72 MB/sec execute 464 sec latency 131.076 ms 2024-06-13T07:23:32.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 285840 25.74 MB/sec execute 465 sec latency 69.239 ms 2024-06-13T07:23:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:32 smithi026 ceph-mon[32041]: pgmap v1200: 129 pgs: 129 active+clean; 669 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.4 MiB/s wr, 468 op/s 2024-06-13T07:23:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:32 smithi135 ceph-mon[36276]: pgmap v1200: 129 pgs: 129 active+clean; 669 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.4 MiB/s wr, 468 op/s 2024-06-13T07:23:32.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:32 smithi136 ceph-mon[37229]: pgmap v1200: 129 pgs: 129 active+clean; 669 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.4 MiB/s wr, 468 op/s 2024-06-13T07:23:33.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 286406 25.74 MB/sec execute 466 sec latency 58.988 ms 2024-06-13T07:23:34.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 286949 25.75 MB/sec execute 467 sec latency 68.313 ms 2024-06-13T07:23:34.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:34 smithi026 ceph-mon[32041]: pgmap v1201: 129 pgs: 129 active+clean; 671 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:23:34.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:34 smithi135 ceph-mon[36276]: pgmap v1201: 129 pgs: 129 active+clean; 671 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:23:34.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:34 smithi136 ceph-mon[37229]: pgmap v1201: 129 pgs: 129 active+clean; 671 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:23:35.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 287330 25.73 MB/sec execute 468 sec latency 205.566 ms 2024-06-13T07:23:36.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 287801 25.72 MB/sec execute 469 sec latency 132.743 ms 2024-06-13T07:23:36.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:36 smithi026 ceph-mon[32041]: pgmap v1202: 129 pgs: 129 active+clean; 676 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.1 MiB/s wr, 460 op/s 2024-06-13T07:23:36.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:36 smithi135 ceph-mon[36276]: pgmap v1202: 129 pgs: 129 active+clean; 676 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.1 MiB/s wr, 460 op/s 2024-06-13T07:23:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:36 smithi136 ceph-mon[37229]: pgmap v1202: 129 pgs: 129 active+clean; 676 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.1 MiB/s wr, 460 op/s 2024-06-13T07:23:37.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 288411 25.74 MB/sec execute 470 sec latency 90.737 ms 2024-06-13T07:23:37.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:37 smithi135 ceph-mon[36276]: pgmap v1203: 129 pgs: 129 active+clean; 681 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:23:37.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:37 smithi136 ceph-mon[37229]: pgmap v1203: 129 pgs: 129 active+clean; 681 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:23:38.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:37 smithi026 ceph-mon[32041]: pgmap v1203: 129 pgs: 129 active+clean; 681 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.3 MiB/s wr, 481 op/s 2024-06-13T07:23:38.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 289178 25.77 MB/sec execute 471 sec latency 66.621 ms 2024-06-13T07:23:39.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 289403 25.75 MB/sec execute 472 sec latency 302.625 ms 2024-06-13T07:23:40.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 289404 25.70 MB/sec execute 473 sec latency 1302.696 ms 2024-06-13T07:23:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:40 smithi026 ceph-mon[32041]: pgmap v1204: 129 pgs: 129 active+clean; 686 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 477 op/s 2024-06-13T07:23:40.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:40 smithi135 ceph-mon[36276]: pgmap v1204: 129 pgs: 129 active+clean; 686 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 477 op/s 2024-06-13T07:23:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:40 smithi136 ceph-mon[37229]: pgmap v1204: 129 pgs: 129 active+clean; 686 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 477 op/s 2024-06-13T07:23:41.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 289404 25.64 MB/sec execute 474 sec latency 2302.759 ms 2024-06-13T07:23:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.221+0000 7f075cc00640 1 -- 172.21.15.26:0/1018162965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 msgr2=0x7f0758109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.221+0000 7f075cc00640 1 --2- 172.21.15.26:0/1018162965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 0x7f0758109990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0750009be0 tx=0x7f075002a6c0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.221+0000 7f075cc00640 1 -- 172.21.15.26:0/1018162965 shutdown_connections 2024-06-13T07:23:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.221+0000 7f075cc00640 1 --2- 172.21.15.26:0/1018162965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0758104c70 0x7f0758107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.221+0000 7f075cc00640 1 --2- 172.21.15.26:0/1018162965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 0x7f0758109990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.221+0000 7f075cc00640 1 --2- 172.21.15.26:0/1018162965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0758102340 0x7f0758104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.221+0000 7f075cc00640 1 -- 172.21.15.26:0/1018162965 >> 172.21.15.26:0/1018162965 conn(0x7f07580fbc90 msgr2=0x7f07580fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.222+0000 7f075cc00640 1 -- 172.21.15.26:0/1018162965 shutdown_connections 2024-06-13T07:23:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.224+0000 7f075cc00640 1 -- 172.21.15.26:0/1018162965 wait complete. 2024-06-13T07:23:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.225+0000 7f075cc00640 1 Processor -- start 2024-06-13T07:23:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.226+0000 7f075cc00640 1 -- start start 2024-06-13T07:23:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.226+0000 7f075cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0758102340 0x7f075819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.226+0000 7f075cc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0758104c70 0x7f075819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f075cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 0x7f07581a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f0756a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0758104c70 0x7f075819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f0756a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0758104c70 0x7f075819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:58326/0 (socket says 172.21.15.26:58326) 2024-06-13T07:23:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f0757400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0758102340 0x7f075819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f0757e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 0x7f07581a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f0757e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 0x7f07581a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37006/0 (socket says 172.21.15.26:37006) 2024-06-13T07:23:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f0757400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0758102340 0x7f075819e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60998/0 (socket says 172.21.15.26:60998) 2024-06-13T07:23:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0758101e70 con 0x7f0758102340 2024-06-13T07:23:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0758101cf0 con 0x7f0758104c70 2024-06-13T07:23:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0758101ff0 con 0x7f07581075a0 2024-06-13T07:23:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.227+0000 7f0756a00640 1 -- 172.21.15.26:0/1416957208 learned_addr learned my addr 172.21.15.26:0/1416957208 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.228+0000 7f0756a00640 1 -- 172.21.15.26:0/1416957208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 msgr2=0x7f07581a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.228+0000 7f0756a00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 0x7f07581a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.228+0000 7f0756a00640 1 -- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0758102340 msgr2=0x7f075819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.228+0000 7f0756a00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0758102340 0x7f075819e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.228+0000 7f0756a00640 1 -- 172.21.15.26:0/1416957208 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07581a5cb0 con 0x7f0758104c70 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.228+0000 7f0757e00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 0x7f07581a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.228+0000 7f0757400640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0758102340 0x7f075819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.229+0000 7f0756a00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0758104c70 0x7f075819e5d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f074400c970 tx=0x7f074400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.229+0000 7f074fe00640 1 -- 172.21.15.26:0/1416957208 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744014070 con 0x7f0758104c70 2024-06-13T07:23:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.229+0000 7f074fe00640 1 -- 172.21.15.26:0/1416957208 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0744004440 con 0x7f0758104c70 2024-06-13T07:23:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.229+0000 7f074fe00640 1 -- 172.21.15.26:0/1416957208 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744003db0 con 0x7f0758104c70 2024-06-13T07:23:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.229+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07581a5fa0 con 0x7f0758104c70 2024-06-13T07:23:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.229+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07581a6410 con 0x7f0758104c70 2024-06-13T07:23:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.236+0000 7f0760fde740 1 Processor -- start 2024-06-13T07:23:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.237+0000 7f0760fde740 1 -- start start 2024-06-13T07:23:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.238+0000 7f0760fde740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46211dc0 0x562b462121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.238+0000 7f0760fde740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b462127b0 0x562b462172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.238+0000 7f0760fde740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 0x562b46219db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.238+0000 7f0760fde740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562b46175420 con 0x562b462127b0 2024-06-13T07:23:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.238+0000 7f0760fde740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562b4611cc50 con 0x562b46217920 2024-06-13T07:23:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.238+0000 7f0760fde740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562b461713e0 con 0x562b46211dc0 2024-06-13T07:23:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.238+0000 7f0757400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b462127b0 0x562b462172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.238+0000 7f0757e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46211dc0 0x562b462121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.239+0000 7f0757e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46211dc0 0x562b462121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37016/0 (socket says 172.21.15.26:37016) 2024-06-13T07:23:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.239+0000 7f0757e00640 1 -- 172.21.15.26:0/2255774075 learned_addr learned my addr 172.21.15.26:0/2255774075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.239+0000 7f0757e00640 1 -- 172.21.15.26:0/2255774075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 msgr2=0x562b46219db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:23:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.239+0000 7f0757e00640 1 --2- 172.21.15.26:0/2255774075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 0x562b46219db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.239+0000 7f0757e00640 1 -- 172.21.15.26:0/2255774075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b462127b0 msgr2=0x562b462172b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.239+0000 7f0757e00640 1 --2- 172.21.15.26:0/2255774075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b462127b0 0x562b462172b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.240+0000 7f0757e00640 1 -- 172.21.15.26:0/2255774075 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b45e89210 con 0x562b46211dc0 2024-06-13T07:23:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.241+0000 7f0757e00640 1 --2- 172.21.15.26:0/2255774075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46211dc0 0x562b462121e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0750009be0 tx=0x7f07500304a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25cd7fcc15ccff0b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:42.242 INFO:tasks.workunit.client.0.smithi026.stdout: 10 289574 25.62 MB/sec execute 475 sec latency 2743.791 ms 2024-06-13T07:23:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.242+0000 7f074fe00640 1 -- 172.21.15.26:0/1416957208 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0744017070 con 0x7f0758104c70 2024-06-13T07:23:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.243+0000 7f0743e00640 1 -- 172.21.15.26:0/2255774075 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750041030 con 0x562b46211dc0 2024-06-13T07:23:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.243+0000 7f0743e00640 1 -- 172.21.15.26:0/2255774075 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0750003e50 con 0x562b46211dc0 2024-06-13T07:23:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.243+0000 7f0743e00640 1 -- 172.21.15.26:0/2255774075 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750041030 con 0x562b46211dc0 2024-06-13T07:23:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.244+0000 7f0760fde740 1 -- 172.21.15.26:0/2255774075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46211dc0 msgr2=0x562b462121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.245+0000 7f0760fde740 1 --2- 172.21.15.26:0/2255774075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46211dc0 0x562b462121e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0750009be0 tx=0x7f07500304a0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.246+0000 7f0760fde740 1 -- 172.21.15.26:0/2255774075 shutdown_connections 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.246+0000 7f0760fde740 1 --2- 172.21.15.26:0/2255774075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 0x562b46219db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.246+0000 7f0760fde740 1 --2- 172.21.15.26:0/2255774075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b462127b0 0x562b462172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.246+0000 7f0760fde740 1 --2- 172.21.15.26:0/2255774075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46211dc0 0x562b462121e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.246+0000 7f0760fde740 1 -- 172.21.15.26:0/2255774075 >> 172.21.15.26:0/2255774075 conn(0x562b461d9fb0 msgr2=0x562b4620d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.246+0000 7f0760fde740 1 -- 172.21.15.26:0/2255774075 shutdown_connections 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.247+0000 7f0760fde740 1 -- 172.21.15.26:0/2255774075 wait complete. 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.247+0000 7f0760fde740 1 -- start start 2024-06-13T07:23:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.248+0000 7f074fe00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f073c0a2bd0 0x7f073c0a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.248+0000 7f0760fde740 1 Processor -- start 2024-06-13T07:23:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.248+0000 7f0757e00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f073c0a2bd0 0x7f073c0a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.249+0000 7f0760fde740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46211dc0 0x562b4621c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.249+0000 7f0760fde740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b462127b0 0x562b4621c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.249+0000 7f0760fde740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46217920 0x562b46223630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.249+0000 7f0760fde740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562b4621a870 con 0x562b46211dc0 2024-06-13T07:23:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.249+0000 7f0760fde740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562b461300b0 con 0x562b462127b0 2024-06-13T07:23:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.249+0000 7f0760fde740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562b4621a9f0 con 0x562b46217920 2024-06-13T07:23:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.249+0000 7f0760fde740 10 client.0 _ll_register_callbacks cb 0x562b4611bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:23:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.250+0000 7f0757400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46211dc0 0x562b4621c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.250+0000 7f0757400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46211dc0 0x562b4621c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32788/0 (socket says 172.21.15.26:32788) 2024-06-13T07:23:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.250+0000 7f0757400640 1 -- 172.21.15.26:0/3114583137 learned_addr learned my addr 172.21.15.26:0/3114583137 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.250+0000 7f0757400640 1 -- 172.21.15.26:0/3114583137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46217920 msgr2=0x562b46223630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:23:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.250+0000 7f0757400640 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46217920 0x562b46223630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.250+0000 7f0757400640 1 -- 172.21.15.26:0/3114583137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b462127b0 msgr2=0x562b4621c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.250+0000 7f0757400640 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b462127b0 0x562b4621c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.250+0000 7f0757400640 1 -- 172.21.15.26:0/3114583137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b45f55620 con 0x562b46211dc0 2024-06-13T07:23:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.251+0000 7f0757400640 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46211dc0 0x562b4621c0b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0748011d10 tx=0x7f074800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d68bdf259a40b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.251+0000 7f0740c00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:23:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.252+0000 7f0740c00640 1 -- 172.21.15.26:0/3114583137 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748004050 con 0x562b46211dc0 2024-06-13T07:23:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.252+0000 7f0740c00640 1 -- 172.21.15.26:0/3114583137 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f07480041f0 con 0x562b46211dc0 2024-06-13T07:23:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.253+0000 7f0760fde740 1 -- 172.21.15.26:0/3114583137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0750003e50 con 0x562b46211dc0 2024-06-13T07:23:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.253+0000 7f074fe00640 1 -- 172.21.15.26:0/1416957208 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f07440cee00 con 0x7f0758104c70 2024-06-13T07:23:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.253+0000 7f0740c00640 1 -- 172.21.15.26:0/3114583137 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0748004410 con 0x562b46211dc0 2024-06-13T07:23:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.253+0000 7f0760fde740 10 client.16848 fetch_fsmap learned FSMap version 49 2024-06-13T07:23:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.253+0000 7f0760fde740 1 -- 172.21.15.26:0/3114583137 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562b46226a60 con 0x562b46211dc0 2024-06-13T07:23:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.254+0000 7f0740c00640 1 -- 172.21.15.26:0/3114583137 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0750003e50 con 0x562b46211dc0 2024-06-13T07:23:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.255+0000 7f0740c00640 1 -- 172.21.15.26:0/3114583137 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f07480053c0 con 0x562b46211dc0 2024-06-13T07:23:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.255+0000 7f0760fde740 10 client.16848 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:23:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.255+0000 7f0760fde740 10 client.16848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:23:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.255+0000 7f0760fde740 20 client.16848 populate_metadata read hostname 'smithi026' 2024-06-13T07:23:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.256+0000 7f0760fde740 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b46228150 0x562b4622a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.256+0000 7f0757e00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f073c0a2bd0 0x7f073c0a5090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0750030960 tx=0x7f075002a6c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.256+0000 7f0760fde740 4 client.16848 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:23:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.256+0000 7f0760fde740 1 -- 172.21.15.26:0/3114583137 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b46182b50 con 0x562b46228150 2024-06-13T07:23:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.256+0000 7f0756a00640 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b46228150 0x562b4622a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.259+0000 7f0756a00640 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b46228150 0x562b4622a5e0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a81a4f640e598a9 server_cookie=efb1ee565668f082 in_seq=0 out_seq=0 2024-06-13T07:23:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.260+0000 7f0740c00640 10 client.16848 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0740c00640 1 -- 172.21.15.26:0/3114583137 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b46182b50 con 0x562b46228150 2024-06-13T07:23:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0740c00640 10 client.16848 handle_command_reply: tid=0 2024-06-13T07:23:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 client.16848 shutdown 2024-06-13T07:23:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 -- 172.21.15.26:0/3114583137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46211dc0 msgr2=0x562b4621c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46211dc0 0x562b4621c0b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0748011d10 tx=0x7f074800f790 comp rx=0 tx=0).stop 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 -- 172.21.15.26:0/3114583137 shutdown_connections 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b46228150 0x562b4622a5e0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b462127b0 0x562b4621c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b46217920 0x562b46223630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 --2- 172.21.15.26:0/3114583137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46211dc0 0x562b4621c0b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.266+0000 7f0760fde740 1 -- 172.21.15.26:0/3114583137 >> 172.21.15.26:0/3114583137 conn(0x562b461d9fb0 msgr2=0x562b4620c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.267+0000 7f0740c00640 0 client.16848 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.267+0000 7f0760fde740 1 -- 172.21.15.26:0/3114583137 shutdown_connections 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.267+0000 7f0760fde740 1 -- 172.21.15.26:0/3114583137 wait complete. 2024-06-13T07:23:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.267+0000 7f0760fde740 20 client.16848 trim_cache size 0 max 16384 2024-06-13T07:23:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.302+0000 7f0760fde740 1 Processor -- start 2024-06-13T07:23:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.302+0000 7f0760fde740 1 -- start start 2024-06-13T07:23:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.302+0000 7f0760fde740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b4622a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.302+0000 7f0760fde740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 0x562b462235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.302+0000 7f0760fde740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b4620dbb0 0x562b46210ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0757400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b4622a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0757400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b4622a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:37032/0 (socket says 172.21.15.26:37032) 2024-06-13T07:23:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0756a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 0x562b462235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0756a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 0x562b462235f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:58352/0 (socket says 172.21.15.26:58352) 2024-06-13T07:23:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0757400640 1 -- 172.21.15.26:0/1007784773 learned_addr learned my addr 172.21.15.26:0/1007784773 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0760fde740 1 -- 172.21.15.26:0/1007784773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07581b34f0 con 0x562b4620dbb0 2024-06-13T07:23:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0760fde740 1 -- 172.21.15.26:0/1007784773 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07581b0d60 con 0x562b46217920 2024-06-13T07:23:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0760fde740 1 -- 172.21.15.26:0/1007784773 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07581aeb70 con 0x562b462127b0 2024-06-13T07:23:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.303+0000 7f0757e00640 1 --2- 172.21.15.26:0/1007784773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b4620dbb0 0x562b46210ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0757400640 1 -- 172.21.15.26:0/1007784773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 msgr2=0x562b462235f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0757400640 1 --2- 172.21.15.26:0/1007784773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 0x562b462235f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0757400640 1 -- 172.21.15.26:0/1007784773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b4620dbb0 msgr2=0x562b46210ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0757400640 1 --2- 172.21.15.26:0/1007784773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b4620dbb0 0x562b46210ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0757400640 1 -- 172.21.15.26:0/1007784773 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4620e4f0 con 0x562b462127b0 2024-06-13T07:23:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0757400640 1 --2- 172.21.15.26:0/1007784773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b4622a5a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f074800ffd0 tx=0x7f07480041f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d1216d4c8eb0f0c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0743400640 1 -- 172.21.15.26:0/1007784773 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748030020 con 0x562b462127b0 2024-06-13T07:23:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0743400640 1 -- 172.21.15.26:0/1007784773 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0748005520 con 0x562b462127b0 2024-06-13T07:23:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.304+0000 7f0743400640 1 -- 172.21.15.26:0/1007784773 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748000e60 con 0x562b462127b0 2024-06-13T07:23:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.305+0000 7f0760fde740 1 -- 172.21.15.26:0/1007784773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 msgr2=0x562b4622a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.305+0000 7f0760fde740 1 --2- 172.21.15.26:0/1007784773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b4622a5a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f074800ffd0 tx=0x7f07480041f0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.305+0000 7f0760fde740 1 -- 172.21.15.26:0/1007784773 shutdown_connections 2024-06-13T07:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.306+0000 7f0760fde740 1 --2- 172.21.15.26:0/1007784773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b46217920 0x562b462235f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.306+0000 7f0760fde740 1 --2- 172.21.15.26:0/1007784773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b4620dbb0 0x562b46210ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.306+0000 7f0760fde740 1 --2- 172.21.15.26:0/1007784773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b4622a5a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.306+0000 7f0760fde740 1 -- 172.21.15.26:0/1007784773 >> 172.21.15.26:0/1007784773 conn(0x562b461d9fb0 msgr2=0x562b462107d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.306+0000 7f0760fde740 1 -- 172.21.15.26:0/1007784773 shutdown_connections 2024-06-13T07:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.306+0000 7f0760fde740 1 -- 172.21.15.26:0/1007784773 wait complete. 2024-06-13T07:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.307+0000 7f0760fde740 1 -- start start 2024-06-13T07:23:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.307+0000 7f0760fde740 1 Processor -- start 2024-06-13T07:23:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.308+0000 7f0760fde740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b4620dbb0 0x562b46220970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.308+0000 7f0760fde740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b46220eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.308+0000 7f0760fde740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46217920 0x562b46221ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.309+0000 7f0757400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b4620dbb0 0x562b46220970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.309+0000 7f0757400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b4620dbb0 0x562b46220970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:43104/0 (socket says 172.21.15.26:43104) 2024-06-13T07:23:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.309+0000 7f0757e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46217920 0x562b46221ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.309+0000 7f0757400640 1 -- 172.21.15.26:0/3804983667 learned_addr learned my addr 172.21.15.26:0/3804983667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.309+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07581af220 con 0x562b46217920 2024-06-13T07:23:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.309+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07581b0690 con 0x562b4620dbb0 2024-06-13T07:23:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.309+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07581af8f0 con 0x562b462127b0 2024-06-13T07:23:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.310+0000 7f0760fde740 10 client.0 _ll_register_callbacks cb 0x7f0748009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:23:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.311+0000 7f0756a00640 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b46220eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.312+0000 7f0756a00640 1 -- 172.21.15.26:0/3804983667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b4620dbb0 msgr2=0x562b46220970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.312+0000 7f0756a00640 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b4620dbb0 0x562b46220970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.312+0000 7f0756a00640 1 -- 172.21.15.26:0/3804983667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46217920 msgr2=0x562b46221ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.312+0000 7f0756a00640 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46217920 0x562b46221ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.312+0000 7f0756a00640 1 -- 172.21.15.26:0/3804983667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b462224b0 con 0x562b462127b0 2024-06-13T07:23:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.312+0000 7f0757e00640 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46217920 0x562b46221ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:23:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.312+0000 7f0756a00640 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b46220eb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x562b4621d590 tx=0x7f074409ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cac12ab674b51cb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.313+0000 7f0740c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:23:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.313+0000 7f0740c00640 1 -- 172.21.15.26:0/3804983667 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074401e040 con 0x562b462127b0 2024-06-13T07:23:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.313+0000 7f0740c00640 1 -- 172.21.15.26:0/3804983667 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0744097880 con 0x562b462127b0 2024-06-13T07:23:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.313+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0748005520 con 0x562b462127b0 2024-06-13T07:23:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.313+0000 7f0740c00640 1 -- 172.21.15.26:0/3804983667 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f074409b210 con 0x562b462127b0 2024-06-13T07:23:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0760fde740 10 client.27196 fetch_fsmap learned FSMap version 49 2024-06-13T07:23:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562b46223890 con 0x562b462127b0 2024-06-13T07:23:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0740c00640 1 -- 172.21.15.26:0/3804983667 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0748005520 con 0x562b462127b0 2024-06-13T07:23:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0740c00640 1 -- 172.21.15.26:0/3804983667 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f07440520d0 con 0x562b462127b0 2024-06-13T07:23:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0760fde740 10 client.27196 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:23:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0760fde740 10 client.27196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:23:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0760fde740 20 client.27196 populate_metadata read hostname 'smithi026' 2024-06-13T07:23:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0760fde740 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b462935a0 0x562b462959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0760fde740 4 client.27196 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:23:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.314+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b4617cea0 con 0x562b462935a0 2024-06-13T07:23:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.317+0000 7f0757400640 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b462935a0 0x562b462959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.318+0000 7f0757400640 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b462935a0 0x562b462959c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c8e48191030513 server_cookie=52da42d023491bb5 in_seq=0 out_seq=0 2024-06-13T07:23:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:42.318+0000 7f0740c00640 10 client.27196 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:42.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:42 smithi026 ceph-mon[32041]: pgmap v1205: 129 pgs: 129 active+clean; 698 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.8 MiB/s wr, 439 op/s 2024-06-13T07:23:42.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:42 smithi135 ceph-mon[36276]: pgmap v1205: 129 pgs: 129 active+clean; 698 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.8 MiB/s wr, 439 op/s 2024-06-13T07:23:42.633 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:42 smithi136 ceph-mon[37229]: pgmap v1205: 129 pgs: 129 active+clean; 698 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.8 MiB/s wr, 439 op/s 2024-06-13T07:23:43.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 289644 25.58 MB/sec execute 476 sec latency 680.641 ms 2024-06-13T07:23:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.246+0000 7f0740c00640 1 -- 172.21.15.26:0/3804983667 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562b4617cea0 con 0x562b462935a0 2024-06-13T07:23:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.247+0000 7f0740c00640 10 client.27196 handle_command_reply: tid=0 2024-06-13T07:23:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.247+0000 7f0760fde740 1 client.27196 shutdown 2024-06-13T07:23:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.247+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 msgr2=0x562b46220eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.247+0000 7f0760fde740 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b46220eb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x562b4621d590 tx=0x7f074409ab70 comp rx=0 tx=0).stop 2024-06-13T07:23:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.247+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 shutdown_connections 2024-06-13T07:23:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.247+0000 7f0760fde740 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562b462935a0 0x562b462959c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.248+0000 7f0760fde740 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46217920 0x562b46221ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.248+0000 7f0740c00640 0 client.27196 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.248+0000 7f0760fde740 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562b462127b0 0x562b46220eb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.248+0000 7f0760fde740 1 --2- 172.21.15.26:0/3804983667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b4620dbb0 0x562b46220970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.248+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 >> 172.21.15.26:0/3804983667 conn(0x562b461d9fb0 msgr2=0x562b4620c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.248+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 shutdown_connections 2024-06-13T07:23:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.249+0000 7f0760fde740 1 -- 172.21.15.26:0/3804983667 wait complete. 2024-06-13T07:23:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.249+0000 7f0760fde740 20 client.27196 trim_cache size 0 max 16384 2024-06-13T07:23:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:23:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:23:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:23:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f073c0a2bd0 msgr2=0x7f073c0a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f073c0a2bd0 0x7f073c0a5090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0750030960 tx=0x7f075002a6c0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0758104c70 msgr2=0x7f075819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0758104c70 0x7f075819e5d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f074400c970 tx=0x7f074400cd70 comp rx=0 tx=0).stop 2024-06-13T07:23:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 shutdown_connections 2024-06-13T07:23:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f073c0a2bd0 0x7f073c0a5090 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0758104c70 0x7f075819e5d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07581075a0 0x7f07581a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 --2- 172.21.15.26:0/1416957208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0758102340 0x7f075819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.254+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 >> 172.21.15.26:0/1416957208 conn(0x7f07580fbc90 msgr2=0x7f0758102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.255+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 shutdown_connections 2024-06-13T07:23:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:43.255+0000 7f075cc00640 1 -- 172.21.15.26:0/1416957208 wait complete. 2024-06-13T07:23:44.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 289750 25.54 MB/sec execute 477 sec latency 309.876 ms 2024-06-13T07:23:44.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:44 smithi026 ceph-mon[32041]: pgmap v1206: 129 pgs: 129 active+clean; 732 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 419 op/s 2024-06-13T07:23:44.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:44 smithi135 ceph-mon[36276]: pgmap v1206: 129 pgs: 129 active+clean; 732 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 419 op/s 2024-06-13T07:23:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:44 smithi136 ceph-mon[37229]: pgmap v1206: 129 pgs: 129 active+clean; 732 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 419 op/s 2024-06-13T07:23:45.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 289899 25.51 MB/sec execute 478 sec latency 246.019 ms 2024-06-13T07:23:46.246 INFO:tasks.workunit.client.0.smithi026.stdout: 10 290042 25.47 MB/sec execute 479 sec latency 221.769 ms 2024-06-13T07:23:46.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:46 smithi135 ceph-mon[36276]: pgmap v1207: 129 pgs: 129 active+clean; 763 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-06-13T07:23:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:46 smithi136 ceph-mon[37229]: pgmap v1207: 129 pgs: 129 active+clean; 763 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-06-13T07:23:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:46 smithi026 ceph-mon[32041]: pgmap v1207: 129 pgs: 129 active+clean; 763 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-06-13T07:23:47.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 290241 25.44 MB/sec execute 480 sec latency 208.663 ms 2024-06-13T07:23:48.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 290559 25.41 MB/sec execute 481 sec latency 91.656 ms 2024-06-13T07:23:48.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:48 smithi135 ceph-mon[36276]: pgmap v1208: 129 pgs: 129 active+clean; 746 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 540 op/s 2024-06-13T07:23:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:48 smithi136 ceph-mon[37229]: pgmap v1208: 129 pgs: 129 active+clean; 746 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 540 op/s 2024-06-13T07:23:48.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:48 smithi026 ceph-mon[32041]: pgmap v1208: 129 pgs: 129 active+clean; 746 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 540 op/s 2024-06-13T07:23:49.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 290965 25.40 MB/sec execute 482 sec latency 92.119 ms 2024-06-13T07:23:50.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 291537 25.39 MB/sec execute 483 sec latency 123.410 ms 2024-06-13T07:23:50.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:50 smithi135 ceph-mon[36276]: pgmap v1209: 129 pgs: 129 active+clean; 693 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-06-13T07:23:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:50 smithi136 ceph-mon[37229]: pgmap v1209: 129 pgs: 129 active+clean; 693 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-06-13T07:23:50.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:50 smithi026 ceph-mon[32041]: pgmap v1209: 129 pgs: 129 active+clean; 693 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-06-13T07:23:51.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 292288 25.42 MB/sec execute 484 sec latency 85.299 ms 2024-06-13T07:23:52.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 293015 25.48 MB/sec execute 485 sec latency 59.790 ms 2024-06-13T07:23:52.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:52 smithi135 ceph-mon[36276]: pgmap v1210: 129 pgs: 129 active+clean; 689 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 600 op/s 2024-06-13T07:23:52.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:52 smithi136 ceph-mon[37229]: pgmap v1210: 129 pgs: 129 active+clean; 689 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 600 op/s 2024-06-13T07:23:52.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:52 smithi026 ceph-mon[32041]: pgmap v1210: 129 pgs: 129 active+clean; 689 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 600 op/s 2024-06-13T07:23:53.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 293597 25.49 MB/sec execute 486 sec latency 56.771 ms 2024-06-13T07:23:54.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 293860 25.46 MB/sec execute 487 sec latency 91.794 ms 2024-06-13T07:23:54.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:54 smithi135 ceph-mon[36276]: pgmap v1211: 129 pgs: 129 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-06-13T07:23:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:54 smithi136 ceph-mon[37229]: pgmap v1211: 129 pgs: 129 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-06-13T07:23:54.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:54 smithi026 ceph-mon[32041]: pgmap v1211: 129 pgs: 129 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-06-13T07:23:55.244 INFO:tasks.workunit.client.0.smithi026.stdout: 10 294272 25.45 MB/sec execute 488 sec latency 78.466 ms 2024-06-13T07:23:56.243 INFO:tasks.workunit.client.0.smithi026.stdout: 10 294705 25.43 MB/sec execute 489 sec latency 124.080 ms 2024-06-13T07:23:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.484+0000 7f1ce4a00640 1 -- 172.21.15.26:0/3720295655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce0101860 msgr2=0x7f1ce0101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.484+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/3720295655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce0101860 0x7f1ce0101c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009ca0 tx=0x7f1cd002a8f0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.484+0000 7f1ce4a00640 1 -- 172.21.15.26:0/3720295655 shutdown_connections 2024-06-13T07:23:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.484+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/3720295655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce0102a50 0x7f1ce0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.484+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/3720295655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce0103410 0x7f1ce0109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.484+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/3720295655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce0101860 0x7f1ce0101c80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.484+0000 7f1ce4a00640 1 -- 172.21.15.26:0/3720295655 >> 172.21.15.26:0/3720295655 conn(0x7f1ce00fcdd0 msgr2=0x7f1ce00ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.484+0000 7f1ce4a00640 1 -- 172.21.15.26:0/3720295655 shutdown_connections 2024-06-13T07:23:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.485+0000 7f1ce4a00640 1 -- 172.21.15.26:0/3720295655 wait complete. 2024-06-13T07:23:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.486+0000 7f1ce4a00640 1 Processor -- start 2024-06-13T07:23:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.486+0000 7f1ce4a00640 1 -- start start 2024-06-13T07:23:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.487+0000 7f1ce4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce0101860 0x7f1ce0075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.487+0000 7f1ce4a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce0102a50 0x7f1ce0073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.487+0000 7f1ce4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce0103410 0x7f1ce00742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.487+0000 7f1ce4a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce010d580 con 0x7f1ce0101860 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.487+0000 7f1ce4a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce010d400 con 0x7f1ce0102a50 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.487+0000 7f1ce4a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce010d700 con 0x7f1ce0103410 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.487+0000 7f1cdea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce0102a50 0x7f1ce0073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce0102a50 0x7f1ce0073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:40812/0 (socket says 172.21.15.26:40812) 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdea00640 1 -- 172.21.15.26:0/2125564960 learned_addr learned my addr 172.21.15.26:0/2125564960 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdf400640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce0101860 0x7f1ce0075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdfe00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce0103410 0x7f1ce00742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdea00640 1 -- 172.21.15.26:0/2125564960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce0103410 msgr2=0x7f1ce00742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdea00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce0103410 0x7f1ce00742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdea00640 1 -- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce0101860 msgr2=0x7f1ce0075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdea00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce0101860 0x7f1ce0075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdea00640 1 -- 172.21.15.26:0/2125564960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce0074b60 con 0x7f1ce0102a50 2024-06-13T07:23:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.488+0000 7f1cdf400640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce0101860 0x7f1ce0075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:23:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.489+0000 7f1cdea00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce0102a50 0x7f1ce0073d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00d950 tx=0x7f1ccc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.490+0000 7f1cd7e00640 1 -- 172.21.15.26:0/2125564960 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc017070 con 0x7f1ce0102a50 2024-06-13T07:23:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.490+0000 7f1ce4a00640 1 -- 172.21.15.26:0/2125564960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce01a1cc0 con 0x7f1ce0102a50 2024-06-13T07:23:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.490+0000 7f1ce4a00640 1 -- 172.21.15.26:0/2125564960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce01a2230 con 0x7f1ce0102a50 2024-06-13T07:23:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.490+0000 7f1cd7e00640 1 -- 172.21.15.26:0/2125564960 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ccc009c30 con 0x7f1ce0102a50 2024-06-13T07:23:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.490+0000 7f1cd7e00640 1 -- 172.21.15.26:0/2125564960 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc00f6b0 con 0x7f1ce0102a50 2024-06-13T07:23:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.500+0000 7f1ce8e0e740 1 Processor -- start 2024-06-13T07:23:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.501+0000 7f1cd7e00640 1 -- 172.21.15.26:0/2125564960 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ccc00f850 con 0x7f1ce0102a50 2024-06-13T07:23:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.501+0000 7f1ce8e0e740 1 -- start start 2024-06-13T07:23:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.502+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53ac99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.502+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 0x562c53aceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.502+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ad15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.502+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562c53a24420 con 0x562c53ac95d0 2024-06-13T07:23:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.502+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562c539cbc50 con 0x562c53acf130 2024-06-13T07:23:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.502+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562c53a203e0 con 0x562c53ac9fc0 2024-06-13T07:23:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.503+0000 7f1cdea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ad15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.503+0000 7f1cdea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ad15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:40814/0 (socket says 172.21.15.26:40814) 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.503+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 0x562c53aceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.503+0000 7f1cdea00640 1 -- 172.21.15.26:0/2982701844 learned_addr learned my addr 172.21.15.26:0/2982701844 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.503+0000 7f1cdfe00640 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53ac99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.504+0000 7f1cdf400640 1 -- 172.21.15.26:0/2982701844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 msgr2=0x562c53ad15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.504+0000 7f1cdf400640 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ad15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.504+0000 7f1cdf400640 1 -- 172.21.15.26:0/2982701844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 msgr2=0x562c53ac99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.504+0000 7f1cdf400640 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53ac99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.504+0000 7f1cdf400640 1 -- 172.21.15.26:0/2982701844 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c53738210 con 0x562c53ac9fc0 2024-06-13T07:23:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.504+0000 7f1cdfe00640 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53ac99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:23:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.505+0000 7f1cd7e00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1cc40a2c00 0x7f1cc40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.505+0000 7f1cd7e00640 1 -- 172.21.15.26:0/2125564960 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1ccc0cee30 con 0x7f1ce0102a50 2024-06-13T07:23:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.505+0000 7f1cdf400640 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 0x562c53aceac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0031b80 tx=0x7f1cd00098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3162e046df59df7b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.505+0000 7f1cdfe00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1cc40a2c00 0x7f1cc40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.505+0000 7f1ccbe00640 1 -- 172.21.15.26:0/2982701844 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0031740 con 0x562c53ac9fc0 2024-06-13T07:23:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.506+0000 7f1ccbe00640 1 -- 172.21.15.26:0/2982701844 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd0036040 con 0x562c53ac9fc0 2024-06-13T07:23:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.506+0000 7f1ccbe00640 1 -- 172.21.15.26:0/2982701844 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd002f640 con 0x562c53ac9fc0 2024-06-13T07:23:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/2982701844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 msgr2=0x562c53aceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1cdfe00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1cc40a2c00 0x7f1cc40a50c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x562c53aca820 tx=0x7f1cd8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 0x562c53aceac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0031b80 tx=0x7f1cd00098e0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/2982701844 shutdown_connections 2024-06-13T07:23:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ad15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 0x562c53aceac0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/2982701844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53ac99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/2982701844 >> 172.21.15.26:0/2982701844 conn(0x562c53a88fb0 msgr2=0x562c53ac5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/2982701844 shutdown_connections 2024-06-13T07:23:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.507+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/2982701844 wait complete. 2024-06-13T07:23:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.508+0000 7f1ce8e0e740 1 -- start start 2024-06-13T07:23:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.508+0000 7f1ce8e0e740 1 Processor -- start 2024-06-13T07:23:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53abdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 0x562c53abe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ac3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562c53abc590 con 0x562c53ac95d0 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562c539df0b0 con 0x562c53acf130 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562c53abc710 con 0x562c53ac9fc0 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1ce8e0e740 10 client.0 _ll_register_callbacks cb 0x562c539caf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1cdfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ac3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53abdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1cdfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ac3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:40830/0 (socket says 172.21.15.26:40830) 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.509+0000 7f1cdfe00640 1 -- 172.21.15.26:0/1150366607 learned_addr learned my addr 172.21.15.26:0/1150366607 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.510+0000 7f1cdf400640 1 -- 172.21.15.26:0/1150366607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 msgr2=0x562c53abe3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.510+0000 7f1cdf400640 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 0x562c53abe3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.510+0000 7f1cdf400640 1 -- 172.21.15.26:0/1150366607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 msgr2=0x562c53ac3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.510+0000 7f1cdf400640 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ac3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.510+0000 7f1cdf400640 1 -- 172.21.15.26:0/1150366607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c53804620 con 0x562c53ac95d0 2024-06-13T07:23:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.511+0000 7f1cdf400640 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53abdea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562c53ac5f00 tx=0x7f1cd0031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bcce545c2caf1a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.511+0000 7f1cc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:23:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.511+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1150366607 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd002f910 con 0x562c53ac95d0 2024-06-13T07:23:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.511+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1150366607 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1cd0055060 con 0x562c53ac95d0 2024-06-13T07:23:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.512+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1150366607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd0036040 con 0x562c53ac95d0 2024-06-13T07:23:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.512+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1150366607 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1cd002f910 con 0x562c53ac95d0 2024-06-13T07:23:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.512+0000 7f1ce8e0e740 10 client.16860 fetch_fsmap learned FSMap version 49 2024-06-13T07:23:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.512+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1150366607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562c53abc590 con 0x562c53ac95d0 2024-06-13T07:23:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.513+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1150366607 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd002d070 con 0x562c53ac95d0 2024-06-13T07:23:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.513+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1150366607 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f1cd004f070 con 0x562c53ac95d0 2024-06-13T07:23:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.513+0000 7f1ce8e0e740 10 client.16860 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:23:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.513+0000 7f1ce8e0e740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:23:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.513+0000 7f1ce8e0e740 20 client.16860 populate_metadata read hostname 'smithi026' 2024-06-13T07:23:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.513+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562c53ae7c10 0x562c53aea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.513+0000 7f1ce8e0e740 4 client.16860 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:23:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.513+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1150366607 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1cd0036040 con 0x562c53ae7c10 2024-06-13T07:23:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.516+0000 7f1cdea00640 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562c53ae7c10 0x562c53aea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.517+0000 7f1cdea00640 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562c53ae7c10 0x562c53aea090 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2aa93f22f0785e server_cookie=16f02e790935b328 in_seq=0 out_seq=0 2024-06-13T07:23:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.517+0000 7f1cc8c00640 10 client.16860 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.523+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1150366607 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f1cd0036040 con 0x562c53ae7c10 2024-06-13T07:23:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.523+0000 7f1cc8c00640 10 client.16860 handle_command_reply: tid=0 2024-06-13T07:23:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.523+0000 7f1ce8e0e740 1 client.16860 shutdown 2024-06-13T07:23:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.523+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1150366607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 msgr2=0x562c53abdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.523+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53abdea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562c53ac5f00 tx=0x7f1cd0031840 comp rx=0 tx=0).stop 2024-06-13T07:23:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.524+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1150366607 shutdown_connections 2024-06-13T07:23:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.524+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562c53ae7c10 0x562c53aea090 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.524+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53acf130 0x562c53ac3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.524+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac9fc0 0x562c53abe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.524+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1150366607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53ac95d0 0x562c53abdea0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.524+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1150366607 >> 172.21.15.26:0/1150366607 conn(0x562c53a88fb0 msgr2=0x562c53abb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.524+0000 7f1cc8c00640 0 client.16860 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.525+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1150366607 shutdown_connections 2024-06-13T07:23:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.525+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1150366607 wait complete. 2024-06-13T07:23:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.525+0000 7f1ce8e0e740 20 client.16860 trim_cache size 0 max 16384 2024-06-13T07:23:56.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:56 smithi135 ceph-mon[36276]: pgmap v1212: 129 pgs: 129 active+clean; 685 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.8 MiB/s wr, 612 op/s 2024-06-13T07:23:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.557+0000 7f1ce8e0e740 1 Processor -- start 2024-06-13T07:23:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.558+0000 7f1ce8e0e740 1 -- start start 2024-06-13T07:23:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.558+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53ac8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.558+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53abd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.558+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 0x562c53abda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.558+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0012720 con 0x562c53acf130 2024-06-13T07:23:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.558+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000ff90 con 0x562c53ac9fc0 2024-06-13T07:23:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.558+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000dde0 con 0x562c53ac95d0 2024-06-13T07:23:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.559+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53ac8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.559+0000 7f1cdea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53abd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.559+0000 7f1cdfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 0x562c53abda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.559+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53ac8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58908/0 (socket says 172.21.15.26:58908) 2024-06-13T07:23:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.559+0000 7f1cdf400640 1 -- 172.21.15.26:0/886875584 learned_addr learned my addr 172.21.15.26:0/886875584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.560+0000 7f1cdf400640 1 -- 172.21.15.26:0/886875584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 msgr2=0x562c53abd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.560+0000 7f1cdf400640 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53abd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.560+0000 7f1cdf400640 1 -- 172.21.15.26:0/886875584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 msgr2=0x562c53abda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.560+0000 7f1cdf400640 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 0x562c53abda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.560+0000 7f1cdf400640 1 -- 172.21.15.26:0/886875584 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c53acee10 con 0x562c53ac95d0 2024-06-13T07:23:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.560+0000 7f1cdfe00640 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 0x562c53abda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:23:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.560+0000 7f1cdea00640 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53abd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:23:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.561+0000 7f1cdf400640 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53ac8650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1cd002ae10 tx=0x7f1cd0030f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deec4296d661f9cd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.561+0000 7f1ccb400640 1 -- 172.21.15.26:0/886875584 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0040070 con 0x562c53ac95d0 2024-06-13T07:23:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.561+0000 7f1ccb400640 1 -- 172.21.15.26:0/886875584 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd0036040 con 0x562c53ac95d0 2024-06-13T07:23:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.561+0000 7f1ccb400640 1 -- 172.21.15.26:0/886875584 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00305f0 con 0x562c53ac95d0 2024-06-13T07:23:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.562+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/886875584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 msgr2=0x562c53ac8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.562+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53ac8650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1cd002ae10 tx=0x7f1cd0030f70 comp rx=0 tx=0).stop 2024-06-13T07:23:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.562+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/886875584 shutdown_connections 2024-06-13T07:23:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.562+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53abd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.562+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 0x562c53abda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.562+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/886875584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53ac8650 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.562+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/886875584 >> 172.21.15.26:0/886875584 conn(0x562c53a88fb0 msgr2=0x562c53ac7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.563+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/886875584 shutdown_connections 2024-06-13T07:23:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.563+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/886875584 wait complete. 2024-06-13T07:23:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.563+0000 7f1ce8e0e740 1 -- start start 2024-06-13T07:23:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.563+0000 7f1ce8e0e740 1 Processor -- start 2024-06-13T07:23:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53af9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53af9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1ce8e0e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 0x562c53afacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000e450 con 0x562c53acf130 2024-06-13T07:23:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000f8c0 con 0x562c53ac9fc0 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1ce8e0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000eb20 con 0x562c53ac95d0 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1ce8e0e740 10 client.0 _ll_register_callbacks cb 0x7f1cd0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53af9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53af9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:58916/0 (socket says 172.21.15.26:58916) 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.564+0000 7f1cdf400640 1 -- 172.21.15.26:0/1802114671 learned_addr learned my addr 172.21.15.26:0/1802114671 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.565+0000 7f1cdea00640 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53af9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.565+0000 7f1cdf400640 1 -- 172.21.15.26:0/1802114671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 msgr2=0x562c53af9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.565+0000 7f1cdf400640 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53af9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.565+0000 7f1cdf400640 1 -- 172.21.15.26:0/1802114671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 msgr2=0x562c53afacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:23:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.565+0000 7f1cdf400640 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 0x562c53afacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.565+0000 7f1cdf400640 1 -- 172.21.15.26:0/1802114671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c53abc120 con 0x562c53ac95d0 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.565+0000 7f1cdea00640 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53af9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.566+0000 7f1cdf400640 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53af9760 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0031040 tx=0x7f1cd002a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a3339b9c97de61 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.566+0000 7f1cc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.566+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1802114671 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0040040 con 0x562c53ac95d0 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.566+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1802114671 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1cd00305f0 con 0x562c53ac95d0 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.566+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1802114671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd0036040 con 0x562c53ac95d0 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.567+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1802114671 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1cd0031820 con 0x562c53ac95d0 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.567+0000 7f1ce8e0e740 10 client.27211 fetch_fsmap learned FSMap version 49 2024-06-13T07:23:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.567+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1802114671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562c53b637c0 con 0x562c53ac95d0 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.567+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1802114671 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd0036040 con 0x562c53ac95d0 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.567+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1802114671 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f1cd0044070 con 0x562c53ac95d0 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.568+0000 7f1ce8e0e740 10 client.27211 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.568+0000 7f1ce8e0e740 10 client.27211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.568+0000 7f1ce8e0e740 20 client.27211 populate_metadata read hostname 'smithi026' 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.568+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562c53b63d60 0x562c53b661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.568+0000 7f1ce8e0e740 4 client.27211 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.568+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1802114671 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c53a31b50 con 0x562c53b63d60 2024-06-13T07:23:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.568+0000 7f1cdea00640 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562c53b63d60 0x562c53b661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:23:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.569+0000 7f1cdea00640 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562c53b63d60 0x562c53b661a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5404e18ec8d6bd1e server_cookie=ceaa75b177833e49 in_seq=0 out_seq=0 2024-06-13T07:23:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:56.569+0000 7f1cc8c00640 10 client.27211 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:56 smithi136 ceph-mon[37229]: pgmap v1212: 129 pgs: 129 active+clean; 685 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.8 MiB/s wr, 612 op/s 2024-06-13T07:23:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:56 smithi026 ceph-mon[32041]: pgmap v1212: 129 pgs: 129 active+clean; 685 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.8 MiB/s wr, 612 op/s 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1cc8c00640 1 -- 172.21.15.26:0/1802114671 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562c53a31b50 con 0x562c53b63d60 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1cc8c00640 10 client.27211 handle_command_reply: tid=0 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 client.27211 shutdown 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1802114671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 msgr2=0x562c53af9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53af9760 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0031040 tx=0x7f1cd002a8f0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1802114671 shutdown_connections 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x562c53b63d60 0x562c53b661a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c53ac9fc0 0x562c53af9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c53acf130 0x562c53afacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 --2- 172.21.15.26:0/1802114671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c53ac95d0 0x562c53af9760 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.218+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1802114671 >> 172.21.15.26:0/1802114671 conn(0x562c53a88fb0 msgr2=0x562c53abb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.219+0000 7f1cc8c00640 0 client.27211 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:23:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.219+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1802114671 shutdown_connections 2024-06-13T07:23:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.219+0000 7f1ce8e0e740 1 -- 172.21.15.26:0/1802114671 wait complete. 2024-06-13T07:23:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.219+0000 7f1ce8e0e740 20 client.27211 trim_cache size 0 max 16384 2024-06-13T07:23:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:23:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:23:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:23:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 -- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1cc40a2c00 msgr2=0x7f1cc40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1cc40a2c00 0x7f1cc40a50c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x562c53aca820 tx=0x7f1cd8008040 comp rx=0 tx=0).stop 2024-06-13T07:23:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 -- 172.21.15.26:0/2125564960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce0102a50 msgr2=0x7f1ce0073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:23:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce0102a50 0x7f1ce0073d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00d950 tx=0x7f1ccc00de10 comp rx=0 tx=0).stop 2024-06-13T07:23:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 -- 172.21.15.26:0/2125564960 shutdown_connections 2024-06-13T07:23:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f1cc40a2c00 0x7f1cc40a50c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce0102a50 0x7f1ce0073d90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce0103410 0x7f1ce00742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 --2- 172.21.15.26:0/2125564960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce0101860 0x7f1ce0075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:23:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.222+0000 7f1ce4a00640 1 -- 172.21.15.26:0/2125564960 >> 172.21.15.26:0/2125564960 conn(0x7f1ce00fcdd0 msgr2=0x7f1ce0105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:23:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.223+0000 7f1ce4a00640 1 -- 172.21.15.26:0/2125564960 shutdown_connections 2024-06-13T07:23:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:23:57.223+0000 7f1ce4a00640 1 -- 172.21.15.26:0/2125564960 wait complete. 2024-06-13T07:23:57.244 INFO:tasks.workunit.client.0.smithi026.stdout: 10 295064 25.41 MB/sec execute 490 sec latency 431.477 ms 2024-06-13T07:23:58.244 INFO:tasks.workunit.client.0.smithi026.stdout: 10 295717 25.44 MB/sec execute 491 sec latency 106.411 ms 2024-06-13T07:23:58.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:23:58 smithi135 ceph-mon[36276]: pgmap v1213: 129 pgs: 129 active+clean; 679 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 449 op/s 2024-06-13T07:23:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:23:58 smithi136 ceph-mon[37229]: pgmap v1213: 129 pgs: 129 active+clean; 679 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 449 op/s 2024-06-13T07:23:58.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:23:58 smithi026 ceph-mon[32041]: pgmap v1213: 129 pgs: 129 active+clean; 679 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 449 op/s 2024-06-13T07:23:59.244 INFO:tasks.workunit.client.0.smithi026.stdout: 10 295996 25.43 MB/sec execute 492 sec latency 633.469 ms 2024-06-13T07:24:00.244 INFO:tasks.workunit.client.0.smithi026.stdout: 10 296096 25.39 MB/sec execute 493 sec latency 1070.548 ms 2024-06-13T07:24:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:00 smithi135 ceph-mon[36276]: pgmap v1214: 129 pgs: 129 active+clean; 675 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-06-13T07:24:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:00 smithi136 ceph-mon[37229]: pgmap v1214: 129 pgs: 129 active+clean; 675 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-06-13T07:24:00.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:00 smithi026 ceph-mon[32041]: pgmap v1214: 129 pgs: 129 active+clean; 675 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-06-13T07:24:01.245 INFO:tasks.workunit.client.0.smithi026.stdout: 10 296303 25.36 MB/sec execute 494 sec latency 588.792 ms 2024-06-13T07:24:02.245 INFO:tasks.workunit.client.0.smithi026.stdout: 10 296459 25.32 MB/sec execute 495 sec latency 348.093 ms 2024-06-13T07:24:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:02 smithi136 ceph-mon[37229]: pgmap v1215: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.1 MiB/s wr, 442 op/s 2024-06-13T07:24:02.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:02 smithi026 ceph-mon[32041]: pgmap v1215: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.1 MiB/s wr, 442 op/s 2024-06-13T07:24:02.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:02 smithi135 ceph-mon[36276]: pgmap v1215: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.1 MiB/s wr, 442 op/s 2024-06-13T07:24:03.246 INFO:tasks.workunit.client.0.smithi026.stdout: 10 296942 25.33 MB/sec execute 496 sec latency 105.373 ms 2024-06-13T07:24:04.246 INFO:tasks.workunit.client.0.smithi026.stdout: 10 297340 25.31 MB/sec execute 497 sec latency 124.393 ms 2024-06-13T07:24:04.555 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:04 smithi026 ceph-mon[32041]: pgmap v1216: 129 pgs: 129 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.1 MiB/s wr, 496 op/s 2024-06-13T07:24:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:04 smithi136 ceph-mon[37229]: pgmap v1216: 129 pgs: 129 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.1 MiB/s wr, 496 op/s 2024-06-13T07:24:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:04 smithi135 ceph-mon[36276]: pgmap v1216: 129 pgs: 129 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.1 MiB/s wr, 496 op/s 2024-06-13T07:24:05.247 INFO:tasks.workunit.client.0.smithi026.stdout: 10 297752 25.32 MB/sec execute 498 sec latency 195.635 ms 2024-06-13T07:24:06.248 INFO:tasks.workunit.client.0.smithi026.stdout: 10 298269 25.32 MB/sec execute 499 sec latency 79.723 ms 2024-06-13T07:24:06.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:06 smithi136 ceph-mon[37229]: pgmap v1217: 129 pgs: 129 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.2 MiB/s wr, 555 op/s 2024-06-13T07:24:06.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:06 smithi026 ceph-mon[32041]: pgmap v1217: 129 pgs: 129 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.2 MiB/s wr, 555 op/s 2024-06-13T07:24:06.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:06 smithi135 ceph-mon[36276]: pgmap v1217: 129 pgs: 129 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.2 MiB/s wr, 555 op/s 2024-06-13T07:24:07.248 INFO:tasks.workunit.client.0.smithi026.stdout: 10 298815 25.32 MB/sec execute 500 sec latency 128.068 ms 2024-06-13T07:24:08.248 INFO:tasks.workunit.client.0.smithi026.stdout: 10 299484 25.35 MB/sec execute 501 sec latency 76.128 ms 2024-06-13T07:24:08.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:08 smithi136 ceph-mon[37229]: pgmap v1218: 129 pgs: 129 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.9 MiB/s wr, 536 op/s 2024-06-13T07:24:08.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:08 smithi026 ceph-mon[32041]: pgmap v1218: 129 pgs: 129 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.9 MiB/s wr, 536 op/s 2024-06-13T07:24:08.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:08 smithi135 ceph-mon[36276]: pgmap v1218: 129 pgs: 129 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.9 MiB/s wr, 536 op/s 2024-06-13T07:24:09.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 300082 25.37 MB/sec execute 502 sec latency 75.899 ms 2024-06-13T07:24:10.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 300477 25.35 MB/sec execute 503 sec latency 109.508 ms 2024-06-13T07:24:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.499+0000 7f9901400640 1 -- 172.21.15.26:0/4274982014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc110220 msgr2=0x7f98fc112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.499+0000 7f9901400640 1 --2- 172.21.15.26:0/4274982014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc110220 0x7f98fc112610 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009ca0 tx=0x7f98f402a8c0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.500+0000 7f9901400640 1 -- 172.21.15.26:0/4274982014 shutdown_connections 2024-06-13T07:24:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.500+0000 7f9901400640 1 --2- 172.21.15.26:0/4274982014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc110220 0x7f98fc112610 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.500+0000 7f9901400640 1 --2- 172.21.15.26:0/4274982014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98fc109300 0x7f98fc10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.500+0000 7f9901400640 1 --2- 172.21.15.26:0/4274982014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98fc1089a0 0x7f98fc108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.500+0000 7f9901400640 1 -- 172.21.15.26:0/4274982014 >> 172.21.15.26:0/4274982014 conn(0x7f98fc06ea10 msgr2=0x7f98fc070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.500+0000 7f9901400640 1 -- 172.21.15.26:0/4274982014 shutdown_connections 2024-06-13T07:24:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.500+0000 7f9901400640 1 -- 172.21.15.26:0/4274982014 wait complete. 2024-06-13T07:24:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.502+0000 7f9901400640 1 Processor -- start 2024-06-13T07:24:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.502+0000 7f9901400640 1 -- start start 2024-06-13T07:24:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.502+0000 7f9901400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98fc1089a0 0x7f98fc1a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.502+0000 7f9901400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc109300 0x7f98fc1a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.503+0000 7f9901400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98fc110220 0x7f98fc1a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.503+0000 7f9901400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc115ee0 con 0x7f98fc110220 2024-06-13T07:24:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.503+0000 7f9901400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc115d60 con 0x7f98fc1089a0 2024-06-13T07:24:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.503+0000 7f9901400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc116060 con 0x7f98fc109300 2024-06-13T07:24:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.503+0000 7f98f1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc109300 0x7f98fc1a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.503+0000 7f98f1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc109300 0x7f98fc1a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52728/0 (socket says 172.21.15.26:52728) 2024-06-13T07:24:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.503+0000 7f98f1e00640 1 -- 172.21.15.26:0/1620885237 learned_addr learned my addr 172.21.15.26:0/1620885237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.503+0000 7f98fa800640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98fc1089a0 0x7f98fc1a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.504+0000 7f98fa800640 1 -- 172.21.15.26:0/1620885237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc109300 msgr2=0x7f98fc1a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.504+0000 7f98fa800640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc109300 0x7f98fc1a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.504+0000 7f98fa800640 1 -- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98fc110220 msgr2=0x7f98fc1a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.504+0000 7f98fa800640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98fc110220 0x7f98fc1a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.504+0000 7f98fa800640 1 -- 172.21.15.26:0/1620885237 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc1aa130 con 0x7f98fc1089a0 2024-06-13T07:24:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.504+0000 7f98fa800640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98fc1089a0 0x7f98fc1a2510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c970 tx=0x7f98ec00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.505+0000 7f98f3e00640 1 -- 172.21.15.26:0/1620885237 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec007bb0 con 0x7f98fc1089a0 2024-06-13T07:24:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.505+0000 7f9901400640 1 -- 172.21.15.26:0/1620885237 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc1aa420 con 0x7f98fc1089a0 2024-06-13T07:24:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.505+0000 7f9901400640 1 -- 172.21.15.26:0/1620885237 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc1aa9c0 con 0x7f98fc1089a0 2024-06-13T07:24:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.505+0000 7f98f3e00640 1 -- 172.21.15.26:0/1620885237 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ec007d50 con 0x7f98fc1089a0 2024-06-13T07:24:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.505+0000 7f98f3e00640 1 -- 172.21.15.26:0/1620885237 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec007bb0 con 0x7f98fc1089a0 2024-06-13T07:24:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.510+0000 7f99041dc740 1 Processor -- start 2024-06-13T07:24:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.512+0000 7f99041dc740 1 -- start start 2024-06-13T07:24:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.512+0000 7f99041dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533d93020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.512+0000 7f99041dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d935f0 0x557533d980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.512+0000 7f99041dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 0x557533d9abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.512+0000 7f99041dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557533cf6420 con 0x557533d98760 2024-06-13T07:24:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.512+0000 7f99041dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557533c9dc50 con 0x557533d92c00 2024-06-13T07:24:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.512+0000 7f99041dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557533cf23e0 con 0x557533d935f0 2024-06-13T07:24:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.513+0000 7f98f3e00640 1 -- 172.21.15.26:0/1620885237 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f98ec00f450 con 0x7f98fc1089a0 2024-06-13T07:24:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.513+0000 7f98fb200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533d93020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.513+0000 7f98fb200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533d93020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:42136/0 (socket says 172.21.15.26:42136) 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.513+0000 7f98fb200640 1 -- 172.21.15.26:0/3477987881 learned_addr learned my addr 172.21.15.26:0/3477987881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.513+0000 7f98f1e00640 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 0x557533d9abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.514+0000 7f98fb200640 1 -- 172.21.15.26:0/3477987881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d935f0 msgr2=0x557533d980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.514+0000 7f98fb200640 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d935f0 0x557533d980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.514+0000 7f98fb200640 1 -- 172.21.15.26:0/3477987881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 msgr2=0x557533d9abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.514+0000 7f98fb200640 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 0x557533d9abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.514+0000 7f98fb200640 1 -- 172.21.15.26:0/3477987881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557533a0a210 con 0x557533d92c00 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.514+0000 7f98f1e00640 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 0x557533d9abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:24:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.515+0000 7f98fb200640 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533d93020 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f98f4005e30 tx=0x7f98f4005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbe01ecdfe90260b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.516+0000 7f98f3e00640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98e00a2ec0 0x7f98e00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.516+0000 7f98eb400640 1 -- 172.21.15.26:0/3477987881 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4031650 con 0x557533d92c00 2024-06-13T07:24:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.517+0000 7f98eb400640 1 -- 172.21.15.26:0/3477987881 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f403f070 con 0x557533d92c00 2024-06-13T07:24:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.517+0000 7f98eb400640 1 -- 172.21.15.26:0/3477987881 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4004780 con 0x557533d92c00 2024-06-13T07:24:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.517+0000 7f98fa800640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98e00a2ec0 0x7f98e00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.518+0000 7f99041dc740 1 -- 172.21.15.26:0/3477987881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 msgr2=0x557533d93020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.518+0000 7f99041dc740 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533d93020 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f98f4005e30 tx=0x7f98f4005fa0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.518+0000 7f98f3e00640 1 -- 172.21.15.26:0/1620885237 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f98ec0cf3a0 con 0x7f98fc1089a0 2024-06-13T07:24:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.520+0000 7f99041dc740 1 -- 172.21.15.26:0/3477987881 shutdown_connections 2024-06-13T07:24:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.520+0000 7f99041dc740 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 0x557533d9abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.520+0000 7f99041dc740 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d935f0 0x557533d980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.520+0000 7f99041dc740 1 --2- 172.21.15.26:0/3477987881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533d93020 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.520+0000 7f99041dc740 1 -- 172.21.15.26:0/3477987881 >> 172.21.15.26:0/3477987881 conn(0x557533d5afb0 msgr2=0x557533d8e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.520+0000 7f98fa800640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98e00a2ec0 0x7f98e00a5380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x557533d939f0 tx=0x7f98ec00fa10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.521+0000 7f99041dc740 1 -- 172.21.15.26:0/3477987881 shutdown_connections 2024-06-13T07:24:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.521+0000 7f99041dc740 1 -- 172.21.15.26:0/3477987881 wait complete. 2024-06-13T07:24:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.521+0000 7f99041dc740 1 -- start start 2024-06-13T07:24:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.521+0000 7f99041dc740 1 Processor -- start 2024-06-13T07:24:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.522+0000 7f99041dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d92c00 0x557533d9cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.522+0000 7f99041dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533d9d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.522+0000 7f99041dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 0x557533da44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.523+0000 7f99041dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557533d9b6b0 con 0x557533d98760 2024-06-13T07:24:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.523+0000 7f99041dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557533cb10b0 con 0x557533d935f0 2024-06-13T07:24:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.523+0000 7f99041dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557533d9b830 con 0x557533d92c00 2024-06-13T07:24:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.523+0000 7f99041dc740 10 client.0 _ll_register_callbacks cb 0x557533c9cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:24:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.523+0000 7f98fb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d92c00 0x557533d9cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.523+0000 7f98fb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d92c00 0x557533d9cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52740/0 (socket says 172.21.15.26:52740) 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.523+0000 7f98fb200640 1 -- 172.21.15.26:0/218543708 learned_addr learned my addr 172.21.15.26:0/218543708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.523+0000 7f98f1e00640 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533d9d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.524+0000 7f98fb200640 1 -- 172.21.15.26:0/218543708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 msgr2=0x557533d9d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.524+0000 7f98fb200640 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533d9d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.524+0000 7f98fb200640 1 -- 172.21.15.26:0/218543708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 msgr2=0x557533da44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.524+0000 7f98fb200640 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 0x557533da44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.524+0000 7f98fb200640 1 -- 172.21.15.26:0/218543708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557533ad6620 con 0x557533d92c00 2024-06-13T07:24:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.524+0000 7f98f1e00640 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533d9d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:24:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.524+0000 7f98fb200640 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d92c00 0x557533d9cf40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98f4031c10 tx=0x7f98f4004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a2ccaa45cdd363 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.524+0000 7f98dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:24:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.525+0000 7f98dfe00640 1 -- 172.21.15.26:0/218543708 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4054060 con 0x557533d92c00 2024-06-13T07:24:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.525+0000 7f98dfe00640 1 -- 172.21.15.26:0/218543708 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f98f4003c30 con 0x557533d92c00 2024-06-13T07:24:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.525+0000 7f99041dc740 1 -- 172.21.15.26:0/218543708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f403f070 con 0x557533d92c00 2024-06-13T07:24:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.526+0000 7f98dfe00640 1 -- 172.21.15.26:0/218543708 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f98f4003e30 con 0x557533d92c00 2024-06-13T07:24:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.526+0000 7f99041dc740 10 client.27220 fetch_fsmap learned FSMap version 49 2024-06-13T07:24:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.526+0000 7f99041dc740 1 -- 172.21.15.26:0/218543708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557533d9b6b0 con 0x557533d92c00 2024-06-13T07:24:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.526+0000 7f98dfe00640 1 -- 172.21.15.26:0/218543708 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f403f070 con 0x557533d92c00 2024-06-13T07:24:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.527+0000 7f98dfe00640 1 -- 172.21.15.26:0/218543708 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f98f4030650 con 0x557533d92c00 2024-06-13T07:24:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.528+0000 7f99041dc740 10 client.27220 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:24:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.528+0000 7f99041dc740 10 client.27220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:24:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.528+0000 7f99041dc740 20 client.27220 populate_metadata read hostname 'smithi026' 2024-06-13T07:24:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.528+0000 7f99041dc740 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557533da8990 0x557533daad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.528+0000 7f99041dc740 4 client.27220 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:24:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.528+0000 7f99041dc740 1 -- 172.21.15.26:0/218543708 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557533d03b50 con 0x557533da8990 2024-06-13T07:24:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.530+0000 7f98f1e00640 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557533da8990 0x557533daad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.531+0000 7f98f1e00640 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557533da8990 0x557533daad80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa94793548f8823 server_cookie=67324cc111d2a4e3 in_seq=0 out_seq=0 2024-06-13T07:24:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.531+0000 7f98dfe00640 10 client.27220 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.536+0000 7f98dfe00640 1 -- 172.21.15.26:0/218543708 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557533d03b50 con 0x557533da8990 2024-06-13T07:24:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.536+0000 7f98dfe00640 10 client.27220 handle_command_reply: tid=0 2024-06-13T07:24:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.536+0000 7f99041dc740 1 client.27220 shutdown 2024-06-13T07:24:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.537+0000 7f99041dc740 1 -- 172.21.15.26:0/218543708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d92c00 msgr2=0x557533d9cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.537+0000 7f99041dc740 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d92c00 0x557533d9cf40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98f4031c10 tx=0x7f98f4004970 comp rx=0 tx=0).stop 2024-06-13T07:24:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.537+0000 7f99041dc740 1 -- 172.21.15.26:0/218543708 shutdown_connections 2024-06-13T07:24:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f99041dc740 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557533da8990 0x557533daad80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f99041dc740 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533d9d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f99041dc740 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d98760 0x557533da44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f99041dc740 1 --2- 172.21.15.26:0/218543708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d92c00 0x557533d9cf40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f99041dc740 1 -- 172.21.15.26:0/218543708 >> 172.21.15.26:0/218543708 conn(0x557533d5afb0 msgr2=0x557533d8d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f98dfe00640 0 client.27220 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f99041dc740 1 -- 172.21.15.26:0/218543708 shutdown_connections 2024-06-13T07:24:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f99041dc740 1 -- 172.21.15.26:0/218543708 wait complete. 2024-06-13T07:24:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.538+0000 7f99041dc740 20 client.27220 trim_cache size 0 max 16384 2024-06-13T07:24:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.575+0000 7f99041dc740 1 Processor -- start 2024-06-13T07:24:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.576+0000 7f99041dc740 1 -- start start 2024-06-13T07:24:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.576+0000 7f99041dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d92c00 0x557533d9d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.576+0000 7f99041dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533da4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.576+0000 7f99041dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533d8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.576+0000 7f99041dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98d8012670 con 0x557533d92c00 2024-06-13T07:24:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.576+0000 7f99041dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98d800fee0 con 0x557533d935f0 2024-06-13T07:24:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.576+0000 7f99041dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98d800dd30 con 0x557533d98760 2024-06-13T07:24:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.577+0000 7f98fb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533d8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.577+0000 7f98fb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533d8dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:52754/0 (socket says 172.21.15.26:52754) 2024-06-13T07:24:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.577+0000 7f98fb200640 1 -- 172.21.15.26:0/552699405 learned_addr learned my addr 172.21.15.26:0/552699405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.577+0000 7f98fb200640 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d92c00 0x557533d9d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.577+0000 7f98fb200640 1 -- 172.21.15.26:0/552699405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 msgr2=0x557533da4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.577+0000 7f98f1e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533da4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.578+0000 7f98fb200640 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533da4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.578+0000 7f98fb200640 1 -- 172.21.15.26:0/552699405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d92c00 msgr2=0x557533d9d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.578+0000 7f98fb200640 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d92c00 0x557533d9d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.578+0000 7f98fb200640 1 -- 172.21.15.26:0/552699405 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557533dab400 con 0x557533d98760 2024-06-13T07:24:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.578+0000 7f98f1e00640 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533da4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:24:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.578+0000 7f98fb200640 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533d8dfc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f98f402aeb0 tx=0x7f98f4004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91109efef02cd30e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.580+0000 7f98eaa00640 1 -- 172.21.15.26:0/552699405 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4054040 con 0x557533d98760 2024-06-13T07:24:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.580+0000 7f98eaa00640 1 -- 172.21.15.26:0/552699405 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f403f040 con 0x557533d98760 2024-06-13T07:24:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.580+0000 7f98eaa00640 1 -- 172.21.15.26:0/552699405 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f402f610 con 0x557533d98760 2024-06-13T07:24:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 -- 172.21.15.26:0/552699405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 msgr2=0x557533d8dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533d8dfc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f98f402aeb0 tx=0x7f98f4004970 comp rx=0 tx=0).stop 2024-06-13T07:24:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 -- 172.21.15.26:0/552699405 shutdown_connections 2024-06-13T07:24:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d935f0 0x557533da4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533d8dfc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 --2- 172.21.15.26:0/552699405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d92c00 0x557533d9d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 -- 172.21.15.26:0/552699405 >> 172.21.15.26:0/552699405 conn(0x557533d5afb0 msgr2=0x557533d9cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 -- 172.21.15.26:0/552699405 shutdown_connections 2024-06-13T07:24:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.581+0000 7f99041dc740 1 -- 172.21.15.26:0/552699405 wait complete. 2024-06-13T07:24:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.582+0000 7f99041dc740 1 -- start start 2024-06-13T07:24:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.582+0000 7f99041dc740 1 Processor -- start 2024-06-13T07:24:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.584+0000 7f99041dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533da4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.584+0000 7f99041dc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d935f0 0x557533da0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.584+0000 7f99041dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533da1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.584+0000 7f99041dc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98d800e3a0 con 0x557533d935f0 2024-06-13T07:24:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.584+0000 7f99041dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98d800f810 con 0x557533d92c00 2024-06-13T07:24:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.584+0000 7f99041dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98d800ea70 con 0x557533d98760 2024-06-13T07:24:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.584+0000 7f99041dc740 10 client.0 _ll_register_callbacks cb 0x7f98f4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:24:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.584+0000 7f98fb200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533da4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.585+0000 7f98fb200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533da4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:42156/0 (socket says 172.21.15.26:42156) 2024-06-13T07:24:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.585+0000 7f98fb200640 1 -- 172.21.15.26:0/363266132 learned_addr learned my addr 172.21.15.26:0/363266132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.585+0000 7f98fb200640 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533da1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.585+0000 7f98f1e00640 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d935f0 0x557533da0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.588+0000 7f98fb200640 1 -- 172.21.15.26:0/363266132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 msgr2=0x557533da1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.588+0000 7f98fb200640 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533da1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.588+0000 7f98fb200640 1 -- 172.21.15.26:0/363266132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d935f0 msgr2=0x557533da0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.588+0000 7f98fb200640 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d935f0 0x557533da0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.588+0000 7f98fb200640 1 -- 172.21.15.26:0/363266132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557533e11700 con 0x557533d92c00 2024-06-13T07:24:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.590+0000 7f98fb200640 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533da4440 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f98f40048b0 tx=0x7f98f4051040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca0de1f982e080d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.590+0000 7f98dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:24:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.590+0000 7f98dfe00640 1 -- 172.21.15.26:0/363266132 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f40041d0 con 0x557533d92c00 2024-06-13T07:24:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.590+0000 7f98dfe00640 1 -- 172.21.15.26:0/363266132 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f98f402f7b0 con 0x557533d92c00 2024-06-13T07:24:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.591+0000 7f99041dc740 1 -- 172.21.15.26:0/363266132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f403f040 con 0x557533d92c00 2024-06-13T07:24:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.591+0000 7f98dfe00640 1 -- 172.21.15.26:0/363266132 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f98f4064970 con 0x557533d92c00 2024-06-13T07:24:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.591+0000 7f99041dc740 10 client.27164 fetch_fsmap learned FSMap version 49 2024-06-13T07:24:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.591+0000 7f99041dc740 1 -- 172.21.15.26:0/363266132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557533e12960 con 0x557533d92c00 2024-06-13T07:24:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.594+0000 7f98dfe00640 1 -- 172.21.15.26:0/363266132 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f403f040 con 0x557533d92c00 2024-06-13T07:24:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.594+0000 7f98dfe00640 1 -- 172.21.15.26:0/363266132 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f98f4059ad0 con 0x557533d92c00 2024-06-13T07:24:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.594+0000 7f99041dc740 10 client.27164 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:24:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.594+0000 7f99041dc740 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:24:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.594+0000 7f99041dc740 20 client.27164 populate_metadata read hostname 'smithi026' 2024-06-13T07:24:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.594+0000 7f99041dc740 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557533e132a0 0x557533e156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.594+0000 7f99041dc740 4 client.27164 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:24:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.594+0000 7f99041dc740 1 -- 172.21.15.26:0/363266132 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557533d89db0 con 0x557533e132a0 2024-06-13T07:24:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.595+0000 7f98f1e00640 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557533e132a0 0x557533e156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:10.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.599+0000 7f98f1e00640 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557533e132a0 0x557533e156c0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54865425abe4e0a3 server_cookie=32d109c2286bcf80 in_seq=0 out_seq=0 2024-06-13T07:24:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:10.599+0000 7f98dfe00640 10 client.27164 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:10.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:10 smithi136 ceph-mon[37229]: pgmap v1219: 129 pgs: 129 active+clean; 649 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-06-13T07:24:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:10 smithi026 ceph-mon[32041]: pgmap v1219: 129 pgs: 129 active+clean; 649 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-06-13T07:24:10.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:10 smithi135 ceph-mon[36276]: pgmap v1219: 129 pgs: 129 active+clean; 649 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-06-13T07:24:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f98dfe00640 1 -- 172.21.15.26:0/363266132 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557533d89db0 con 0x557533e132a0 2024-06-13T07:24:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f98dfe00640 10 client.27164 handle_command_reply: tid=0 2024-06-13T07:24:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 client.27164 shutdown 2024-06-13T07:24:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 -- 172.21.15.26:0/363266132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 msgr2=0x557533da4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533da4440 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f98f40048b0 tx=0x7f98f4051040 comp rx=0 tx=0).stop 2024-06-13T07:24:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 -- 172.21.15.26:0/363266132 shutdown_connections 2024-06-13T07:24:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x557533e132a0 0x557533e156c0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557533d98760 0x557533da1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557533d935f0 0x557533da0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 --2- 172.21.15.26:0/363266132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557533d92c00 0x557533da4440 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 -- 172.21.15.26:0/363266132 >> 172.21.15.26:0/363266132 conn(0x557533d5afb0 msgr2=0x557533d91570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f98dfe00640 0 client.27164 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 -- 172.21.15.26:0/363266132 shutdown_connections 2024-06-13T07:24:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.164+0000 7f99041dc740 1 -- 172.21.15.26:0/363266132 wait complete. 2024-06-13T07:24:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.165+0000 7f99041dc740 20 client.27164 trim_cache size 0 max 16384 2024-06-13T07:24:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:24:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:24:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:24:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.165+0000 7f9901400640 1 -- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98e00a2ec0 msgr2=0x7f98e00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.165+0000 7f9901400640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98e00a2ec0 0x7f98e00a5380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x557533d939f0 tx=0x7f98ec00fa10 comp rx=0 tx=0).stop 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.165+0000 7f9901400640 1 -- 172.21.15.26:0/1620885237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98fc1089a0 msgr2=0x7f98fc1a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.165+0000 7f9901400640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98fc1089a0 0x7f98fc1a2510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c970 tx=0x7f98ec00ce30 comp rx=0 tx=0).stop 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.166+0000 7f9901400640 1 -- 172.21.15.26:0/1620885237 shutdown_connections 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.166+0000 7f9901400640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f98e00a2ec0 0x7f98e00a5380 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.166+0000 7f9901400640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98fc110220 0x7f98fc1a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.166+0000 7f9901400640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98fc109300 0x7f98fc1a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.166+0000 7f9901400640 1 --2- 172.21.15.26:0/1620885237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98fc1089a0 0x7f98fc1a2510 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.166+0000 7f9901400640 1 -- 172.21.15.26:0/1620885237 >> 172.21.15.26:0/1620885237 conn(0x7f98fc06ea10 msgr2=0x7f98fc109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.166+0000 7f9901400640 1 -- 172.21.15.26:0/1620885237 shutdown_connections 2024-06-13T07:24:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:11.166+0000 7f9901400640 1 -- 172.21.15.26:0/1620885237 wait complete. 2024-06-13T07:24:11.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 300757 25.33 MB/sec execute 504 sec latency 441.391 ms 2024-06-13T07:24:12.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 301347 25.35 MB/sec execute 505 sec latency 67.536 ms 2024-06-13T07:24:12.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:12 smithi136 ceph-mon[37229]: pgmap v1220: 129 pgs: 129 active+clean; 652 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 443 op/s 2024-06-13T07:24:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:12 smithi026 ceph-mon[32041]: pgmap v1220: 129 pgs: 129 active+clean; 652 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 443 op/s 2024-06-13T07:24:12.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:12 smithi135 ceph-mon[36276]: pgmap v1220: 129 pgs: 129 active+clean; 652 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 443 op/s 2024-06-13T07:24:13.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 301878 25.34 MB/sec execute 506 sec latency 124.008 ms 2024-06-13T07:24:14.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 302319 25.34 MB/sec execute 507 sec latency 296.455 ms 2024-06-13T07:24:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:14 smithi136 ceph-mon[37229]: pgmap v1221: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 510 op/s 2024-06-13T07:24:14.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:14 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:24:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:14 smithi026 ceph-mon[32041]: pgmap v1221: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 510 op/s 2024-06-13T07:24:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:14 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:24:14.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:14 smithi135 ceph-mon[36276]: pgmap v1221: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 510 op/s 2024-06-13T07:24:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:14 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:24:15.250 INFO:tasks.workunit.client.0.smithi026.stdout: 10 302784 25.36 MB/sec execute 508 sec latency 332.791 ms 2024-06-13T07:24:15.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:15 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:15 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:15.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:15 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:24:16.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 303209 25.35 MB/sec execute 509 sec latency 92.179 ms 2024-06-13T07:24:16.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:16 smithi136 ceph-mon[37229]: pgmap v1222: 129 pgs: 129 active+clean; 658 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-06-13T07:24:16.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:16 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:24:16.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:16 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:24:16.683 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:16 smithi135 ceph-mon[36276]: pgmap v1222: 129 pgs: 129 active+clean; 658 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-06-13T07:24:16.684 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:16 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:24:16.684 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:16 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:24:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:16 smithi026 ceph-mon[32041]: pgmap v1222: 129 pgs: 129 active+clean; 658 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-06-13T07:24:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:16 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi026 to 177.5M 2024-06-13T07:24:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:16 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi026 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:24:17.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 303688 25.34 MB/sec execute 510 sec latency 113.496 ms 2024-06-13T07:24:18.249 INFO:tasks.workunit.client.0.smithi026.stdout: 10 304118 25.33 MB/sec execute 511 sec latency 111.987 ms 2024-06-13T07:24:18.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:18 smithi135 ceph-mon[36276]: pgmap v1223: 129 pgs: 129 active+clean; 664 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 442 op/s 2024-06-13T07:24:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:18 smithi136 ceph-mon[37229]: pgmap v1223: 129 pgs: 129 active+clean; 664 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 442 op/s 2024-06-13T07:24:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:18 smithi026 ceph-mon[32041]: pgmap v1223: 129 pgs: 129 active+clean; 664 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 442 op/s 2024-06-13T07:24:19.251 INFO:tasks.workunit.client.0.smithi026.stdout: 10 304771 25.36 MB/sec execute 512 sec latency 71.966 ms 2024-06-13T07:24:20.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 304889 25.32 MB/sec execute 513 sec latency 433.331 ms 2024-06-13T07:24:20.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:20 smithi136 ceph-mon[37229]: pgmap v1224: 129 pgs: 129 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-06-13T07:24:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:20 smithi026 ceph-mon[32041]: pgmap v1224: 129 pgs: 129 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-06-13T07:24:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:20 smithi135 ceph-mon[36276]: pgmap v1224: 129 pgs: 129 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-06-13T07:24:21.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 305294 25.32 MB/sec execute 514 sec latency 134.493 ms 2024-06-13T07:24:22.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 305795 25.31 MB/sec execute 515 sec latency 88.305 ms 2024-06-13T07:24:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:22 smithi135 ceph-mon[36276]: pgmap v1225: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 403 op/s 2024-06-13T07:24:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:22.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:22 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:22 smithi136 ceph-mon[37229]: pgmap v1225: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 403 op/s 2024-06-13T07:24:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:22 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:22 smithi026 ceph-mon[32041]: pgmap v1225: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 403 op/s 2024-06-13T07:24:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:22 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:23.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 306231 25.31 MB/sec execute 516 sec latency 84.323 ms 2024-06-13T07:24:24.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 306820 25.33 MB/sec execute 517 sec latency 96.736 ms 2024-06-13T07:24:24.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:24:24.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:24:24.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:24 smithi136 ceph-mon[37229]: pgmap v1226: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 433 op/s 2024-06-13T07:24:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.393+0000 7f33b6400640 1 -- 172.21.15.26:0/275198014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 msgr2=0x7f33b01071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.393+0000 7f33b6400640 1 --2- 172.21.15.26:0/275198014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 0x7f33b01071a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f339c009900 tx=0x7f339c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.394+0000 7f33b6400640 1 -- 172.21.15.26:0/275198014 shutdown_connections 2024-06-13T07:24:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.394+0000 7f33b6400640 1 --2- 172.21.15.26:0/275198014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 0x7f33b01071a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.394+0000 7f33b6400640 1 --2- 172.21.15.26:0/275198014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33b01076e0 0x7f33b0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.394+0000 7f33b6400640 1 --2- 172.21.15.26:0/275198014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b0102480 0x7f33b0104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.394+0000 7f33b6400640 1 -- 172.21.15.26:0/275198014 >> 172.21.15.26:0/275198014 conn(0x7f33b00fbe70 msgr2=0x7f33b00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.394+0000 7f33b6400640 1 -- 172.21.15.26:0/275198014 shutdown_connections 2024-06-13T07:24:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.394+0000 7f33b6400640 1 -- 172.21.15.26:0/275198014 wait complete. 2024-06-13T07:24:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.396+0000 7f33b6400640 1 Processor -- start 2024-06-13T07:24:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.396+0000 7f33b6400640 1 -- start start 2024-06-13T07:24:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.396+0000 7f33b6400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33b0102480 0x7f33b0199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.396+0000 7f33b6400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 0x7f33b019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.396+0000 7f33b6400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b01076e0 0x7f33b01a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.397+0000 7f33b6400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33b010d730 con 0x7f33b01076e0 2024-06-13T07:24:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.397+0000 7f33b6400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33b010d5b0 con 0x7f33b0104db0 2024-06-13T07:24:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.397+0000 7f33b6400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33b010d8b0 con 0x7f33b0102480 2024-06-13T07:24:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.397+0000 7f33afe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b01076e0 0x7f33b01a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.397+0000 7f33afe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b01076e0 0x7f33b01a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47692/0 (socket says 172.21.15.26:47692) 2024-06-13T07:24:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.397+0000 7f33afe00640 1 -- 172.21.15.26:0/982320775 learned_addr learned my addr 172.21.15.26:0/982320775 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.399+0000 7f33aea00640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 0x7f33b019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.399+0000 7f33af400640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33b0102480 0x7f33b0199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.399+0000 7f33aea00640 1 -- 172.21.15.26:0/982320775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33b0102480 msgr2=0x7f33b0199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.400+0000 7f33aea00640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33b0102480 0x7f33b0199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.400+0000 7f33aea00640 1 -- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b01076e0 msgr2=0x7f33b01a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.400+0000 7f33aea00640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b01076e0 0x7f33b01a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.400+0000 7f33aea00640 1 -- 172.21.15.26:0/982320775 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b01a1900 con 0x7f33b0104db0 2024-06-13T07:24:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.400+0000 7f33af400640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33b0102480 0x7f33b0199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:24:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.400+0000 7f33aea00640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 0x7f33b019a230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f339c009850 tx=0x7f339c003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.401+0000 7f33a7e00640 1 -- 172.21.15.26:0/982320775 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c03f030 con 0x7f33b0104db0 2024-06-13T07:24:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.401+0000 7f33b6400640 1 -- 172.21.15.26:0/982320775 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b01a1b90 con 0x7f33b0104db0 2024-06-13T07:24:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.401+0000 7f33a7e00640 1 -- 172.21.15.26:0/982320775 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c032dc0 con 0x7f33b0104db0 2024-06-13T07:24:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.401+0000 7f33a7e00640 1 -- 172.21.15.26:0/982320775 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c035070 con 0x7f33b0104db0 2024-06-13T07:24:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.401+0000 7f33b6400640 1 -- 172.21.15.26:0/982320775 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b01a20a0 con 0x7f33b0104db0 2024-06-13T07:24:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.414+0000 7f33b916f740 1 Processor -- start 2024-06-13T07:24:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.414+0000 7f33a7e00640 1 -- 172.21.15.26:0/982320775 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f339c042030 con 0x7f33b0104db0 2024-06-13T07:24:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.416+0000 7f33b916f740 1 -- start start 2024-06-13T07:24:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.416+0000 7f33b916f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d72830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.417+0000 7f33b916f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d77900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.417+0000 7f33b916f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d7a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.417+0000 7f33b916f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f44ccd420 con 0x559f44d72410 2024-06-13T07:24:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.417+0000 7f33b916f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559f44c74c50 con 0x559f44d77f70 2024-06-13T07:24:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.417+0000 7f33b916f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f44cc93e0 con 0x559f44d72e00 2024-06-13T07:24:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.417+0000 7f33aea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d7a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.417+0000 7f33aea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d7a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54246/0 (socket says 172.21.15.26:54246) 2024-06-13T07:24:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.417+0000 7f33aea00640 1 -- 172.21.15.26:0/2781685870 learned_addr learned my addr 172.21.15.26:0/2781685870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.418+0000 7f33aea00640 1 -- 172.21.15.26:0/2781685870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 msgr2=0x559f44d77900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:24:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.418+0000 7f33aea00640 1 --2- 172.21.15.26:0/2781685870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d77900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.418+0000 7f33aea00640 1 -- 172.21.15.26:0/2781685870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 msgr2=0x559f44d72830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.418+0000 7f33aea00640 1 --2- 172.21.15.26:0/2781685870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d72830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.418+0000 7f33aea00640 1 -- 172.21.15.26:0/2781685870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f449e1210 con 0x559f44d77f70 2024-06-13T07:24:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.419+0000 7f33aea00640 1 --2- 172.21.15.26:0/2781685870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d7a400 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f339c0f3fd0 tx=0x7f339c0f83b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6baae5ba12f64c73 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.419+0000 7f33a7e00640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f33940a2cb0 0x7f33940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.421+0000 7f33a4c00640 1 -- 172.21.15.26:0/2781685870 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0f8810 con 0x559f44d77f70 2024-06-13T07:24:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.421+0000 7f33a4c00640 1 -- 172.21.15.26:0/2781685870 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c0fd030 con 0x559f44d77f70 2024-06-13T07:24:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.421+0000 7f33a4c00640 1 -- 172.21.15.26:0/2781685870 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c1013d0 con 0x559f44d77f70 2024-06-13T07:24:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.423+0000 7f33a7e00640 1 -- 172.21.15.26:0/982320775 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f339c0f00d0 con 0x7f33b0104db0 2024-06-13T07:24:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.424+0000 7f33afe00640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f33940a2cb0 0x7f33940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.425+0000 7f33b916f740 1 -- 172.21.15.26:0/2781685870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 msgr2=0x559f44d7a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.425+0000 7f33b916f740 1 --2- 172.21.15.26:0/2781685870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d7a400 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f339c0f3fd0 tx=0x7f339c0f83b0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.425+0000 7f33b916f740 1 -- 172.21.15.26:0/2781685870 shutdown_connections 2024-06-13T07:24:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.425+0000 7f33b916f740 1 --2- 172.21.15.26:0/2781685870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d7a400 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.425+0000 7f33b916f740 1 --2- 172.21.15.26:0/2781685870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d77900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.425+0000 7f33b916f740 1 --2- 172.21.15.26:0/2781685870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d72830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.425+0000 7f33b916f740 1 -- 172.21.15.26:0/2781685870 >> 172.21.15.26:0/2781685870 conn(0x559f44d31fb0 msgr2=0x559f44d6de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.428+0000 7f33b916f740 1 -- 172.21.15.26:0/2781685870 shutdown_connections 2024-06-13T07:24:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.428+0000 7f33b916f740 1 -- 172.21.15.26:0/2781685870 wait complete. 2024-06-13T07:24:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.428+0000 7f33b916f740 1 -- start start 2024-06-13T07:24:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.429+0000 7f33b916f740 1 Processor -- start 2024-06-13T07:24:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.429+0000 7f33b916f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d72410 0x559f44d7c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.429+0000 7f33b916f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d7ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.429+0000 7f33b916f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d77f70 0x559f44d83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.429+0000 7f33b916f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f44d7ace0 con 0x559f44d77f70 2024-06-13T07:24:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.429+0000 7f33b916f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559f44c880b0 con 0x559f44d72410 2024-06-13T07:24:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.429+0000 7f33b916f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f44d7ae60 con 0x559f44d72e00 2024-06-13T07:24:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.430+0000 7f33b916f740 10 client.0 _ll_register_callbacks cb 0x559f44c73f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:24:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.431+0000 7f33afe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d77f70 0x559f44d83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.432+0000 7f33afe00640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f33940a2cb0 0x7f33940a5170 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005c40 tx=0x7f33a8005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.432+0000 7f33afe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d77f70 0x559f44d83a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47706/0 (socket says 172.21.15.26:47706) 2024-06-13T07:24:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.432+0000 7f33afe00640 1 -- 172.21.15.26:0/1674833990 learned_addr learned my addr 172.21.15.26:0/1674833990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.433+0000 7f33aea00640 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d7ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.434+0000 7f33aea00640 1 -- 172.21.15.26:0/1674833990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d72410 msgr2=0x559f44d7c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.434+0000 7f33aea00640 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d72410 0x559f44d7c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.434+0000 7f33aea00640 1 -- 172.21.15.26:0/1674833990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d77f70 msgr2=0x559f44d83a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.434+0000 7f33aea00640 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d77f70 0x559f44d83a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.434+0000 7f33aea00640 1 -- 172.21.15.26:0/1674833990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f44aad620 con 0x559f44d72e00 2024-06-13T07:24:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.435+0000 7f33aea00640 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d7ca40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f339c0bc240 tx=0x7f339c03ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73598c05aa8b08a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.435+0000 7f3399600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:24:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.437+0000 7f3399600640 1 -- 172.21.15.26:0/1674833990 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c049050 con 0x559f44d72e00 2024-06-13T07:24:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.438+0000 7f3399600640 1 -- 172.21.15.26:0/1674833990 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f339c0ba840 con 0x559f44d72e00 2024-06-13T07:24:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.438+0000 7f33b916f740 1 -- 172.21.15.26:0/1674833990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339c0fd030 con 0x559f44d72e00 2024-06-13T07:24:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.440+0000 7f3399600640 1 -- 172.21.15.26:0/1674833990 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f339c0b8750 con 0x559f44d72e00 2024-06-13T07:24:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.440+0000 7f33b916f740 10 client.27232 fetch_fsmap learned FSMap version 49 2024-06-13T07:24:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.440+0000 7f33b916f740 1 -- 172.21.15.26:0/1674833990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559f44d7ace0 con 0x559f44d72e00 2024-06-13T07:24:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.441+0000 7f3399600640 1 -- 172.21.15.26:0/1674833990 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c0fd030 con 0x559f44d72e00 2024-06-13T07:24:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.443+0000 7f3399600640 1 -- 172.21.15.26:0/1674833990 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f339c0b8b20 con 0x559f44d72e00 2024-06-13T07:24:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.443+0000 7f33b916f740 10 client.27232 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:24:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.443+0000 7f33b916f740 10 client.27232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:24:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.443+0000 7f33b916f740 20 client.27232 populate_metadata read hostname 'smithi026' 2024-06-13T07:24:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.443+0000 7f33b916f740 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559f44d98f10 0x559f44d9b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.443+0000 7f33b916f740 4 client.27232 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:24:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.443+0000 7f33b916f740 1 -- 172.21.15.26:0/1674833990 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f44cdab50 con 0x559f44d98f10 2024-06-13T07:24:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.444+0000 7f33af400640 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559f44d98f10 0x559f44d9b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.446+0000 7f33af400640 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559f44d98f10 0x559f44d9b350 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d069927044774c4a server_cookie=468afd1603b0b951 in_seq=0 out_seq=0 2024-06-13T07:24:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.446+0000 7f3399600640 10 client.27232 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.455+0000 7f3399600640 1 -- 172.21.15.26:0/1674833990 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f44cdab50 con 0x559f44d98f10 2024-06-13T07:24:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.455+0000 7f3399600640 10 client.27232 handle_command_reply: tid=0 2024-06-13T07:24:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.455+0000 7f33b916f740 1 client.27232 shutdown 2024-06-13T07:24:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f33b916f740 1 -- 172.21.15.26:0/1674833990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 msgr2=0x559f44d7ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f33b916f740 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d7ca40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f339c0bc240 tx=0x7f339c03ba70 comp rx=0 tx=0).stop 2024-06-13T07:24:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f33b916f740 1 -- 172.21.15.26:0/1674833990 shutdown_connections 2024-06-13T07:24:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f33b916f740 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559f44d98f10 0x559f44d9b350 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f33b916f740 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d77f70 0x559f44d83a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f33b916f740 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d7ca40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f33b916f740 1 --2- 172.21.15.26:0/1674833990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d72410 0x559f44d7c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f33b916f740 1 -- 172.21.15.26:0/1674833990 >> 172.21.15.26:0/1674833990 conn(0x559f44d31fb0 msgr2=0x559f44d64840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.459+0000 7f3399600640 0 client.27232 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.462+0000 7f33b916f740 1 -- 172.21.15.26:0/1674833990 shutdown_connections 2024-06-13T07:24:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.462+0000 7f33b916f740 1 -- 172.21.15.26:0/1674833990 wait complete. 2024-06-13T07:24:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.463+0000 7f33b916f740 20 client.27232 trim_cache size 0 max 16384 2024-06-13T07:24:24.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.506 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:24:24.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:24:24.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:24 smithi026 ceph-mon[32041]: pgmap v1226: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 433 op/s 2024-06-13T07:24:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.510+0000 7f33b916f740 1 Processor -- start 2024-06-13T07:24:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.510+0000 7f33b916f740 1 -- start start 2024-06-13T07:24:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33b916f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d70d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33b916f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33b916f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33b916f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33b01a2460 con 0x559f44d72410 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33b916f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33b0071630 con 0x559f44d77f70 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33b916f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33b006f480 con 0x559f44d72e00 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33afe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33aea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33aea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d839d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41098/0 (socket says 172.21.15.26:41098) 2024-06-13T07:24:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.511+0000 7f33aea00640 1 -- 172.21.15.26:0/2273473404 learned_addr learned my addr 172.21.15.26:0/2273473404 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.512+0000 7f33aea00640 1 -- 172.21.15.26:0/2273473404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 msgr2=0x559f44d712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.512+0000 7f33aea00640 1 --2- 172.21.15.26:0/2273473404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.512+0000 7f33aea00640 1 -- 172.21.15.26:0/2273473404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 msgr2=0x559f44d70d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.512+0000 7f33aea00640 1 --2- 172.21.15.26:0/2273473404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d70d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.512+0000 7f33aea00640 1 -- 172.21.15.26:0/2273473404 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f44d9b9d0 con 0x559f44d72e00 2024-06-13T07:24:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.512+0000 7f33afe00640 1 --2- 172.21.15.26:0/2273473404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:24:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.518+0000 7f33aea00640 1 --2- 172.21.15.26:0/2273473404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d839d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f339c009900 tx=0x7f339c032f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da3bc934ad41b8aa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.518+0000 7f339be00640 1 -- 172.21.15.26:0/2273473404 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c049070 con 0x559f44d72e00 2024-06-13T07:24:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.518+0000 7f339be00640 1 -- 172.21.15.26:0/2273473404 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c0fd070 con 0x559f44d72e00 2024-06-13T07:24:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.518+0000 7f339be00640 1 -- 172.21.15.26:0/2273473404 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0ba680 con 0x559f44d72e00 2024-06-13T07:24:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 -- 172.21.15.26:0/2273473404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 msgr2=0x559f44d839d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 --2- 172.21.15.26:0/2273473404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d839d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f339c009900 tx=0x7f339c032f70 comp rx=0 tx=0).stop 2024-06-13T07:24:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 -- 172.21.15.26:0/2273473404 shutdown_connections 2024-06-13T07:24:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 --2- 172.21.15.26:0/2273473404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 --2- 172.21.15.26:0/2273473404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d839d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 --2- 172.21.15.26:0/2273473404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d70d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 -- 172.21.15.26:0/2273473404 >> 172.21.15.26:0/2273473404 conn(0x559f44d31fb0 msgr2=0x559f44d741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 -- 172.21.15.26:0/2273473404 shutdown_connections 2024-06-13T07:24:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.520+0000 7f33b916f740 1 -- 172.21.15.26:0/2273473404 wait complete. 2024-06-13T07:24:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.521+0000 7f33b916f740 1 -- start start 2024-06-13T07:24:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.521+0000 7f33b916f740 1 Processor -- start 2024-06-13T07:24:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.525+0000 7f33b916f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d81020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.525+0000 7f33b916f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d81560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.526+0000 7f33aea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d81560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.526+0000 7f33b916f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d824b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.527+0000 7f33aea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d81560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:41106/0 (socket says 172.21.15.26:41106) 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.527+0000 7f33b916f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33b006faf0 con 0x559f44d72410 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.527+0000 7f33b916f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33b0070f60 con 0x559f44d77f70 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.527+0000 7f33b916f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00701c0 con 0x559f44d72e00 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.527+0000 7f33b916f740 10 client.0 _ll_register_callbacks cb 0x7f339c0f8170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.528+0000 7f33afe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d824b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.528+0000 7f33afe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d824b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:54288/0 (socket says 172.21.15.26:54288) 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.528+0000 7f33afe00640 1 -- 172.21.15.26:0/1330581726 learned_addr learned my addr 172.21.15.26:0/1330581726 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.528+0000 7f33af400640 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d81020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.530+0000 7f33afe00640 1 -- 172.21.15.26:0/1330581726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 msgr2=0x559f44d81560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.530+0000 7f33afe00640 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d81560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.530+0000 7f33afe00640 1 -- 172.21.15.26:0/1330581726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 msgr2=0x559f44d81020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.530+0000 7f33afe00640 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d81020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.530+0000 7f33afe00640 1 -- 172.21.15.26:0/1330581726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f44d82aa0 con 0x559f44d77f70 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.530+0000 7f33aea00640 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d81560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.530+0000 7f33afe00640 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d824b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f33a8010f70 tx=0x7f33a80075e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca86838460323208 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.530+0000 7f3399600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.531+0000 7f3399600640 1 -- 172.21.15.26:0/1330581726 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a802b060 con 0x559f44d77f70 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.531+0000 7f3399600640 1 -- 172.21.15.26:0/1330581726 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f33a801b6f0 con 0x559f44d77f70 2024-06-13T07:24:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.532+0000 7f33b916f740 1 -- 172.21.15.26:0/1330581726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339c0fd070 con 0x559f44d77f70 2024-06-13T07:24:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:24:24.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:24:24.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:24:24.534 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:24 smithi135 ceph-mon[36276]: pgmap v1226: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 433 op/s 2024-06-13T07:24:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.533+0000 7f3399600640 1 -- 172.21.15.26:0/1330581726 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f33a801b8f0 con 0x559f44d77f70 2024-06-13T07:24:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.533+0000 7f33b916f740 10 client.27179 fetch_fsmap learned FSMap version 49 2024-06-13T07:24:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.533+0000 7f33b916f740 1 -- 172.21.15.26:0/1330581726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559f44e03c30 con 0x559f44d77f70 2024-06-13T07:24:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.534+0000 7f3399600640 1 -- 172.21.15.26:0/1330581726 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c0fd070 con 0x559f44d77f70 2024-06-13T07:24:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.534+0000 7f3399600640 1 -- 172.21.15.26:0/1330581726 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f33a802f3a0 con 0x559f44d77f70 2024-06-13T07:24:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.535+0000 7f33b916f740 10 client.27179 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:24:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.536+0000 7f33b916f740 10 client.27179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:24:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.536+0000 7f33b916f740 20 client.27179 populate_metadata read hostname 'smithi026' 2024-06-13T07:24:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.536+0000 7f33b916f740 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559f44e043f0 0x559f44e06810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.536+0000 7f33b916f740 4 client.27179 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:24:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.536+0000 7f33b916f740 1 -- 172.21.15.26:0/1330581726 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f44cd4ea0 con 0x559f44e043f0 2024-06-13T07:24:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.537+0000 7f33af400640 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559f44e043f0 0x559f44e06810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.540+0000 7f33af400640 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559f44e043f0 0x559f44e06810 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487ccd9fd14d5f8e server_cookie=f522121f4b24b32 in_seq=0 out_seq=0 2024-06-13T07:24:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:24.541+0000 7f3399600640 10 client.27179 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:25.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 306918 25.29 MB/sec execute 518 sec latency 903.866 ms 2024-06-13T07:24:26.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 306951 25.25 MB/sec execute 519 sec latency 1858.325 ms 2024-06-13T07:24:26.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:26 smithi026 ceph-mon[32041]: pgmap v1227: 129 pgs: 129 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-13T07:24:26.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:26 smithi135 ceph-mon[36276]: pgmap v1227: 129 pgs: 129 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-13T07:24:26.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:26 smithi136 ceph-mon[37229]: pgmap v1227: 129 pgs: 129 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-13T07:24:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.170+0000 7f3399600640 1 -- 172.21.15.26:0/1330581726 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559f44cd4ea0 con 0x559f44e043f0 2024-06-13T07:24:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.170+0000 7f3399600640 10 client.27179 handle_command_reply: tid=0 2024-06-13T07:24:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.171+0000 7f33b916f740 1 client.27179 shutdown 2024-06-13T07:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.171+0000 7f33b916f740 1 -- 172.21.15.26:0/1330581726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 msgr2=0x559f44d824b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.172+0000 7f33b916f740 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d824b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f33a8010f70 tx=0x7f33a80075e0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.172+0000 7f33b916f740 1 -- 172.21.15.26:0/1330581726 shutdown_connections 2024-06-13T07:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.172+0000 7f33b916f740 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559f44e043f0 0x559f44e06810 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.172+0000 7f33b916f740 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f44d77f70 0x559f44d824b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.172+0000 7f33b916f740 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f44d72e00 0x559f44d81560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.172+0000 7f33b916f740 1 --2- 172.21.15.26:0/1330581726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f44d72410 0x559f44d81020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.172+0000 7f33b916f740 1 -- 172.21.15.26:0/1330581726 >> 172.21.15.26:0/1330581726 conn(0x559f44d31fb0 msgr2=0x559f44d70d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.172+0000 7f3399600640 0 client.27179 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.173+0000 7f33b916f740 1 -- 172.21.15.26:0/1330581726 shutdown_connections 2024-06-13T07:24:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.173+0000 7f33b916f740 1 -- 172.21.15.26:0/1330581726 wait complete. 2024-06-13T07:24:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.173+0000 7f33b916f740 20 client.27179 trim_cache size 0 max 16384 2024-06-13T07:24:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:24:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:24:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:24:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.175+0000 7f33b6400640 1 -- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f33940a2cb0 msgr2=0x7f33940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.175+0000 7f33b6400640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f33940a2cb0 0x7f33940a5170 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005c40 tx=0x7f33a8005bd0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 -- 172.21.15.26:0/982320775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 msgr2=0x7f33b019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 0x7f33b019a230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f339c009850 tx=0x7f339c003f80 comp rx=0 tx=0).stop 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 -- 172.21.15.26:0/982320775 shutdown_connections 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f33940a2cb0 0x7f33940a5170 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b0104db0 0x7f33b019a230 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b01076e0 0x7f33b01a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 --2- 172.21.15.26:0/982320775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33b0102480 0x7f33b0199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 -- 172.21.15.26:0/982320775 >> 172.21.15.26:0/982320775 conn(0x7f33b00fbe70 msgr2=0x7f33b0100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 -- 172.21.15.26:0/982320775 shutdown_connections 2024-06-13T07:24:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:27.176+0000 7f33b6400640 1 -- 172.21.15.26:0/982320775 wait complete. 2024-06-13T07:24:27.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 307091 25.21 MB/sec execute 520 sec latency 2196.964 ms 2024-06-13T07:24:28.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 307243 25.18 MB/sec execute 521 sec latency 230.348 ms 2024-06-13T07:24:28.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:28 smithi026 ceph-mon[32041]: pgmap v1228: 129 pgs: 129 active+clean; 699 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 407 op/s 2024-06-13T07:24:28.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:28 smithi135 ceph-mon[36276]: pgmap v1228: 129 pgs: 129 active+clean; 699 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 407 op/s 2024-06-13T07:24:28.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:28 smithi136 ceph-mon[37229]: pgmap v1228: 129 pgs: 129 active+clean; 699 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 407 op/s 2024-06-13T07:24:29.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 307439 25.15 MB/sec execute 522 sec latency 210.280 ms 2024-06-13T07:24:30.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 307736 25.14 MB/sec execute 523 sec latency 184.787 ms 2024-06-13T07:24:30.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:30 smithi026 ceph-mon[32041]: pgmap v1229: 129 pgs: 129 active+clean; 719 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 8.6 MiB/s wr, 534 op/s 2024-06-13T07:24:30.532 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:30 smithi135 ceph-mon[36276]: pgmap v1229: 129 pgs: 129 active+clean; 719 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 8.6 MiB/s wr, 534 op/s 2024-06-13T07:24:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:30 smithi136 ceph-mon[37229]: pgmap v1229: 129 pgs: 129 active+clean; 719 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 8.6 MiB/s wr, 534 op/s 2024-06-13T07:24:31.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 308287 25.14 MB/sec execute 524 sec latency 171.188 ms 2024-06-13T07:24:32.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 308895 25.15 MB/sec execute 525 sec latency 85.365 ms 2024-06-13T07:24:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:32 smithi135 ceph-mon[36276]: pgmap v1230: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.3 MiB/s wr, 523 op/s 2024-06-13T07:24:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:32 smithi136 ceph-mon[37229]: pgmap v1230: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.3 MiB/s wr, 523 op/s 2024-06-13T07:24:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:32 smithi026 ceph-mon[32041]: pgmap v1230: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.3 MiB/s wr, 523 op/s 2024-06-13T07:24:33.252 INFO:tasks.workunit.client.0.smithi026.stdout: 10 309377 25.15 MB/sec execute 526 sec latency 144.648 ms 2024-06-13T07:24:34.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 309984 25.18 MB/sec execute 527 sec latency 64.115 ms 2024-06-13T07:24:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:34 smithi135 ceph-mon[36276]: pgmap v1231: 129 pgs: 129 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.5 MiB/s wr, 601 op/s 2024-06-13T07:24:34.554 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:34 smithi026 ceph-mon[32041]: pgmap v1231: 129 pgs: 129 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.5 MiB/s wr, 601 op/s 2024-06-13T07:24:34.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:34 smithi136 ceph-mon[37229]: pgmap v1231: 129 pgs: 129 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.5 MiB/s wr, 601 op/s 2024-06-13T07:24:35.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 310501 25.18 MB/sec execute 528 sec latency 83.070 ms 2024-06-13T07:24:36.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 310833 25.16 MB/sec execute 529 sec latency 143.955 ms 2024-06-13T07:24:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:36 smithi135 ceph-mon[36276]: pgmap v1232: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 614 op/s 2024-06-13T07:24:36.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:36 smithi136 ceph-mon[37229]: pgmap v1232: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 614 op/s 2024-06-13T07:24:36.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:36 smithi026 ceph-mon[32041]: pgmap v1232: 129 pgs: 129 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 614 op/s 2024-06-13T07:24:37.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 311349 25.16 MB/sec execute 530 sec latency 87.617 ms 2024-06-13T07:24:38.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 312030 25.18 MB/sec execute 531 sec latency 69.056 ms 2024-06-13T07:24:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:38 smithi135 ceph-mon[36276]: pgmap v1233: 129 pgs: 129 active+clean; 678 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.9 MiB/s wr, 589 op/s 2024-06-13T07:24:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:38 smithi136 ceph-mon[37229]: pgmap v1233: 129 pgs: 129 active+clean; 678 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.9 MiB/s wr, 589 op/s 2024-06-13T07:24:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:38 smithi026 ceph-mon[32041]: pgmap v1233: 129 pgs: 129 active+clean; 678 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.9 MiB/s wr, 589 op/s 2024-06-13T07:24:39.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 312497 25.19 MB/sec execute 532 sec latency 285.494 ms 2024-06-13T07:24:40.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 313073 25.20 MB/sec execute 533 sec latency 125.134 ms 2024-06-13T07:24:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.416+0000 7f89c2a00640 1 -- 172.21.15.26:0/3426851138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89bc1089a0 msgr2=0x7f89bc10add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.416+0000 7f89c2a00640 1 --2- 172.21.15.26:0/3426851138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89bc1089a0 0x7f89bc10add0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f89ac01a910 tx=0x7f89ac03b450 comp rx=0 tx=0).stop 2024-06-13T07:24:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.416+0000 7f89c2a00640 1 -- 172.21.15.26:0/3426851138 shutdown_connections 2024-06-13T07:24:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.416+0000 7f89c2a00640 1 --2- 172.21.15.26:0/3426851138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89bc10dcc0 0x7f89bc10e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.416+0000 7f89c2a00640 1 --2- 172.21.15.26:0/3426851138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc10b310 0x7f89bc10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.416+0000 7f89c2a00640 1 --2- 172.21.15.26:0/3426851138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89bc1089a0 0x7f89bc10add0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.416+0000 7f89c2a00640 1 -- 172.21.15.26:0/3426851138 >> 172.21.15.26:0/3426851138 conn(0x7f89bc06ea10 msgr2=0x7f89bc070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.417+0000 7f89c2a00640 1 -- 172.21.15.26:0/3426851138 shutdown_connections 2024-06-13T07:24:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.417+0000 7f89c2a00640 1 -- 172.21.15.26:0/3426851138 wait complete. 2024-06-13T07:24:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.418+0000 7f89c2a00640 1 Processor -- start 2024-06-13T07:24:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.419+0000 7f89c2a00640 1 -- start start 2024-06-13T07:24:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.419+0000 7f89c2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1089a0 0x7f89bc19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.419+0000 7f89c2a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89bc10b310 0x7f89bc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.419+0000 7f89c2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89bc10dcc0 0x7f89bc1a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89c2a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc111ca0 con 0x7f89bc1089a0 2024-06-13T07:24:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89bc10b310 0x7f89bc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89c2a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc111b20 con 0x7f89bc10b310 2024-06-13T07:24:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89c2a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc111e20 con 0x7f89bc10dcc0 2024-06-13T07:24:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89bc10b310 0x7f89bc19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:48346/0 (socket says 172.21.15.26:48346) 2024-06-13T07:24:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89b3e00640 1 -- 172.21.15.26:0/1951689919 learned_addr learned my addr 172.21.15.26:0/1951689919 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89c0a00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89bc10dcc0 0x7f89bc1a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89b3e00640 1 -- 172.21.15.26:0/1951689919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89bc10dcc0 msgr2=0x7f89bc1a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89b3e00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89bc10dcc0 0x7f89bc1a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89b3e00640 1 -- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1089a0 msgr2=0x7f89bc19e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89b3e00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1089a0 0x7f89bc19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.420+0000 7f89b3e00640 1 -- 172.21.15.26:0/1951689919 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc1a5e10 con 0x7f89bc10b310 2024-06-13T07:24:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.421+0000 7f89b3e00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89bc10b310 0x7f89bc19e730 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d950 tx=0x7f89a400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.422+0000 7f89b9600640 1 -- 172.21.15.26:0/1951689919 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4017070 con 0x7f89bc10b310 2024-06-13T07:24:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.422+0000 7f89b9600640 1 -- 172.21.15.26:0/1951689919 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a4009c30 con 0x7f89bc10b310 2024-06-13T07:24:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.422+0000 7f89b9600640 1 -- 172.21.15.26:0/1951689919 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a400f6b0 con 0x7f89bc10b310 2024-06-13T07:24:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.422+0000 7f89c2a00640 1 -- 172.21.15.26:0/1951689919 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc1a6100 con 0x7f89bc10b310 2024-06-13T07:24:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.423+0000 7f89c2a00640 1 -- 172.21.15.26:0/1951689919 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc1a6600 con 0x7f89bc10b310 2024-06-13T07:24:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.429+0000 7f89c57f5740 1 Processor -- start 2024-06-13T07:24:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.429+0000 7f89c57f5740 1 -- start start 2024-06-13T07:24:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.430+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 0x55db9b4bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.430+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.430+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4c2760 0x55db9b4c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.430+0000 7f89c57f5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55db9b420420 con 0x55db9b4bd5f0 2024-06-13T07:24:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.430+0000 7f89c57f5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db9b3c7c50 con 0x55db9b4bcc00 2024-06-13T07:24:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.430+0000 7f89c57f5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55db9b41c3e0 con 0x55db9b4c2760 2024-06-13T07:24:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.431+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 0x55db9b4bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.431+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.431+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51706/0 (socket says 172.21.15.26:51706) 2024-06-13T07:24:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.431+0000 7f89bbe00640 1 -- 172.21.15.26:0/1256808778 learned_addr learned my addr 172.21.15.26:0/1256808778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.432+0000 7f89bbe00640 1 -- 172.21.15.26:0/1256808778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4c2760 msgr2=0x55db9b4c4bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:24:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.432+0000 7f89bbe00640 1 --2- 172.21.15.26:0/1256808778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4c2760 0x55db9b4c4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.432+0000 7f89bbe00640 1 -- 172.21.15.26:0/1256808778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 msgr2=0x55db9b4bd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.432+0000 7f89bbe00640 1 --2- 172.21.15.26:0/1256808778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 0x55db9b4bd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.432+0000 7f89bbe00640 1 -- 172.21.15.26:0/1256808778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9b134210 con 0x55db9b4bd5f0 2024-06-13T07:24:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.433+0000 7f89bbe00640 1 --2- 172.21.15.26:0/1256808778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c20f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0035f0 tx=0x7f89ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d248be588271b745 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.434+0000 7f89b1200640 1 -- 172.21.15.26:0/1256808778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac051070 con 0x55db9b4bd5f0 2024-06-13T07:24:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.434+0000 7f89b1200640 1 -- 172.21.15.26:0/1256808778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac0046c0 con 0x55db9b4bd5f0 2024-06-13T07:24:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.434+0000 7f89b1200640 1 -- 172.21.15.26:0/1256808778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac042700 con 0x55db9b4bd5f0 2024-06-13T07:24:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.435+0000 7f89c57f5740 1 -- 172.21.15.26:0/1256808778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bd5f0 msgr2=0x55db9b4c20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.437+0000 7f89c57f5740 1 --2- 172.21.15.26:0/1256808778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c20f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0035f0 tx=0x7f89ac000bc0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.439+0000 7f89b9600640 1 -- 172.21.15.26:0/1951689919 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89a40109d0 con 0x7f89bc10b310 2024-06-13T07:24:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.441+0000 7f89c57f5740 1 -- 172.21.15.26:0/1256808778 shutdown_connections 2024-06-13T07:24:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.441+0000 7f89c57f5740 1 --2- 172.21.15.26:0/1256808778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4c2760 0x55db9b4c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.441+0000 7f89c57f5740 1 --2- 172.21.15.26:0/1256808778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c20f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.441+0000 7f89c57f5740 1 --2- 172.21.15.26:0/1256808778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 0x55db9b4bd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.441+0000 7f89c57f5740 1 -- 172.21.15.26:0/1256808778 >> 172.21.15.26:0/1256808778 conn(0x55db9b484fb0 msgr2=0x55db9b4b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.444+0000 7f89c57f5740 1 -- 172.21.15.26:0/1256808778 shutdown_connections 2024-06-13T07:24:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.445+0000 7f89b9600640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f89a00a2d00 0x7f89a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.445+0000 7f89b9600640 1 -- 172.21.15.26:0/1951689919 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f89a40cef90 con 0x7f89bc10b310 2024-06-13T07:24:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.449+0000 7f89c57f5740 1 -- 172.21.15.26:0/1256808778 wait complete. 2024-06-13T07:24:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.449+0000 7f89c57f5740 1 -- start start 2024-06-13T07:24:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.450+0000 7f89c57f5740 1 Processor -- start 2024-06-13T07:24:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.451+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bcc00 0x55db9b4c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.451+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.451+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4c2760 0x55db9b4ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.451+0000 7f89c57f5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55db9b4c56b0 con 0x55db9b4bcc00 2024-06-13T07:24:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.451+0000 7f89c57f5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db9b3db0b0 con 0x55db9b4bd5f0 2024-06-13T07:24:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.451+0000 7f89c57f5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55db9b4c5830 con 0x55db9b4c2760 2024-06-13T07:24:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.451+0000 7f89c57f5740 10 client.0 _ll_register_callbacks cb 0x55db9b3c6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:24:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.452+0000 7f89bbe00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f89a00a2d00 0x7f89a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.453+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.453+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:48358/0 (socket says 172.21.15.26:48358) 2024-06-13T07:24:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.453+0000 7f89b3e00640 1 -- 172.21.15.26:0/749469072 learned_addr learned my addr 172.21.15.26:0/749469072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.453+0000 7f89b3e00640 1 -- 172.21.15.26:0/749469072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4c2760 msgr2=0x55db9b4ce480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:24:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.453+0000 7f89b3e00640 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4c2760 0x55db9b4ce480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.453+0000 7f89b3e00640 1 -- 172.21.15.26:0/749469072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bcc00 msgr2=0x55db9b4c6f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:24:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.453+0000 7f89b3e00640 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bcc00 0x55db9b4c6f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.453+0000 7f89b3e00640 1 -- 172.21.15.26:0/749469072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9b200620 con 0x55db9b4bd5f0 2024-06-13T07:24:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.454+0000 7f89b3e00640 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c7440 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f89a40a3b40 tx=0x7f89a4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6584eb6c5d1e2aa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.454+0000 7f89a9600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:24:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.454+0000 7f89a9600640 1 -- 172.21.15.26:0/749469072 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4020050 con 0x55db9b4bd5f0 2024-06-13T07:24:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.454+0000 7f89a9600640 1 -- 172.21.15.26:0/749469072 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f89a40ccb60 con 0x55db9b4bd5f0 2024-06-13T07:24:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.455+0000 7f89c57f5740 1 -- 172.21.15.26:0/749469072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac0046c0 con 0x55db9b4bd5f0 2024-06-13T07:24:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.455+0000 7f89a9600640 1 -- 172.21.15.26:0/749469072 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f89a409b400 con 0x55db9b4bd5f0 2024-06-13T07:24:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.455+0000 7f89c57f5740 10 client.27191 fetch_fsmap learned FSMap version 49 2024-06-13T07:24:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.455+0000 7f89c57f5740 1 -- 172.21.15.26:0/749469072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55db9b4d1860 con 0x55db9b4bd5f0 2024-06-13T07:24:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.456+0000 7f89a9600640 1 -- 172.21.15.26:0/749469072 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac0046c0 con 0x55db9b4bd5f0 2024-06-13T07:24:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.456+0000 7f89a9600640 1 -- 172.21.15.26:0/749469072 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f89a409a850 con 0x55db9b4bd5f0 2024-06-13T07:24:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.456+0000 7f89c57f5740 10 client.27191 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:24:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.456+0000 7f89c57f5740 10 client.27191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:24:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.456+0000 7f89c57f5740 20 client.27191 populate_metadata read hostname 'smithi026' 2024-06-13T07:24:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.456+0000 7f89c57f5740 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db9b4d2f70 0x55db9b4d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.456+0000 7f89c57f5740 4 client.27191 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:24:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.456+0000 7f89c57f5740 1 -- 172.21.15.26:0/749469072 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9b42db50 con 0x55db9b4d2f70 2024-06-13T07:24:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.459+0000 7f89bbe00640 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db9b4d2f70 0x55db9b4d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.461+0000 7f89bbe00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f89a00a2d00 0x7f89a00a51c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f89ac01a910 tx=0x7f89ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.463+0000 7f89bbe00640 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db9b4d2f70 0x55db9b4d5360 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234afdfe13a70cec server_cookie=961e592f6c6b7bc2 in_seq=0 out_seq=0 2024-06-13T07:24:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.464+0000 7f89a9600640 10 client.27191 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.472+0000 7f89a9600640 1 -- 172.21.15.26:0/749469072 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db9b42db50 con 0x55db9b4d2f70 2024-06-13T07:24:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.473+0000 7f89a9600640 10 client.27191 handle_command_reply: tid=0 2024-06-13T07:24:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.475+0000 7f89c57f5740 1 client.27191 shutdown 2024-06-13T07:24:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.476+0000 7f89c57f5740 1 -- 172.21.15.26:0/749469072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 msgr2=0x55db9b4c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.476+0000 7f89c57f5740 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c7440 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f89a40a3b40 tx=0x7f89a4005f90 comp rx=0 tx=0).stop 2024-06-13T07:24:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.476+0000 7f89c57f5740 1 -- 172.21.15.26:0/749469072 shutdown_connections 2024-06-13T07:24:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.476+0000 7f89c57f5740 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db9b4d2f70 0x55db9b4d5360 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.476+0000 7f89c57f5740 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4c7440 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.476+0000 7f89a9600640 0 client.27191 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.477+0000 7f89c57f5740 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4c2760 0x55db9b4ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.477+0000 7f89c57f5740 1 --2- 172.21.15.26:0/749469072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4bcc00 0x55db9b4c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.477+0000 7f89c57f5740 1 -- 172.21.15.26:0/749469072 >> 172.21.15.26:0/749469072 conn(0x55db9b484fb0 msgr2=0x55db9b4b7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.478+0000 7f89c57f5740 1 -- 172.21.15.26:0/749469072 shutdown_connections 2024-06-13T07:24:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.478+0000 7f89c57f5740 1 -- 172.21.15.26:0/749469072 wait complete. 2024-06-13T07:24:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.478+0000 7f89c57f5740 20 client.27191 trim_cache size 0 max 16384 2024-06-13T07:24:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.528+0000 7f89c57f5740 1 Processor -- start 2024-06-13T07:24:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.529+0000 7f89c57f5740 1 -- start start 2024-06-13T07:24:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.529+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 0x55db9b4b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.529+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bd5f0 0x55db9b4ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.529+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4c6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.529+0000 7f89c57f5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b35d0 con 0x55db9b4c2760 2024-06-13T07:24:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.529+0000 7f89c57f5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b0e40 con 0x55db9b4bcc00 2024-06-13T07:24:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.529+0000 7f89c57f5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1aec90 con 0x55db9b4bd5f0 2024-06-13T07:24:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.531+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bd5f0 0x55db9b4ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.531+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bd5f0 0x55db9b4ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:45846/0 (socket says 172.21.15.26:45846) 2024-06-13T07:24:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.532+0000 7f89b3e00640 1 -- 172.21.15.26:0/1864374948 learned_addr learned my addr 172.21.15.26:0/1864374948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.533+0000 7f89c0a00640 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.533+0000 7f89bbe00640 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 0x55db9b4b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.533+0000 7f89b3e00640 1 -- 172.21.15.26:0/1864374948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 msgr2=0x55db9b4b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.533+0000 7f89b3e00640 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 0x55db9b4b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.533+0000 7f89b3e00640 1 -- 172.21.15.26:0/1864374948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 msgr2=0x55db9b4c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.533+0000 7f89b3e00640 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.533+0000 7f89b3e00640 1 -- 172.21.15.26:0/1864374948 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9b4bde80 con 0x55db9b4bd5f0 2024-06-13T07:24:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.534+0000 7f89b3e00640 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bd5f0 0x55db9b4ce440 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009610 tx=0x7f89a4099910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e141f119dcaddd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.534+0000 7f89abe00640 1 -- 172.21.15.26:0/1864374948 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4020040 con 0x55db9b4bd5f0 2024-06-13T07:24:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.535+0000 7f89abe00640 1 -- 172.21.15.26:0/1864374948 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a40b4200 con 0x55db9b4bd5f0 2024-06-13T07:24:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.535+0000 7f89abe00640 1 -- 172.21.15.26:0/1864374948 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4097a70 con 0x55db9b4bd5f0 2024-06-13T07:24:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.536+0000 7f89c57f5740 1 -- 172.21.15.26:0/1864374948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bd5f0 msgr2=0x55db9b4ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.536+0000 7f89c57f5740 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bd5f0 0x55db9b4ce440 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009610 tx=0x7f89a4099910 comp rx=0 tx=0).stop 2024-06-13T07:24:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.536+0000 7f89c57f5740 1 -- 172.21.15.26:0/1864374948 shutdown_connections 2024-06-13T07:24:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.536+0000 7f89c57f5740 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.536+0000 7f89c57f5740 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bd5f0 0x55db9b4ce440 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.536+0000 7f89c57f5740 1 --2- 172.21.15.26:0/1864374948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bcc00 0x55db9b4b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.536+0000 7f89c57f5740 1 -- 172.21.15.26:0/1864374948 >> 172.21.15.26:0/1864374948 conn(0x55db9b484fb0 msgr2=0x55db9b4b8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.537+0000 7f89c57f5740 1 -- 172.21.15.26:0/1864374948 shutdown_connections 2024-06-13T07:24:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.537+0000 7f89c57f5740 1 -- 172.21.15.26:0/1864374948 wait complete. 2024-06-13T07:24:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.538+0000 7f89c57f5740 1 -- start start 2024-06-13T07:24:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.538+0000 7f89c57f5740 1 Processor -- start 2024-06-13T07:24:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.539+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bcc00 0x55db9b4cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.539+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.539+0000 7f89c57f5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.540+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.540+0000 7f89c57f5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1af300 con 0x55db9b4c2760 2024-06-13T07:24:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.540+0000 7f89c57f5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b0770 con 0x55db9b4bd5f0 2024-06-13T07:24:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.540+0000 7f89c57f5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1af9d0 con 0x55db9b4bcc00 2024-06-13T07:24:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.540+0000 7f89c57f5740 10 client.0 _ll_register_callbacks cb 0x7f89a400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:24:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.540+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51714/0 (socket says 172.21.15.26:51714) 2024-06-13T07:24:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.540+0000 7f89c0a00640 1 -- 172.21.15.26:0/2070667687 learned_addr learned my addr 172.21.15.26:0/2070667687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.540+0000 7f89bbe00640 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bcc00 0x55db9b4cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.541+0000 7f89b3e00640 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.541+0000 7f89c0a00640 1 -- 172.21.15.26:0/2070667687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bcc00 msgr2=0x55db9b4cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.541+0000 7f89c0a00640 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bcc00 0x55db9b4cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.541+0000 7f89c0a00640 1 -- 172.21.15.26:0/2070667687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 msgr2=0x55db9b4cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.541+0000 7f89c0a00640 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.541+0000 7f89c0a00640 1 -- 172.21.15.26:0/2070667687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9b4cd4b0 con 0x55db9b4c2760 2024-06-13T07:24:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.542+0000 7f89c0a00640 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4ccec0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f89b40042f0 tx=0x7f89b40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57acee93d6141350 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.542+0000 7f89a9600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:24:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.543+0000 7f89a9600640 1 -- 172.21.15.26:0/2070667687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4009e30 con 0x55db9b4c2760 2024-06-13T07:24:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.544+0000 7f89a9600640 1 -- 172.21.15.26:0/2070667687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f89b4011ad0 con 0x55db9b4c2760 2024-06-13T07:24:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.544+0000 7f89c57f5740 1 -- 172.21.15.26:0/2070667687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a40b4200 con 0x55db9b4c2760 2024-06-13T07:24:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.548+0000 7f89a9600640 1 -- 172.21.15.26:0/2070667687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f89b4011cf0 con 0x55db9b4c2760 2024-06-13T07:24:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.549+0000 7f89c57f5740 10 client.16893 fetch_fsmap learned FSMap version 49 2024-06-13T07:24:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.550+0000 7f89c57f5740 1 -- 172.21.15.26:0/2070667687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55db9b53dc20 con 0x55db9b4c2760 2024-06-13T07:24:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.551+0000 7f89a9600640 1 -- 172.21.15.26:0/2070667687 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a40b4200 con 0x55db9b4c2760 2024-06-13T07:24:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.551+0000 7f89a9600640 1 -- 172.21.15.26:0/2070667687 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f89b400f4b0 con 0x55db9b4c2760 2024-06-13T07:24:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.551+0000 7f89c57f5740 10 client.16893 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:24:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.551+0000 7f89c57f5740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:24:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.551+0000 7f89c57f5740 20 client.16893 populate_metadata read hostname 'smithi026' 2024-06-13T07:24:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.551+0000 7f89c57f5740 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db9b4d2f70 0x55db9b540650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.551+0000 7f89c57f5740 4 client.16893 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:24:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.551+0000 7f89c57f5740 1 -- 172.21.15.26:0/2070667687 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db9b4c6cf0 con 0x55db9b4d2f70 2024-06-13T07:24:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.552+0000 7f89bbe00640 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db9b4d2f70 0x55db9b540650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.556+0000 7f89bbe00640 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db9b4d2f70 0x55db9b540650 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cdcd5df595d1ae server_cookie=388b0d3b7ff802ff in_seq=0 out_seq=0 2024-06-13T07:24:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:40.556+0000 7f89a9600640 10 client.16893 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:40 smithi136 ceph-mon[37229]: pgmap v1234: 129 pgs: 129 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-06-13T07:24:40.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:40 smithi026 ceph-mon[32041]: pgmap v1234: 129 pgs: 129 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-06-13T07:24:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:40 smithi135 ceph-mon[36276]: pgmap v1234: 129 pgs: 129 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-06-13T07:24:41.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 313149 25.17 MB/sec execute 534 sec latency 836.079 ms 2024-06-13T07:24:42.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 313264 25.13 MB/sec execute 535 sec latency 1079.301 ms 2024-06-13T07:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.378+0000 7f89a9600640 1 -- 172.21.15.26:0/2070667687 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55db9b4c6cf0 con 0x55db9b4d2f70 2024-06-13T07:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.378+0000 7f89a9600640 10 client.16893 handle_command_reply: tid=0 2024-06-13T07:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.378+0000 7f89c57f5740 1 client.16893 shutdown 2024-06-13T07:24:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 -- 172.21.15.26:0/2070667687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 msgr2=0x55db9b4ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4ccec0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f89b40042f0 tx=0x7f89b40047b0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 -- 172.21.15.26:0/2070667687 shutdown_connections 2024-06-13T07:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55db9b4d2f70 0x55db9b540650 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db9b4bd5f0 0x55db9b4cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db9b4c2760 0x55db9b4ccec0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 --2- 172.21.15.26:0/2070667687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55db9b4bcc00 0x55db9b4cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 -- 172.21.15.26:0/2070667687 >> 172.21.15.26:0/2070667687 conn(0x55db9b484fb0 msgr2=0x55db9b4b8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89a9600640 0 client.16893 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.380+0000 7f89c57f5740 1 -- 172.21.15.26:0/2070667687 shutdown_connections 2024-06-13T07:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.381+0000 7f89c57f5740 1 -- 172.21.15.26:0/2070667687 wait complete. 2024-06-13T07:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.381+0000 7f89c57f5740 20 client.16893 trim_cache size 0 max 16384 2024-06-13T07:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 -- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f89a00a2d00 msgr2=0x7f89a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f89a00a2d00 0x7f89a00a51c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f89ac01a910 tx=0x7f89ac005fd0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 -- 172.21.15.26:0/1951689919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89bc10b310 msgr2=0x7f89bc19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89bc10b310 0x7f89bc19e730 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d950 tx=0x7f89a400de10 comp rx=0 tx=0).stop 2024-06-13T07:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 -- 172.21.15.26:0/1951689919 shutdown_connections 2024-06-13T07:24:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f89a00a2d00 0x7f89a00a51c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89bc10b310 0x7f89bc19e730 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89bc10dcc0 0x7f89bc1a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 --2- 172.21.15.26:0/1951689919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1089a0 0x7f89bc19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.382+0000 7f89c2a00640 1 -- 172.21.15.26:0/1951689919 >> 172.21.15.26:0/1951689919 conn(0x7f89bc06ea10 msgr2=0x7f89bc109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.383+0000 7f89c2a00640 1 -- 172.21.15.26:0/1951689919 shutdown_connections 2024-06-13T07:24:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:42.383+0000 7f89c2a00640 1 -- 172.21.15.26:0/1951689919 wait complete. 2024-06-13T07:24:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:42 smithi136 ceph-mon[37229]: pgmap v1235: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 387 op/s 2024-06-13T07:24:42.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:42 smithi026 ceph-mon[32041]: pgmap v1235: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 387 op/s 2024-06-13T07:24:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:42 smithi135 ceph-mon[36276]: pgmap v1235: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 387 op/s 2024-06-13T07:24:43.253 INFO:tasks.workunit.client.0.smithi026.stdout: 10 313480 25.10 MB/sec execute 536 sec latency 233.522 ms 2024-06-13T07:24:44.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 313725 25.08 MB/sec execute 537 sec latency 172.434 ms 2024-06-13T07:24:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:44 smithi136 ceph-mon[37229]: pgmap v1236: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.7 MiB/s wr, 467 op/s 2024-06-13T07:24:44.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:44 smithi026 ceph-mon[32041]: pgmap v1236: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.7 MiB/s wr, 467 op/s 2024-06-13T07:24:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:44 smithi135 ceph-mon[36276]: pgmap v1236: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.7 MiB/s wr, 467 op/s 2024-06-13T07:24:45.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 314259 25.08 MB/sec execute 538 sec latency 111.027 ms 2024-06-13T07:24:46.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 314721 25.08 MB/sec execute 539 sec latency 315.010 ms 2024-06-13T07:24:46.584 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:46 smithi135 ceph-mon[36276]: pgmap v1237: 129 pgs: 129 active+clean; 705 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 524 op/s 2024-06-13T07:24:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:46 smithi136 ceph-mon[37229]: pgmap v1237: 129 pgs: 129 active+clean; 705 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 524 op/s 2024-06-13T07:24:46.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:46 smithi026 ceph-mon[32041]: pgmap v1237: 129 pgs: 129 active+clean; 705 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 524 op/s 2024-06-13T07:24:47.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 314987 25.06 MB/sec execute 540 sec latency 339.785 ms 2024-06-13T07:24:48.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 315332 25.04 MB/sec execute 541 sec latency 233.555 ms 2024-06-13T07:24:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:48 smithi136 ceph-mon[37229]: pgmap v1238: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.3 MiB/s wr, 519 op/s 2024-06-13T07:24:48.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:48 smithi026 ceph-mon[32041]: pgmap v1238: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.3 MiB/s wr, 519 op/s 2024-06-13T07:24:48.754 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:48 smithi135 ceph-mon[36276]: pgmap v1238: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.3 MiB/s wr, 519 op/s 2024-06-13T07:24:49.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 315946 25.08 MB/sec execute 542 sec latency 83.937 ms 2024-06-13T07:24:50.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 316307 25.07 MB/sec execute 543 sec latency 97.516 ms 2024-06-13T07:24:50.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:50 smithi136 ceph-mon[37229]: pgmap v1239: 129 pgs: 129 active+clean; 690 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-06-13T07:24:50.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:50 smithi026 ceph-mon[32041]: pgmap v1239: 129 pgs: 129 active+clean; 690 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-06-13T07:24:50.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:50 smithi135 ceph-mon[36276]: pgmap v1239: 129 pgs: 129 active+clean; 690 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-06-13T07:24:51.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 316713 25.05 MB/sec execute 544 sec latency 82.677 ms 2024-06-13T07:24:52.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 317288 25.05 MB/sec execute 545 sec latency 65.751 ms 2024-06-13T07:24:52.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:52 smithi136 ceph-mon[37229]: pgmap v1240: 129 pgs: 129 active+clean; 683 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.6 MiB/s wr, 577 op/s 2024-06-13T07:24:52.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:52 smithi026 ceph-mon[32041]: pgmap v1240: 129 pgs: 129 active+clean; 683 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.6 MiB/s wr, 577 op/s 2024-06-13T07:24:52.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:52 smithi135 ceph-mon[36276]: pgmap v1240: 129 pgs: 129 active+clean; 683 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.6 MiB/s wr, 577 op/s 2024-06-13T07:24:53.254 INFO:tasks.workunit.client.0.smithi026.stdout: 10 317834 25.05 MB/sec execute 546 sec latency 148.240 ms 2024-06-13T07:24:54.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 318622 25.10 MB/sec execute 547 sec latency 49.153 ms 2024-06-13T07:24:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:54 smithi136 ceph-mon[37229]: pgmap v1241: 129 pgs: 129 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.0 MiB/s wr, 638 op/s 2024-06-13T07:24:54.751 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:54 smithi026 ceph-mon[32041]: pgmap v1241: 129 pgs: 129 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.0 MiB/s wr, 638 op/s 2024-06-13T07:24:54.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:54 smithi135 ceph-mon[36276]: pgmap v1241: 129 pgs: 129 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.0 MiB/s wr, 638 op/s 2024-06-13T07:24:55.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 319122 25.11 MB/sec execute 548 sec latency 54.891 ms 2024-06-13T07:24:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.575+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1678929604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec8074400 msgr2=0x7f2ec8074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.575+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1678929604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec8074400 0x7f2ec8074880 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009900 tx=0x7f2eb402a400 comp rx=0 tx=0).stop 2024-06-13T07:24:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.576+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1678929604 shutdown_connections 2024-06-13T07:24:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.576+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1678929604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ec810dfe0 0x7f2ec81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.576+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1678929604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec8074400 0x7f2ec8074880 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.576+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1678929604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec80759e0 0x7f2ec8073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.576+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1678929604 >> 172.21.15.26:0/1678929604 conn(0x7f2ec80fcf60 msgr2=0x7f2ec80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.576+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1678929604 shutdown_connections 2024-06-13T07:24:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.576+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1678929604 wait complete. 2024-06-13T07:24:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.577+0000 7f2ecfe00640 1 Processor -- start 2024-06-13T07:24:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.577+0000 7f2ecfe00640 1 -- start start 2024-06-13T07:24:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.577+0000 7f2ecfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ec8074400 0x7f2ec819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.577+0000 7f2ecfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec80759e0 0x7f2ec819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.577+0000 7f2ecfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec810dfe0 0x7f2ec81a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.577+0000 7f2ecfe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8113c60 con 0x7f2ec810dfe0 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.577+0000 7f2ecfe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8113ae0 con 0x7f2ec8074400 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.577+0000 7f2ecfe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8113de0 con 0x7f2ec80759e0 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ec8074400 0x7f2ec819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ec8074400 0x7f2ec819e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56352/0 (socket says 172.21.15.26:56352) 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 -- 172.21.15.26:0/1706659030 learned_addr learned my addr 172.21.15.26:0/1706659030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecca00640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec80759e0 0x7f2ec819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecde00640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec810dfe0 0x7f2ec81a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 -- 172.21.15.26:0/1706659030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec80759e0 msgr2=0x7f2ec819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec80759e0 0x7f2ec819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 -- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec810dfe0 msgr2=0x7f2ec81a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec810dfe0 0x7f2ec81a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 -- 172.21.15.26:0/1706659030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec81a5e30 con 0x7f2ec8074400 2024-06-13T07:24:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.578+0000 7f2ecd400640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ec8074400 0x7f2ec819e210 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000ebc0 tx=0x7f2ec000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.579+0000 7f2ec6000640 1 -- 172.21.15.26:0/1706659030 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0010630 con 0x7f2ec8074400 2024-06-13T07:24:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.579+0000 7f2ec6000640 1 -- 172.21.15.26:0/1706659030 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec0010cb0 con 0x7f2ec8074400 2024-06-13T07:24:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.579+0000 7f2ec6000640 1 -- 172.21.15.26:0/1706659030 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0018bd0 con 0x7f2ec8074400 2024-06-13T07:24:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.579+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1706659030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec81a6120 con 0x7f2ec8074400 2024-06-13T07:24:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.579+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1706659030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81a65f0 con 0x7f2ec8074400 2024-06-13T07:24:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.583+0000 7f2ed2c27740 1 Processor -- start 2024-06-13T07:24:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.583+0000 7f2ed2c27740 1 -- start start 2024-06-13T07:24:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.584+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9aaf7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.584+0000 7f2ec6000640 1 -- 172.21.15.26:0/1706659030 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ec00107d0 con 0x7f2ec8074400 2024-06-13T07:24:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9aafc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 0x560d9aafebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d9aa5a420 con 0x560d9aaf75f0 2024-06-13T07:24:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560d9aa01c50 con 0x560d9aafc760 2024-06-13T07:24:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d9aa563e0 con 0x560d9aaf6c00 2024-06-13T07:24:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ecde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9aaf7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ecde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9aaf7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:34514/0 (socket says 172.21.15.26:34514) 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ecde00640 1 -- 172.21.15.26:0/1110674969 learned_addr learned my addr 172.21.15.26:0/1110674969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.585+0000 7f2ecca00640 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 0x560d9aafebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.586+0000 7f2ecd400640 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9aafc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.586+0000 7f2ecde00640 1 -- 172.21.15.26:0/1110674969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 msgr2=0x560d9aafebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.586+0000 7f2ecde00640 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 0x560d9aafebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.586+0000 7f2ecde00640 1 -- 172.21.15.26:0/1110674969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 msgr2=0x560d9aafc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.586+0000 7f2ecde00640 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9aafc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.586+0000 7f2ecde00640 1 -- 172.21.15.26:0/1110674969 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9a76e210 con 0x560d9aaf6c00 2024-06-13T07:24:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.586+0000 7f2ecd400640 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9aafc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:24:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.587+0000 7f2ecde00640 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9aaf7020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800cbe0 tx=0x7f2eb800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2603b7d0e45916a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.587+0000 7f2ebe600640 1 -- 172.21.15.26:0/1110674969 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8005220 con 0x560d9aaf6c00 2024-06-13T07:24:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.587+0000 7f2ec6000640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2eac0a2cb0 0x7f2eac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.587+0000 7f2ebe600640 1 -- 172.21.15.26:0/1110674969 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb8004c80 con 0x560d9aaf6c00 2024-06-13T07:24:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.587+0000 7f2ebe600640 1 -- 172.21.15.26:0/1110674969 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8010810 con 0x560d9aaf6c00 2024-06-13T07:24:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.587+0000 7f2ec6000640 1 -- 172.21.15.26:0/1706659030 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2ec00cfa80 con 0x7f2ec8074400 2024-06-13T07:24:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.588+0000 7f2ecd400640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2eac0a2cb0 0x7f2eac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.588+0000 7f2ed2c27740 1 -- 172.21.15.26:0/1110674969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 msgr2=0x560d9aaf7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.588+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9aaf7020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800cbe0 tx=0x7f2eb800e720 comp rx=0 tx=0).stop 2024-06-13T07:24:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.589+0000 7f2ed2c27740 1 -- 172.21.15.26:0/1110674969 shutdown_connections 2024-06-13T07:24:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.589+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 0x560d9aafebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.589+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9aafc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.589+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/1110674969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9aaf7020 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.589+0000 7f2ed2c27740 1 -- 172.21.15.26:0/1110674969 >> 172.21.15.26:0/1110674969 conn(0x560d9aabefb0 msgr2=0x560d9aaf2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.589+0000 7f2ed2c27740 1 -- 172.21.15.26:0/1110674969 shutdown_connections 2024-06-13T07:24:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.589+0000 7f2ed2c27740 1 -- 172.21.15.26:0/1110674969 wait complete. 2024-06-13T07:24:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.589+0000 7f2ecd400640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2eac0a2cb0 0x7f2eac0a5170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x560d9aaf79f0 tx=0x7f2ec000cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.590+0000 7f2ed2c27740 1 -- start start 2024-06-13T07:24:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.590+0000 7f2ed2c27740 1 Processor -- start 2024-06-13T07:24:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.592+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9ab00f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.592+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab01440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.592+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 0x560d9ab08480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.592+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d9aaff710 con 0x560d9aaf75f0 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.592+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560d9aa150b0 con 0x560d9aafc760 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.592+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d9aaff890 con 0x560d9aaf6c00 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.593+0000 7f2ecca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab01440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab01440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45854/0 (socket says 172.21.15.26:45854) 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecca00640 1 -- 172.21.15.26:0/78462786 learned_addr learned my addr 172.21.15.26:0/78462786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ed2c27740 10 client.0 _ll_register_callbacks cb 0x560d9aa00f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:24:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecde00640 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 0x560d9ab08480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecde00640 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9ab00f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecca00640 1 -- 172.21.15.26:0/78462786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 msgr2=0x560d9ab00f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecca00640 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9ab00f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecca00640 1 -- 172.21.15.26:0/78462786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 msgr2=0x560d9ab08480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecca00640 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 0x560d9ab08480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.594+0000 7f2ecca00640 1 -- 172.21.15.26:0/78462786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9a83a620 con 0x560d9aaf75f0 2024-06-13T07:24:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.595+0000 7f2ecca00640 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab01440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2eb402a920 tx=0x7f2eb40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50b542ea15dee6e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.595+0000 7f2eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:24:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.596+0000 7f2eb2a00640 1 -- 172.21.15.26:0/78462786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4035030 con 0x560d9aaf75f0 2024-06-13T07:24:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.596+0000 7f2eb2a00640 1 -- 172.21.15.26:0/78462786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2eb4031a50 con 0x560d9aaf75f0 2024-06-13T07:24:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.596+0000 7f2ed2c27740 1 -- 172.21.15.26:0/78462786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb8004c80 con 0x560d9aaf75f0 2024-06-13T07:24:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.597+0000 7f2eb2a00640 1 -- 172.21.15.26:0/78462786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2eb4031c50 con 0x560d9aaf75f0 2024-06-13T07:24:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.597+0000 7f2ed2c27740 10 client.16902 fetch_fsmap learned FSMap version 49 2024-06-13T07:24:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.597+0000 7f2ed2c27740 1 -- 172.21.15.26:0/78462786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560d9ab0b7d0 con 0x560d9aaf75f0 2024-06-13T07:24:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.598+0000 7f2eb2a00640 1 -- 172.21.15.26:0/78462786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb8004c80 con 0x560d9aaf75f0 2024-06-13T07:24:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.598+0000 7f2eb2a00640 1 -- 172.21.15.26:0/78462786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f2eb403c510 con 0x560d9aaf75f0 2024-06-13T07:24:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.598+0000 7f2ed2c27740 10 client.16902 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:24:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.598+0000 7f2ed2c27740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:24:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.598+0000 7f2ed2c27740 20 client.16902 populate_metadata read hostname 'smithi026' 2024-06-13T07:24:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.598+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560d9ab0cea0 0x560d9ab0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.598+0000 7f2ed2c27740 4 client.16902 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:24:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.598+0000 7f2ed2c27740 1 -- 172.21.15.26:0/78462786 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d9aa67b50 con 0x560d9ab0cea0 2024-06-13T07:24:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.599+0000 7f2ecde00640 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560d9ab0cea0 0x560d9ab0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.601+0000 7f2ecde00640 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560d9ab0cea0 0x560d9ab0f2b0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c527f49e6fe1548 server_cookie=abbbcb58bd59ce6d in_seq=0 out_seq=0 2024-06-13T07:24:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.601+0000 7f2eb2a00640 10 client.16902 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.610+0000 7f2eb2a00640 1 -- 172.21.15.26:0/78462786 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d9aa67b50 con 0x560d9ab0cea0 2024-06-13T07:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.610+0000 7f2eb2a00640 10 client.16902 handle_command_reply: tid=0 2024-06-13T07:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 client.16902 shutdown 2024-06-13T07:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 -- 172.21.15.26:0/78462786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 msgr2=0x560d9ab01440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab01440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2eb402a920 tx=0x7f2eb40042e0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 -- 172.21.15.26:0/78462786 shutdown_connections 2024-06-13T07:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560d9ab0cea0 0x560d9ab0f2b0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aafc760 0x560d9ab08480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab01440 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/78462786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aaf6c00 0x560d9ab00f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2eb2a00640 0 client.16902 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 -- 172.21.15.26:0/78462786 >> 172.21.15.26:0/78462786 conn(0x560d9aabefb0 msgr2=0x560d9aaf1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.611+0000 7f2ed2c27740 1 -- 172.21.15.26:0/78462786 shutdown_connections 2024-06-13T07:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.612+0000 7f2ed2c27740 1 -- 172.21.15.26:0/78462786 wait complete. 2024-06-13T07:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.612+0000 7f2ed2c27740 20 client.16902 trim_cache size 0 max 16384 2024-06-13T07:24:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.637+0000 7f2ed2c27740 1 Processor -- start 2024-06-13T07:24:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.637+0000 7f2ed2c27740 1 -- start start 2024-06-13T07:24:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.638+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf6c00 0x560d9aaf2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.638+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf75f0 0x560d9ab08440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.638+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9aaed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.638+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec81b3640 con 0x560d9aaf6c00 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.638+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec81b0eb0 con 0x560d9aaf75f0 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.638+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec81aed00 con 0x560d9aafc760 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf75f0 0x560d9ab08440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf75f0 0x560d9ab08440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56372/0 (socket says 172.21.15.26:56372) 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecca00640 1 -- 172.21.15.26:0/4070344907 learned_addr learned my addr 172.21.15.26:0/4070344907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecde00640 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9aaed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecde00640 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf6c00 0x560d9aaf2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecca00640 1 -- 172.21.15.26:0/4070344907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 msgr2=0x560d9aaed8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecca00640 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9aaed8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecca00640 1 -- 172.21.15.26:0/4070344907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf6c00 msgr2=0x560d9aaf2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecca00640 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf6c00 0x560d9aaf2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecca00640 1 -- 172.21.15.26:0/4070344907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9aaf7e80 con 0x560d9aaf75f0 2024-06-13T07:24:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.639+0000 7f2ecde00640 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9aaed8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:24:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.640+0000 7f2ecca00640 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf75f0 0x560d9ab08440 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009900 tx=0x7f2eb402a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f811758a2435aa21 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.640+0000 7f2ebdc00640 1 -- 172.21.15.26:0/4070344907 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4035040 con 0x560d9aaf75f0 2024-06-13T07:24:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.641+0000 7f2ebdc00640 1 -- 172.21.15.26:0/4070344907 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4040070 con 0x560d9aaf75f0 2024-06-13T07:24:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.641+0000 7f2ebdc00640 1 -- 172.21.15.26:0/4070344907 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4031ab0 con 0x560d9aaf75f0 2024-06-13T07:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.641+0000 7f2ed2c27740 1 -- 172.21.15.26:0/4070344907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf75f0 msgr2=0x560d9ab08440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.641+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf75f0 0x560d9ab08440 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009900 tx=0x7f2eb402a400 comp rx=0 tx=0).stop 2024-06-13T07:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.642+0000 7f2ed2c27740 1 -- 172.21.15.26:0/4070344907 shutdown_connections 2024-06-13T07:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.642+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf75f0 0x560d9ab08440 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.642+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9aaed8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.642+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/4070344907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf6c00 0x560d9aaf2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.642+0000 7f2ed2c27740 1 -- 172.21.15.26:0/4070344907 >> 172.21.15.26:0/4070344907 conn(0x560d9aabefb0 msgr2=0x560d9aaf2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.642+0000 7f2ed2c27740 1 -- 172.21.15.26:0/4070344907 shutdown_connections 2024-06-13T07:24:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.642+0000 7f2ed2c27740 1 -- 172.21.15.26:0/4070344907 wait complete. 2024-06-13T07:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.643+0000 7f2ed2c27740 1 -- start start 2024-06-13T07:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.643+0000 7f2ed2c27740 1 Processor -- start 2024-06-13T07:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf6c00 0x560d9ab05740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab05c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ed2c27740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9ab06c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec81af370 con 0x560d9aaf75f0 2024-06-13T07:24:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec81b07e0 con 0x560d9aaf6c00 2024-06-13T07:24:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ed2c27740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec81afa40 con 0x560d9aafc760 2024-06-13T07:24:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ed2c27740 10 client.0 _ll_register_callbacks cb 0x7f2eb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ecde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf6c00 0x560d9ab05740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.644+0000 7f2ecca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab05c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf6c00 0x560d9ab05740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:56376/0 (socket says 172.21.15.26:56376) 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab05c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45882/0 (socket says 172.21.15.26:45882) 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecde00640 1 -- 172.21.15.26:0/3884717179 learned_addr learned my addr 172.21.15.26:0/3884717179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecde00640 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9ab06c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecca00640 1 -- 172.21.15.26:0/3884717179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 msgr2=0x560d9ab06c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecca00640 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9ab06c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecca00640 1 -- 172.21.15.26:0/3884717179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf6c00 msgr2=0x560d9ab05740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecca00640 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf6c00 0x560d9ab05740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.645+0000 7f2ecca00640 1 -- 172.21.15.26:0/3884717179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9ab07280 con 0x560d9aaf75f0 2024-06-13T07:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.646+0000 7f2ecde00640 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9ab06c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.646+0000 7f2eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:24:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.646+0000 7f2ecca00640 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab05c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4031470 tx=0x7f2eb402a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34b651b688b534 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:24:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.646+0000 7f2eb2a00640 1 -- 172.21.15.26:0/3884717179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb403c640 con 0x560d9aaf75f0 2024-06-13T07:24:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.646+0000 7f2eb2a00640 1 -- 172.21.15.26:0/3884717179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2eb4005020 con 0x560d9aaf75f0 2024-06-13T07:24:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.647+0000 7f2ed2c27740 1 -- 172.21.15.26:0/3884717179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb4040070 con 0x560d9aaf75f0 2024-06-13T07:24:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.647+0000 7f2eb2a00640 1 -- 172.21.15.26:0/3884717179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2eb4005850 con 0x560d9aaf75f0 2024-06-13T07:24:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.647+0000 7f2ed2c27740 10 client.16908 fetch_fsmap learned FSMap version 49 2024-06-13T07:24:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.647+0000 7f2ed2c27740 1 -- 172.21.15.26:0/3884717179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560d9ab08590 con 0x560d9aaf75f0 2024-06-13T07:24:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.648+0000 7f2eb2a00640 1 -- 172.21.15.26:0/3884717179 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4040070 con 0x560d9aaf75f0 2024-06-13T07:24:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.648+0000 7f2eb2a00640 1 -- 172.21.15.26:0/3884717179 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f2eb4003780 con 0x560d9aaf75f0 2024-06-13T07:24:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.648+0000 7f2ed2c27740 10 client.16908 fetch_fsmap finished waiting for FSMap version 49 2024-06-13T07:24:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.648+0000 7f2ed2c27740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:24:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.648+0000 7f2ed2c27740 20 client.16908 populate_metadata read hostname 'smithi026' 2024-06-13T07:24:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.648+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560d9ab78040 0x560d9ab7a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:24:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.648+0000 7f2ed2c27740 4 client.16908 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:24:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.648+0000 7f2ed2c27740 1 -- 172.21.15.26:0/3884717179 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d9aa61ea0 con 0x560d9ab78040 2024-06-13T07:24:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.649+0000 7f2ecde00640 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560d9ab78040 0x560d9ab7a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:24:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.650+0000 7f2ecde00640 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560d9ab78040 0x560d9ab7a460 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b777a4874324a6 server_cookie=4fb44e504e332d20 in_seq=0 out_seq=0 2024-06-13T07:24:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:55.650+0000 7f2eb2a00640 10 client.16908 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:56.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 319398 25.09 MB/sec execute 549 sec latency 370.913 ms 2024-06-13T07:24:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.352+0000 7f2eb2a00640 1 -- 172.21.15.26:0/3884717179 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560d9aa61ea0 con 0x560d9ab78040 2024-06-13T07:24:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.352+0000 7f2eb2a00640 10 client.16908 handle_command_reply: tid=0 2024-06-13T07:24:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.352+0000 7f2ed2c27740 1 client.16908 shutdown 2024-06-13T07:24:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.352+0000 7f2ed2c27740 1 -- 172.21.15.26:0/3884717179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 msgr2=0x560d9ab05c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.352+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab05c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4031470 tx=0x7f2eb402a9a0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.353+0000 7f2ed2c27740 1 -- 172.21.15.26:0/3884717179 shutdown_connections 2024-06-13T07:24:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.353+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560d9ab78040 0x560d9ab7a460 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.353+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d9aafc760 0x560d9ab06c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.353+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d9aaf75f0 0x560d9ab05c80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.353+0000 7f2ed2c27740 1 --2- 172.21.15.26:0/3884717179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d9aaf6c00 0x560d9ab05740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.353+0000 7f2ed2c27740 1 -- 172.21.15.26:0/3884717179 >> 172.21.15.26:0/3884717179 conn(0x560d9aabefb0 msgr2=0x560d9aaf0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.353+0000 7f2eb2a00640 0 client.16908 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:24:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.353+0000 7f2ed2c27740 1 -- 172.21.15.26:0/3884717179 shutdown_connections 2024-06-13T07:24:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.354+0000 7f2ed2c27740 1 -- 172.21.15.26:0/3884717179 wait complete. 2024-06-13T07:24:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.354+0000 7f2ed2c27740 20 client.16908 trim_cache size 0 max 16384 2024-06-13T07:24:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:24:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:24:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2eac0a2cb0 msgr2=0x7f2eac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2eac0a2cb0 0x7f2eac0a5170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x560d9aaf79f0 tx=0x7f2ec000cf90 comp rx=0 tx=0).stop 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1706659030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ec8074400 msgr2=0x7f2ec819e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ec8074400 0x7f2ec819e210 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000ebc0 tx=0x7f2ec000c6a0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1706659030 shutdown_connections 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f2eac0a2cb0 0x7f2eac0a5170 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ec810dfe0 0x7f2ec81a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec80759e0 0x7f2ec819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 --2- 172.21.15.26:0/1706659030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ec8074400 0x7f2ec819e210 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:24:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.355+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1706659030 >> 172.21.15.26:0/1706659030 conn(0x7f2ec80fcf60 msgr2=0x7f2ec8101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:24:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.356+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1706659030 shutdown_connections 2024-06-13T07:24:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:24:56.356+0000 7f2ecfe00640 1 -- 172.21.15.26:0/1706659030 wait complete. 2024-06-13T07:24:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:56 smithi136 ceph-mon[37229]: pgmap v1242: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 555 op/s 2024-06-13T07:24:56.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:56 smithi026 ceph-mon[32041]: pgmap v1242: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 555 op/s 2024-06-13T07:24:56.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:56 smithi135 ceph-mon[36276]: pgmap v1242: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 555 op/s 2024-06-13T07:24:57.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 319785 25.09 MB/sec execute 550 sec latency 221.981 ms 2024-06-13T07:24:58.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 320209 25.08 MB/sec execute 551 sec latency 104.243 ms 2024-06-13T07:24:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:24:58 smithi136 ceph-mon[37229]: pgmap v1243: 129 pgs: 129 active+clean; 671 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-13T07:24:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:24:58 smithi026 ceph-mon[32041]: pgmap v1243: 129 pgs: 129 active+clean; 671 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-13T07:24:58.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:24:58 smithi135 ceph-mon[36276]: pgmap v1243: 129 pgs: 129 active+clean; 671 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-13T07:24:59.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 320721 25.06 MB/sec execute 552 sec latency 153.060 ms 2024-06-13T07:25:00.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 321285 25.08 MB/sec execute 553 sec latency 99.694 ms 2024-06-13T07:25:00.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:00 smithi135 ceph-mon[36276]: pgmap v1244: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-06-13T07:25:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:00 smithi136 ceph-mon[37229]: pgmap v1244: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-06-13T07:25:00.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:00 smithi026 ceph-mon[32041]: pgmap v1244: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-06-13T07:25:01.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 322072 25.11 MB/sec execute 554 sec latency 67.720 ms 2024-06-13T07:25:02.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 322703 25.15 MB/sec execute 555 sec latency 63.966 ms 2024-06-13T07:25:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:02 smithi136 ceph-mon[37229]: pgmap v1245: 129 pgs: 129 active+clean; 653 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 448 op/s 2024-06-13T07:25:02.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:02 smithi026 ceph-mon[32041]: pgmap v1245: 129 pgs: 129 active+clean; 653 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 448 op/s 2024-06-13T07:25:02.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:02 smithi135 ceph-mon[36276]: pgmap v1245: 129 pgs: 129 active+clean; 653 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 448 op/s 2024-06-13T07:25:03.255 INFO:tasks.workunit.client.0.smithi026.stdout: 10 323056 25.14 MB/sec execute 556 sec latency 118.275 ms 2024-06-13T07:25:04.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 323333 25.12 MB/sec execute 557 sec latency 357.043 ms 2024-06-13T07:25:04.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:04 smithi026 ceph-mon[32041]: pgmap v1246: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.2 MiB/s wr, 501 op/s 2024-06-13T07:25:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:04 smithi135 ceph-mon[36276]: pgmap v1246: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.2 MiB/s wr, 501 op/s 2024-06-13T07:25:04.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:04 smithi136 ceph-mon[37229]: pgmap v1246: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.2 MiB/s wr, 501 op/s 2024-06-13T07:25:05.258 INFO:tasks.workunit.client.0.smithi026.stdout: 10 323701 25.09 MB/sec execute 558 sec latency 116.884 ms 2024-06-13T07:25:06.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 324225 25.11 MB/sec execute 559 sec latency 112.710 ms 2024-06-13T07:25:06.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:06 smithi026 ceph-mon[32041]: pgmap v1247: 129 pgs: 129 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.8 MiB/s wr, 425 op/s 2024-06-13T07:25:06.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:06 smithi135 ceph-mon[36276]: pgmap v1247: 129 pgs: 129 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.8 MiB/s wr, 425 op/s 2024-06-13T07:25:06.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:06 smithi136 ceph-mon[37229]: pgmap v1247: 129 pgs: 129 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.8 MiB/s wr, 425 op/s 2024-06-13T07:25:07.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 324278 25.07 MB/sec execute 560 sec latency 827.487 ms 2024-06-13T07:25:08.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 324388 25.03 MB/sec execute 561 sec latency 1530.961 ms 2024-06-13T07:25:08.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:08 smithi026 ceph-mon[32041]: pgmap v1248: 129 pgs: 129 active+clean; 671 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 438 op/s 2024-06-13T07:25:08.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:08 smithi135 ceph-mon[36276]: pgmap v1248: 129 pgs: 129 active+clean; 671 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 438 op/s 2024-06-13T07:25:08.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:08 smithi136 ceph-mon[37229]: pgmap v1248: 129 pgs: 129 active+clean; 671 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 438 op/s 2024-06-13T07:25:09.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 324664 25.00 MB/sec execute 562 sec latency 466.167 ms 2024-06-13T07:25:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.592+0000 7f8d8f400640 1 -- 172.21.15.26:0/1132432023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88075980 msgr2=0x7f8d88073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.592+0000 7f8d8f400640 1 --2- 172.21.15.26:0/1132432023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88075980 0x7f8d88073dd0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009be0 tx=0x7f8d8002a750 comp rx=0 tx=0).stop 2024-06-13T07:25:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.595+0000 7f8d8f400640 1 -- 172.21.15.26:0/1132432023 shutdown_connections 2024-06-13T07:25:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.595+0000 7f8d8f400640 1 --2- 172.21.15.26:0/1132432023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d88109890 0x7f8d8810bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.595+0000 7f8d8f400640 1 --2- 172.21.15.26:0/1132432023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d880743a0 0x7f8d88074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.595+0000 7f8d8f400640 1 --2- 172.21.15.26:0/1132432023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88075980 0x7f8d88073dd0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.595+0000 7f8d8f400640 1 -- 172.21.15.26:0/1132432023 >> 172.21.15.26:0/1132432023 conn(0x7f8d880fcf90 msgr2=0x7f8d880ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.596+0000 7f8d8f400640 1 -- 172.21.15.26:0/1132432023 shutdown_connections 2024-06-13T07:25:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.596+0000 7f8d8f400640 1 -- 172.21.15.26:0/1132432023 wait complete. 2024-06-13T07:25:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.598+0000 7f8d8f400640 1 Processor -- start 2024-06-13T07:25:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.599+0000 7f8d8f400640 1 -- start start 2024-06-13T07:25:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.599+0000 7f8d8f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d880743a0 0x7f8d88199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.599+0000 7f8d8f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d88075980 0x7f8d8819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.599+0000 7f8d8f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88109890 0x7f8d881a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.599+0000 7f8d8f400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8810f870 con 0x7f8d88075980 2024-06-13T07:25:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.599+0000 7f8d8f400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8810f6f0 con 0x7f8d88109890 2024-06-13T07:25:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.599+0000 7f8d8f400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8810f9f0 con 0x7f8d880743a0 2024-06-13T07:25:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.600+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d88075980 0x7f8d8819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.600+0000 7f8d8d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88109890 0x7f8d881a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.601+0000 7f8d8d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88109890 0x7f8d881a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59754/0 (socket says 172.21.15.26:59754) 2024-06-13T07:25:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.601+0000 7f8d8d400640 1 -- 172.21.15.26:0/2248061174 learned_addr learned my addr 172.21.15.26:0/2248061174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.601+0000 7f8d8d400640 1 -- 172.21.15.26:0/2248061174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d880743a0 msgr2=0x7f8d88199ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:25:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.601+0000 7f8d8d400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d880743a0 0x7f8d88199ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.601+0000 7f8d8d400640 1 -- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d88075980 msgr2=0x7f8d8819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.602+0000 7f8d8d400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d88075980 0x7f8d8819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.602+0000 7f8d8d400640 1 -- 172.21.15.26:0/2248061174 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d881a1ac0 con 0x7f8d88109890 2024-06-13T07:25:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.603+0000 7f8d8d400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88109890 0x7f8d881a1450 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400e9b0 tx=0x7f8d7400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.605+0000 7f8d85600640 1 -- 172.21.15.26:0/2248061174 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400cd80 con 0x7f8d88109890 2024-06-13T07:25:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.605+0000 7f8d85600640 1 -- 172.21.15.26:0/2248061174 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d740035f0 con 0x7f8d88109890 2024-06-13T07:25:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.605+0000 7f8d85600640 1 -- 172.21.15.26:0/2248061174 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74010690 con 0x7f8d88109890 2024-06-13T07:25:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.606+0000 7f8d8f400640 1 -- 172.21.15.26:0/2248061174 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d88101470 con 0x7f8d88109890 2024-06-13T07:25:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.606+0000 7f8d8f400640 1 -- 172.21.15.26:0/2248061174 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d88101970 con 0x7f8d88109890 2024-06-13T07:25:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.615+0000 7f8d85600640 1 -- 172.21.15.26:0/2248061174 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d740037a0 con 0x7f8d88109890 2024-06-13T07:25:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.617+0000 7f8d85600640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8d6c0a2d00 0x7f8d6c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.618+0000 7f8d85600640 1 -- 172.21.15.26:0/2248061174 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8d740cf890 con 0x7f8d88109890 2024-06-13T07:25:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.618+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8d6c0a2d00 0x7f8d6c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.619+0000 7f8d92390740 1 Processor -- start 2024-06-13T07:25:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.619+0000 7f8d92390740 1 -- start start 2024-06-13T07:25:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.619+0000 7f8d92390740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430cc00 0x55996430d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.619+0000 7f8d92390740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.619+0000 7f8d92390740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x559964314bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.619+0000 7f8d92390740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559964270420 con 0x559964312760 2024-06-13T07:25:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.619+0000 7f8d92390740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559964217c50 con 0x55996430d5f0 2024-06-13T07:25:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.619+0000 7f8d92390740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55996426c3e0 con 0x55996430cc00 2024-06-13T07:25:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.620+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.621+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59760/0 (socket says 172.21.15.26:59760) 2024-06-13T07:25:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.621+0000 7f8d87e00640 1 -- 172.21.15.26:0/2267114076 learned_addr learned my addr 172.21.15.26:0/2267114076 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.621+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8d6c0a2d00 0x7f8d6c0a51c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8d800099a0 tx=0x7f8d80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.622+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430cc00 0x55996430d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.622+0000 7f8d87e00640 1 -- 172.21.15.26:0/2267114076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430cc00 msgr2=0x55996430d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.622+0000 7f8d87e00640 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430cc00 0x55996430d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.622+0000 7f8d8d400640 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x559964314bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.623+0000 7f8d87e00640 1 -- 172.21.15.26:0/2267114076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 msgr2=0x559964314bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.623+0000 7f8d87e00640 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x559964314bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.623+0000 7f8d87e00640 1 -- 172.21.15.26:0/2267114076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559963f84210 con 0x55996430d5f0 2024-06-13T07:25:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.623+0000 7f8d8d400640 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x559964314bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:25:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.623+0000 7f8d87e00640 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643120f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800dcb0 tx=0x7f8d7800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad83dde0a2d389b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.624+0000 7f8d7dc00640 1 -- 172.21.15.26:0/2267114076 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78011440 con 0x55996430d5f0 2024-06-13T07:25:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.625+0000 7f8d7dc00640 1 -- 172.21.15.26:0/2267114076 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d78011ac0 con 0x55996430d5f0 2024-06-13T07:25:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.625+0000 7f8d7dc00640 1 -- 172.21.15.26:0/2267114076 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7801bb10 con 0x55996430d5f0 2024-06-13T07:25:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.626+0000 7f8d92390740 1 -- 172.21.15.26:0/2267114076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 msgr2=0x5599643120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.626+0000 7f8d92390740 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643120f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800dcb0 tx=0x7f8d7800f730 comp rx=0 tx=0).stop 2024-06-13T07:25:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.627+0000 7f8d92390740 1 -- 172.21.15.26:0/2267114076 shutdown_connections 2024-06-13T07:25:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.628+0000 7f8d92390740 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643120f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.628+0000 7f8d92390740 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x559964314bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.628+0000 7f8d92390740 1 --2- 172.21.15.26:0/2267114076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430cc00 0x55996430d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.628+0000 7f8d92390740 1 -- 172.21.15.26:0/2267114076 >> 172.21.15.26:0/2267114076 conn(0x5599642d4fb0 msgr2=0x559964308680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.628+0000 7f8d92390740 1 -- 172.21.15.26:0/2267114076 shutdown_connections 2024-06-13T07:25:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.629+0000 7f8d92390740 1 -- 172.21.15.26:0/2267114076 wait complete. 2024-06-13T07:25:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.630+0000 7f8d92390740 1 -- start start 2024-06-13T07:25:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.634+0000 7f8d92390740 1 Processor -- start 2024-06-13T07:25:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.636+0000 7f8d92390740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x559964316fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.636+0000 7f8d92390740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.636+0000 7f8d92390740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559964312760 0x55996431e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.636+0000 7f8d92390740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559964315710 con 0x55996430cc00 2024-06-13T07:25:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.636+0000 7f8d92390740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55996422b0b0 con 0x55996430d5f0 2024-06-13T07:25:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.636+0000 7f8d92390740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559964315890 con 0x559964312760 2024-06-13T07:25:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.636+0000 7f8d92390740 10 client.0 _ll_register_callbacks cb 0x559964216f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:25:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.637+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.638+0000 7f8d8ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x559964316fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.638+0000 7f8d8d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559964312760 0x55996431e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.638+0000 7f8d8ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x559964316fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53226/0 (socket says 172.21.15.26:53226) 2024-06-13T07:25:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.638+0000 7f8d8ca00640 1 -- 172.21.15.26:0/1201813006 learned_addr learned my addr 172.21.15.26:0/1201813006 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.639+0000 7f8d8d400640 1 -- 172.21.15.26:0/1201813006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 msgr2=0x5599643174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.639+0000 7f8d8d400640 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.639+0000 7f8d8d400640 1 -- 172.21.15.26:0/1201813006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 msgr2=0x559964316fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.639+0000 7f8d8d400640 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x559964316fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.639+0000 7f8d8d400640 1 -- 172.21.15.26:0/1201813006 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559964050620 con 0x559964312760 2024-06-13T07:25:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.639+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x559964316fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:25:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.640+0000 7f8d8d400640 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559964312760 0x55996431e520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d740ccff0 tx=0x7f8d7409bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e001efa13e325f73 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.640+0000 7f8d72000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:25:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.641+0000 7f8d72000640 1 -- 172.21.15.26:0/1201813006 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74003b10 con 0x559964312760 2024-06-13T07:25:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.641+0000 7f8d72000640 1 -- 172.21.15.26:0/1201813006 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8d740ba5e0 con 0x559964312760 2024-06-13T07:25:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.641+0000 7f8d92390740 1 -- 172.21.15.26:0/1201813006 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d78011ac0 con 0x559964312760 2024-06-13T07:25:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.641+0000 7f8d72000640 1 -- 172.21.15.26:0/1201813006 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8d7407ab70 con 0x559964312760 2024-06-13T07:25:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.644+0000 7f8d92390740 10 client.27277 fetch_fsmap learned FSMap version 50 2024-06-13T07:25:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.644+0000 7f8d92390740 1 -- 172.21.15.26:0/1201813006 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x559964321870 con 0x559964312760 2024-06-13T07:25:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.645+0000 7f8d72000640 1 -- 172.21.15.26:0/1201813006 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d78011ac0 con 0x559964312760 2024-06-13T07:25:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.645+0000 7f8d72000640 1 -- 172.21.15.26:0/1201813006 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7f8d74097a50 con 0x559964312760 2024-06-13T07:25:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.645+0000 7f8d92390740 10 client.27277 fetch_fsmap finished waiting for FSMap version 50 2024-06-13T07:25:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.645+0000 7f8d92390740 10 client.27277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:25:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.645+0000 7f8d92390740 20 client.27277 populate_metadata read hostname 'smithi026' 2024-06-13T07:25:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.645+0000 7f8d92390740 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559964322f90 0x5599643253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.645+0000 7f8d92390740 4 client.27277 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:25:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.645+0000 7f8d92390740 1 -- 172.21.15.26:0/1201813006 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55996427db50 con 0x559964322f90 2024-06-13T07:25:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.650+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559964322f90 0x5599643253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.651+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559964322f90 0x5599643253c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b958d08b86667622 server_cookie=5da8ec1383a39ece in_seq=0 out_seq=0 2024-06-13T07:25:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.651+0000 7f8d72000640 10 client.27277 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.660+0000 7f8d72000640 1 -- 172.21.15.26:0/1201813006 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55996427db50 con 0x559964322f90 2024-06-13T07:25:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.660+0000 7f8d72000640 10 client.27277 handle_command_reply: tid=0 2024-06-13T07:25:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.660+0000 7f8d92390740 1 client.27277 shutdown 2024-06-13T07:25:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.661+0000 7f8d92390740 1 -- 172.21.15.26:0/1201813006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559964312760 msgr2=0x55996431e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.661+0000 7f8d92390740 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559964312760 0x55996431e520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d740ccff0 tx=0x7f8d7409bdc0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.662+0000 7f8d92390740 1 -- 172.21.15.26:0/1201813006 shutdown_connections 2024-06-13T07:25:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.663+0000 7f8d92390740 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x559964322f90 0x5599643253c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.663+0000 7f8d92390740 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x5599643174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.664+0000 7f8d92390740 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559964312760 0x55996431e520 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.664+0000 7f8d92390740 1 --2- 172.21.15.26:0/1201813006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x559964316fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.664+0000 7f8d92390740 1 -- 172.21.15.26:0/1201813006 >> 172.21.15.26:0/1201813006 conn(0x5599642d4fb0 msgr2=0x559964307840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.663+0000 7f8d72000640 0 client.27277 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.664+0000 7f8d92390740 1 -- 172.21.15.26:0/1201813006 shutdown_connections 2024-06-13T07:25:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.665+0000 7f8d92390740 1 -- 172.21.15.26:0/1201813006 wait complete. 2024-06-13T07:25:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.666+0000 7f8d92390740 20 client.27277 trim_cache size 0 max 16384 2024-06-13T07:25:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.705+0000 7f8d92390740 1 Processor -- start 2024-06-13T07:25:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.705+0000 7f8d92390740 1 -- start start 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.705+0000 7f8d92390740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x55996430bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.705+0000 7f8d92390740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430d5f0 0x55996431e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.705+0000 7f8d92390740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559964312760 0x559964329cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.705+0000 7f8d92390740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881b3710 con 0x55996430cc00 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.705+0000 7f8d92390740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881b0fb0 con 0x559964312760 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.705+0000 7f8d92390740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881aee00 con 0x55996430d5f0 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.706+0000 7f8d8ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x55996430bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.706+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430d5f0 0x55996431e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.706+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430d5f0 0x55996431e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:51186/0 (socket says 172.21.15.26:51186) 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.706+0000 7f8d87e00640 1 -- 172.21.15.26:0/1849425977 learned_addr learned my addr 172.21.15.26:0/1849425977 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.706+0000 7f8d8d400640 1 --2- 172.21.15.26:0/1849425977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559964312760 0x559964329cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.706+0000 7f8d8ca00640 1 -- 172.21.15.26:0/1849425977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430d5f0 msgr2=0x55996431e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.706+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/1849425977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430d5f0 0x55996431e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.707+0000 7f8d8ca00640 1 -- 172.21.15.26:0/1849425977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559964312760 msgr2=0x559964329cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.707+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/1849425977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559964312760 0x559964329cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.707+0000 7f8d8ca00640 1 -- 172.21.15.26:0/1849425977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559964309330 con 0x55996430cc00 2024-06-13T07:25:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.707+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/1849425977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x55996430bc80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x559964325c10 tx=0x7f8d8003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cef7db10a8c998e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.708+0000 7f8d7d200640 1 -- 172.21.15.26:0/1849425977 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80036040 con 0x55996430cc00 2024-06-13T07:25:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.709+0000 7f8d7d200640 1 -- 172.21.15.26:0/1849425977 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80044040 con 0x55996430cc00 2024-06-13T07:25:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.709+0000 7f8d7d200640 1 -- 172.21.15.26:0/1849425977 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8004b6a0 con 0x55996430cc00 2024-06-13T07:25:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.709+0000 7f8d92390740 1 -- 172.21.15.26:0/1849425977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 msgr2=0x55996430bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.709+0000 7f8d92390740 1 --2- 172.21.15.26:0/1849425977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x55996430bc80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x559964325c10 tx=0x7f8d8003a000 comp rx=0 tx=0).stop 2024-06-13T07:25:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.710+0000 7f8d92390740 1 -- 172.21.15.26:0/1849425977 shutdown_connections 2024-06-13T07:25:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.710+0000 7f8d92390740 1 --2- 172.21.15.26:0/1849425977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559964312760 0x559964329cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.710+0000 7f8d92390740 1 --2- 172.21.15.26:0/1849425977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996430d5f0 0x55996431e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.710+0000 7f8d92390740 1 --2- 172.21.15.26:0/1849425977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55996430cc00 0x55996430bc80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.710+0000 7f8d92390740 1 -- 172.21.15.26:0/1849425977 >> 172.21.15.26:0/1849425977 conn(0x5599642d4fb0 msgr2=0x55996430b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.710+0000 7f8d92390740 1 -- 172.21.15.26:0/1849425977 shutdown_connections 2024-06-13T07:25:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.710+0000 7f8d92390740 1 -- 172.21.15.26:0/1849425977 wait complete. 2024-06-13T07:25:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.711+0000 7f8d92390740 1 -- start start 2024-06-13T07:25:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.711+0000 7f8d92390740 1 Processor -- start 2024-06-13T07:25:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.712+0000 7f8d92390740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x55996431be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.712+0000 7f8d92390740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x55996431c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.712+0000 7f8d92390740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996431d320 0x55996431d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d8ca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x55996431be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d8ca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x55996431be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:59782/0 (socket says 172.21.15.26:59782) 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881af470 con 0x559964312760 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881b08e0 con 0x55996430d5f0 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881afb40 con 0x55996431d320 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d87e00640 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x55996431c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d92390740 10 client.0 _ll_register_callbacks cb 0x7f8d7409bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d8d400640 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996431d320 0x55996431d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.713+0000 7f8d8ca00640 1 -- 172.21.15.26:0/3636370962 learned_addr learned my addr 172.21.15.26:0/3636370962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.717+0000 7f8d87e00640 1 -- 172.21.15.26:0/3636370962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996431d320 msgr2=0x55996431d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.717+0000 7f8d87e00640 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996431d320 0x55996431d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.717+0000 7f8d87e00640 1 -- 172.21.15.26:0/3636370962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 msgr2=0x55996431be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.717+0000 7f8d87e00640 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x55996431be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.717+0000 7f8d87e00640 1 -- 172.21.15.26:0/3636370962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55996431dce0 con 0x559964312760 2024-06-13T07:25:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.717+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x55996431be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:25:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.717+0000 7f8d87e00640 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x55996431c380 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x559964317c70 tx=0x7f8d7800d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e57403c8b19406a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.718+0000 7f8d72000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:25:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.718+0000 7f8d72000640 1 -- 172.21.15.26:0/3636370962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78005450 con 0x559964312760 2024-06-13T07:25:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.719+0000 7f8d72000640 1 -- 172.21.15.26:0/3636370962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8d780055f0 con 0x559964312760 2024-06-13T07:25:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.719+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d80044040 con 0x559964312760 2024-06-13T07:25:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.725+0000 7f8d72000640 1 -- 172.21.15.26:0/3636370962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8d7800d8b0 con 0x559964312760 2024-06-13T07:25:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.725+0000 7f8d92390740 10 client.16923 fetch_fsmap learned FSMap version 50 2024-06-13T07:25:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.725+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55996438e210 con 0x559964312760 2024-06-13T07:25:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.727+0000 7f8d72000640 1 -- 172.21.15.26:0/3636370962 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80044040 con 0x559964312760 2024-06-13T07:25:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.727+0000 7f8d72000640 1 -- 172.21.15.26:0/3636370962 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7f8d78011940 con 0x559964312760 2024-06-13T07:25:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.729+0000 7f8d92390740 10 client.16923 fetch_fsmap finished waiting for FSMap version 50 2024-06-13T07:25:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.729+0000 7f8d92390740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:25:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.729+0000 7f8d92390740 20 client.16923 populate_metadata read hostname 'smithi026' 2024-06-13T07:25:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.729+0000 7f8d92390740 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55996438e820 0x559964390c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.729+0000 7f8d92390740 4 client.16923 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:25:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.729+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559964316d90 con 0x55996438e820 2024-06-13T07:25:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.729+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55996438e820 0x559964390c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.731+0000 7f8d8ca00640 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55996438e820 0x559964390c10 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba73d39d2ad2141 server_cookie=fcf2273e0817e784 in_seq=0 out_seq=0 2024-06-13T07:25:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:09.731+0000 7f8d72000640 10 client.16923 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:09.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:09 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:09.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:09 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:09.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:09 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:10.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 324837 24.96 MB/sec execute 563 sec latency 335.793 ms 2024-06-13T07:25:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.494+0000 7f8d72000640 1 -- 172.21.15.26:0/3636370962 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559964316d90 con 0x55996438e820 2024-06-13T07:25:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.494+0000 7f8d72000640 10 client.16923 handle_command_reply: tid=0 2024-06-13T07:25:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.494+0000 7f8d92390740 1 client.16923 shutdown 2024-06-13T07:25:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.495+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 msgr2=0x55996431c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.495+0000 7f8d92390740 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x55996431c380 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x559964317c70 tx=0x7f8d7800d7c0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.495+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 shutdown_connections 2024-06-13T07:25:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.495+0000 7f8d92390740 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x55996438e820 0x559964390c10 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.495+0000 7f8d92390740 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55996431d320 0x55996431d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.495+0000 7f8d92390740 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559964312760 0x55996431c380 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.495+0000 7f8d92390740 1 --2- 172.21.15.26:0/3636370962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55996430d5f0 0x55996431be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.495+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 >> 172.21.15.26:0/3636370962 conn(0x5599642d4fb0 msgr2=0x559964306fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.496+0000 7f8d72000640 0 client.16923 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.497+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 shutdown_connections 2024-06-13T07:25:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.498+0000 7f8d92390740 1 -- 172.21.15.26:0/3636370962 wait complete. 2024-06-13T07:25:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.498+0000 7f8d92390740 20 client.16923 trim_cache size 0 max 16384 2024-06-13T07:25:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:25:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:25:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:25:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 -- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8d6c0a2d00 msgr2=0x7f8d6c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8d6c0a2d00 0x7f8d6c0a51c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8d800099a0 tx=0x7f8d80005fd0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 -- 172.21.15.26:0/2248061174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88109890 msgr2=0x7f8d881a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88109890 0x7f8d881a1450 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400e9b0 tx=0x7f8d7400ee70 comp rx=0 tx=0).stop 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 -- 172.21.15.26:0/2248061174 shutdown_connections 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f8d6c0a2d00 0x7f8d6c0a51c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d88109890 0x7f8d881a1450 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d88075980 0x7f8d8819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 --2- 172.21.15.26:0/2248061174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d880743a0 0x7f8d88199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 -- 172.21.15.26:0/2248061174 >> 172.21.15.26:0/2248061174 conn(0x7f8d880fcf90 msgr2=0x7f8d88105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 -- 172.21.15.26:0/2248061174 shutdown_connections 2024-06-13T07:25:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:10.500+0000 7f8d8f400640 1 -- 172.21.15.26:0/2248061174 wait complete. 2024-06-13T07:25:10.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:10 smithi026 ceph-mon[32041]: pgmap v1249: 129 pgs: 129 active+clean; 713 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 483 op/s 2024-06-13T07:25:10.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:10 smithi135 ceph-mon[36276]: pgmap v1249: 129 pgs: 129 active+clean; 713 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 483 op/s 2024-06-13T07:25:10.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:10 smithi136 ceph-mon[37229]: pgmap v1249: 129 pgs: 129 active+clean; 713 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 483 op/s 2024-06-13T07:25:11.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 325222 24.96 MB/sec execute 564 sec latency 230.857 ms 2024-06-13T07:25:12.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 325475 24.95 MB/sec execute 565 sec latency 344.137 ms 2024-06-13T07:25:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:12 smithi026 ceph-mon[32041]: pgmap v1250: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.6 MiB/s wr, 438 op/s 2024-06-13T07:25:12.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:12 smithi135 ceph-mon[36276]: pgmap v1250: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.6 MiB/s wr, 438 op/s 2024-06-13T07:25:12.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:12 smithi136 ceph-mon[37229]: pgmap v1250: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.6 MiB/s wr, 438 op/s 2024-06-13T07:25:13.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 325705 24.93 MB/sec execute 566 sec latency 654.810 ms 2024-06-13T07:25:14.257 INFO:tasks.workunit.client.0.smithi026.stdout: 10 325972 24.92 MB/sec execute 567 sec latency 363.971 ms 2024-06-13T07:25:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:14 smithi026 ceph-mon[32041]: pgmap v1251: 129 pgs: 129 active+clean; 705 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.0 MiB/s wr, 492 op/s 2024-06-13T07:25:14.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:14 smithi135 ceph-mon[36276]: pgmap v1251: 129 pgs: 129 active+clean; 705 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.0 MiB/s wr, 492 op/s 2024-06-13T07:25:14.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:14 smithi136 ceph-mon[37229]: pgmap v1251: 129 pgs: 129 active+clean; 705 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.0 MiB/s wr, 492 op/s 2024-06-13T07:25:15.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 326594 24.94 MB/sec execute 568 sec latency 102.647 ms 2024-06-13T07:25:16.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 326962 24.93 MB/sec execute 569 sec latency 101.591 ms 2024-06-13T07:25:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:16 smithi026 ceph-mon[32041]: pgmap v1252: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.9 MiB/s wr, 565 op/s 2024-06-13T07:25:16.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:16 smithi135 ceph-mon[36276]: pgmap v1252: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.9 MiB/s wr, 565 op/s 2024-06-13T07:25:16.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:16 smithi136 ceph-mon[37229]: pgmap v1252: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.9 MiB/s wr, 565 op/s 2024-06-13T07:25:17.256 INFO:tasks.workunit.client.0.smithi026.stdout: 10 327399 24.93 MB/sec execute 570 sec latency 65.541 ms 2024-06-13T07:25:18.257 INFO:tasks.workunit.client.0.smithi026.stdout: 10 327952 24.92 MB/sec execute 571 sec latency 112.569 ms 2024-06-13T07:25:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:18 smithi026 ceph-mon[32041]: pgmap v1253: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.9 MiB/s wr, 570 op/s 2024-06-13T07:25:18.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:18 smithi135 ceph-mon[36276]: pgmap v1253: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.9 MiB/s wr, 570 op/s 2024-06-13T07:25:18.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:18 smithi136 ceph-mon[37229]: pgmap v1253: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.9 MiB/s wr, 570 op/s 2024-06-13T07:25:19.257 INFO:tasks.workunit.client.0.smithi026.stdout: 10 328563 24.94 MB/sec execute 572 sec latency 82.301 ms 2024-06-13T07:25:20.257 INFO:tasks.workunit.client.0.smithi026.stdout: 10 329128 24.95 MB/sec execute 573 sec latency 121.307 ms 2024-06-13T07:25:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:20 smithi026 ceph-mon[32041]: pgmap v1254: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-06-13T07:25:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:20 smithi135 ceph-mon[36276]: pgmap v1254: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-06-13T07:25:20.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:20 smithi136 ceph-mon[37229]: pgmap v1254: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-06-13T07:25:21.257 INFO:tasks.workunit.client.0.smithi026.stdout: 10 329754 24.97 MB/sec execute 574 sec latency 49.702 ms 2024-06-13T07:25:22.259 INFO:tasks.workunit.client.0.smithi026.stdout: 10 330288 24.97 MB/sec execute 575 sec latency 81.568 ms 2024-06-13T07:25:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:22 smithi026 ceph-mon[32041]: pgmap v1255: 129 pgs: 129 active+clean; 667 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 535 op/s 2024-06-13T07:25:22.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:22 smithi135 ceph-mon[36276]: pgmap v1255: 129 pgs: 129 active+clean; 667 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 535 op/s 2024-06-13T07:25:22.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:22 smithi136 ceph-mon[37229]: pgmap v1255: 129 pgs: 129 active+clean; 667 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 535 op/s 2024-06-13T07:25:23.259 INFO:tasks.workunit.client.0.smithi026.stdout: 10 330597 24.96 MB/sec execute 576 sec latency 154.413 ms 2024-06-13T07:25:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.708+0000 7fd41ee00640 1 -- 172.21.15.26:0/3457675679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 msgr2=0x7fd4181070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.708+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3457675679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 0x7fd4181070d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd408009900 tx=0x7fd40802a400 comp rx=0 tx=0).stop 2024-06-13T07:25:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.709+0000 7fd41ee00640 1 -- 172.21.15.26:0/3457675679 shutdown_connections 2024-06-13T07:25:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.709+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3457675679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 0x7fd4181070d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.709+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3457675679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd418107660 0x7fd418109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.709+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3457675679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180ff770 0x7fd4180ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.709+0000 7fd41ee00640 1 -- 172.21.15.26:0/3457675679 >> 172.21.15.26:0/3457675679 conn(0x7fd4180fb1c0 msgr2=0x7fd4180fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.709+0000 7fd41ee00640 1 -- 172.21.15.26:0/3457675679 shutdown_connections 2024-06-13T07:25:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.709+0000 7fd41ee00640 1 -- 172.21.15.26:0/3457675679 wait complete. 2024-06-13T07:25:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.711+0000 7fd41ee00640 1 Processor -- start 2024-06-13T07:25:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.711+0000 7fd41ee00640 1 -- start start 2024-06-13T07:25:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.711+0000 7fd41ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180ff770 0x7fd418197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.711+0000 7fd41ee00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 0x7fd4181981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.712+0000 7fd417e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180ff770 0x7fd418197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.712+0000 7fd417400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 0x7fd4181981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.712+0000 7fd417e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180ff770 0x7fd418197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49858/0 (socket says 172.21.15.26:49858) 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.712+0000 7fd417400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 0x7fd4181981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35100/0 (socket says 172.21.15.26:35100) 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.711+0000 7fd41ee00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd418107660 0x7fd41819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.713+0000 7fd41ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810d690 con 0x7fd4180ff770 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.713+0000 7fd417e00640 1 -- 172.21.15.26:0/3026795354 learned_addr learned my addr 172.21.15.26:0/3026795354 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.713+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810d510 con 0x7fd4181000d0 2024-06-13T07:25:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.713+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810d810 con 0x7fd418107660 2024-06-13T07:25:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.714+0000 7fd41ce00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd418107660 0x7fd41819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.715+0000 7fd41ce00640 1 -- 172.21.15.26:0/3026795354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 msgr2=0x7fd4181981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.715+0000 7fd41ce00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 0x7fd4181981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.715+0000 7fd41ce00640 1 -- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180ff770 msgr2=0x7fd418197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.715+0000 7fd41ce00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180ff770 0x7fd418197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.715+0000 7fd41ce00640 1 -- 172.21.15.26:0/3026795354 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd418100ea0 con 0x7fd418107660 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.715+0000 7fd417e00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180ff770 0x7fd418197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.715+0000 7fd417400640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 0x7fd4181981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.716+0000 7fd41ce00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd418107660 0x7fd41819f1c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e940 tx=0x7fd41000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.716+0000 7fd414c00640 1 -- 172.21.15.26:0/3026795354 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410004d20 con 0x7fd418107660 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.716+0000 7fd414c00640 1 -- 172.21.15.26:0/3026795354 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd410005020 con 0x7fd418107660 2024-06-13T07:25:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.716+0000 7fd414c00640 1 -- 172.21.15.26:0/3026795354 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100116d0 con 0x7fd418107660 2024-06-13T07:25:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.716+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd418101190 con 0x7fd418107660 2024-06-13T07:25:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.717+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181015a0 con 0x7fd418107660 2024-06-13T07:25:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.723+0000 7fd421d90740 1 Processor -- start 2024-06-13T07:25:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.723+0000 7fd414c00640 1 -- 172.21.15.26:0/3026795354 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4100054d0 con 0x7fd418107660 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.724+0000 7fd421d90740 1 -- start start 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.725+0000 7fd421d90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad19020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.725+0000 7fd421d90740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad1e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.725+0000 7fd421d90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad1e760 0x558faad20bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.727+0000 7fd421d90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558faac7c420 con 0x558faad1e760 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.727+0000 7fd421d90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558faac23c50 con 0x558faad18c00 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.727+0000 7fd421d90740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558faac783e0 con 0x558faad195f0 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.725+0000 7fd417400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad19020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.726+0000 7fd417e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad1e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.727+0000 7fd417400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad19020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35110/0 (socket says 172.21.15.26:35110) 2024-06-13T07:25:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.727+0000 7fd417400640 1 -- 172.21.15.26:0/1728834698 learned_addr learned my addr 172.21.15.26:0/1728834698 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.727+0000 7fd417e00640 1 -- 172.21.15.26:0/1728834698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 msgr2=0x558faad19020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.728+0000 7fd417e00640 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad19020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.728+0000 7fd417e00640 1 -- 172.21.15.26:0/1728834698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad1e760 msgr2=0x558faad20bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:25:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.728+0000 7fd417400640 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad1e760 0x558faad20bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.728+0000 7fd414c00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3fc0a2e20 0x7fd3fc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.729+0000 7fd417e00640 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad1e760 0x558faad20bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.729+0000 7fd417e00640 1 -- 172.21.15.26:0/1728834698 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558faa990210 con 0x558faad195f0 2024-06-13T07:25:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.729+0000 7fd417400640 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad1e760 0x558faad20bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:25:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.730+0000 7fd414c00640 1 -- 172.21.15.26:0/3026795354 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd4100cf4a0 con 0x7fd418107660 2024-06-13T07:25:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.730+0000 7fd417e00640 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad1e0f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd418199120 tx=0x7fd4040119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8c004e2aa0563e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.731+0000 7fd40d600640 1 -- 172.21.15.26:0/1728834698 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40401e070 con 0x558faad195f0 2024-06-13T07:25:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.731+0000 7fd40d600640 1 -- 172.21.15.26:0/1728834698 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4040035f0 con 0x558faad195f0 2024-06-13T07:25:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.731+0000 7fd40d600640 1 -- 172.21.15.26:0/1728834698 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40401e070 con 0x558faad195f0 2024-06-13T07:25:23.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.732+0000 7fd417400640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3fc0a2e20 0x7fd3fc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.732+0000 7fd421d90740 1 -- 172.21.15.26:0/1728834698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 msgr2=0x558faad1e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.732+0000 7fd421d90740 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad1e0f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd418199120 tx=0x7fd4040119a0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.733+0000 7fd421d90740 1 -- 172.21.15.26:0/1728834698 shutdown_connections 2024-06-13T07:25:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.733+0000 7fd421d90740 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad1e760 0x558faad20bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.733+0000 7fd421d90740 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad1e0f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.733+0000 7fd421d90740 1 --2- 172.21.15.26:0/1728834698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad19020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.733+0000 7fd421d90740 1 -- 172.21.15.26:0/1728834698 >> 172.21.15.26:0/1728834698 conn(0x558faace0fb0 msgr2=0x558faad14680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.733+0000 7fd421d90740 1 -- 172.21.15.26:0/1728834698 shutdown_connections 2024-06-13T07:25:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.733+0000 7fd417400640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3fc0a2e20 0x7fd3fc0a52e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd408000c00 tx=0x7fd4080315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.734+0000 7fd421d90740 1 -- 172.21.15.26:0/1728834698 wait complete. 2024-06-13T07:25:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.737+0000 7fd421d90740 1 -- start start 2024-06-13T07:25:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.737+0000 7fd421d90740 1 Processor -- start 2024-06-13T07:25:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.739+0000 7fd421d90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad33fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.739+0000 7fd421d90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad195f0 0x558faad344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.739+0000 7fd421d90740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad1e760 0x558faad3b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.739+0000 7fd421d90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558faad32770 con 0x558faad195f0 2024-06-13T07:25:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.739+0000 7fd421d90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558faac370b0 con 0x558faad18c00 2024-06-13T07:25:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.739+0000 7fd421d90740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558faad328f0 con 0x558faad1e760 2024-06-13T07:25:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.739+0000 7fd421d90740 10 client.0 _ll_register_callbacks cb 0x558faac22f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:25:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.740+0000 7fd417400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad195f0 0x558faad344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.740+0000 7fd417e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad33fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.740+0000 7fd417400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad195f0 0x558faad344e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49868/0 (socket says 172.21.15.26:49868) 2024-06-13T07:25:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.740+0000 7fd417400640 1 -- 172.21.15.26:0/3731096077 learned_addr learned my addr 172.21.15.26:0/3731096077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.741+0000 7fd41ce00640 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad1e760 0x558faad3b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.741+0000 7fd417e00640 1 -- 172.21.15.26:0/3731096077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad1e760 msgr2=0x558faad3b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.741+0000 7fd417e00640 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad1e760 0x558faad3b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.741+0000 7fd417e00640 1 -- 172.21.15.26:0/3731096077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad195f0 msgr2=0x558faad344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.741+0000 7fd417e00640 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad195f0 0x558faad344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.741+0000 7fd417e00640 1 -- 172.21.15.26:0/3731096077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558faaa5c620 con 0x558faad18c00 2024-06-13T07:25:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.741+0000 7fd41ce00640 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad1e760 0x558faad3b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:25:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.742+0000 7fd417e00640 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad33fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x558faad15530 tx=0x7fd4040042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c1c25e287863975 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.742+0000 7fd402000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:25:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.742+0000 7fd402000640 1 -- 172.21.15.26:0/3731096077 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40401e040 con 0x558faad18c00 2024-06-13T07:25:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.742+0000 7fd402000640 1 -- 172.21.15.26:0/3731096077 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd404005050 con 0x558faad18c00 2024-06-13T07:25:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.743+0000 7fd421d90740 1 -- 172.21.15.26:0/3731096077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4040035f0 con 0x558faad18c00 2024-06-13T07:25:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.745+0000 7fd402000640 1 -- 172.21.15.26:0/3731096077 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd404009cd0 con 0x558faad18c00 2024-06-13T07:25:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.747+0000 7fd421d90740 10 client.27233 fetch_fsmap learned FSMap version 50 2024-06-13T07:25:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.747+0000 7fd421d90740 1 -- 172.21.15.26:0/3731096077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558faad3e7e0 con 0x558faad18c00 2024-06-13T07:25:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.748+0000 7fd402000640 1 -- 172.21.15.26:0/3731096077 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd404027070 con 0x558faad18c00 2024-06-13T07:25:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.748+0000 7fd402000640 1 -- 172.21.15.26:0/3731096077 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7fd404021070 con 0x558faad18c00 2024-06-13T07:25:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.751+0000 7fd421d90740 10 client.27233 fetch_fsmap finished waiting for FSMap version 50 2024-06-13T07:25:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.751+0000 7fd421d90740 10 client.27233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:25:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.751+0000 7fd421d90740 20 client.27233 populate_metadata read hostname 'smithi026' 2024-06-13T07:25:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.751+0000 7fd421d90740 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558faad3fc80 0x558faad42070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.751+0000 7fd421d90740 4 client.27233 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:25:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.751+0000 7fd421d90740 1 -- 172.21.15.26:0/3731096077 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd4040035f0 con 0x558faad3fc80 2024-06-13T07:25:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.751+0000 7fd417400640 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558faad3fc80 0x558faad42070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.752+0000 7fd402000640 10 client.27233 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.752+0000 7fd417400640 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558faad3fc80 0x558faad42070 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdcc950e8d6c9337 server_cookie=fc05b76a468658ef in_seq=0 out_seq=0 2024-06-13T07:25:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:23 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:25:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.754+0000 7fd402000640 1 -- 172.21.15.26:0/3731096077 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fd4040035f0 con 0x558faad3fc80 2024-06-13T07:25:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.754+0000 7fd402000640 10 client.27233 handle_command_reply: tid=0 2024-06-13T07:25:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.754+0000 7fd421d90740 1 client.27233 shutdown 2024-06-13T07:25:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd421d90740 1 -- 172.21.15.26:0/3731096077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 msgr2=0x558faad33fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd421d90740 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad33fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x558faad15530 tx=0x7fd4040042b0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd421d90740 1 -- 172.21.15.26:0/3731096077 shutdown_connections 2024-06-13T07:25:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd421d90740 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558faad3fc80 0x558faad42070 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd421d90740 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad1e760 0x558faad3b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd421d90740 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad195f0 0x558faad344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd421d90740 1 --2- 172.21.15.26:0/3731096077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad18c00 0x558faad33fa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd421d90740 1 -- 172.21.15.26:0/3731096077 >> 172.21.15.26:0/3731096077 conn(0x558faace0fb0 msgr2=0x558faad13840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.756+0000 7fd402000640 0 client.27233 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.757+0000 7fd421d90740 1 -- 172.21.15.26:0/3731096077 shutdown_connections 2024-06-13T07:25:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.757+0000 7fd421d90740 1 -- 172.21.15.26:0/3731096077 wait complete. 2024-06-13T07:25:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.757+0000 7fd421d90740 20 client.27233 trim_cache size 0 max 16384 2024-06-13T07:25:23.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:23 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:25:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.793+0000 7fd421d90740 1 Processor -- start 2024-06-13T07:25:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.794+0000 7fd421d90740 1 -- start start 2024-06-13T07:25:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.794+0000 7fd421d90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 0x558faad17650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.794+0000 7fd421d90740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad3b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.794+0000 7fd421d90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad17b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.794+0000 7fd421d90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181aae70 con 0x558faad18c00 2024-06-13T07:25:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.794+0000 7fd421d90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181a8710 con 0x558faad1e760 2024-06-13T07:25:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.794+0000 7fd421d90740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181a6560 con 0x558faad195f0 2024-06-13T07:25:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.795+0000 7fd41ce00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad17b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.795+0000 7fd41ce00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad17b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:35128/0 (socket says 172.21.15.26:35128) 2024-06-13T07:25:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.795+0000 7fd417400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad3b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.795+0000 7fd417e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 0x558faad17650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.799+0000 7fd417400640 1 -- 172.21.15.26:0/1574753096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 msgr2=0x558faad17b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.799+0000 7fd417400640 1 --2- 172.21.15.26:0/1574753096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad17b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.799+0000 7fd417400640 1 -- 172.21.15.26:0/1574753096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 msgr2=0x558faad17650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.799+0000 7fd417400640 1 --2- 172.21.15.26:0/1574753096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 0x558faad17650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.799+0000 7fd417400640 1 -- 172.21.15.26:0/1574753096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558faad426a0 con 0x558faad195f0 2024-06-13T07:25:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.796+0000 7fd41ce00640 1 -- 172.21.15.26:0/1574753096 learned_addr learned my addr 172.21.15.26:0/1574753096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.799+0000 7fd41ce00640 1 --2- 172.21.15.26:0/1574753096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad17b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-13T07:25:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.800+0000 7fd417400640 1 --2- 172.21.15.26:0/1574753096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad3b470 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd408009230 tx=0x7fd40803f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac78f6e9c180fde server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.800+0000 7fd40cc00640 1 -- 172.21.15.26:0/1574753096 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408035040 con 0x558faad195f0 2024-06-13T07:25:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.801+0000 7fd40cc00640 1 -- 172.21.15.26:0/1574753096 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4040035f0 con 0x558faad195f0 2024-06-13T07:25:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.801+0000 7fd40cc00640 1 -- 172.21.15.26:0/1574753096 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408003850 con 0x558faad195f0 2024-06-13T07:25:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.802+0000 7fd421d90740 1 -- 172.21.15.26:0/1574753096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 msgr2=0x558faad3b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.802+0000 7fd421d90740 1 --2- 172.21.15.26:0/1574753096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad3b470 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd408009230 tx=0x7fd40803f040 comp rx=0 tx=0).stop 2024-06-13T07:25:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.802+0000 7fd421d90740 1 -- 172.21.15.26:0/1574753096 shutdown_connections 2024-06-13T07:25:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.802+0000 7fd421d90740 1 --2- 172.21.15.26:0/1574753096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad17b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.802+0000 7fd421d90740 1 --2- 172.21.15.26:0/1574753096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad3b470 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.802+0000 7fd421d90740 1 --2- 172.21.15.26:0/1574753096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 0x558faad17650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.802+0000 7fd421d90740 1 -- 172.21.15.26:0/1574753096 >> 172.21.15.26:0/1574753096 conn(0x558faace0fb0 msgr2=0x558faad14100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.803+0000 7fd421d90740 1 -- 172.21.15.26:0/1574753096 shutdown_connections 2024-06-13T07:25:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.803+0000 7fd421d90740 1 -- 172.21.15.26:0/1574753096 wait complete. 2024-06-13T07:25:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.805+0000 7fd421d90740 1 -- start start 2024-06-13T07:25:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.806+0000 7fd421d90740 1 Processor -- start 2024-06-13T07:25:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.808+0000 7fd421d90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 0x558faad38630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.808+0000 7fd421d90740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad38b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.809+0000 7fd417400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad38b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.809+0000 7fd417400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad38b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:33722/0 (socket says 172.21.15.26:33722) 2024-06-13T07:25:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.809+0000 7fd417400640 1 -- 172.21.15.26:0/1547513299 learned_addr learned my addr 172.21.15.26:0/1547513299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.809+0000 7fd421d90740 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad39b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.810+0000 7fd417e00640 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 0x558faad38630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.810+0000 7fd41ce00640 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.810+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181a6bd0 con 0x558faad18c00 2024-06-13T07:25:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.810+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181a8040 con 0x558faad1e760 2024-06-13T07:25:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.810+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181a72a0 con 0x558faad195f0 2024-06-13T07:25:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.810+0000 7fd421d90740 10 client.0 _ll_register_callbacks cb 0x7fd40401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:25:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.812+0000 7fd41ce00640 1 -- 172.21.15.26:0/1547513299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 msgr2=0x558faad38b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.812+0000 7fd41ce00640 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad38b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.812+0000 7fd41ce00640 1 -- 172.21.15.26:0/1547513299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 msgr2=0x558faad38630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.812+0000 7fd41ce00640 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 0x558faad38630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.812+0000 7fd41ce00640 1 -- 172.21.15.26:0/1547513299 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558faad3a170 con 0x558faad1e760 2024-06-13T07:25:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.813+0000 7fd41ce00640 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad39b80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd4100982a0 tx=0x7fd410099aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7cde22d7c4479f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.813+0000 7fd402000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:25:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.814+0000 7fd402000640 1 -- 172.21.15.26:0/1547513299 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100b4060 con 0x558faad1e760 2024-06-13T07:25:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.814+0000 7fd402000640 1 -- 172.21.15.26:0/1547513299 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fd41009cbe0 con 0x558faad1e760 2024-06-13T07:25:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.816+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4040035f0 con 0x558faad1e760 2024-06-13T07:25:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.816+0000 7fd402000640 1 -- 172.21.15.26:0/1547513299 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd4100b9050 con 0x558faad1e760 2024-06-13T07:25:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.816+0000 7fd421d90740 10 client.27236 fetch_fsmap learned FSMap version 50 2024-06-13T07:25:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.816+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558faad3b460 con 0x558faad1e760 2024-06-13T07:25:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.816+0000 7fd402000640 1 -- 172.21.15.26:0/1547513299 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4040035f0 con 0x558faad1e760 2024-06-13T07:25:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.816+0000 7fd402000640 1 -- 172.21.15.26:0/1547513299 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7fd41000f040 con 0x558faad1e760 2024-06-13T07:25:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.817+0000 7fd421d90740 10 client.27236 fetch_fsmap finished waiting for FSMap version 50 2024-06-13T07:25:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.817+0000 7fd421d90740 10 client.27236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:25:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.817+0000 7fd421d90740 20 client.27236 populate_metadata read hostname 'smithi026' 2024-06-13T07:25:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.817+0000 7fd421d90740 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558faadaabf0 0x558faadad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.817+0000 7fd421d90740 4 client.27236 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:25:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.817+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558faac89b50 con 0x558faadaabf0 2024-06-13T07:25:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.819+0000 7fd417e00640 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558faadaabf0 0x558faadad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.822+0000 7fd417e00640 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558faadaabf0 0x558faadad010 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a66c8373a2cb077 server_cookie=1795ac03e84a1f00 in_seq=0 out_seq=0 2024-06-13T07:25:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:23.822+0000 7fd402000640 10 client.27236 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:23.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:23 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:25:24.259 INFO:tasks.workunit.client.0.smithi026.stdout: 10 330862 24.94 MB/sec execute 577 sec latency 319.628 ms 2024-06-13T07:25:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.727+0000 7fd402000640 1 -- 172.21.15.26:0/1547513299 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558faac89b50 con 0x558faadaabf0 2024-06-13T07:25:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.727+0000 7fd402000640 10 client.27236 handle_command_reply: tid=0 2024-06-13T07:25:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.727+0000 7fd421d90740 1 client.27236 shutdown 2024-06-13T07:25:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.727+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 msgr2=0x558faad39b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.727+0000 7fd421d90740 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad39b80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd4100982a0 tx=0x7fd410099aa0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 shutdown_connections 2024-06-13T07:25:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd421d90740 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x558faadaabf0 0x558faadad010 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd421d90740 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558faad1e760 0x558faad39b80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd421d90740 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558faad195f0 0x558faad38b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd421d90740 1 --2- 172.21.15.26:0/1547513299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558faad18c00 0x558faad38630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 >> 172.21.15.26:0/1547513299 conn(0x558faace0fb0 msgr2=0x558faad17650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd402000640 0 client.27236 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 shutdown_connections 2024-06-13T07:25:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.728+0000 7fd421d90740 1 -- 172.21.15.26:0/1547513299 wait complete. 2024-06-13T07:25:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.729+0000 7fd421d90740 20 client.27236 trim_cache size 0 max 16384 2024-06-13T07:25:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:25:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:25:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:25:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3fc0a2e20 msgr2=0x7fd3fc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3fc0a2e20 0x7fd3fc0a52e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd408000c00 tx=0x7fd4080315b0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd418107660 msgr2=0x7fd41819f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd418107660 0x7fd41819f1c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e940 tx=0x7fd41000ee00 comp rx=0 tx=0).stop 2024-06-13T07:25:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 shutdown_connections 2024-06-13T07:25:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd3fc0a2e20 0x7fd3fc0a52e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4181000d0 0x7fd4181981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd418107660 0x7fd41819f1c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 --2- 172.21.15.26:0/3026795354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180ff770 0x7fd418197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 >> 172.21.15.26:0/3026795354 conn(0x7fd4180fb1c0 msgr2=0x7fd4181030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.731+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 shutdown_connections 2024-06-13T07:25:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:24.732+0000 7fd41ee00640 1 -- 172.21.15.26:0/3026795354 wait complete. 2024-06-13T07:25:24.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: pgmap v1256: 129 pgs: 129 active+clean; 673 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 543 op/s 2024-06-13T07:25:24.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.756 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.757 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.757 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.757 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.757 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.757 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.757 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:24 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: pgmap v1256: 129 pgs: 129 active+clean; 673 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 543 op/s 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:24 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: pgmap v1256: 129 pgs: 129 active+clean; 673 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 543 op/s 2024-06-13T07:25:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:24.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:24 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:25:25.259 INFO:tasks.workunit.client.0.smithi026.stdout: 10 331212 24.94 MB/sec execute 578 sec latency 684.134 ms 2024-06-13T07:25:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:25 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:25:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:25 smithi026 ceph-mon[32041]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:25:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:25 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:25:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:25 smithi026 ceph-mon[32041]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:25:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:25:25.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:25:25.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:25 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:25:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:25 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:25:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:25 smithi135 ceph-mon[36276]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:25:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:25 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:25:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:25 smithi135 ceph-mon[36276]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:25:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:25:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:25:25.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:25 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:25:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:25 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-06-13T07:25:25.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:25 smithi136 ceph-mon[37229]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-13T07:25:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:25 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:25:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:25 smithi136 ceph-mon[37229]: Unable to set osd_memory_target on smithi136 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-13T07:25:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:25:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:25:25.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:25 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:25:26.259 INFO:tasks.workunit.client.0.smithi026.stdout: 10 331636 24.92 MB/sec execute 579 sec latency 93.524 ms 2024-06-13T07:25:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:26 smithi026 ceph-mon[32041]: pgmap v1257: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.3 MiB/s wr, 520 op/s 2024-06-13T07:25:26.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:26 smithi135 ceph-mon[36276]: pgmap v1257: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.3 MiB/s wr, 520 op/s 2024-06-13T07:25:26.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:26 smithi136 ceph-mon[37229]: pgmap v1257: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.3 MiB/s wr, 520 op/s 2024-06-13T07:25:27.262 INFO:tasks.workunit.client.0.smithi026.stdout: 10 332113 24.92 MB/sec execute 580 sec latency 98.140 ms 2024-06-13T07:25:28.260 INFO:tasks.workunit.client.0.smithi026.stdout: 10 332628 24.93 MB/sec execute 581 sec latency 116.367 ms 2024-06-13T07:25:28.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:28 smithi135 ceph-mon[36276]: pgmap v1258: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.7 MiB/s wr, 367 op/s 2024-06-13T07:25:28.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:28 smithi136 ceph-mon[37229]: pgmap v1258: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.7 MiB/s wr, 367 op/s 2024-06-13T07:25:29.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:28 smithi026 ceph-mon[32041]: pgmap v1258: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.7 MiB/s wr, 367 op/s 2024-06-13T07:25:29.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 333081 24.94 MB/sec execute 582 sec latency 155.082 ms 2024-06-13T07:25:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:29 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:29.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:29 smithi135 ceph-mon[36276]: pgmap v1259: 129 pgs: 129 active+clean; 684 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 536 op/s 2024-06-13T07:25:29.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:29 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:29.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:29 smithi136 ceph-mon[37229]: pgmap v1259: 129 pgs: 129 active+clean; 684 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 536 op/s 2024-06-13T07:25:30.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:29 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:30.005 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:29 smithi026 ceph-mon[32041]: pgmap v1259: 129 pgs: 129 active+clean; 684 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 536 op/s 2024-06-13T07:25:30.260 INFO:tasks.workunit.client.0.smithi026.stdout: 10 333687 24.94 MB/sec execute 583 sec latency 80.504 ms 2024-06-13T07:25:31.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 334168 24.95 MB/sec execute 584 sec latency 82.596 ms 2024-06-13T07:25:32.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 334812 24.97 MB/sec execute 585 sec latency 65.425 ms 2024-06-13T07:25:32.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:32 smithi135 ceph-mon[36276]: pgmap v1260: 129 pgs: 129 active+clean; 685 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 424 op/s 2024-06-13T07:25:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:32 smithi136 ceph-mon[37229]: pgmap v1260: 129 pgs: 129 active+clean; 685 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 424 op/s 2024-06-13T07:25:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:32 smithi026 ceph-mon[32041]: pgmap v1260: 129 pgs: 129 active+clean; 685 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 424 op/s 2024-06-13T07:25:33.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 335366 24.97 MB/sec execute 586 sec latency 137.621 ms 2024-06-13T07:25:34.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 335800 24.97 MB/sec execute 587 sec latency 178.120 ms 2024-06-13T07:25:34.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:34 smithi135 ceph-mon[36276]: pgmap v1261: 129 pgs: 129 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.8 MiB/s wr, 444 op/s 2024-06-13T07:25:34.553 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:34 smithi026 ceph-mon[32041]: pgmap v1261: 129 pgs: 129 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.8 MiB/s wr, 444 op/s 2024-06-13T07:25:34.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:34 smithi136 ceph-mon[37229]: pgmap v1261: 129 pgs: 129 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.8 MiB/s wr, 444 op/s 2024-06-13T07:25:35.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 336335 24.98 MB/sec execute 588 sec latency 71.462 ms 2024-06-13T07:25:36.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 336912 24.98 MB/sec execute 589 sec latency 58.518 ms 2024-06-13T07:25:36.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:36 smithi135 ceph-mon[36276]: pgmap v1262: 129 pgs: 129 active+clean; 698 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.1 MiB/s wr, 483 op/s 2024-06-13T07:25:36.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:36 smithi136 ceph-mon[37229]: pgmap v1262: 129 pgs: 129 active+clean; 698 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.1 MiB/s wr, 483 op/s 2024-06-13T07:25:36.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:36 smithi026 ceph-mon[32041]: pgmap v1262: 129 pgs: 129 active+clean; 698 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.1 MiB/s wr, 483 op/s 2024-06-13T07:25:37.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 337373 24.98 MB/sec execute 590 sec latency 109.800 ms 2024-06-13T07:25:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 -- 172.21.15.26:0/1614301744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78101a00 msgr2=0x7fae78101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 --2- 172.21.15.26:0/1614301744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78101a00 0x7fae78101e20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fae68009900 tx=0x7fae6802a410 comp rx=0 tx=0).stop 2024-06-13T07:25:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 -- 172.21.15.26:0/1614301744 shutdown_connections 2024-06-13T07:25:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 --2- 172.21.15.26:0/1614301744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae781035b0 0x7fae78109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 --2- 172.21.15.26:0/1614301744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae78102bf0 0x7fae78103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 --2- 172.21.15.26:0/1614301744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78101a00 0x7fae78101e20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 -- 172.21.15.26:0/1614301744 >> 172.21.15.26:0/1614301744 conn(0x7fae780fcf70 msgr2=0x7fae780ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 -- 172.21.15.26:0/1614301744 shutdown_connections 2024-06-13T07:25:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.967+0000 7fae7e000640 1 -- 172.21.15.26:0/1614301744 wait complete. 2024-06-13T07:25:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.974+0000 7fae7e000640 1 Processor -- start 2024-06-13T07:25:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.974+0000 7fae7e000640 1 -- start start 2024-06-13T07:25:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.975+0000 7fae7e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78101a00 0x7fae78195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae7e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae78102bf0 0x7fae7819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae7e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae781035b0 0x7fae78196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae7e000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae7810d750 con 0x7fae781035b0 2024-06-13T07:25:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae7e000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae7810d5d0 con 0x7fae78101a00 2024-06-13T07:25:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae7e000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae7810d8d0 con 0x7fae78102bf0 2024-06-13T07:25:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae76a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae78102bf0 0x7fae7819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae76a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae78102bf0 0x7fae7819b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48246/0 (socket says 172.21.15.26:48246) 2024-06-13T07:25:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae76a00640 1 -- 172.21.15.26:0/1898189588 learned_addr learned my addr 172.21.15.26:0/1898189588 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.976+0000 7fae77e00640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae781035b0 0x7fae78196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.977+0000 7fae77e00640 1 -- 172.21.15.26:0/1898189588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae78102bf0 msgr2=0x7fae7819b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.977+0000 7fae77e00640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae78102bf0 0x7fae7819b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.977+0000 7fae77e00640 1 -- 172.21.15.26:0/1898189588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78101a00 msgr2=0x7fae78195b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:25:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.977+0000 7fae77e00640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78101a00 0x7fae78195b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.977+0000 7fae77e00640 1 -- 172.21.15.26:0/1898189588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae781ae760 con 0x7fae781035b0 2024-06-13T07:25:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.978+0000 7fae77e00640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae781035b0 0x7fae78196120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae7000be20 tx=0x7fae7000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.979+0000 7fae6fe00640 1 -- 172.21.15.26:0/1898189588 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70019070 con 0x7fae781035b0 2024-06-13T07:25:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.979+0000 7fae6fe00640 1 -- 172.21.15.26:0/1898189588 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae70014920 con 0x7fae781035b0 2024-06-13T07:25:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.979+0000 7fae6fe00640 1 -- 172.21.15.26:0/1898189588 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70012920 con 0x7fae781035b0 2024-06-13T07:25:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.979+0000 7fae7e000640 1 -- 172.21.15.26:0/1898189588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae781aea50 con 0x7fae781035b0 2024-06-13T07:25:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.980+0000 7fae7e000640 1 -- 172.21.15.26:0/1898189588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae781aefd0 con 0x7fae781035b0 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.984+0000 7fae80d98740 1 Processor -- start 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.985+0000 7fae80d98740 1 -- start start 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.985+0000 7fae80d98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.985+0000 7fae80d98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f9a600 0x560921f9f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.985+0000 7fae80d98740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fa1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.985+0000 7fae80d98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560921eec420 con 0x560921f9a600 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.985+0000 7fae80d98740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560921e93c50 con 0x560921f9f770 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.985+0000 7fae80d98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560921ee83e0 con 0x560921f99c10 2024-06-13T07:25:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48262/0 (socket says 172.21.15.26:48262) 2024-06-13T07:25:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 -- 172.21.15.26:0/2884723268 learned_addr learned my addr 172.21.15.26:0/2884723268 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 --2- 172.21.15.26:0/2884723268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fa1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 -- 172.21.15.26:0/2884723268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 msgr2=0x560921fa1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 --2- 172.21.15.26:0/2884723268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fa1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 -- 172.21.15.26:0/2884723268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f9a600 msgr2=0x560921f9f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 --2- 172.21.15.26:0/2884723268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f9a600 0x560921f9f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.986+0000 7fae76a00640 1 -- 172.21.15.26:0/2884723268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560921c00210 con 0x560921f99c10 2024-06-13T07:25:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.987+0000 7fae76a00640 1 --2- 172.21.15.26:0/2884723268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9a030 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fae600177a0 tx=0x7fae600095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d2fa0bfcf1b596 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.987+0000 7fae67e00640 1 -- 172.21.15.26:0/2884723268 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae600157c0 con 0x560921f99c10 2024-06-13T07:25:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.988+0000 7fae67e00640 1 -- 172.21.15.26:0/2884723268 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae6001d070 con 0x560921f99c10 2024-06-13T07:25:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.988+0000 7fae67e00640 1 -- 172.21.15.26:0/2884723268 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae600157c0 con 0x560921f99c10 2024-06-13T07:25:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.989+0000 7fae80d98740 1 -- 172.21.15.26:0/2884723268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 msgr2=0x560921f9a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.989+0000 7fae80d98740 1 --2- 172.21.15.26:0/2884723268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9a030 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fae600177a0 tx=0x7fae600095c0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.990+0000 7fae80d98740 1 -- 172.21.15.26:0/2884723268 shutdown_connections 2024-06-13T07:25:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.990+0000 7fae80d98740 1 --2- 172.21.15.26:0/2884723268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fa1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.990+0000 7fae80d98740 1 --2- 172.21.15.26:0/2884723268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f9a600 0x560921f9f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.990+0000 7fae80d98740 1 --2- 172.21.15.26:0/2884723268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9a030 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.990+0000 7fae80d98740 1 -- 172.21.15.26:0/2884723268 >> 172.21.15.26:0/2884723268 conn(0x560921f50fb0 msgr2=0x560921f84680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.994+0000 7fae80d98740 1 -- 172.21.15.26:0/2884723268 shutdown_connections 2024-06-13T07:25:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.994+0000 7fae6fe00640 1 -- 172.21.15.26:0/1898189588 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae7001c070 con 0x7fae781035b0 2024-06-13T07:25:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.997+0000 7fae6fe00640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae5c0a2d30 0x7fae5c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.998+0000 7fae80d98740 1 -- 172.21.15.26:0/2884723268 wait complete. 2024-06-13T07:25:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.998+0000 7fae80d98740 1 -- start start 2024-06-13T07:25:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.998+0000 7fae6fe00640 1 -- 172.21.15.26:0/1898189588 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fae700cebe0 con 0x7fae781035b0 2024-06-13T07:25:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.998+0000 7fae80d98740 1 Processor -- start 2024-06-13T07:25:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.999+0000 7fae77400640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae5c0a2d30 0x7fae5c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.999+0000 7fae80d98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.999+0000 7fae80d98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9a600 0x560921fa4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:37.999+0000 7fae80d98740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae76a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9a600 0x560921fa4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae76a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9a600 0x560921fa4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48278/0 (socket says 172.21.15.26:48278) 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae76a00640 1 -- 172.21.15.26:0/581890828 learned_addr learned my addr 172.21.15.26:0/581890828 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560921fa26c0 con 0x560921f99c10 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560921ea70b0 con 0x560921f9f770 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560921fa2840 con 0x560921f9a600 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae77e00640 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae80d98740 10 client.0 _ll_register_callbacks cb 0x560921e92f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae77e00640 1 -- 172.21.15.26:0/581890828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9a600 msgr2=0x560921fa4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae77e00640 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9a600 0x560921fa4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.000+0000 7fae77e00640 1 -- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 msgr2=0x560921fa3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.001+0000 7fae77400640 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.003+0000 7fae77e00640 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.003+0000 7fae77e00640 1 -- 172.21.15.26:0/581890828 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560921ccc620 con 0x560921f9f770 2024-06-13T07:25:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.003+0000 7fae77400640 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:25:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.003+0000 7fae77e00640 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fab4c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fae700cc290 tx=0x7fae70004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751e5a60e307836b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.003+0000 7fae64c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:25:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.004+0000 7fae64c00640 1 -- 172.21.15.26:0/581890828 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700a37d0 con 0x560921f9f770 2024-06-13T07:25:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.004+0000 7fae64c00640 1 -- 172.21.15.26:0/581890828 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fae700a3970 con 0x560921f9f770 2024-06-13T07:25:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.004+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae6001d070 con 0x560921f9f770 2024-06-13T07:25:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.004+0000 7fae64c00640 1 -- 172.21.15.26:0/581890828 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fae7009a4f0 con 0x560921f9f770 2024-06-13T07:25:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.004+0000 7fae80d98740 10 client.27245 fetch_fsmap learned FSMap version 51 2024-06-13T07:25:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.004+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x560921fae880 con 0x560921f9f770 2024-06-13T07:25:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.005+0000 7fae64c00640 1 -- 172.21.15.26:0/581890828 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae6001d070 con 0x560921f9f770 2024-06-13T07:25:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.006+0000 7fae77400640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae5c0a2d30 0x7fae5c0a51f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fae6802a930 tx=0x7fae6802a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.006+0000 7fae64c00640 1 -- 172.21.15.26:0/581890828 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 51) ==== 8346+0+0 (secure 0 0 0) 0x7fae700b7050 con 0x560921f9f770 2024-06-13T07:25:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.007+0000 7fae80d98740 10 client.27245 fetch_fsmap finished waiting for FSMap version 51 2024-06-13T07:25:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.007+0000 7fae80d98740 10 client.27245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:25:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.007+0000 7fae80d98740 20 client.27245 populate_metadata read hostname 'smithi026' 2024-06-13T07:25:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.007+0000 7fae80d98740 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560921faff60 0x560921fb2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.007+0000 7fae80d98740 4 client.27245 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:25:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.007+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560921ef9b50 con 0x560921faff60 2024-06-13T07:25:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.008+0000 7fae77400640 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560921faff60 0x560921fb2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.009+0000 7fae77400640 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560921faff60 0x560921fb2380 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4840cbd78b0b8381 server_cookie=6365be444bc779db in_seq=0 out_seq=0 2024-06-13T07:25:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.009+0000 7fae64c00640 10 client.27245 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.026+0000 7fae64c00640 1 -- 172.21.15.26:0/581890828 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560921ef9b50 con 0x560921faff60 2024-06-13T07:25:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.026+0000 7fae64c00640 10 client.27245 handle_command_reply: tid=0 2024-06-13T07:25:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.026+0000 7fae80d98740 1 client.27245 shutdown 2024-06-13T07:25:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 msgr2=0x560921fab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae80d98740 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fab4c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fae700cc290 tx=0x7fae70004b50 comp rx=0 tx=0).stop 2024-06-13T07:25:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 shutdown_connections 2024-06-13T07:25:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae80d98740 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x560921faff60 0x560921fb2380 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae80d98740 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9f770 0x560921fab4c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae80d98740 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9a600 0x560921fa4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae64c00640 0 client.27245 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae80d98740 1 --2- 172.21.15.26:0/581890828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.027+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 >> 172.21.15.26:0/581890828 conn(0x560921f50fb0 msgr2=0x560921f83840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.029+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 shutdown_connections 2024-06-13T07:25:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.029+0000 7fae80d98740 1 -- 172.21.15.26:0/581890828 wait complete. 2024-06-13T07:25:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.029+0000 7fae80d98740 20 client.27245 trim_cache size 0 max 16384 2024-06-13T07:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.071+0000 7fae80d98740 1 Processor -- start 2024-06-13T07:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.071+0000 7fae80d98740 1 -- start start 2024-06-13T07:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.071+0000 7fae80d98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.071+0000 7fae80d98740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.071+0000 7fae80d98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f9f770 0x560921f9bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae80d98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae781b3780 con 0x560921f9f770 2024-06-13T07:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae80d98740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae781b1050 con 0x560921f9a600 2024-06-13T07:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae80d98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae78073810 con 0x560921f99c10 2024-06-13T07:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae76a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae76a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:58912/0 (socket says 172.21.15.26:58912) 2024-06-13T07:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae76a00640 1 -- 172.21.15.26:0/2232615762 learned_addr learned my addr 172.21.15.26:0/2232615762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae76a00640 1 -- 172.21.15.26:0/2232615762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 msgr2=0x560921f9b980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae76a00640 1 --2- 172.21.15.26:0/2232615762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae76a00640 1 -- 172.21.15.26:0/2232615762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f9f770 msgr2=0x560921f9bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae76a00640 1 --2- 172.21.15.26:0/2232615762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f9f770 0x560921f9bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.072+0000 7fae76a00640 1 -- 172.21.15.26:0/2232615762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560921f9ec20 con 0x560921f9a600 2024-06-13T07:25:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.073+0000 7fae76a00640 1 --2- 172.21.15.26:0/2232615762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fab480 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fae600175d0 tx=0x7fae600046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2f2ac45c854aea server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.073+0000 7fae67400640 1 -- 172.21.15.26:0/2232615762 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60027070 con 0x560921f9a600 2024-06-13T07:25:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.073+0000 7fae67400640 1 -- 172.21.15.26:0/2232615762 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae6001d040 con 0x560921f9a600 2024-06-13T07:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.073+0000 7fae67400640 1 -- 172.21.15.26:0/2232615762 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60018d20 con 0x560921f9a600 2024-06-13T07:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.074+0000 7fae80d98740 1 -- 172.21.15.26:0/2232615762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 msgr2=0x560921fab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.074+0000 7fae80d98740 1 --2- 172.21.15.26:0/2232615762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fab480 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fae600175d0 tx=0x7fae600046c0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.075+0000 7fae80d98740 1 -- 172.21.15.26:0/2232615762 shutdown_connections 2024-06-13T07:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.075+0000 7fae80d98740 1 --2- 172.21.15.26:0/2232615762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fab480 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.075+0000 7fae80d98740 1 --2- 172.21.15.26:0/2232615762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f9f770 0x560921f9bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.075+0000 7fae80d98740 1 --2- 172.21.15.26:0/2232615762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f99c10 0x560921f9b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.075+0000 7fae80d98740 1 -- 172.21.15.26:0/2232615762 >> 172.21.15.26:0/2232615762 conn(0x560921f50fb0 msgr2=0x560921f84100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.075+0000 7fae80d98740 1 -- 172.21.15.26:0/2232615762 shutdown_connections 2024-06-13T07:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.075+0000 7fae80d98740 1 -- 172.21.15.26:0/2232615762 wait complete. 2024-06-13T07:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.075+0000 7fae80d98740 1 -- start start 2024-06-13T07:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.076+0000 7fae80d98740 1 Processor -- start 2024-06-13T07:25:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.077+0000 7fae80d98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.077+0000 7fae80d98740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fa8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae80d98740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9f770 0x560921fa9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae76a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fa8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae80d98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae781af510 con 0x560921f99c10 2024-06-13T07:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae80d98740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae781b0980 con 0x560921f9a600 2024-06-13T07:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae80d98740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae781afbe0 con 0x560921f9f770 2024-06-13T07:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae80d98740 10 client.0 _ll_register_callbacks cb 0x7fae70097340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae77e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9f770 0x560921fa9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae77e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9f770 0x560921fa9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:48286/0 (socket says 172.21.15.26:48286) 2024-06-13T07:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.078+0000 7fae77e00640 1 -- 172.21.15.26:0/1969035610 learned_addr learned my addr 172.21.15.26:0/1969035610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.079+0000 7fae76a00640 1 -- 172.21.15.26:0/1969035610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9f770 msgr2=0x560921fa9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.079+0000 7fae76a00640 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9f770 0x560921fa9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.079+0000 7fae76a00640 1 -- 172.21.15.26:0/1969035610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 msgr2=0x560921fa8630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.083+0000 7fae77400640 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.083+0000 7fae76a00640 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.083+0000 7fae76a00640 1 -- 172.21.15.26:0/1969035610 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560921faa120 con 0x560921f9a600 2024-06-13T07:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.084+0000 7fae64c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-13T07:25:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.084+0000 7fae76a00640 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fa8b70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fae60006d10 tx=0x7fae60018c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927f1b0be3975748 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.085+0000 7fae64c00640 1 -- 172.21.15.26:0/1969035610 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6003a060 con 0x560921f9a600 2024-06-13T07:25:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.085+0000 7fae64c00640 1 -- 172.21.15.26:0/1969035610 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fae60015990 con 0x560921f9a600 2024-06-13T07:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.085+0000 7fae80d98740 1 -- 172.21.15.26:0/1969035610 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae6001d040 con 0x560921f9a600 2024-06-13T07:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.086+0000 7fae64c00640 1 -- 172.21.15.26:0/1969035610 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fae60027040 con 0x560921f9a600 2024-06-13T07:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.086+0000 7fae80d98740 10 client.27251 fetch_fsmap learned FSMap version 51 2024-06-13T07:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.086+0000 7fae80d98740 1 -- 172.21.15.26:0/1969035610 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x560921fab430 con 0x560921f9a600 2024-06-13T07:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.087+0000 7fae64c00640 1 -- 172.21.15.26:0/1969035610 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae6001d040 con 0x560921f9a600 2024-06-13T07:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.087+0000 7fae64c00640 1 -- 172.21.15.26:0/1969035610 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 51) ==== 8346+0+0 (secure 0 0 0) 0x7fae60034070 con 0x560921f9a600 2024-06-13T07:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.087+0000 7fae80d98740 10 client.27251 fetch_fsmap finished waiting for FSMap version 51 2024-06-13T07:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.087+0000 7fae80d98740 10 client.27251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.087+0000 7fae80d98740 20 client.27251 populate_metadata read hostname 'smithi026' 2024-06-13T07:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.087+0000 7fae80d98740 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56092201ad00 0x56092201d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.087+0000 7fae80d98740 4 client.27251 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.087+0000 7fae80d98740 1 -- 172.21.15.26:0/1969035610 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560921ef3ea0 con 0x56092201ad00 2024-06-13T07:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.088+0000 7fae77400640 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56092201ad00 0x56092201d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.089+0000 7fae77400640 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56092201ad00 0x56092201d0f0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3bffb9b3343199d server_cookie=9c96182211bf0da3 in_seq=0 out_seq=0 2024-06-13T07:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:38.089+0000 7fae64c00640 10 client.27251 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:38.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 337590 24.96 MB/sec execute 591 sec latency 672.831 ms 2024-06-13T07:25:38.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:38 smithi135 ceph-mon[36276]: pgmap v1263: 129 pgs: 129 active+clean; 699 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.5 MiB/s wr, 406 op/s 2024-06-13T07:25:38.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:38 smithi136 ceph-mon[37229]: pgmap v1263: 129 pgs: 129 active+clean; 699 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.5 MiB/s wr, 406 op/s 2024-06-13T07:25:38.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:38 smithi026 ceph-mon[32041]: pgmap v1263: 129 pgs: 129 active+clean; 699 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.5 MiB/s wr, 406 op/s 2024-06-13T07:25:39.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 337667 24.93 MB/sec execute 592 sec latency 1178.437 ms 2024-06-13T07:25:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.978+0000 7fae64c00640 1 -- 172.21.15.26:0/1969035610 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560921ef3ea0 con 0x56092201ad00 2024-06-13T07:25:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.978+0000 7fae64c00640 10 client.27251 handle_command_reply: tid=0 2024-06-13T07:25:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.978+0000 7fae80d98740 1 client.27251 shutdown 2024-06-13T07:25:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae80d98740 1 -- 172.21.15.26:0/1969035610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 msgr2=0x560921fa8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae80d98740 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fa8b70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fae60006d10 tx=0x7fae60018c50 comp rx=0 tx=0).stop 2024-06-13T07:25:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae80d98740 1 -- 172.21.15.26:0/1969035610 shutdown_connections 2024-06-13T07:25:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae80d98740 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x56092201ad00 0x56092201d0f0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae80d98740 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560921f9a600 0x560921fa8b70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae80d98740 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560921f9f770 0x560921fa9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae80d98740 1 --2- 172.21.15.26:0/1969035610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560921f99c10 0x560921fa8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae80d98740 1 -- 172.21.15.26:0/1969035610 >> 172.21.15.26:0/1969035610 conn(0x560921f50fb0 msgr2=0x560921f9b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.983+0000 7fae64c00640 0 client.27251 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.984+0000 7fae80d98740 1 -- 172.21.15.26:0/1969035610 shutdown_connections 2024-06-13T07:25:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.984+0000 7fae80d98740 1 -- 172.21.15.26:0/1969035610 wait complete. 2024-06-13T07:25:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.984+0000 7fae80d98740 20 client.27251 trim_cache size 0 max 16384 2024-06-13T07:25:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:25:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:25:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:25:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.985+0000 7fae7e000640 1 -- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae5c0a2d30 msgr2=0x7fae5c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.985+0000 7fae7e000640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae5c0a2d30 0x7fae5c0a51f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fae6802a930 tx=0x7fae6802a410 comp rx=0 tx=0).stop 2024-06-13T07:25:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.985+0000 7fae7e000640 1 -- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae781035b0 msgr2=0x7fae78196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.985+0000 7fae7e000640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae781035b0 0x7fae78196120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae7000be20 tx=0x7fae7000bf10 comp rx=0 tx=0).stop 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.986+0000 7fae7e000640 1 -- 172.21.15.26:0/1898189588 shutdown_connections 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.986+0000 7fae7e000640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fae5c0a2d30 0x7fae5c0a51f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.986+0000 7fae7e000640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae781035b0 0x7fae78196120 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.986+0000 7fae7e000640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae78102bf0 0x7fae7819b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.986+0000 7fae7e000640 1 --2- 172.21.15.26:0/1898189588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78101a00 0x7fae78195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.986+0000 7fae7e000640 1 -- 172.21.15.26:0/1898189588 >> 172.21.15.26:0/1898189588 conn(0x7fae780fcf70 msgr2=0x7fae78103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.986+0000 7fae7e000640 1 -- 172.21.15.26:0/1898189588 shutdown_connections 2024-06-13T07:25:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:39.986+0000 7fae7e000640 1 -- 172.21.15.26:0/1898189588 wait complete. 2024-06-13T07:25:40.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 337918 24.91 MB/sec execute 593 sec latency 1370.423 ms 2024-06-13T07:25:40.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:40 smithi135 ceph-mon[36276]: pgmap v1264: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-06-13T07:25:40.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:40 smithi136 ceph-mon[37229]: pgmap v1264: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-06-13T07:25:40.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:40 smithi026 ceph-mon[32041]: pgmap v1264: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-06-13T07:25:41.261 INFO:tasks.workunit.client.0.smithi026.stdout: 10 338323 24.90 MB/sec execute 594 sec latency 150.676 ms 2024-06-13T07:25:42.262 INFO:tasks.workunit.client.0.smithi026.stdout: 10 338827 24.91 MB/sec execute 595 sec latency 70.834 ms 2024-06-13T07:25:42.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:42 smithi136 ceph-mon[37229]: pgmap v1265: 129 pgs: 129 active+clean; 698 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 386 op/s 2024-06-13T07:25:42.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:42 smithi026 ceph-mon[32041]: pgmap v1265: 129 pgs: 129 active+clean; 698 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 386 op/s 2024-06-13T07:25:42.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:42 smithi135 ceph-mon[36276]: pgmap v1265: 129 pgs: 129 active+clean; 698 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 386 op/s 2024-06-13T07:25:43.262 INFO:tasks.workunit.client.0.smithi026.stdout: 10 339333 24.92 MB/sec execute 596 sec latency 104.433 ms 2024-06-13T07:25:44.262 INFO:tasks.workunit.client.0.smithi026.stdout: 10 339347 24.88 MB/sec execute 597 sec latency 940.211 ms 2024-06-13T07:25:44.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:44 smithi136 ceph-mon[37229]: pgmap v1266: 129 pgs: 129 active+clean; 688 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.9 MiB/s wr, 413 op/s 2024-06-13T07:25:44.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:44 smithi026 ceph-mon[32041]: pgmap v1266: 129 pgs: 129 active+clean; 688 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.9 MiB/s wr, 413 op/s 2024-06-13T07:25:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:44 smithi135 ceph-mon[36276]: pgmap v1266: 129 pgs: 129 active+clean; 688 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.9 MiB/s wr, 413 op/s 2024-06-13T07:25:45.265 INFO:tasks.workunit.client.0.smithi026.stdout: 10 339384 24.84 MB/sec execute 598 sec latency 1833.906 ms 2024-06-13T07:25:46.265 INFO:tasks.workunit.client.0.smithi026.stdout: 10 339561 24.82 MB/sec execute 599 sec latency 2203.958 ms 2024-06-13T07:25:46.584 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:46 smithi135 ceph-mon[36276]: pgmap v1267: 129 pgs: 129 active+clean; 706 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 567 op/s 2024-06-13T07:25:46.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:46 smithi136 ceph-mon[37229]: pgmap v1267: 129 pgs: 129 active+clean; 706 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 567 op/s 2024-06-13T07:25:46.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:46 smithi026 ceph-mon[32041]: pgmap v1267: 129 pgs: 129 active+clean; 706 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 567 op/s 2024-06-13T07:25:47.265 INFO:tasks.workunit.client.0.smithi026.stdout: 10 cleanup 600 sec 2024-06-13T07:25:48.265 INFO:tasks.workunit.client.0.smithi026.stdout: 7 cleanup 601 sec 2024-06-13T07:25:48.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:48 smithi136 ceph-mon[37229]: pgmap v1268: 129 pgs: 129 active+clean; 713 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.2 MiB/s wr, 465 op/s 2024-06-13T07:25:48.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:48 smithi026 ceph-mon[32041]: pgmap v1268: 129 pgs: 129 active+clean; 713 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.2 MiB/s wr, 465 op/s 2024-06-13T07:25:48.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:48 smithi135 ceph-mon[36276]: pgmap v1268: 129 pgs: 129 active+clean; 713 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.2 MiB/s wr, 465 op/s 2024-06-13T07:25:48.999 INFO:tasks.workunit.client.0.smithi026.stdout: 0 cleanup 602 sec 2024-06-13T07:25:48.999 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-06-13T07:25:48.999 INFO:tasks.workunit.client.0.smithi026.stdout: Operation Count AvgLat MaxLat 2024-06-13T07:25:48.999 INFO:tasks.workunit.client.0.smithi026.stdout: ---------------------------------------- 2024-06-13T07:25:48.999 INFO:tasks.workunit.client.0.smithi026.stdout: NTCreateX 476969 5.119 2488.635 2024-06-13T07:25:48.999 INFO:tasks.workunit.client.0.smithi026.stdout: Close 350222 0.008 2.759 2024-06-13T07:25:48.999 INFO:tasks.workunit.client.0.smithi026.stdout: Rename 20213 21.837 2258.494 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: Unlink 96369 5.512 674.989 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: Qpathinfo 432613 3.493 2743.785 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: Qfileinfo 75456 0.003 4.114 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: Qfsinfo 79319 0.603 332.778 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: Sfileinfo 38804 0.350 303.065 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: Find 167181 0.408 1525.940 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: WriteX 235673 0.182 1320.343 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: ReadX 748979 0.142 1959.968 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: LockX 1560 3.383 188.013 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: UnlockX 1560 2.726 51.110 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: Flush 33403 23.075 2519.629 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-06-13T07:25:49.000 INFO:tasks.workunit.client.0.smithi026.stdout:Throughput 24.8159 MB/sec 10 clients 10 procs max_latency=2743.791 ms 2024-06-13T07:25:49.019 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-13T07:25:49.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-13T07:25:50.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:50 smithi136 ceph-mon[37229]: pgmap v1269: 129 pgs: 129 active+clean; 714 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 635 op/s 2024-06-13T07:25:50.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:50 smithi026 ceph-mon[32041]: pgmap v1269: 129 pgs: 129 active+clean; 714 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 635 op/s 2024-06-13T07:25:50.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:50 smithi135 ceph-mon[36276]: pgmap v1269: 129 pgs: 129 active+clean; 714 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 635 op/s 2024-06-13T07:25:51.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:51 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:51.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:51 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:51.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:51 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:51.871 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-06-13T07:25:51.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-13T07:25:52.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:52 smithi136 ceph-mon[37229]: pgmap v1270: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 610 op/s 2024-06-13T07:25:52.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:52 smithi026 ceph-mon[32041]: pgmap v1270: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 610 op/s 2024-06-13T07:25:52.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:52 smithi135 ceph-mon[36276]: pgmap v1270: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 610 op/s 2024-06-13T07:25:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.144+0000 7f698ce00640 1 -- 172.21.15.26:0/3026882345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988102300 msgr2=0x7f69881046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.144+0000 7f698ce00640 1 --2- 172.21.15.26:0/3026882345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988102300 0x7f69881046f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6978009900 tx=0x7f697802a410 comp rx=0 tx=0).stop 2024-06-13T07:25:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.145+0000 7f698ce00640 1 -- 172.21.15.26:0/3026882345 shutdown_connections 2024-06-13T07:25:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.145+0000 7f698ce00640 1 --2- 172.21.15.26:0/3026882345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988107560 0x7f6988109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.145+0000 7f698ce00640 1 --2- 172.21.15.26:0/3026882345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6988104c30 0x7f6988107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.145+0000 7f698ce00640 1 --2- 172.21.15.26:0/3026882345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988102300 0x7f69881046f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.145+0000 7f698ce00640 1 -- 172.21.15.26:0/3026882345 >> 172.21.15.26:0/3026882345 conn(0x7f69880fbc90 msgr2=0x7f69880fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.145+0000 7f698ce00640 1 -- 172.21.15.26:0/3026882345 shutdown_connections 2024-06-13T07:25:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.145+0000 7f698ce00640 1 -- 172.21.15.26:0/3026882345 wait complete. 2024-06-13T07:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f698ce00640 1 Processor -- start 2024-06-13T07:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f698ce00640 1 -- start start 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f698ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6988102300 0x7f6988101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f698ce00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988104c30 0x7f69881002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f698ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988107560 0x7f69881007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f698ce00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f698810d4e0 con 0x7f6988107560 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f698ce00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f698810d360 con 0x7f6988104c30 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f698ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f698810d660 con 0x7f6988102300 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f6987400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6988102300 0x7f6988101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f6987400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6988102300 0x7f6988101c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35120/0 (socket says 172.21.15.26:35120) 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f6987400640 1 -- 172.21.15.26:0/2590086238 learned_addr learned my addr 172.21.15.26:0/2590086238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f6986a00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988104c30 0x7f69881002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.146+0000 7f6987e00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988107560 0x7f69881007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.147+0000 7f6987400640 1 -- 172.21.15.26:0/2590086238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988104c30 msgr2=0x7f69881002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.147+0000 7f6987400640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988104c30 0x7f69881002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.147+0000 7f6987400640 1 -- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988107560 msgr2=0x7f69881007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.147+0000 7f6987400640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988107560 0x7f69881007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.147+0000 7f6987400640 1 -- 172.21.15.26:0/2590086238 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6988101080 con 0x7f6988102300 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.147+0000 7f6986a00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988104c30 0x7f69881002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.147+0000 7f6987400640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6988102300 0x7f6988101c70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6978031440 tx=0x7f6978032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.147+0000 7f697fe00640 1 -- 172.21.15.26:0/2590086238 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697803f030 con 0x7f6988102300 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.148+0000 7f697fe00640 1 -- 172.21.15.26:0/2590086238 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6978003ce0 con 0x7f6988102300 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.148+0000 7f697fe00640 1 -- 172.21.15.26:0/2590086238 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978035070 con 0x7f6988102300 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.148+0000 7f698ce00640 1 -- 172.21.15.26:0/2590086238 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a1c70 con 0x7f6988102300 2024-06-13T07:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.148+0000 7f698ce00640 1 -- 172.21.15.26:0/2590086238 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a21c0 con 0x7f6988102300 2024-06-13T07:25:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.152+0000 7f6991266740 1 Processor -- start 2024-06-13T07:25:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.152+0000 7f6991266740 1 -- start start 2024-06-13T07:25:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.152+0000 7f697fe00640 1 -- 172.21.15.26:0/2590086238 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6978042030 con 0x7f6988102300 2024-06-13T07:25:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.152+0000 7f6991266740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae0dc0 0x5593b6ae11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.152+0000 7f6991266740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6ae62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6991266740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae6920 0x5593b6ae8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6991266740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5593b6a44420 con 0x5593b6ae0dc0 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6991266740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5593b69ebc50 con 0x5593b6ae17b0 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6991266740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5593b6a403e0 con 0x5593b6ae6920 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae0dc0 0x5593b6ae11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae0dc0 0x5593b6ae11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50140/0 (socket says 172.21.15.26:50140) 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987e00640 1 -- 172.21.15.26:0/4250337973 learned_addr learned my addr 172.21.15.26:0/4250337973 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987400640 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6ae62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6986a00640 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae6920 0x5593b6ae8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987e00640 1 -- 172.21.15.26:0/4250337973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae6920 msgr2=0x5593b6ae8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987e00640 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae6920 0x5593b6ae8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987e00640 1 -- 172.21.15.26:0/4250337973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 msgr2=0x5593b6ae62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987e00640 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6ae62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.153+0000 7f6987e00640 1 -- 172.21.15.26:0/4250337973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b6758210 con 0x5593b6ae0dc0 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.154+0000 7f6987e00640 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae0dc0 0x5593b6ae11e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6980009fd0 tx=0x7f69800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702e88ba3d6c6f14 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.154+0000 7f697cc00640 1 -- 172.21.15.26:0/4250337973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980004d20 con 0x5593b6ae0dc0 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.154+0000 7f697cc00640 1 -- 172.21.15.26:0/4250337973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6980004050 con 0x5593b6ae0dc0 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.154+0000 7f697cc00640 1 -- 172.21.15.26:0/4250337973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980005240 con 0x5593b6ae0dc0 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.154+0000 7f697fe00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f696c0a2c00 0x7f696c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f697fe00640 1 -- 172.21.15.26:0/2590086238 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f69780efc40 con 0x7f6988102300 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6987400640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f696c0a2c00 0x7f696c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 -- 172.21.15.26:0/4250337973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae0dc0 msgr2=0x5593b6ae11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae0dc0 0x5593b6ae11e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6980009fd0 tx=0x7f69800118d0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 -- 172.21.15.26:0/4250337973 shutdown_connections 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6ae62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae6920 0x5593b6ae8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 --2- 172.21.15.26:0/4250337973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae0dc0 0x5593b6ae11e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 -- 172.21.15.26:0/4250337973 >> 172.21.15.26:0/4250337973 conn(0x5593b6aa8fb0 msgr2=0x5593b6adc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 -- 172.21.15.26:0/4250337973 shutdown_connections 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6991266740 1 -- 172.21.15.26:0/4250337973 wait complete. 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.155+0000 7f6987400640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f696c0a2c00 0x7f696c0a50c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f69780b8ad0 tx=0x7f69780f3680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 1 -- start start 2024-06-13T07:25:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 1 Processor -- start 2024-06-13T07:25:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6aeb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae17b0 0x5593b6aeb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae6920 0x5593b6af2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5593b6ae98d0 con 0x5593b6ae17b0 2024-06-13T07:25:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5593b69ff0b0 con 0x5593b6ae6920 2024-06-13T07:25:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5593b6ae9a50 con 0x5593b6ae0dc0 2024-06-13T07:25:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.156+0000 7f6991266740 10 client.0 _ll_register_callbacks cb 0x5593b69eaf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:25:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae17b0 0x5593b6aeb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae17b0 0x5593b6aeb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50150/0 (socket says 172.21.15.26:50150) 2024-06-13T07:25:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 -- 172.21.15.26:0/3085427808 learned_addr learned my addr 172.21.15.26:0/3085427808 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6987e00640 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6aeb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 -- 172.21.15.26:0/3085427808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 msgr2=0x5593b6aeb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6aeb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 -- 172.21.15.26:0/3085427808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae6920 msgr2=0x5593b6af2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae6920 0x5593b6af2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 -- 172.21.15.26:0/3085427808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b6824620 con 0x5593b6ae17b0 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6987e00640 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6aeb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6986a00640 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae17b0 0x5593b6aeb610 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6974012d20 tx=0x7f697400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7da93b3fedc225 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.157+0000 7f6971600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.158+0000 7f6971600640 1 -- 172.21.15.26:0/3085427808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974012110 con 0x5593b6ae17b0 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.158+0000 7f6971600640 1 -- 172.21.15.26:0/3085427808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6974004050 con 0x5593b6ae17b0 2024-06-13T07:25:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.158+0000 7f6991266740 1 -- 172.21.15.26:0/3085427808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6980004050 con 0x5593b6ae17b0 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.158+0000 7f6971600640 1 -- 172.21.15.26:0/3085427808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6974025050 con 0x5593b6ae17b0 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.158+0000 7f6991266740 10 client.16953 fetch_fsmap learned FSMap version 52 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.158+0000 7f6991266740 1 -- 172.21.15.26:0/3085427808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5593b6af5a90 con 0x5593b6ae17b0 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.158+0000 7f6971600640 1 -- 172.21.15.26:0/3085427808 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6980004050 con 0x5593b6ae17b0 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.158+0000 7f6971600640 1 -- 172.21.15.26:0/3085427808 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f6974010430 con 0x5593b6ae17b0 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.159+0000 7f6991266740 10 client.16953 fetch_fsmap finished waiting for FSMap version 52 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.159+0000 7f6991266740 10 client.16953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.159+0000 7f6991266740 20 client.16953 populate_metadata read hostname 'smithi026' 2024-06-13T07:25:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.159+0000 7f6991266740 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5593b6af7220 0x5593b6af9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.159+0000 7f6991266740 4 client.16953 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.159+0000 7f6991266740 1 -- 172.21.15.26:0/3085427808 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593b6a51b50 con 0x5593b6af7220 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.159+0000 7f6987e00640 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5593b6af7220 0x5593b6af9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.160+0000 7f6971600640 10 client.16953 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.159+0000 7f6987e00640 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5593b6af7220 0x5593b6af9660 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c0cb659fd16754 server_cookie=4cef445aad25672c in_seq=0 out_seq=0 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.160+0000 7f6971600640 1 -- 172.21.15.26:0/3085427808 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5593b6a51b50 con 0x5593b6af7220 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.160+0000 7f6971600640 10 client.16953 handle_command_reply: tid=0 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.160+0000 7f6991266740 1 client.16953 shutdown 2024-06-13T07:25:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 -- 172.21.15.26:0/3085427808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae17b0 msgr2=0x5593b6aeb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae17b0 0x5593b6aeb610 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6974012d20 tx=0x7f697400f790 comp rx=0 tx=0).stop 2024-06-13T07:25:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 -- 172.21.15.26:0/3085427808 shutdown_connections 2024-06-13T07:25:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5593b6af7220 0x5593b6af9660 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae6920 0x5593b6af2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae17b0 0x5593b6aeb610 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 --2- 172.21.15.26:0/3085427808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6aeb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6971600640 0 client.16953 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 -- 172.21.15.26:0/3085427808 >> 172.21.15.26:0/3085427808 conn(0x5593b6aa8fb0 msgr2=0x5593b6adb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 -- 172.21.15.26:0/3085427808 shutdown_connections 2024-06-13T07:25:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 1 -- 172.21.15.26:0/3085427808 wait complete. 2024-06-13T07:25:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.161+0000 7f6991266740 20 client.16953 trim_cache size 0 max 16384 2024-06-13T07:25:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.177+0000 7f6991266740 1 Processor -- start 2024-06-13T07:25:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6991266740 1 -- start start 2024-06-13T07:25:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6991266740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6adc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6991266740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6af2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6991266740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6ae2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6991266740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69881af1b0 con 0x5593b6ae6920 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6991266740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69881aca50 con 0x5593b6ae17b0 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6991266740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69881aa8a0 con 0x5593b6ae0dc0 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6987e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6adc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6987e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6adc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35154/0 (socket says 172.21.15.26:35154) 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6987e00640 1 -- 172.21.15.26:0/3616615587 learned_addr learned my addr 172.21.15.26:0/3616615587 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6986a00640 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6af2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.178+0000 7f6987e00640 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6ae2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6987e00640 1 -- 172.21.15.26:0/3616615587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 msgr2=0x5593b6af2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6987e00640 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6af2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6987e00640 1 -- 172.21.15.26:0/3616615587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 msgr2=0x5593b6ae2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6987e00640 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6ae2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6987e00640 1 -- 172.21.15.26:0/3616615587 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b6ae2040 con 0x5593b6ae0dc0 2024-06-13T07:25:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6986a00640 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6af2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6987e00640 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6adc2c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6980009f70 tx=0x7f6980000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3496813637cd53b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6973e00640 1 -- 172.21.15.26:0/3616615587 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980005050 con 0x5593b6ae0dc0 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6973e00640 1 -- 172.21.15.26:0/3616615587 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69800056d0 con 0x5593b6ae0dc0 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.179+0000 7f6973e00640 1 -- 172.21.15.26:0/3616615587 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698000fba0 con 0x5593b6ae0dc0 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 -- 172.21.15.26:0/3616615587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 msgr2=0x5593b6adc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6adc2c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6980009f70 tx=0x7f6980000f40 comp rx=0 tx=0).stop 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 -- 172.21.15.26:0/3616615587 shutdown_connections 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae17b0 0x5593b6af2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6ae2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 --2- 172.21.15.26:0/3616615587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae0dc0 0x5593b6adc2c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 -- 172.21.15.26:0/3616615587 >> 172.21.15.26:0/3616615587 conn(0x5593b6aa8fb0 msgr2=0x5593b6ae6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 -- 172.21.15.26:0/3616615587 shutdown_connections 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 -- 172.21.15.26:0/3616615587 wait complete. 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.180+0000 7f6991266740 1 -- start start 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6991266740 1 Processor -- start 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6991266740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae0dc0 0x5593b6aef850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6991266740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae17b0 0x5593b6aefd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6991266740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6af0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6991266740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69881aaf10 con 0x5593b6ae6920 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6991266740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69881ac380 con 0x5593b6ae0dc0 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6991266740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69881ab5e0 con 0x5593b6ae17b0 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6991266740 10 client.0 _ll_register_callbacks cb 0x7f6974009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6986a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae17b0 0x5593b6aefd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6987e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6af0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6986a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae17b0 0x5593b6aefd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35170/0 (socket says 172.21.15.26:35170) 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6986a00640 1 -- 172.21.15.26:0/1167167999 learned_addr learned my addr 172.21.15.26:0/1167167999 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.181+0000 7f6987e00640 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae0dc0 0x5593b6aef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6986a00640 1 -- 172.21.15.26:0/1167167999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae0dc0 msgr2=0x5593b6aef850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6986a00640 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae0dc0 0x5593b6aef850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6986a00640 1 -- 172.21.15.26:0/1167167999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 msgr2=0x5593b6af0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6986a00640 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6af0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6986a00640 1 -- 172.21.15.26:0/1167167999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b6af1320 con 0x5593b6ae17b0 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6987e00640 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6af0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6986a00640 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae17b0 0x5593b6aefd90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5593b6ad6ff0 tx=0x7f697400fef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f841f5a5855f3a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6971600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6971600640 1 -- 172.21.15.26:0/1167167999 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974010cd0 con 0x5593b6ae17b0 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6971600640 1 -- 172.21.15.26:0/1167167999 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6974018040 con 0x5593b6ae17b0 2024-06-13T07:25:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.182+0000 7f6991266740 1 -- 172.21.15.26:0/1167167999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69800056d0 con 0x5593b6ae17b0 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6971600640 1 -- 172.21.15.26:0/1167167999 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f697403a050 con 0x5593b6ae17b0 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6991266740 10 client.27328 fetch_fsmap learned FSMap version 52 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6991266740 1 -- 172.21.15.26:0/1167167999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5593b6af26d0 con 0x5593b6ae17b0 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6971600640 1 -- 172.21.15.26:0/1167167999 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69800056d0 con 0x5593b6ae17b0 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6971600640 1 -- 172.21.15.26:0/1167167999 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f6974003680 con 0x5593b6ae17b0 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6991266740 10 client.27328 fetch_fsmap finished waiting for FSMap version 52 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6991266740 10 client.27328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.g 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6991266740 20 client.27328 populate_metadata read hostname 'smithi026' 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6991266740 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5593b6b623d0 0x5593b6b647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6991266740 4 client.27328 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:25:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.183+0000 7f6991266740 1 -- 172.21.15.26:0/1167167999 --> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593b6a4bea0 con 0x5593b6b623d0 2024-06-13T07:25:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.184+0000 7f6987e00640 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5593b6b623d0 0x5593b6b647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.184+0000 7f6987e00640 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5593b6b623d0 0x5593b6b647f0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7faa8223a7e3d59b server_cookie=810071a18448cd0d in_seq=0 out_seq=0 2024-06-13T07:25:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.184+0000 7f6971600640 10 client.27328 ms_handle_connect on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.473+0000 7f6971600640 1 -- 172.21.15.26:0/1167167999 <== mds.0 v2:172.21.15.26:6838/173910348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5593b6a4bea0 con 0x5593b6b623d0 2024-06-13T07:25:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.473+0000 7f6971600640 10 client.27328 handle_command_reply: tid=0 2024-06-13T07:25:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.473+0000 7f6991266740 1 client.27328 shutdown 2024-06-13T07:25:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 -- 172.21.15.26:0/1167167999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae17b0 msgr2=0x5593b6aefd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae17b0 0x5593b6aefd90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5593b6ad6ff0 tx=0x7f697400fef0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 -- 172.21.15.26:0/1167167999 shutdown_connections 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.26:6838/173910348,v1:172.21.15.26:6839/173910348] conn(0x5593b6b623d0 0x5593b6b647f0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593b6ae6920 0x5593b6af0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5593b6ae17b0 0x5593b6aefd90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 --2- 172.21.15.26:0/1167167999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593b6ae0dc0 0x5593b6aef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 -- 172.21.15.26:0/1167167999 >> 172.21.15.26:0/1167167999 conn(0x5593b6aa8fb0 msgr2=0x5593b6aeb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6971600640 0 client.27328 ms_handle_reset on v2:172.21.15.26:6838/173910348 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 -- 172.21.15.26:0/1167167999 shutdown_connections 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 1 -- 172.21.15.26:0/1167167999 wait complete. 2024-06-13T07:25:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.474+0000 7f6991266740 20 client.27328 trim_cache size 0 max 16384 2024-06-13T07:25:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-06-13T07:25:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-06-13T07:25:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-06-13T07:25:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 -- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f696c0a2c00 msgr2=0x7f696c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f696c0a2c00 0x7f696c0a50c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f69780b8ad0 tx=0x7f69780f3680 comp rx=0 tx=0).stop 2024-06-13T07:25:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 -- 172.21.15.26:0/2590086238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6988102300 msgr2=0x7f6988101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6988102300 0x7f6988101c70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6978031440 tx=0x7f6978032ce0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 -- 172.21.15.26:0/2590086238 shutdown_connections 2024-06-13T07:25:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f696c0a2c00 0x7f696c0a50c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988104c30 0x7f69881002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988107560 0x7f69881007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 --2- 172.21.15.26:0/2590086238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6988102300 0x7f6988101c70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.475+0000 7f698ce00640 1 -- 172.21.15.26:0/2590086238 >> 172.21.15.26:0/2590086238 conn(0x7f69880fbc90 msgr2=0x7f6988102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.476+0000 7f698ce00640 1 -- 172.21.15.26:0/2590086238 shutdown_connections 2024-06-13T07:25:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-06-13T07:25:53.476+0000 7f698ce00640 1 -- 172.21.15.26:0/2590086238 wait complete. 2024-06-13T07:25:53.585 DEBUG:teuthology.parallel:result is None 2024-06-13T07:25:53.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-13T07:25:53.671 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-13T07:25:53.671 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-13T07:25:53.682 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T07:25:53.707 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-13T07:25:53.728 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-13T07:25:53.728 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-06-13T07:25:53.729 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-13T07:25:53.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 -- 172.21.15.26:0/2615975301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4100bc0 msgr2=0x7effb4101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 --2- 172.21.15.26:0/2615975301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4100bc0 0x7effb4101020 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7effa0009900 tx=0x7effa002a400 comp rx=0 tx=0).stop 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 -- 172.21.15.26:0/2615975301 shutdown_connections 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 --2- 172.21.15.26:0/2615975301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb4101560 0x7effb41019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 --2- 172.21.15.26:0/2615975301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4100bc0 0x7effb4101020 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 --2- 172.21.15.26:0/2615975301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effb4100260 0x7effb4100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 -- 172.21.15.26:0/2615975301 >> 172.21.15.26:0/2615975301 conn(0x7effb40fbe60 msgr2=0x7effb40fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 -- 172.21.15.26:0/2615975301 shutdown_connections 2024-06-13T07:25:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.963+0000 7effbb400640 1 -- 172.21.15.26:0/2615975301 wait complete. 2024-06-13T07:25:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.964+0000 7effbb400640 1 Processor -- start 2024-06-13T07:25:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.964+0000 7effbb400640 1 -- start start 2024-06-13T07:25:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.964+0000 7effbb400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb4100260 0x7effb4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.964+0000 7effbb400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effb4100bc0 0x7effb419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.964+0000 7effbb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4101560 0x7effb41a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.964+0000 7effbb400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7effb410f880 con 0x7effb4101560 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.964+0000 7effbb400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effb410f700 con 0x7effb4100bc0 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.964+0000 7effbb400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effb410fa00 con 0x7effb4100260 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4101560 0x7effb41a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4101560 0x7effb41a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50192/0 (socket says 172.21.15.26:50192) 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 -- 172.21.15.26:0/2360278162 learned_addr learned my addr 172.21.15.26:0/2360278162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb8a00640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb4100260 0x7effb4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb3e00640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effb4100bc0 0x7effb419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 -- 172.21.15.26:0/2360278162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb4100260 msgr2=0x7effb4199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb4100260 0x7effb4199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 -- 172.21.15.26:0/2360278162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effb4100bc0 msgr2=0x7effb419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effb4100bc0 0x7effb419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 -- 172.21.15.26:0/2360278162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb41a1b10 con 0x7effb4101560 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.965+0000 7effb9400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4101560 0x7effb41a14a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7effac009fd0 tx=0x7effac00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.966+0000 7effb1600640 1 -- 172.21.15.26:0/2360278162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac017070 con 0x7effb4101560 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.966+0000 7effb1600640 1 -- 172.21.15.26:0/2360278162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effac003e50 con 0x7effb4101560 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.966+0000 7effb1600640 1 -- 172.21.15.26:0/2360278162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0106c0 con 0x7effb4101560 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.966+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb41a1e00 con 0x7effb4101560 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.966+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb41a2270 con 0x7effb4101560 2024-06-13T07:25:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.967+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb410b160 con 0x7effb4101560 2024-06-13T07:25:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.969+0000 7effb1600640 1 -- 172.21.15.26:0/2360278162 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effac003970 con 0x7effb4101560 2024-06-13T07:25:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.970+0000 7effb1600640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff9c0a2c10 0x7eff9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.970+0000 7effb1600640 1 -- 172.21.15.26:0/2360278162 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7effac0cf090 con 0x7effb4101560 2024-06-13T07:25:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.971+0000 7effb8a00640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff9c0a2c10 0x7eff9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:53.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.971+0000 7effb8a00640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff9c0a2c10 0x7eff9c0a50d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7effa800bbe0 tx=0x7effa800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:53.986+0000 7effb1600640 1 -- 172.21.15.26:0/2360278162 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7effac09b5b0 con 0x7effb4101560 2024-06-13T07:25:54.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.198+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effb4062640 con 0x7effb4101560 2024-06-13T07:25:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.201+0000 7effb1600640 1 -- 172.21.15.26:0/2360278162 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7effac09f8b0 con 0x7effb4101560 2024-06-13T07:25:54.202 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:25:54.202 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff9c0a2c10 msgr2=0x7eff9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff9c0a2c10 0x7eff9c0a50d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7effa800bbe0 tx=0x7effa800b390 comp rx=0 tx=0).stop 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4101560 msgr2=0x7effb41a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4101560 0x7effb41a14a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7effac009fd0 tx=0x7effac00b260 comp rx=0 tx=0).stop 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 shutdown_connections 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7eff9c0a2c10 0x7eff9c0a50d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effb4100bc0 0x7effb419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb4101560 0x7effb41a14a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 --2- 172.21.15.26:0/2360278162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb4100260 0x7effb4199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 >> 172.21.15.26:0/2360278162 conn(0x7effb40fbe60 msgr2=0x7effb40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 shutdown_connections 2024-06-13T07:25:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.204+0000 7effbb400640 1 -- 172.21.15.26:0/2360278162 wait complete. 2024-06-13T07:25:54.223 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-13T06:49:18.857067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-13T06:54:52.893467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-13T06:54:57.814451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-13T06:55:10.534099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-13T07:25:54.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-13T07:25:54.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 -- 172.21.15.26:0/4112376378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd941034e0 msgr2=0x7fbd94109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 --2- 172.21.15.26:0/4112376378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd941034e0 0x7fbd94109d70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009ca0 tx=0x7fbd8c02a8b0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 -- 172.21.15.26:0/4112376378 shutdown_connections 2024-06-13T07:25:54.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 --2- 172.21.15.26:0/4112376378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd94102b20 0x7fbd94102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 --2- 172.21.15.26:0/4112376378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd941034e0 0x7fbd94109d70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 --2- 172.21.15.26:0/4112376378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd94101930 0x7fbd94101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 -- 172.21.15.26:0/4112376378 >> 172.21.15.26:0/4112376378 conn(0x7fbd940fcea0 msgr2=0x7fbd940ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:54.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 -- 172.21.15.26:0/4112376378 shutdown_connections 2024-06-13T07:25:54.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.441+0000 7fbd99e00640 1 -- 172.21.15.26:0/4112376378 wait complete. 2024-06-13T07:25:54.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.442+0000 7fbd99e00640 1 Processor -- start 2024-06-13T07:25:54.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.442+0000 7fbd99e00640 1 -- start start 2024-06-13T07:25:54.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.442+0000 7fbd99e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd94101930 0x7fbd94199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:54.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd99e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd94102b20 0x7fbd9419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd99e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd941034e0 0x7fbd941a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd99e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9410d520 con 0x7fbd94102b20 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd99e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9410d3a0 con 0x7fbd94101930 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd99e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9410d6a0 con 0x7fbd941034e0 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd98a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd94101930 0x7fbd94199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd93e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd94102b20 0x7fbd9419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd98a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd94101930 0x7fbd94199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:41858/0 (socket says 172.21.15.26:41858) 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd98a00640 1 -- 172.21.15.26:0/2187576929 learned_addr learned my addr 172.21.15.26:0/2187576929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd99400640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd941034e0 0x7fbd941a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd98a00640 1 -- 172.21.15.26:0/2187576929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd941034e0 msgr2=0x7fbd941a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd98a00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd941034e0 0x7fbd941a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd98a00640 1 -- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd94102b20 msgr2=0x7fbd9419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd98a00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd94102b20 0x7fbd9419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.443+0000 7fbd98a00640 1 -- 172.21.15.26:0/2187576929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd941a1980 con 0x7fbd94101930 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.444+0000 7fbd98a00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd94101930 0x7fbd94199d60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800e9c0 tx=0x7fbd8800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.444+0000 7fbd91600640 1 -- 172.21.15.26:0/2187576929 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800cd80 con 0x7fbd94101930 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.444+0000 7fbd91600640 1 -- 172.21.15.26:0/2187576929 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd880035f0 con 0x7fbd94101930 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.444+0000 7fbd91600640 1 -- 172.21.15.26:0/2187576929 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88010690 con 0x7fbd94101930 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.444+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd941a1c70 con 0x7fbd94101930 2024-06-13T07:25:54.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.444+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd941a2170 con 0x7fbd94101930 2024-06-13T07:25:54.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.445+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd941a23a0 con 0x7fbd94101930 2024-06-13T07:25:54.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.448+0000 7fbd91600640 1 -- 172.21.15.26:0/2187576929 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbd880037a0 con 0x7fbd94101930 2024-06-13T07:25:54.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.449+0000 7fbd91600640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd7c0a2c10 0x7fbd7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:54.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.449+0000 7fbd91600640 1 -- 172.21.15.26:0/2187576929 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fbd880cf7f0 con 0x7fbd94101930 2024-06-13T07:25:54.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.449+0000 7fbd93e00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd7c0a2c10 0x7fbd7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:54.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.450+0000 7fbd93e00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd7c0a2c10 0x7fbd7c0a50d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd9419b270 tx=0x7fbd80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:54.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.453+0000 7fbd91600640 1 -- 172.21.15.26:0/2187576929 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd8809c300 con 0x7fbd94101930 2024-06-13T07:25:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:54 smithi026 ceph-mon[32041]: pgmap v1271: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 554 op/s 2024-06-13T07:25:54.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:54 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:54.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:54 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2360278162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:25:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:54 smithi136 ceph-mon[37229]: pgmap v1271: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 554 op/s 2024-06-13T07:25:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:54 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:54.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:54 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2360278162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:25:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.653+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbd941a2610 con 0x7fbd94101930 2024-06-13T07:25:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.654+0000 7fbd91600640 1 -- 172.21.15.26:0/2187576929 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fbd8809bc00 con 0x7fbd94101930 2024-06-13T07:25:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-06-13T07:25:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd7c0a2c10 msgr2=0x7fbd7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd7c0a2c10 0x7fbd7c0a50d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd9419b270 tx=0x7fbd80005f90 comp rx=0 tx=0).stop 2024-06-13T07:25:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd94101930 msgr2=0x7fbd94199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd94101930 0x7fbd94199d60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800e9c0 tx=0x7fbd8800ee80 comp rx=0 tx=0).stop 2024-06-13T07:25:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 shutdown_connections 2024-06-13T07:25:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fbd7c0a2c10 0x7fbd7c0a50d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd941034e0 0x7fbd941a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd94102b20 0x7fbd9419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 --2- 172.21.15.26:0/2187576929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd94101930 0x7fbd94199d60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 >> 172.21.15.26:0/2187576929 conn(0x7fbd940fcea0 msgr2=0x7fbd94073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 shutdown_connections 2024-06-13T07:25:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.657+0000 7fbd99e00640 1 -- 172.21.15.26:0/2187576929 wait complete. 2024-06-13T07:25:54.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-13T07:25:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:54 smithi135 ceph-mon[36276]: pgmap v1271: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 554 op/s 2024-06-13T07:25:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:54 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:54 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2360278162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:25:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.897+0000 7f6df7e00640 1 -- 172.21.15.26:0/2848775823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8104cd0 msgr2=0x7f6df8105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.897+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2848775823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8104cd0 0x7f6df8105150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009900 tx=0x7f6de002a410 comp rx=0 tx=0).stop 2024-06-13T07:25:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.897+0000 7f6df7e00640 1 -- 172.21.15.26:0/2848775823 shutdown_connections 2024-06-13T07:25:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.897+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2848775823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6df8105690 0x7f6df810bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.897+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2848775823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8104cd0 0x7f6df8105150 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.897+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2848775823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df8103ae0 0x7f6df8103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.897+0000 7f6df7e00640 1 -- 172.21.15.26:0/2848775823 >> 172.21.15.26:0/2848775823 conn(0x7f6df80ff050 msgr2=0x7f6df8101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.898+0000 7f6df7e00640 1 -- 172.21.15.26:0/2848775823 shutdown_connections 2024-06-13T07:25:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.898+0000 7f6df7e00640 1 -- 172.21.15.26:0/2848775823 wait complete. 2024-06-13T07:25:54.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df7e00640 1 Processor -- start 2024-06-13T07:25:54.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df7e00640 1 -- start start 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6df8103ae0 0x7f6df8199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8104cd0 0x7f6df819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df8105690 0x7f6df81a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6df810f740 con 0x7f6df8104cd0 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df7e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6df810f5c0 con 0x7f6df8105690 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6df810f8c0 con 0x7f6df8103ae0 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6df8103ae0 0x7f6df8199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6dee000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8104cd0 0x7f6df819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6df8103ae0 0x7f6df8199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35210/0 (socket says 172.21.15.26:35210) 2024-06-13T07:25:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.899+0000 7f6df6a00640 1 -- 172.21.15.26:0/2288581952 learned_addr learned my addr 172.21.15.26:0/2288581952 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6df7400640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df8105690 0x7f6df81a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6df6a00640 1 -- 172.21.15.26:0/2288581952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df8105690 msgr2=0x7f6df81a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6df6a00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df8105690 0x7f6df81a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6df6a00640 1 -- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8104cd0 msgr2=0x7f6df819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6df6a00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8104cd0 0x7f6df819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6df6a00640 1 -- 172.21.15.26:0/2288581952 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df81a1a10 con 0x7f6df8103ae0 2024-06-13T07:25:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6df6a00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6df8103ae0 0x7f6df8199df0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6de800e9c0 tx=0x7f6de800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:54.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6dfca00640 1 -- 172.21.15.26:0/2288581952 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de800ccc0 con 0x7f6df8103ae0 2024-06-13T07:25:54.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6dfca00640 1 -- 172.21.15.26:0/2288581952 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de80035f0 con 0x7f6df8103ae0 2024-06-13T07:25:54.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6dfca00640 1 -- 172.21.15.26:0/2288581952 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de800ce60 con 0x7f6df8103ae0 2024-06-13T07:25:54.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.900+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df81a1d00 con 0x7f6df8103ae0 2024-06-13T07:25:54.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.901+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df81a21d0 con 0x7f6df8103ae0 2024-06-13T07:25:54.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.902+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df810af20 con 0x7f6df8103ae0 2024-06-13T07:25:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.904+0000 7f6dfca00640 1 -- 172.21.15.26:0/2288581952 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6de80037a0 con 0x7f6df8103ae0 2024-06-13T07:25:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.905+0000 7f6dfca00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ddc0a2c10 0x7f6ddc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.905+0000 7f6dfca00640 1 -- 172.21.15.26:0/2288581952 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6de80cf730 con 0x7f6df8103ae0 2024-06-13T07:25:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.905+0000 7f6dee000640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ddc0a2c10 0x7f6ddc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.906+0000 7f6dee000640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ddc0a2c10 0x7f6ddc0a50d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6de00314e0 tx=0x7f6de0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:54.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:54.909+0000 7f6dfca00640 1 -- 172.21.15.26:0/2288581952 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6de809c240 con 0x7f6df8103ae0 2024-06-13T07:25:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.119+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6df802d590 con 0x7f6df8103ae0 2024-06-13T07:25:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.120+0000 7f6dfca00640 1 -- 172.21.15.26:0/2288581952 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f6de809bb40 con 0x7f6df8103ae0 2024-06-13T07:25:55.120 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-06-13T07:25:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.123+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ddc0a2c10 msgr2=0x7f6ddc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.123+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ddc0a2c10 0x7f6ddc0a50d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6de00314e0 tx=0x7f6de0000bc0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.123+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6df8103ae0 msgr2=0x7f6df8199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.123+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6df8103ae0 0x7f6df8199df0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6de800e9c0 tx=0x7f6de800edc0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.124+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 shutdown_connections 2024-06-13T07:25:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.124+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f6ddc0a2c10 0x7f6ddc0a50d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.124+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df8105690 0x7f6df81a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.124+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8104cd0 0x7f6df819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.124+0000 7f6df7e00640 1 --2- 172.21.15.26:0/2288581952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6df8103ae0 0x7f6df8199df0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.124+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 >> 172.21.15.26:0/2288581952 conn(0x7f6df80ff050 msgr2=0x7f6df81080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.124+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 shutdown_connections 2024-06-13T07:25:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.124+0000 7f6df7e00640 1 -- 172.21.15.26:0/2288581952 wait complete. 2024-06-13T07:25:55.143 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 -- 172.21.15.26:0/984999374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920104db0 msgr2=0x7f19201071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 --2- 172.21.15.26:0/984999374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920104db0 0x7f19201071a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f190c009900 tx=0x7f190c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 -- 172.21.15.26:0/984999374 shutdown_connections 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 --2- 172.21.15.26:0/984999374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19201076e0 0x7f1920109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 --2- 172.21.15.26:0/984999374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920104db0 0x7f19201071a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 --2- 172.21.15.26:0/984999374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1920102480 0x7f1920104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 -- 172.21.15.26:0/984999374 >> 172.21.15.26:0/984999374 conn(0x7f19200fbe70 msgr2=0x7f19200fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 -- 172.21.15.26:0/984999374 shutdown_connections 2024-06-13T07:25:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.362+0000 7f1927800640 1 -- 172.21.15.26:0/984999374 wait complete. 2024-06-13T07:25:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.363+0000 7f1927800640 1 Processor -- start 2024-06-13T07:25:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.363+0000 7f1927800640 1 -- start start 2024-06-13T07:25:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.363+0000 7f1927800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920102480 0x7f1920199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.363+0000 7f1927800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1920104db0 0x7f192019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.363+0000 7f1927800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19201076e0 0x7f19201a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.363+0000 7f1927800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f192010d6c0 con 0x7f1920104db0 2024-06-13T07:25:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.363+0000 7f1927800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f192010d540 con 0x7f19201076e0 2024-06-13T07:25:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.363+0000 7f1927800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f192010d840 con 0x7f1920102480 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f191fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1920104db0 0x7f192019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f191fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1920104db0 0x7f192019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50240/0 (socket says 172.21.15.26:50240) 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f191fe00640 1 -- 172.21.15.26:0/1589984769 learned_addr learned my addr 172.21.15.26:0/1589984769 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f1924e00640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920102480 0x7f1920199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f1925800640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19201076e0 0x7f19201a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f191fe00640 1 -- 172.21.15.26:0/1589984769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920102480 msgr2=0x7f1920199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f191fe00640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920102480 0x7f1920199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f191fe00640 1 -- 172.21.15.26:0/1589984769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19201076e0 msgr2=0x7f19201a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f191fe00640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19201076e0 0x7f19201a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f191fe00640 1 -- 172.21.15.26:0/1589984769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19201a1b90 con 0x7f1920104db0 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f1924e00640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920102480 0x7f1920199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.364+0000 7f1925800640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19201076e0 0x7f19201a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:55.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.365+0000 7f191fe00640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1920104db0 0x7f192019a4b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f190c031440 tx=0x7f190c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.365+0000 7f191d600640 1 -- 172.21.15.26:0/1589984769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c03f030 con 0x7f1920104db0 2024-06-13T07:25:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.365+0000 7f191d600640 1 -- 172.21.15.26:0/1589984769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f190c003ce0 con 0x7f1920104db0 2024-06-13T07:25:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.365+0000 7f191d600640 1 -- 172.21.15.26:0/1589984769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c035070 con 0x7f1920104db0 2024-06-13T07:25:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.365+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19201a1e20 con 0x7f1920104db0 2024-06-13T07:25:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.365+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19201a2230 con 0x7f1920104db0 2024-06-13T07:25:55.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.366+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1920109230 con 0x7f1920104db0 2024-06-13T07:25:55.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.369+0000 7f191d600640 1 -- 172.21.15.26:0/1589984769 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f190c042030 con 0x7f1920104db0 2024-06-13T07:25:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.371+0000 7f191d600640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f19080a2c10 0x7f19080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.371+0000 7f191d600640 1 -- 172.21.15.26:0/1589984769 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f190c0ef940 con 0x7f1920104db0 2024-06-13T07:25:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.371+0000 7f1924e00640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f19080a2c10 0x7f19080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.372+0000 7f1924e00640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f19080a2c10 0x7f19080a50d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1914009fc0 tx=0x7f1914005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:55.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.378+0000 7f191d600640 1 -- 172.21.15.26:0/1589984769 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f190c0bbe00 con 0x7f1920104db0 2024-06-13T07:25:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:55 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2187576929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:25:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:55 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2288581952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:25:55.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.582+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1920062670 con 0x7f1920104db0 2024-06-13T07:25:55.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.584+0000 7f191d600640 1 -- 172.21.15.26:0/1589984769 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f190c0c0100 con 0x7f1920104db0 2024-06-13T07:25:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f19080a2c10 msgr2=0x7f19080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f19080a2c10 0x7f19080a50d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1914009fc0 tx=0x7f1914005f90 comp rx=0 tx=0).stop 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1920104db0 msgr2=0x7f192019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1920104db0 0x7f192019a4b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f190c031440 tx=0x7f190c032ce0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 shutdown_connections 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f19080a2c10 0x7f19080a50d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19201076e0 0x7f19201a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1920104db0 0x7f192019a4b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 --2- 172.21.15.26:0/1589984769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1920102480 0x7f1920199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.586+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 >> 172.21.15.26:0/1589984769 conn(0x7f19200fbe70 msgr2=0x7f1920101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:55.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.587+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 shutdown_connections 2024-06-13T07:25:55.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.587+0000 7f1927800640 1 -- 172.21.15.26:0/1589984769 wait complete. 2024-06-13T07:25:55.606 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-13T07:25:55.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:55 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2187576929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:25:55.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:55 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2288581952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 -- 172.21.15.26:0/1630257938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18077f40 msgr2=0x7faf180787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 --2- 172.21.15.26:0/1630257938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18077f40 0x7faf180787e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faf10009ca0 tx=0x7faf1002a8b0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 -- 172.21.15.26:0/1630257938 shutdown_connections 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 --2- 172.21.15.26:0/1630257938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18077f40 0x7faf180787e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 --2- 172.21.15.26:0/1630257938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf180775a0 0x7faf18077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 --2- 172.21.15.26:0/1630257938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf18076360 0x7faf18076780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 -- 172.21.15.26:0/1630257938 >> 172.21.15.26:0/1630257938 conn(0x7faf180ff050 msgr2=0x7faf18101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 -- 172.21.15.26:0/1630257938 shutdown_connections 2024-06-13T07:25:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.819+0000 7faf17e00640 1 -- 172.21.15.26:0/1630257938 wait complete. 2024-06-13T07:25:55.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.820+0000 7faf17e00640 1 Processor -- start 2024-06-13T07:25:55.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.820+0000 7faf17e00640 1 -- start start 2024-06-13T07:25:55.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf17e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18076360 0x7faf1819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:55.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf17e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf180775a0 0x7faf1819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf17e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf18077f40 0x7faf181a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf17e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf18113bb0 con 0x7faf180775a0 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf17e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf18113a30 con 0x7faf18077f40 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf17e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf18113d30 con 0x7faf18076360 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf16a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18076360 0x7faf1819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf16000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf180775a0 0x7faf1819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf16a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18076360 0x7faf1819e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35236/0 (socket says 172.21.15.26:35236) 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf16a00640 1 -- 172.21.15.26:0/79628347 learned_addr learned my addr 172.21.15.26:0/79628347 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf17400640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf18077f40 0x7faf181a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf16a00640 1 -- 172.21.15.26:0/79628347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf18077f40 msgr2=0x7faf181a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf16a00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf18077f40 0x7faf181a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf16a00640 1 -- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf180775a0 msgr2=0x7faf1819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:55.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.821+0000 7faf16a00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf180775a0 0x7faf1819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:55.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf16a00640 1 -- 172.21.15.26:0/79628347 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf181a5da0 con 0x7faf18076360 2024-06-13T07:25:55.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf17400640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf18077f40 0x7faf181a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:25:55.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf16000640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf180775a0 0x7faf1819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:25:55.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf16a00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18076360 0x7faf1819e180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faf0800d950 tx=0x7faf0800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:55.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf0f400640 1 -- 172.21.15.26:0/79628347 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08017030 con 0x7faf18076360 2024-06-13T07:25:55.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf0f400640 1 -- 172.21.15.26:0/79628347 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08004020 con 0x7faf18076360 2024-06-13T07:25:55.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf0f400640 1 -- 172.21.15.26:0/79628347 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080047a0 con 0x7faf18076360 2024-06-13T07:25:55.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf181a6090 con 0x7faf18076360 2024-06-13T07:25:55.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.822+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf181a6630 con 0x7faf18076360 2024-06-13T07:25:55.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.824+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf18110590 con 0x7faf18076360 2024-06-13T07:25:55.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.826+0000 7faf0f400640 1 -- 172.21.15.26:0/79628347 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf080041d0 con 0x7faf18076360 2024-06-13T07:25:55.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.827+0000 7faf0f400640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faefc0a2c10 0x7faefc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:55.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.827+0000 7faf0f400640 1 -- 172.21.15.26:0/79628347 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faf080cf110 con 0x7faf18076360 2024-06-13T07:25:55.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.827+0000 7faf16000640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faefc0a2c10 0x7faefc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:55.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.828+0000 7faf16000640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faefc0a2c10 0x7faefc0a50d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faf1819f690 tx=0x7faf00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:55.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:55.829+0000 7faf0f400640 1 -- 172.21.15.26:0/79628347 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faf0809bca0 con 0x7faf18076360 2024-06-13T07:25:55.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:55 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2187576929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:25:55.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:55 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2288581952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:25:56.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.024+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faf180625a0 con 0x7faf18076360 2024-06-13T07:25:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.024+0000 7faf0f400640 1 -- 172.21.15.26:0/79628347 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) ==== 103+0+11 (secure 0 0 0) 0x7faf0809b5a0 con 0x7faf18076360 2024-06-13T07:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-06-13T07:25:56.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faefc0a2c10 msgr2=0x7faefc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:56.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faefc0a2c10 0x7faefc0a50d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faf1819f690 tx=0x7faf00005f90 comp rx=0 tx=0).stop 2024-06-13T07:25:56.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18076360 msgr2=0x7faf1819e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:56.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18076360 0x7faf1819e180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faf0800d950 tx=0x7faf0800de10 comp rx=0 tx=0).stop 2024-06-13T07:25:56.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 shutdown_connections 2024-06-13T07:25:56.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faefc0a2c10 0x7faefc0a50d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf18077f40 0x7faf181a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf180775a0 0x7faf1819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 --2- 172.21.15.26:0/79628347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf18076360 0x7faf1819e180 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 >> 172.21.15.26:0/79628347 conn(0x7faf180ff050 msgr2=0x7faf181005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:56.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 shutdown_connections 2024-06-13T07:25:56.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.028+0000 7faf17e00640 1 -- 172.21.15.26:0/79628347 wait complete. 2024-06-13T07:25:56.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-13T07:25:56.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 -- 172.21.15.26:0/110676888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4108f70 msgr2=0x7f77a41093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:56.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 --2- 172.21.15.26:0/110676888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4108f70 0x7f77a41093f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7790009860 tx=0x7f779002a340 comp rx=0 tx=0).stop 2024-06-13T07:25:56.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 -- 172.21.15.26:0/110676888 shutdown_connections 2024-06-13T07:25:56.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 --2- 172.21.15.26:0/110676888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4109930 0x7f77a4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 --2- 172.21.15.26:0/110676888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4108f70 0x7f77a41093f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 --2- 172.21.15.26:0/110676888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a4107e20 0x7f77a4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 -- 172.21.15.26:0/110676888 >> 172.21.15.26:0/110676888 conn(0x7f77a4076de0 msgr2=0x7f77a4079200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:56.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 -- 172.21.15.26:0/110676888 shutdown_connections 2024-06-13T07:25:56.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.243+0000 7f77a3200640 1 -- 172.21.15.26:0/110676888 wait complete. 2024-06-13T07:25:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.244+0000 7f77a3200640 1 Processor -- start 2024-06-13T07:25:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.244+0000 7f77a3200640 1 -- start start 2024-06-13T07:25:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a3200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4107e20 0x7f77a419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a4108f70 0x7f77a419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a3200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4109930 0x7f77a41a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4113ad0 con 0x7f77a4108f70 2024-06-13T07:25:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a3200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4113950 con 0x7f77a4109930 2024-06-13T07:25:56.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a3200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4113c50 con 0x7f77a4107e20 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a4108f70 0x7f77a419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4107e20 0x7f77a419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a4108f70 0x7f77a419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50296/0 (socket says 172.21.15.26:50296) 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4107e20 0x7f77a419e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:35258/0 (socket says 172.21.15.26:35258) 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a1400640 1 -- 172.21.15.26:0/2374199868 learned_addr learned my addr 172.21.15.26:0/2374199868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.245+0000 7f77a2800640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4109930 0x7f77a41a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f77a1400640 1 -- 172.21.15.26:0/2374199868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4107e20 msgr2=0x7f77a419e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f77a1400640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4107e20 0x7f77a419e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f77a1400640 1 -- 172.21.15.26:0/2374199868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4109930 msgr2=0x7f77a41a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f77a1400640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4109930 0x7f77a41a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f77a1400640 1 -- 172.21.15.26:0/2374199868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a41a5d60 con 0x7f77a4108f70 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f77a1400640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a4108f70 0x7f77a419e680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7790035040 tx=0x7f7790033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:56.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f779aa00640 1 -- 172.21.15.26:0/2374199868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790037040 con 0x7f77a4108f70 2024-06-13T07:25:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f779aa00640 1 -- 172.21.15.26:0/2374199868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7790032770 con 0x7f77a4108f70 2024-06-13T07:25:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a41a5ff0 con 0x7f77a4108f70 2024-06-13T07:25:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.246+0000 7f779aa00640 1 -- 172.21.15.26:0/2374199868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77900046c0 con 0x7f77a4108f70 2024-06-13T07:25:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.247+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a6450 con 0x7f77a4108f70 2024-06-13T07:25:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.248+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a410f130 con 0x7f77a4108f70 2024-06-13T07:25:56.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.250+0000 7f779aa00640 1 -- 172.21.15.26:0/2374199868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7790004050 con 0x7f77a4108f70 2024-06-13T07:25:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.251+0000 7f779aa00640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77880a2c10 0x7f77880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:25:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.251+0000 7f779aa00640 1 -- 172.21.15.26:0/2374199868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f77900ef2d0 con 0x7f77a4108f70 2024-06-13T07:25:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.251+0000 7f77a1e00640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77880a2c10 0x7f77880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:25:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.252+0000 7f77a1e00640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77880a2c10 0x7f77880a50d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7794005fd0 tx=0x7f7794009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:25:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.256+0000 7f779aa00640 1 -- 172.21.15.26:0/2374199868 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f77900bbde0 con 0x7f77a4108f70 2024-06-13T07:25:56.383 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:56 smithi136 ceph-mon[37229]: pgmap v1272: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.7 MiB/s wr, 640 op/s 2024-06-13T07:25:56.383 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:56 smithi136 ceph-mon[37229]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:56.383 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:56 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1589984769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:25:56.383 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:56 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/79628347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:25:56.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.518+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f77a40626e0 con 0x7f77a4108f70 2024-06-13T07:25:56.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.531+0000 7f779aa00640 1 -- 172.21.15.26:0/2374199868 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740366 (secure 0 0 0) 0x7f77900bb6e0 con 0x7f77a4108f70 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 18, 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "active_gid": 14152, 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "active_name": "x", 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addrs": { 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-06-13T07:25:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6800", 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 662324914 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6801", 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 662324914 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addr": "172.21.15.26:6801/662324914", 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "active_change": "2024-06-13T06:46:28.110058+0000", 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "active_mgr_features": 4540701547738038271, 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "standbys": [ 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24109, 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "z", 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540701547738038271, 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-06-13T07:25:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-06-13T07:25:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-06-13T07:25:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-06-13T07:25:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-13T07:25:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.583 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.586 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-13T07:25:56.587 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.590 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-06-13T07:25:56.591 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-06-13T07:25:56.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-06-13T07:25:56.597 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:56.598 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-13T07:25:56.600 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.602 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-06-13T07:25:56.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-06-13T07:25:56.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.605 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.607 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.608 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-06-13T07:25:56.610 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-06-13T07:25:56.611 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.612 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.613 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-06-13T07:25:56.614 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.616 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.617 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-06-13T07:25:56.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-06-13T07:25:56.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-06-13T07:25:56.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.622 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-13T07:25:56.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.624 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.625 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-13T07:25:56.627 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.628 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.630 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-06-13T07:25:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-06-13T07:25:56.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.641 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-13T07:25:56.642 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.645 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.647 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.649 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.650 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.652 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.656 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.658 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.659 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.661 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.664 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.665 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.666 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.667 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.669 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.670 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-13T07:25:56.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.672 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.678 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.679 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.681 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.683 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-13T07:25:56.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-06-13T07:25:56.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.686 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.687 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.689 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.693 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.694 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.696 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-13T07:25:56.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-13T07:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-06-13T07:25:56.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.701 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.703 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.708 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.712 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.716 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.720 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.721 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.722 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.723 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.725 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.726 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.727 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.733 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.735 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.737 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-06-13T07:25:56.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.740 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.742 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.743 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.744 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.745 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.746 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-06-13T07:25:56.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-06-13T07:25:56.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.750 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.752 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.753 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.757 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.758 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.759 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.761 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.763 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:25:56.764 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-13T07:25:56.765 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.766 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.767 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.768 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.769 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.771 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.772 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.773 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.775 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.776 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.778 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.780 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-06-13T07:25:56.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.782 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.783 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.784 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.785 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.787 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.788 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.791 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.792 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.793 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.794 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.795 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-06-13T07:25:56.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.797 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.800 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.801 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.802 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.804 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.806 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.809 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.810 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.811 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-06-13T07:25:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-06-13T07:25:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-06-13T07:25:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-06-13T07:25:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-13T07:25:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-06-13T07:25:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-06-13T07:25:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.872 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.873 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.877 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.878 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.880 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.881 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.883 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.884 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.887 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.889 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.890 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.891 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.893 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.897 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.898 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.899 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-06-13T07:25:56.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.903 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.906 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.907 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.908 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.909 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.910 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-06-13T07:25:56.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-06-13T07:25:56.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.914 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.917 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.919 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.921 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.924 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.925 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.926 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.927 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.929 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.930 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.935 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.936 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.937 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.938 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.939 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.940 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.941 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.943 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.945 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.946 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.947 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.948 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-13T07:25:56.950 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.951 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.952 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.953 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.954 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.955 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-13T07:25:56.956 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.957 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.958 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-06-13T07:25:56.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.962 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.963 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.964 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.965 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.966 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.967 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.967 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.969 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.969 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.969 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.969 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.971 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.972 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.975 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.980 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.981 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.983 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.986 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.987 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-13T07:25:56.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-06-13T07:25:56.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-06-13T07:25:56.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.991 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.993 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.994 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.995 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.996 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:56.998 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:56.999 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24116, 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "y", 2024-06-13T07:25:57.001 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540701547738038271, 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.002 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.003 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.005 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.006 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.007 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-06-13T07:25:57.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-06-13T07:25:57.009 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.010 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.011 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-06-13T07:25:57.016 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-06-13T07:25:57.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-06-13T07:25:57.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.020 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.021 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-13T07:25:57.022 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-13T07:25:57.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-06-13T07:25:57.024 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.026 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.029 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.030 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.031 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.033 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-13T07:25:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.036 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.037 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.038 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-13T07:25:57.042 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-13T07:25:57.043 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-13T07:25:57.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.045 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.046 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.047 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.048 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-06-13T07:25:57.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-06-13T07:25:57.050 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.051 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.052 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.055 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.056 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-06-13T07:25:57.057 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-06-13T07:25:57.058 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.059 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.060 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.061 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-13T07:25:57.062 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-13T07:25:57.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.064 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.067 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.068 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-06-13T07:25:57.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-06-13T07:25:57.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.071 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.074 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.075 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-13T07:25:57.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.077 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.078 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-06-13T07:25:57.079 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.082 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.083 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.084 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-13T07:25:57.085 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.086 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.089 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-06-13T07:25:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-13T07:25:57.092 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.093 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.094 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.095 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-06-13T07:25:57.096 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-06-13T07:25:57.097 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.100 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.101 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-13T07:25:57.102 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.103 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.104 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.106 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.107 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.109 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.111 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-06-13T07:25:57.113 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.114 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.115 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.116 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.117 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.119 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.120 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.121 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.122 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.123 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.125 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.126 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-13T07:25:57.127 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-13T07:25:57.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.130 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.131 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.134 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.135 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.136 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.137 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.138 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.139 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.140 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.141 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.142 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.143 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-13T07:25:57.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.148 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.149 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.150 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.151 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.152 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.154 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.155 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.156 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-13T07:25:57.157 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-13T07:25:57.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.159 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-06-13T07:25:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-06-13T07:25:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-06-13T07:25:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-06-13T07:25:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.188 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.189 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.191 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.194 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.195 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.197 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.198 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-06-13T07:25:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-06-13T07:25:57.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.203 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.204 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.205 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.207 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.208 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.210 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.211 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.212 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.214 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.215 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.216 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.217 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:57.218 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.219 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-06-13T07:25:57.220 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.222 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.223 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.225 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.226 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.227 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.228 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.229 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.230 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.231 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.232 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-06-13T07:25:57.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-06-13T07:25:57.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.235 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.236 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.237 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.239 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.240 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.241 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.242 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.243 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.245 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.248 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.249 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-06-13T07:25:57.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-06-13T07:25:57.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.254 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.255 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.256 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.257 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.260 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.262 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.263 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.264 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.269 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.270 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.272 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.275 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.276 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.277 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-06-13T07:25:57.278 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.279 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.280 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.281 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.281 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.281 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.281 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.281 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.282 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.283 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.284 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.286 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.288 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.290 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.292 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.293 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.294 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.295 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.296 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.299 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-13T07:25:57.300 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.301 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-06-13T07:25:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.307 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.309 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.310 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.311 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-13T07:25:57.313 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.314 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.317 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.318 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.319 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.320 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.321 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:57.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:57.323 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-06-13T07:25:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.325 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-06-13T07:25:57.326 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-06-13T07:25:57.327 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.329 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.330 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.332 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.333 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.334 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.335 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.336 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-06-13T07:25:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.339 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.344 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-06-13T07:25:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.370 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.371 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.372 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.373 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.374 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.375 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-06-13T07:25:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-06-13T07:25:57.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.379 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.380 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.381 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.383 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.384 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.385 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.387 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.389 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.391 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.392 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.393 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.395 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.396 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.397 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.398 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.399 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.400 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.402 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.403 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.405 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.406 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.407 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.410 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.411 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.412 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-06-13T07:25:57.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.414 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.415 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.417 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.418 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-06-13T07:25:57.419 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-06-13T07:25:57.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.422 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.424 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.425 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.426 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.427 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.428 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.429 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.430 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.431 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.432 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.433 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.435 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.436 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.437 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.438 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.439 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.440 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.443 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.444 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.446 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.447 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-06-13T07:25:57.448 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.449 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.452 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.453 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.454 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.455 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.456 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-06-13T07:25:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-06-13T07:25:57.458 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.459 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.460 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard", 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-06-13T07:25:57.461 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.462 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.463 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.464 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.465 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.467 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.468 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.470 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.471 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-06-13T07:25:57.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-06-13T07:25:57.473 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.474 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.475 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.476 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.477 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.478 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.479 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-13T07:25:57.480 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.481 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.482 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-06-13T07:25:57.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.484 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.486 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.488 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.489 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.490 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-06-13T07:25:57.493 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.495 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.496 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.497 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.499 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-06-13T07:25:57.500 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.502 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-06-13T07:25:57.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.504 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-06-13T07:25:57.506 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.507 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.508 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.509 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.510 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.511 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.512 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.513 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-06-13T07:25:57.514 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.515 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.517 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-06-13T07:25:57.518 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.519 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.522 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.523 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.524 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-06-13T07:25:57.525 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.527 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-06-13T07:25:57.528 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.529 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-06-13T07:25:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-06-13T07:25:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-06-13T07:25:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-06-13T07:25:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-06-13T07:25:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-06-13T07:25:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.557 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-06-13T07:25:57.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-13T07:25:57.562 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-13T07:25:57.563 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.565 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.566 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.567 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.574 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.575 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.577 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.578 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-13T07:25:57.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-13T07:25:57.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.581 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.581 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.582 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-13T07:25:57.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-06-13T07:25:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.585 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.590 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.592 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.594 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.595 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.597 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-13T07:25:57.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-13T07:25:57.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.600 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.601 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.602 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.603 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.605 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.606 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.607 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.608 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-13T07:25:57.610 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-13T07:25:57.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-06-13T07:25:57.612 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.613 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.615 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.616 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.618 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-06-13T07:25:57.619 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.621 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.622 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-13T07:25:57.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-13T07:25:57.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.625 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.626 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-06-13T07:25:57.628 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.630 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.632 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.635 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-06-13T07:25:57.637 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.639 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.642 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-06-13T07:25:57.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.646 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.647 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.650 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.652 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.653 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-13T07:25:57.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-13T07:25:57.655 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.657 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.659 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-06-13T07:25:57.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.661 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.662 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.665 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.666 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.667 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.669 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-06-13T07:25:57.671 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-06-13T07:25:57.672 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.674 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.676 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.677 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.678 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.680 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.684 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.688 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.690 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.694 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.696 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.697 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-06-13T07:25:57.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-06-13T07:25:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.702 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.706 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.711 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.713 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-06-13T07:25:57.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.736 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.737 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.738 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.739 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.740 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.742 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-06-13T07:25:57.744 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.745 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.746 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.748 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.749 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.750 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.751 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.752 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.753 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.754 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.755 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.759 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.760 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-06-13T07:25:57.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.766 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.769 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.770 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.771 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.772 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.776 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.780 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.782 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.784 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-06-13T07:25:57.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.785 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.785 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.785 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.786 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.787 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-06-13T07:25:57.788 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.790 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-06-13T07:25:57.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.795 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.796 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.797 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-06-13T07:25:57.798 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.799 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.801 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.802 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.803 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.804 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.805 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.806 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.807 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.809 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.810 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.812 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.813 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.814 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.815 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.816 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.817 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.818 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.819 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.820 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.821 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-06-13T07:25:57.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.827 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.828 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.830 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-06-13T07:25:57.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-06-13T07:25:57.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.833 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.836 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.837 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.838 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.839 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.840 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.841 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.842 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.843 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.845 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.846 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.847 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.848 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.849 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.850 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.852 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.853 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.854 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.855 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.856 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.857 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.858 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.861 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.862 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-06-13T07:25:57.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.864 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.866 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.869 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.870 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-06-13T07:25:57.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-06-13T07:25:57.872 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-06-13T07:25:57.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.874 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.877 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.878 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.880 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.881 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.882 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.883 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.884 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.886 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.887 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-13T07:25:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-13T07:25:57.909 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-06-13T07:25:57.910 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.911 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.912 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.914 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.915 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.916 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.917 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.918 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.919 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.920 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-06-13T07:25:57.921 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: "services": { 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard": "https://172.21.15.26:8443/" 2024-06-13T07:25:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "always_on_modules": { 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus": [ 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.923 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific": [ 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy": [ 2024-06-13T07:25:57.924 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "reef": [ 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-06-13T07:25:57.925 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "squid": [ 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-06-13T07:25:57.926 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "last_failure_osd_epoch": 3, 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "active_clients": [ 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.927 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1507751277 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "libcephsqlite", 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3379045443 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3099403779 2024-06-13T07:25:57.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 63559790 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:25:57.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-06-13T07:25:57.931 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:25:57.932 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:57 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2374199868' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-13T07:25:57.932 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:56 smithi135 ceph-mon[36276]: pgmap v1272: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.7 MiB/s wr, 640 op/s 2024-06-13T07:25:57.932 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:56 smithi135 ceph-mon[36276]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:57.932 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:56 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1589984769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:25:57.932 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:56 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/79628347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:25:57.932 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:57 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2374199868' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-13T07:25:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77880a2c10 msgr2=0x7f77880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77880a2c10 0x7f77880a50d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7794005fd0 tx=0x7f7794009260 comp rx=0 tx=0).stop 2024-06-13T07:25:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a4108f70 msgr2=0x7f77a419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:25:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a4108f70 0x7f77a419e680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7790035040 tx=0x7f7790033d90 comp rx=0 tx=0).stop 2024-06-13T07:25:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 shutdown_connections 2024-06-13T07:25:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f77880a2c10 0x7f77880a50d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4109930 0x7f77a41a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a4108f70 0x7f77a419e680 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 --2- 172.21.15.26:0/2374199868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77a4107e20 0x7f77a419e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:25:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 >> 172.21.15.26:0/2374199868 conn(0x7f77a4076de0 msgr2=0x7f77a410c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:25:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 shutdown_connections 2024-06-13T07:25:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:25:56.541+0000 7f77a3200640 1 -- 172.21.15.26:0/2374199868 wait complete. 2024-06-13T07:25:57.940 DEBUG:tasks.check_counter:Getting stats from a 2024-06-13T07:25:57.940 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-13T07:25:57.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:56 smithi026 ceph-mon[32041]: pgmap v1272: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.7 MiB/s wr, 640 op/s 2024-06-13T07:25:57.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:56 smithi026 ceph-mon[32041]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:25:57.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:56 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1589984769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:25:57.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:56 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/79628347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:25:57.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:57 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2374199868' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-13T07:25:58.422 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:25:58.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:25:58 smithi026 ceph-mon[32041]: pgmap v1273: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-06-13T07:25:58.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:25:58 smithi135 ceph-mon[36276]: pgmap v1273: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-06-13T07:25:58.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:25:58 smithi136 ceph-mon[37229]: pgmap v1273: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-06-13T07:26:00.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:00 smithi026 ceph-mon[32041]: pgmap v1274: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 445 op/s 2024-06-13T07:26:00.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:00 smithi135 ceph-mon[36276]: pgmap v1274: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 445 op/s 2024-06-13T07:26:00.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:00 smithi136 ceph-mon[37229]: pgmap v1274: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 445 op/s 2024-06-13T07:26:01.972 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 3953, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 3953, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 177307547, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 1256237, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 8, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 1.199439049, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.228121756, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.657982063, 2024-06-13T07:26:01.973 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.243381117, 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3952, 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.445197273, 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000112651 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-13T07:26:01.974 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1364, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 1744, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 142952524, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 1247034, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 8, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.702971286, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.145360456, 2024-06-13T07:26:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.428603874, 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.095156531, 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1742, 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.240809706, 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000138237 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 813856 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1723, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 1691, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 139882171, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 687546, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.656993976, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.123724787, 2024-06-13T07:26:01.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.410521233, 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.100868273, 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1690, 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.227817223, 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000134803 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 700128, 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 310080 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 10960, 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.648020734, 2024-06-13T07:26:01.979 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000059125 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-06-13T07:26:01.980 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2024-06-13T07:26:01.981 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 50848, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 19314, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 31534, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 23256, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 1, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2024-06-13T07:26:01.982 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-06-13T07:26:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2024-06-13T07:26:01.984 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2024-06-13T07:26:01.985 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-06-13T07:26:01.986 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2024-06-13T07:26:01.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 242123, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 132, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 21, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 239039744, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 461842275, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 461841137, 2024-06-13T07:26:01.988 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 242123 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 173, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 50611, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 50438, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 26, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 89, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 63, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 50848, 2024-06-13T07:26:01.989 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 51183, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 335, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 97136, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 215420 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2024-06-13T07:26:01.990 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-06-13T07:26:01.991 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-06-13T07:26:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.995 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-06-13T07:26:01.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.995 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-06-13T07:26:01.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-06-13T07:26:01.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.001 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:26:02.002 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 2, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 1, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 1, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2024-06-13T07:26:02.003 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 3050, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:02.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:02.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 248718, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 244, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:02.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 9176, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 176, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 36255790, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 224637, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:02.008 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 6498, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 21, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 3, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 6498, 2024-06-13T07:26:02.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 6498, 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 36.896480953, 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005678128 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 6498, 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 6499 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 6498, 2024-06-13T07:26:02.010 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 6498, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 3408, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 3088, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2024-06-13T07:26:02.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 1, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:26:02.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-06-13T07:26:02.013 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 0, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 0, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 11, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 12, 2024-06-13T07:26:02.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 1, 2024-06-13T07:26:02.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-06-13T07:26:02.015 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-06-13T07:26:02.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 0, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:02.017 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 7036, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 459478421, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 7036, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 7036, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 459478421, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:02.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 6498, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 457648798, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 6498, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 457648798, 2024-06-13T07:26:02.019 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:02.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 6498, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 6498, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 6498, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 6498, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.022 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x55ed1957e780": { 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-06-13T07:26:02.023 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-06-13T07:26:02.024 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-06-13T07:26:02.024 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:02.024 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:02.024 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:02.024 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:02.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:02.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:02.024 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:26:02.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:02 smithi026 ceph-mon[32041]: pgmap v1275: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 847 KiB/s wr, 271 op/s 2024-06-13T07:26:02.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:02 smithi135 ceph-mon[36276]: pgmap v1275: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 847 KiB/s wr, 271 op/s 2024-06-13T07:26:02.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:02 smithi136 ceph-mon[37229]: pgmap v1275: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 847 KiB/s wr, 271 op/s 2024-06-13T07:26:03.028 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-06-13T07:26:03.029 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-06-13T07:26:03.029 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-06-13T07:26:03.029 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-06-13T07:26:03.029 DEBUG:tasks.check_counter:Getting stats from d 2024-06-13T07:26:03.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-13T07:26:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:26:04.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:04 smithi136 ceph-mon[37229]: pgmap v1276: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 93 KiB/s wr, 191 op/s 2024-06-13T07:26:04.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:04 smithi026 ceph-mon[32041]: pgmap v1276: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 93 KiB/s wr, 191 op/s 2024-06-13T07:26:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:04 smithi135 ceph-mon[36276]: pgmap v1276: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 93 KiB/s wr, 191 op/s 2024-06-13T07:26:06.483 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:26:06.483 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:06.483 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-06-13T07:26:06.483 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-06-13T07:26:06.483 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-06-13T07:26:06.483 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-06-13T07:26:06.483 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.003703872, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000656511, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:06.484 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 514, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 509, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 564640, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 314800, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.136264996, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.045871004, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.063147342, 2024-06-13T07:26:06.485 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.024838327, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 509, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.061623592, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000121067 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 564640, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 314800 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.486 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 3, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 389, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 480, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 302640, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.070524415, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.057642058, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.005492448, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:06.487 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 387, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.061968779, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000160126 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 302640 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-06-13T07:26:06.488 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:06.489 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:06.490 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:06.491 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148951, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 28, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:06.492 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:06.493 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 513, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 489499, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 513, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 513, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 489499, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:06.494 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:26:06.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:06 smithi026 ceph-mon[32041]: pgmap v1277: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 92 KiB/s wr, 163 op/s 2024-06-13T07:26:06.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:06 smithi135 ceph-mon[36276]: pgmap v1277: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 92 KiB/s wr, 163 op/s 2024-06-13T07:26:06.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:06 smithi136 ceph-mon[37229]: pgmap v1277: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 92 KiB/s wr, 163 op/s 2024-06-13T07:26:07.368 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-13T07:26:07.368 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-13T07:26:07.368 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-06-13T07:26:07.368 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-06-13T07:26:07.368 DEBUG:tasks.check_counter:Getting stats from g 2024-06-13T07:26:07.369 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-13T07:26:07.860 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:26:08.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:08 smithi026 ceph-mon[32041]: pgmap v1278: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 34 op/s 2024-06-13T07:26:08.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:08 smithi135 ceph-mon[36276]: pgmap v1278: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 34 op/s 2024-06-13T07:26:08.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:08 smithi136 ceph-mon[37229]: pgmap v1278: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 34 op/s 2024-06-13T07:26:10.661 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:10 smithi026 ceph-mon[32041]: pgmap v1279: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 KiB/s wr, 19 op/s 2024-06-13T07:26:10.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:10 smithi135 ceph-mon[36276]: pgmap v1279: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 KiB/s wr, 19 op/s 2024-06-13T07:26:10.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:10 smithi136 ceph-mon[37229]: pgmap v1279: 129 pgs: 129 active+clean; 517 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 KiB/s wr, 19 op/s 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 358360, 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 391152, 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 86214884, 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 312707495, 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 250, 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 7, 2024-06-13T07:26:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 57.751188656, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 23.586443035, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 30.879114787, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 6.866866945, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 391151, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 90.838387494, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000232233 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 597005, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.254736069, 2024-06-13T07:26:11.262 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000426 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 733584, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 325808 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 571880, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 788502, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 211970036, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 759144722, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 16, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 196.983916247, 2024-06-13T07:26:11.263 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 139.137128266, 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 62.582757717, 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 8.031405940, 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 788500, 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 180.187798336, 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000228519 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1043762, 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.634631773, 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000608 2024-06-13T07:26:11.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 189784064, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 574167888 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 374054, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 388259, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 90116549, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 254417995, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 11, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 61.142080300, 2024-06-13T07:26:11.265 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 25.033043322, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 33.766728395, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 7.071231896, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 388258, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 94.916419195, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000244467 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 597125, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.268180034, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000449 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.266 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 138559, 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 345.109478261, 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002490704 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 71577, 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 27.409067017, 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000382931 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 341272, 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 285546, 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 285546, 2024-06-13T07:26:11.268 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 2192.114022435, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.007676920 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 55726, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 4, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 1, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 3829, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 3, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 5, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 192, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 17, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 42, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 133, 2024-06-13T07:26:11.269 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 167, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 1573, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 31, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 31, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 4, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 548345, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 300957, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 55696, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 102, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 6156, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 959, 2024-06-13T07:26:11.270 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-06-13T07:26:11.271 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 17, 2024-06-13T07:26:11.271 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 1186, 2024-06-13T07:26:11.271 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 16, 2024-06-13T07:26:11.271 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 214, 2024-06-13T07:26:11.271 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-06-13T07:26:11.271 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 81, 2024-06-13T07:26:11.273 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-06-13T07:26:11.273 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 4, 2024-06-13T07:26:11.273 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 234, 2024-06-13T07:26:11.273 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-06-13T07:26:11.273 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 81, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 37, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 3, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 34, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 44, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 48, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 57496, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 173824, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:26:11.274 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 123558, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 35159, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 15830, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 192850 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 526, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 197, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 118, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 214, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 1531, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 37, 2024-06-13T07:26:11.275 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 42222, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 42185, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:26:11.276 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 3, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 18, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 8, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 229297, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 229067, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 229067, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 230, 2024-06-13T07:26:11.277 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 2918, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 338, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 337, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 337, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 1, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 1, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 4, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 484668982, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 484971396, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 0, 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-06-13T07:26:11.278 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 75741, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 958.185568823, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.012650817 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 0 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 161, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 43027, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 42866, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 38, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 289, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 251, 2024-06-13T07:26:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 243, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 88454, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 88211, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 30, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 47650, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 47620, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 87704, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 231804 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 483948, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 31524, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 341272, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 5031, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 68267, 2024-06-13T07:26:11.280 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 42693, 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 369.922323613, 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.008664706 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 33942, 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 465.344406477, 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.013709987 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 87299, 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 353.662379494, 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004051161 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 119, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1.388276484, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.011666188 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 10, 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.025356399, 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002535639 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 68073, 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 624.598107979, 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.009175416 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 9964, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 82.418326983, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.008271610 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 54, 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1.587906396, 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.029405674 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 129, 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 4.510210623, 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.034962873 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1076, 2024-06-13T07:26:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 4.233724402, 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.003934688 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 8, 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.028385942, 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.003548242 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-06-13T07:26:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 42179, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 284.394617643, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.006742564 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 3, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 2, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 1, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 21938, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 21938, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 1548, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-06-13T07:26:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 360, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 360, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 307295, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 268, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 440, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 5, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-06-13T07:26:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 9176, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 176, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 686503, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 21798, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-06-13T07:26:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 211716, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 521310539, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 37, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 211716, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 211716, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 11216.086967094, 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.052977039 2024-06-13T07:26:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 211716, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 285191 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 211716, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 1707, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 210009, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 3670, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 40846, 2024-06-13T07:26:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 1618, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 77356, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 1243, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 3, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 85103, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 89, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 61940, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:26:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-06-13T07:26:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 99, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 11, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 12, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 12, 2024-06-13T07:26:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 10286, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 10, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 3027 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 1, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 16, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 12, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 84335, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 42185, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1304290, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 247307805, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1304290, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1304290, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 247307805, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-06-13T07:26:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 211716, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 525908356, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 211716, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 525908356, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.305 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 211716, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 211716, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 211716, 2024-06-13T07:26:11.306 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 211716, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 42185, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 4597885, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 42185, 2024-06-13T07:26:11.307 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1615, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 4597885, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x562d11f5c500": { 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 55707, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:11.308 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 229297, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 480777092, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 229297, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 75741, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 480721385, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:11.309 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:26:12.216 INFO:tasks.check_counter:Daemon mds.g mds.exported=17 2024-06-13T07:26:12.216 INFO:tasks.check_counter:Daemon mds.g mds.imported=16 2024-06-13T07:26:12.216 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=526 2024-06-13T07:26:12.216 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=197 2024-06-13T07:26:12.217 DEBUG:tasks.check_counter:Getting stats from j 2024-06-13T07:26:12.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-13T07:26:12.690 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:26:12.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:12 smithi026 ceph-mon[32041]: pgmap v1280: 129 pgs: 129 active+clean; 521 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 615 KiB/s wr, 14 op/s 2024-06-13T07:26:12.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:12 smithi135 ceph-mon[36276]: pgmap v1280: 129 pgs: 129 active+clean; 521 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 615 KiB/s wr, 14 op/s 2024-06-13T07:26:12.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:12 smithi136 ceph-mon[37229]: pgmap v1280: 129 pgs: 129 active+clean; 521 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 615 KiB/s wr, 14 op/s 2024-06-13T07:26:14.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:14 smithi026 ceph-mon[32041]: pgmap v1281: 129 pgs: 129 active+clean; 516 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 22 op/s 2024-06-13T07:26:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:14 smithi135 ceph-mon[36276]: pgmap v1281: 129 pgs: 129 active+clean; 516 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 22 op/s 2024-06-13T07:26:14.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:14 smithi136 ceph-mon[37229]: pgmap v1281: 129 pgs: 129 active+clean; 516 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 22 op/s 2024-06-13T07:26:16.157 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-06-13T07:26:16.157 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:16.157 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-06-13T07:26:16.157 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-06-13T07:26:16.157 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-06-13T07:26:16.157 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.003951589, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000804324, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.158 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 517, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 512, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 565456, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 316720, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-06-13T07:26:16.159 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.140648123, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.048085163, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.065106855, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.023431941, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 512, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.077093893, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000150574 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:16.160 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 565456, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 316720 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 3, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 392, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 480, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 303168, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.076765298, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.063308467, 2024-06-13T07:26:16.161 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.005550180, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 390, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.060449214, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000154997 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-13T07:26:16.162 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 303168 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:16.163 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:16.164 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:16.165 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148951, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 28, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:16.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-06-13T07:26:16.167 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 516, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 489889, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 516, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 516, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 489889, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-06-13T07:26:16.168 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-06-13T07:26:16.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:16 smithi026 ceph-mon[32041]: pgmap v1282: 129 pgs: 129 active+clean; 493 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-06-13T07:26:16.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:16 smithi135 ceph-mon[36276]: pgmap v1282: 129 pgs: 129 active+clean; 493 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-06-13T07:26:16.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:16 smithi136 ceph-mon[37229]: pgmap v1282: 129 pgs: 129 active+clean; 493 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-06-13T07:26:17.006 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-13T07:26:17.006 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-13T07:26:17.006 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-06-13T07:26:17.006 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-06-13T07:26:17.007 DEBUG:tasks.check_counter:Getting stats from b 2024-06-13T07:26:17.007 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-13T07:26:17.355 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T07:26:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:18 smithi026 ceph-mon[32041]: pgmap v1283: 129 pgs: 129 active+clean; 490 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.1 MiB/s wr, 65 op/s 2024-06-13T07:26:18.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:18 smithi135 ceph-mon[36276]: pgmap v1283: 129 pgs: 129 active+clean; 490 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.1 MiB/s wr, 65 op/s 2024-06-13T07:26:18.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:18 smithi136 ceph-mon[37229]: pgmap v1283: 129 pgs: 129 active+clean; 490 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.1 MiB/s wr, 65 op/s 2024-06-13T07:26:19.586 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 377843, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 344993, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 93686824, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 278489903, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 14, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 22.086378016, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 8.534212904, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 13.912430947, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 2.568042994, 2024-06-13T07:26:19.587 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 344994, 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 15.647498073, 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000045355 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 584133, 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.096151818, 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000164 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 714624, 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 346096 2024-06-13T07:26:19.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 553437, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 787320, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 202098190, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 762553951, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 15, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 5, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 36.796815952, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 16.227537720, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 23.171283145, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 2.682856117, 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 787318, 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 44.028712199, 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000055922 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 994252, 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.171054521, 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000172 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 189437808, 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 601271312 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.590 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 260968, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 271801, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 67026098, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 230761828, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 9, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 7, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 16.477903396, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 6.234864370, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 8.084344638, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 3.150018432, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 271798, 2024-06-13T07:26:19.591 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 14.928862647, 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000054926 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 404110, 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.066711611, 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000165 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-06-13T07:26:19.592 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 110976, 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 184.211616673, 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001659923 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-13T07:26:19.593 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 90781, 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 14.029405913, 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000154541 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 341629, 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 294812, 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 294812, 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1867.444205066, 2024-06-13T07:26:19.594 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.006334356 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 46814, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 50, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 3, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 3, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 604, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 244, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 188, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 172, 2024-06-13T07:26:19.595 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 214, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 559, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 1, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 1, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 4, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 556300, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 279038, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 46813, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 211, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 3830, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 1000, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 1, 2024-06-13T07:26:19.596 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 11, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 201, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 11, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 746, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 31, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 4, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 234, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 31, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:26:19.597 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 31, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 32, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 43493, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 189691, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 123508, 2024-06-13T07:26:19.598 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 40314, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 17637, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 203542 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 214, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 1255, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 701, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 1194, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 558, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 12, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:26:19.599 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 48364, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 48352, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 13, 2024-06-13T07:26:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 6, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 242210, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 135031, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 122901, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 119309, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 12130, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 294, 2024-06-13T07:26:19.601 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 166, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 150, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 144, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 10, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 16, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 243007584, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 493481558, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 63893, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 928.537132149, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.014532689 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.602 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 92, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 48694, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 48602, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 39, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 671, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 632, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 468, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 92318, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 91850, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 1, 2024-06-13T07:26:19.603 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 54890, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 54889, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 85188, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 223612 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 476661, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 5856, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 341629, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 5727, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 19399, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 48046, 2024-06-13T07:26:19.604 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 341.824071978, 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.007114516 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 17104, 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 389.177270129, 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.022753582 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-06-13T07:26:19.605 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 97281, 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 206.899163154, 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002126819 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000117377, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000117377 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.608 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-06-13T07:26:19.608 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.608 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.608 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.608 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.609 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-06-13T07:26:19.610 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 72647, 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 660.355683094, 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.009089923 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 9753, 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 74.626741489, 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.007651670 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 40, 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.547197813, 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.013679945 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.612 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 112, 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4.491657055, 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.040104080 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.613 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1504, 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 3.500814493, 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002327669 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.614 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 48324, 2024-06-13T07:26:19.615 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 186.021488484, 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003849463 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-06-13T07:26:19.616 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 1, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 27433, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 27433, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 1537, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:19.617 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 1717633, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 633, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 440, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 5, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:19.620 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 9176, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 176, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 1118857, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 35044, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:19.621 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 165709, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 499970737, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 33, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 165709, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 165709, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 3159.647382694, 2024-06-13T07:26:19.622 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.019067445 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 165709, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 167885 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 165709, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 1439, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 164270, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 12, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 21, 2024-06-13T07:26:19.623 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 1408, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 65306, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 753, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 3, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 96886, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 31, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 42, 2024-06-13T07:26:19.624 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-06-13T07:26:19.625 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 99, 2024-06-13T07:26:19.626 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 1, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 12, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 12, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 2521, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 902 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-06-13T07:26:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 16, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 14, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 96676, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 48352, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.628 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 1192244, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 231948206, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 1192244, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1192244, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 231948206, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.629 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 165709, 2024-06-13T07:26:19.630 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 505240881, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 165709, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 505240881, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-13T07:26:19.631 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 165709, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 165709, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 165709, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 165709, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.632 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 48352, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 5270144, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 48352, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1413, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 5270144, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.633 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x5649ec924500": { 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 31148, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 242210, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 489287254, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 242210, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-13T07:26:19.634 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 63893, 2024-06-13T07:26:19.635 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 489256106, 2024-06-13T07:26:19.635 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-13T07:26:19.635 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:19.635 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:19.635 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:19.635 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.635 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:19.635 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-13T07:26:20.556 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2024-06-13T07:26:20.556 INFO:tasks.check_counter:Daemon mds.b mds.imported=11 2024-06-13T07:26:20.556 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=214 2024-06-13T07:26:20.556 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=1255 2024-06-13T07:26:20.556 DEBUG:tasks.check_counter:Getting stats from e 2024-06-13T07:26:20.556 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-13T07:26:20.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:20 smithi026 ceph-mon[32041]: pgmap v1284: 129 pgs: 129 active+clean; 471 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-06-13T07:26:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:20 smithi135 ceph-mon[36276]: pgmap v1284: 129 pgs: 129 active+clean; 471 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-06-13T07:26:20.805 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T07:26:20.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:20 smithi136 ceph-mon[37229]: pgmap v1284: 129 pgs: 129 active+clean; 471 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-06-13T07:26:22.578 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:22 smithi135 ceph-mon[36276]: pgmap v1285: 129 pgs: 129 active+clean; 466 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.5 MiB/s wr, 95 op/s 2024-06-13T07:26:22.716 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-13T07:26:22.716 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:22.716 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 517, 2024-06-13T07:26:22.716 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 512, 2024-06-13T07:26:22.716 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 565456, 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 316736, 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.064762171, 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.025157761, 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.023309954, 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.011490260, 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 512, 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.024845699, 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000048526 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 565456, 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 316736 2024-06-13T07:26:22.718 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 2, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 390, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 320, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 224640, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.018296874, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.013771379, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.001054294, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 388, 2024-06-13T07:26:22.719 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.017492543, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000045083 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 224640 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 0, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 2, 2024-06-13T07:26:22.720 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 0, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 0, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.000204955, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000000000, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000130812, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.721 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.722 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:22.723 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:22.724 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 147898, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 27, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-13T07:26:22.725 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2024-06-13T07:26:22.726 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 515, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 489862, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 515, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-13T07:26:22.727 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 515, 2024-06-13T07:26:22.728 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 489862, 2024-06-13T07:26:22.728 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-13T07:26:22.728 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:22.728 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:22.728 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:22.728 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:22.728 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:22.728 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-13T07:26:22.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:22 smithi026 ceph-mon[32041]: pgmap v1285: 129 pgs: 129 active+clean; 466 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.5 MiB/s wr, 95 op/s 2024-06-13T07:26:22.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:22 smithi136 ceph-mon[37229]: pgmap v1285: 129 pgs: 129 active+clean; 466 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.5 MiB/s wr, 95 op/s 2024-06-13T07:26:23.430 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-06-13T07:26:23.430 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-06-13T07:26:23.430 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-06-13T07:26:23.430 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-06-13T07:26:23.430 DEBUG:tasks.check_counter:Getting stats from h 2024-06-13T07:26:23.431 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-13T07:26:23.673 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T07:26:24.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:24 smithi026 ceph-mon[32041]: pgmap v1286: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.9 MiB/s wr, 89 op/s 2024-06-13T07:26:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:24 smithi135 ceph-mon[36276]: pgmap v1286: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.9 MiB/s wr, 89 op/s 2024-06-13T07:26:24.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:24 smithi136 ceph-mon[37229]: pgmap v1286: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.9 MiB/s wr, 89 op/s 2024-06-13T07:26:25.951 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-13T07:26:25.951 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:25.951 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 512, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 509, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 562160, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 315648, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.063859421, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.025056071, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.022281779, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.011434799, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 508, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.024499620, 2024-06-13T07:26:25.952 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000048227 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 562160, 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 315648 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:25.953 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 4, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 392, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 640, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 381344, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 6, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.018995240, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.014170354, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.001592643, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 390, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.024758143, 2024-06-13T07:26:25.954 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000063482 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 640, 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 381344 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 4, 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 4, 2024-06-13T07:26:25.955 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 3024, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 448, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.000631829, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000034925, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000579503, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000731131, 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000243710 2024-06-13T07:26:25.956 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-13T07:26:25.957 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:25.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:25.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:25.960 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 147920, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 27, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:25.961 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-13T07:26:25.962 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 516, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 489786, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 516, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 516, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 489786, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:25.963 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-13T07:26:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:26 smithi135 ceph-mon[36276]: pgmap v1287: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 81 op/s 2024-06-13T07:26:26.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:26 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:26:26.607 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.h 2024-06-13T07:26:26.607 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.h 2024-06-13T07:26:26.607 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.h 2024-06-13T07:26:26.607 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.h 2024-06-13T07:26:26.607 DEBUG:tasks.check_counter:Getting stats from k 2024-06-13T07:26:26.607 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-13T07:26:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:26 smithi026 ceph-mon[32041]: pgmap v1287: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 81 op/s 2024-06-13T07:26:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:26 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:26:26.842 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T07:26:26.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:26 smithi136 ceph-mon[37229]: pgmap v1287: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 81 op/s 2024-06-13T07:26:26.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:26 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:26:27.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:27 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:26:27.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:27 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:26:27.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:27 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:26:27.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:27 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:26:27.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:27 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:26:27.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:27 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:26:27.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:27 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:26:27.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:27 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:26:27.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:27 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' 2024-06-13T07:26:28.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:28 smithi026 ceph-mon[32041]: pgmap v1288: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 447 KiB/s wr, 40 op/s 2024-06-13T07:26:28.768 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:28 smithi135 ceph-mon[36276]: pgmap v1288: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 447 KiB/s wr, 40 op/s 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 518, 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 515, 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 563792, 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 319488, 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2024-06-13T07:26:28.769 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.065506372, 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.025361440, 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.023295945, 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.011915782, 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 514, 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.021830551, 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000042471 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:28.770 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 563792, 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 319488 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 3, 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 395, 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 480, 2024-06-13T07:26:28.771 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 303696, 2024-06-13T07:26:28.772 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 5, 2024-06-13T07:26:28.772 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2024-06-13T07:26:28.772 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.019412769, 2024-06-13T07:26:28.772 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.014626017, 2024-06-13T07:26:28.772 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.001137467, 2024-06-13T07:26:28.772 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:28.773 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:28.773 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 393, 2024-06-13T07:26:28.773 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.021788053, 2024-06-13T07:26:28.773 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000055440 2024-06-13T07:26:28.773 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.773 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:28.773 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:28.774 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:28.774 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:28.774 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.774 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-13T07:26:28.774 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 303696 2024-06-13T07:26:28.774 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.774 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:28.775 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 4, 2024-06-13T07:26:28.775 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 4, 2024-06-13T07:26:28.775 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 3024, 2024-06-13T07:26:28.775 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 448, 2024-06-13T07:26:28.775 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2024-06-13T07:26:28.775 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2024-06-13T07:26:28.775 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.000555068, 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000033764, 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000493450, 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3, 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000686973, 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000228991 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:28.776 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:28.777 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:28.778 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:28.779 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 148682, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 29, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-13T07:26:28.780 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-13T07:26:28.781 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 521, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 490539, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 521, 2024-06-13T07:26:28.782 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 521, 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 490539, 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-13T07:26:28.783 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-13T07:26:28.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:28 smithi136 ceph-mon[37229]: pgmap v1288: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 447 KiB/s wr, 40 op/s 2024-06-13T07:26:29.507 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-06-13T07:26:29.507 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-06-13T07:26:29.508 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-06-13T07:26:29.508 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-06-13T07:26:29.508 DEBUG:tasks.check_counter:Getting stats from c 2024-06-13T07:26:29.508 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-13T07:26:29.831 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T07:26:30.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:30 smithi136 ceph-mon[37229]: pgmap v1289: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 382 KiB/s wr, 38 op/s 2024-06-13T07:26:30.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:30 smithi026 ceph-mon[32041]: pgmap v1289: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 382 KiB/s wr, 38 op/s 2024-06-13T07:26:30.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:30 smithi135 ceph-mon[36276]: pgmap v1289: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 382 KiB/s wr, 38 op/s 2024-06-13T07:26:31.836 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2187, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2189, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 210268220, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 695473, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.268521386, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.051224685, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.147000475, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.055631870, 2024-06-13T07:26:31.837 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2187, 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.081516741, 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000037273 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.838 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 4648, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 4616, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 215800086, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1621463, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.485371641, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.116558296, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.222202000, 2024-06-13T07:26:31.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.117463164, 2024-06-13T07:26:31.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:31.840 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4615, 2024-06-13T07:26:31.840 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.176534148, 2024-06-13T07:26:31.840 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000038252 2024-06-13T07:26:31.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:31.840 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 703552, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 317920 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 600, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 988, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 57387047, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1092519, 2024-06-13T07:26:31.841 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.093219404, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.026229334, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.046126432, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.014012699, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 986, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.043572784, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000044191 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:31.843 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.843 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.843 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.843 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-06-13T07:26:31.843 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 903936 2024-06-13T07:26:31.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.843 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 11662, 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.639404364, 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000054828 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-06-13T07:26:31.844 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.845 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 42363, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 29625, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 12738, 2024-06-13T07:26:31.846 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 163, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 2, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-06-13T07:26:31.847 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-06-13T07:26:31.848 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:26:31.849 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-06-13T07:26:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:26:31.851 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-06-13T07:26:31.852 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 229356, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-06-13T07:26:31.853 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 112, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 484668982, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 485051121, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 485050522, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.854 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 229356 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 31, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 42876, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 42845, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 26, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 163, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 137, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 42363, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 43935, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 1572, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-06-13T07:26:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 90856, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215420 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.857 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.859 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.861 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-06-13T07:26:31.861 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.861 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.863 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.864 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.865 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.866 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-06-13T07:26:31.867 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 2, 2024-06-13T07:26:31.869 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 3328, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:31.870 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:31.871 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:31.872 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:31.872 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:31.872 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:31.872 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:31.872 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:31.873 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:31.873 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:31.873 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:31.873 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:31.873 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:31.873 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:31.874 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:31.874 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:31.874 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-06-13T07:26:31.874 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:31.874 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:31.874 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 241273, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 90, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:31.875 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 9176, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 176, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 29928011, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 170738, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:31.876 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 6879, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 26, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 4, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 6879, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6879, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 28.598432731, 2024-06-13T07:26:31.877 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004157353 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6879, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 6880 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 6879, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 6879, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-06-13T07:26:31.878 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 3610, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 3266, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-06-13T07:26:31.879 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-06-13T07:26:31.880 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-06-13T07:26:31.880 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-06-13T07:26:31.880 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 2, 2024-06-13T07:26:31.880 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-06-13T07:26:31.880 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:26:31.880 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-06-13T07:26:31.881 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:26:31.881 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-06-13T07:26:31.881 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-06-13T07:26:31.881 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-06-13T07:26:31.881 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:26:31.881 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-06-13T07:26:31.882 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:26:31.882 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-06-13T07:26:31.882 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-06-13T07:26:31.882 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-06-13T07:26:31.882 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-06-13T07:26:31.882 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-06-13T07:26:31.882 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-06-13T07:26:31.883 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 0, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 0, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 11, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 11, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-06-13T07:26:31.884 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-06-13T07:26:31.885 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 7431, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 482755269, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 7431, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 7431, 2024-06-13T07:26:31.886 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 482755269, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:31.887 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6879, 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 480856827, 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6879, 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 480856827, 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-06-13T07:26:31.888 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6879, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 6879, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6879, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 6879, 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:31.889 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:31.890 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x556d054606e0": { 2024-06-13T07:26:31.891 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-06-13T07:26:31.892 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:31.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:31.893 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:31.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:31.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:31.893 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-13T07:26:32.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:32 smithi136 ceph-mon[37229]: pgmap v1290: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-06-13T07:26:32.665 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-13T07:26:32.665 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-13T07:26:32.665 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-06-13T07:26:32.665 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-06-13T07:26:32.665 DEBUG:tasks.check_counter:Getting stats from f 2024-06-13T07:26:32.666 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-13T07:26:32.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:32 smithi026 ceph-mon[32041]: pgmap v1290: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-06-13T07:26:32.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:32 smithi135 ceph-mon[36276]: pgmap v1290: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-06-13T07:26:32.925 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T07:26:34.581 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:34 smithi136 ceph-mon[37229]: pgmap v1291: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-06-13T07:26:34.717 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-13T07:26:34.717 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:34.717 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1981, 2024-06-13T07:26:34.717 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1983, 2024-06-13T07:26:34.717 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 210224185, 2024-06-13T07:26:34.717 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 630022, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 9, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.273707157, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.038972422, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.171259607, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.049697151, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1981, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.067008718, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000033825 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.718 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2293, 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2261, 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 175866765, 2024-06-13T07:26:34.719 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 872997, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.279937672, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.065860515, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.144195708, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.055300130, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2260, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.082877459, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000036671 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:34.720 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 703664, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 318400 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2699, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 3087, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 105689093, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1682134, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 7, 2024-06-13T07:26:34.721 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.298328482, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.075326275, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.122503839, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.077733646, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3086, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.119593769, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000038753 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.722 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 825760 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-13T07:26:34.723 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 10988, 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.292364378, 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000026607 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-06-13T07:26:34.725 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 48597, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 16416, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 32181, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 25145, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 1, 2024-06-13T07:26:34.726 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-06-13T07:26:34.727 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:26:34.728 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-06-13T07:26:34.729 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-06-13T07:26:34.730 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.731 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 242210, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 144, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 21, 2024-06-13T07:26:34.732 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 243007584, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 493481558, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 493480959, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 242210 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.733 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 25, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 48469, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 48444, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 23, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 106, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 83, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 48597, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 49153, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 556, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-06-13T07:26:34.734 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 97748, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215420 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.735 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.736 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-06-13T07:26:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-06-13T07:26:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-06-13T07:26:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-06-13T07:26:34.742 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.744 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.745 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.746 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.747 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 1, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 0, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 3329, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.748 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:34.749 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:34.750 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 242973, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 97, 2024-06-13T07:26:34.751 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 9176, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 176, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:34.752 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 34300332, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 195914, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 6418, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 21, 2024-06-13T07:26:34.753 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 3, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 6418, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6418, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 43.168143030, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006726105 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6418, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 6419 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 6418, 2024-06-13T07:26:34.754 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 6418, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 3422, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 2994, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-06-13T07:26:34.755 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 1, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:26:34.756 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-06-13T07:26:34.757 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 11, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 1, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-06-13T07:26:34.758 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 6969, 2024-06-13T07:26:34.760 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 491121576, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 6969, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6969, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 491121576, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.761 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6418, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 489298359, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6418, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 489298359, 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:34.762 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6418, 2024-06-13T07:26:34.763 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 6418, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6418, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 6418, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:34.764 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:34.765 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x564697c146e0": { 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:34.766 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:34.767 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:34.767 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:34.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:34.767 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-13T07:26:34.767 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:34 smithi026 ceph-mon[32041]: pgmap v1291: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-06-13T07:26:34.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:34 smithi135 ceph-mon[36276]: pgmap v1291: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-06-13T07:26:35.609 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-13T07:26:35.609 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-13T07:26:35.609 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-06-13T07:26:35.609 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-06-13T07:26:35.609 DEBUG:tasks.check_counter:Getting stats from i 2024-06-13T07:26:35.610 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-13T07:26:35.849 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T07:26:36.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:36 smithi026 ceph-mon[32041]: pgmap v1292: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-06-13T07:26:36.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:36 smithi135 ceph-mon[36276]: pgmap v1292: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-06-13T07:26:36.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:36 smithi136 ceph-mon[37229]: pgmap v1292: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 740875, 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 964631, 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 244250515, 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 776517479, 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 19, 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-06-13T07:26:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 44.991269481, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 19.815549907, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 29.487677350, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 2.243128835, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 964630, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 54.615570809, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000056618 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1386959, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.247519937, 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000178 2024-06-13T07:26:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 182542832, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 572597936 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 353152, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 339334, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 88347241, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 167216456, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 10, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 19.581457630, 2024-06-13T07:26:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 7.787203437, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 12.822726145, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 1.560529832, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 339333, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 14.951149034, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000044060 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 586940, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.095247376, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000162 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-13T07:26:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 906752 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 87398, 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 87367, 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 28089811, 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 269827584, 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 10, 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 7, 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 8.347067406, 2024-06-13T07:26:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 2.628237717, 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 1.501940002, 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 3.797851525, 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 87364, 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 5.099375971, 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000058369 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000232, 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000116 2024-06-13T07:26:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 720416, 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 342144 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 122938, 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 165.125301436, 2024-06-13T07:26:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001343159 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 94695, 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 11.135001454, 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000117588 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-06-13T07:26:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 349714, 2024-06-13T07:26:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 299858, 2024-06-13T07:26:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-06-13T07:26:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 299858, 2024-06-13T07:26:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1523.365155918, 2024-06-13T07:26:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005080288 2024-06-13T07:26:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 49856, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 51, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 3, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 3, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 381, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 247, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 109, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 25, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 28, 2024-06-13T07:26:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 317, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 1, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 1, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 4, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 605693, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 301542, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 49853, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 199, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 8181, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 7, 2024-06-13T07:26:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 69, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 8, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 496, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 20, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 20, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:26:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 20, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 37, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 23276, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 175664, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:26:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 92804, 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 39220, 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 15246, 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 235117 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 469, 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 1300, 2024-06-13T07:26:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 724, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 1217, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 536, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 15, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 50572, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 50557, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:26:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 7, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 6, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 242415, 2024-06-13T07:26:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 138112, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 135191, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 107224, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 2921, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 297, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 169, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 165, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 132, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 9, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-06-13T07:26:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 4, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 239039744, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 462237333, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 67041, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 718.048873201, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.010710593 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 40, 2024-06-13T07:26:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 50778, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 50738, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 39, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 687, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 648, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 381, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 85192, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 84811, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 1, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 54568, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 54567, 2024-06-13T07:26:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 82960, 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 223612 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 518486, 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 40, 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 349714, 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 6545, 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 24539, 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 50310, 2024-06-13T07:26:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 361.963624799, 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.007194665 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 20483, 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 329.441105252, 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.016083635 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 88307, 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 184.323054674, 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002087298 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000251773, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000251773 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-06-13T07:26:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 78403, 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 375.658361261, 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004791377 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 9754, 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 80.457091427, 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.008248625 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-06-13T07:26:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 30, 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.235245989, 2024-06-13T07:26:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.007841532 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 80, 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2.532742735, 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.031659284 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1948, 2024-06-13T07:26:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.821105189, 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000934858 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 50542, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 186.932572819, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003698559 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:26:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 2, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 24161, 2024-06-13T07:26:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 24161, 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 1573, 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 284594, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 129, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 616, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 7, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 9176, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 176, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 434383, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 6384, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-06-13T07:26:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 173624, 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 469022143, 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 28, 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 173624, 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 173624, 2024-06-13T07:26:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2956.875397406, 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.017030337 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 173624, 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 176064 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 173624, 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 1516, 2024-06-13T07:26:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 172108, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 12, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 31, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 1496, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 68537, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 801, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-06-13T07:26:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 3, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 101301, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 20, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 57, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-06-13T07:26:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-06-13T07:26:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 99, 2024-06-13T07:26:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 2, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 2823, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 983 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-06-13T07:26:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 17, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 10, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 101099, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 50557, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 1181421, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 231339306, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 1181421, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1181421, 2024-06-13T07:26:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 231339306, 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 173624, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 474532736, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 173624, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 474532736, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 173624, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 173624, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 173624, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 173624, 2024-06-13T07:26:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 50557, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 5510593, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 50557, 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1496, 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 5510593, 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55d18672c500": { 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 242415, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 458043029, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 242415, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 67041, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 458043029, 2024-06-13T07:26:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:37.692 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-13T07:26:38.454 INFO:tasks.check_counter:Daemon mds.i mds.exported=7 2024-06-13T07:26:38.454 INFO:tasks.check_counter:Daemon mds.i mds.imported=8 2024-06-13T07:26:38.454 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=469 2024-06-13T07:26:38.454 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1300 2024-06-13T07:26:38.454 DEBUG:tasks.check_counter:Getting stats from l 2024-06-13T07:26:38.455 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-13T07:26:38.705 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T07:26:38.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:38 smithi135 ceph-mon[36276]: pgmap v1293: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-06-13T07:26:38.827 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:38 smithi136 ceph-mon[37229]: pgmap v1293: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-06-13T07:26:39.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:38 smithi026 ceph-mon[32041]: pgmap v1293: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-06-13T07:26:40.597 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-13T07:26:40.597 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:26:40.597 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 4, 2024-06-13T07:26:40.598 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 4, 2024-06-13T07:26:40.598 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 3024, 2024-06-13T07:26:40.598 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 448, 2024-06-13T07:26:40.598 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 4, 2024-06-13T07:26:40.598 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 0, 2024-06-13T07:26:40.598 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.001285256, 2024-06-13T07:26:40.598 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.000029589, 2024-06-13T07:26:40.599 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.000461681, 2024-06-13T07:26:40.599 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:40.599 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:40.599 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3, 2024-06-13T07:26:40.599 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000630321, 2024-06-13T07:26:40.599 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000210107 2024-06-13T07:26:40.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.600 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:40.600 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:40.600 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:40.600 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:40.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.600 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-13T07:26:40.600 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:26:40.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.601 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:26:40.601 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 522, 2024-06-13T07:26:40.601 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 519, 2024-06-13T07:26:40.601 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 564880, 2024-06-13T07:26:40.601 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 322048, 2024-06-13T07:26:40.601 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 2, 2024-06-13T07:26:40.601 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 1, 2024-06-13T07:26:40.601 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.064054193, 2024-06-13T07:26:40.602 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.026025974, 2024-06-13T07:26:40.602 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.022057474, 2024-06-13T07:26:40.602 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.011413153, 2024-06-13T07:26:40.602 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:40.602 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 518, 2024-06-13T07:26:40.602 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.026966927, 2024-06-13T07:26:40.602 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000052059 2024-06-13T07:26:40.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.603 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:40.603 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:40.603 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:40.603 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:40.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.603 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 564880, 2024-06-13T07:26:40.603 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 322048 2024-06-13T07:26:40.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.604 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:26:40.604 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 4, 2024-06-13T07:26:40.604 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 400, 2024-06-13T07:26:40.604 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 640, 2024-06-13T07:26:40.604 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 382752, 2024-06-13T07:26:40.604 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-06-13T07:26:40.604 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-06-13T07:26:40.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.019026301, 2024-06-13T07:26:40.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.014002576, 2024-06-13T07:26:40.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.001485000, 2024-06-13T07:26:40.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:26:40.605 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:26:40.605 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 398, 2024-06-13T07:26:40.605 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.024808080, 2024-06-13T07:26:40.605 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000062331 2024-06-13T07:26:40.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.606 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:26:40.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:40.606 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:40.606 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:40.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.606 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 640, 2024-06-13T07:26:40.606 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 382752 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 0, 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-06-13T07:26:40.607 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:26:40.608 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:26:40.608 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:26:40.608 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:26:40.608 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:26:40.608 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:26:40.608 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:26:40.608 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:26:40.608 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:26:40.609 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:26:40.609 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:26:40.609 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:26:40.609 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:26:40.609 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-06-13T07:26:40.609 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:26:40.609 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-06-13T07:26:40.609 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:26:40.610 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:26:40.610 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:26:40.610 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:26:40.610 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:26:40.610 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:26:40.610 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:26:40.610 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-06-13T07:26:40.610 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:26:40.611 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:26:40.611 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:26:40.611 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-06-13T07:26:40.611 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-06-13T07:26:40.611 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-06-13T07:26:40.611 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:26:40.611 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:26:40.611 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:26:40.612 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:26:40.612 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 147898, 2024-06-13T07:26:40.612 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 27, 2024-06-13T07:26:40.612 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:26:40.612 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-06-13T07:26:40.612 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-06-13T07:26:40.612 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:26:40.613 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 928, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 2, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-13T07:26:40.614 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 526, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 491086, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 526, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 526, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 491086, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:40.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-13T07:26:40.616 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-13T07:26:40.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:40 smithi135 ceph-mon[36276]: pgmap v1294: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-13T07:26:40.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:40 smithi136 ceph-mon[37229]: pgmap v1294: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-13T07:26:41.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:40 smithi026 ceph-mon[32041]: pgmap v1294: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-13T07:26:41.347 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-06-13T07:26:41.347 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-06-13T07:26:41.347 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-06-13T07:26:41.348 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-06-13T07:26:41.348 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-13T07:26:41.359 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-13T07:26:41.359 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T07:26:41.359 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T07:26:41.433 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T07:26:41.434 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T07:26:41.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd blocklist ls 2024-06-13T07:26:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:26:42.667 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:42 smithi026 ceph-mon[32041]: pgmap v1295: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:26:42.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:42 smithi135 ceph-mon[36276]: pgmap v1295: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:26:42.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:42 smithi136 ceph-mon[37229]: pgmap v1295: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-13T07:26:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.444+0000 7f489c000700 1 -- 172.21.15.26:0/3983431789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 msgr2=0x7f4894108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.444+0000 7f489c000700 1 --2- 172.21.15.26:0/3983431789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 0x7f4894108960 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f488800b0d0 tx=0x7f488800b3e0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.444+0000 7f489c000700 1 -- 172.21.15.26:0/3983431789 shutdown_connections 2024-06-13T07:26:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.444+0000 7f489c000700 1 --2- 172.21.15.26:0/3983431789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4894102e90 0x7f489410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.444+0000 7f489c000700 1 --2- 172.21.15.26:0/3983431789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 0x7f4894108960 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.444+0000 7f489c000700 1 --2- 172.21.15.26:0/3983431789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48941024d0 0x7f4894102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.444+0000 7f489c000700 1 -- 172.21.15.26:0/3983431789 >> 172.21.15.26:0/3983431789 conn(0x7f48940fdf60 msgr2=0x7f4894100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:26:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.444+0000 7f489c000700 1 -- 172.21.15.26:0/3983431789 shutdown_connections 2024-06-13T07:26:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.445+0000 7f489c000700 1 -- 172.21.15.26:0/3983431789 wait complete. 2024-06-13T07:26:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.446+0000 7f489c000700 1 Processor -- start 2024-06-13T07:26:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.446+0000 7f489c000700 1 -- start start 2024-06-13T07:26:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.446+0000 7f489c000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48941024d0 0x7f489419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.446+0000 7f489c000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4894102e90 0x7f489419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.446+0000 7f489c000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 0x7f489419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.446+0000 7f489c000700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f48941127d0 con 0x7f4894108580 2024-06-13T07:26:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.446+0000 7f489c000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4894112650 con 0x7f4894102e90 2024-06-13T07:26:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.446+0000 7f489c000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4894112950 con 0x7f48941024d0 2024-06-13T07:26:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f489a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 0x7f489419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f4898c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4894102e90 0x7f489419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f4899600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48941024d0 0x7f489419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f489a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 0x7f489419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47360/0 (socket says 172.21.15.26:47360) 2024-06-13T07:26:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f489a000700 1 -- 172.21.15.26:0/2970414574 learned_addr learned my addr 172.21.15.26:0/2970414574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:26:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f4898c00700 1 -- 172.21.15.26:0/2970414574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48941024d0 msgr2=0x7f489419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f4898c00700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48941024d0 0x7f489419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f4898c00700 1 -- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 msgr2=0x7f489419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f4898c00700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 0x7f489419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f4898c00700 1 -- 172.21.15.26:0/2970414574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489419fdb0 con 0x7f4894102e90 2024-06-13T07:26:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f4899600700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48941024d0 0x7f489419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:26:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.447+0000 7f489a000700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 0x7f489419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:26:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.448+0000 7f4898c00700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4894102e90 0x7f489419b2e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f488000e9f0 tx=0x7f488000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:26:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.448+0000 7f4892000700 1 -- 172.21.15.26:0/2970414574 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880004220 con 0x7f4894102e90 2024-06-13T07:26:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.448+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48941a00a0 con 0x7f4894102e90 2024-06-13T07:26:44.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.448+0000 7f4892000700 1 -- 172.21.15.26:0/2970414574 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4880004d10 con 0x7f4894102e90 2024-06-13T07:26:44.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.448+0000 7f4892000700 1 -- 172.21.15.26:0/2970414574 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880005020 con 0x7f4894102e90 2024-06-13T07:26:44.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.448+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48941a7920 con 0x7f4894102e90 2024-06-13T07:26:44.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.449+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4894069180 con 0x7f4894102e90 2024-06-13T07:26:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.452+0000 7f4892000700 1 -- 172.21.15.26:0/2970414574 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4880004750 con 0x7f4894102e90 2024-06-13T07:26:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.453+0000 7f4892000700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f487c0a2c20 0x7f487c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.453+0000 7f4892000700 1 -- 172.21.15.26:0/2970414574 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f48800cf3f0 con 0x7f4894102e90 2024-06-13T07:26:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.453+0000 7f4899600700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f487c0a2c20 0x7f487c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.454+0000 7f4899600700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f487c0a2c20 0x7f487c0a50e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4888009fd0 tx=0x7f488800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:26:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.455+0000 7f4892000700 1 -- 172.21.15.26:0/2970414574 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f488009bf90 con 0x7f4894102e90 2024-06-13T07:26:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.701+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f489419c100 con 0x7f4894102e90 2024-06-13T07:26:44.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:44 smithi026 ceph-mon[32041]: pgmap v1296: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:26:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.702+0000 7f4892000700 1 -- 172.21.15.26:0/2970414574 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v99) ==== 81+0+591 (secure 0 0 0) 0x7f489419c100 con 0x7f4894102e90 2024-06-13T07:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3363905776 2024-06-14T06:46:28.109797+0000 2024-06-13T07:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/3558618334 2024-06-14T06:46:28.109797+0000 2024-06-13T07:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2900186702 2024-06-14T06:46:28.109797+0000 2024-06-13T07:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/2776242437 2024-06-14T06:45:51.299898+0000 2024-06-13T07:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/2776242437 2024-06-14T06:45:51.299898+0000 2024-06-13T07:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/3558618334 2024-06-14T06:46:28.109797+0000 2024-06-13T07:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/106059480 2024-06-14T06:45:51.299898+0000 2024-06-13T07:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2539477633 2024-06-14T06:45:51.299898+0000 2024-06-13T07:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3002519462 2024-06-14T06:45:51.299898+0000 2024-06-13T07:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2082657826 2024-06-14T06:46:28.109797+0000 2024-06-13T07:26:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f487c0a2c20 msgr2=0x7f487c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f487c0a2c20 0x7f487c0a50e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4888009fd0 tx=0x7f488800b040 comp rx=0 tx=0).stop 2024-06-13T07:26:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4894102e90 msgr2=0x7f489419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4894102e90 0x7f489419b2e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f488000e9f0 tx=0x7f488000ed00 comp rx=0 tx=0).stop 2024-06-13T07:26:44.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 shutdown_connections 2024-06-13T07:26:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f487c0a2c20 0x7f487c0a50e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894108580 0x7f489419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48941024d0 0x7f489419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 --2- 172.21.15.26:0/2970414574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4894102e90 0x7f489419b2e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.705+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 >> 172.21.15.26:0/2970414574 conn(0x7f48940fdf60 msgr2=0x7f48940ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:26:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.706+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 shutdown_connections 2024-06-13T07:26:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:44.706+0000 7f489c000700 1 -- 172.21.15.26:0/2970414574 wait complete. 2024-06-13T07:26:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:listed 10 entries 2024-06-13T07:26:44.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:44 smithi135 ceph-mon[36276]: pgmap v1296: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:26:44.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:44 smithi136 ceph-mon[37229]: pgmap v1296: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:26:45.691 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-13T07:26:45.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:26:45.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-13T07:26:45.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:45 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2970414574' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T07:26:45.782 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:45 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2970414574' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T07:26:45.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:45 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2970414574' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T07:26:45.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T07:26:45.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:26:45.970 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-13T07:26:46.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:26:46.054 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-06-13T07:26:46.136 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-13T07:26:46.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:26:46.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-13T07:26:46.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:26:46.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-06-13T07:26:46.363 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-13T07:26:46.363 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-06-13T07:26:46.363 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T07:26:46.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/1337e8c5d9094e3deb947bac572c8db6588a1b86008e9de02c485f40cf519395/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/1337e8c5d9094e3deb947bac572c8db6588a1b86008e9de02c485f40cf519395/diff,workdir=/var/lib/containers/storage/overlay/1337e8c5d9094e3deb947bac572c8db6588a1b86008e9de02c485f40cf519395/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/774019c5ed061b6316b32373d60de9652338cd7a4e1272ca7c049e249476e04c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/774019c5ed061b6316b32373d60de9652338cd7a4e1272ca7c049e249476e04c/diff,workdir=/var/lib/containers/storage/overlay/774019c5ed061b6316b32373d60de9652338cd7a4e1272ca7c049e249476e04c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/66a0dd08edb1df6c5a62e79ce198f5ac402bf54a867b06c77de35438a9adb0cf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/66a0dd08edb1df6c5a62e79ce198f5ac402bf54a867b06c77de35438a9adb0cf/diff,workdir=/var/lib/containers/storage/overlay/66a0dd08edb1df6c5a62e79ce198f5ac402bf54a867b06c77de35438a9adb0cf/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1519aa908434d0d5d394039a605cc4c776d1f81de848ae3527078b2ae64d14d6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/1519aa908434d0d5d394039a605cc4c776d1f81de848ae3527078b2ae64d14d6/diff,workdir=/var/lib/containers/storage/overlay/1519aa908434d0d5d394039a605cc4c776d1f81de848ae3527078b2ae64d14d6/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0fa84763735caf5588d2e9049a4973b968242480c019d0bfe70fc685f5e9c5fe/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/0fa84763735caf5588d2e9049a4973b968242480c019d0bfe70fc685f5e9c5fe/diff,workdir=/var/lib/containers/storage/overlay/0fa84763735caf5588d2e9049a4973b968242480c019d0bfe70fc685f5e9c5fe/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/39c8d35054f10f6166506ffa72bb99e88d663bd42485b9935572a008c03c6f2f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/39c8d35054f10f6166506ffa72bb99e88d663bd42485b9935572a008c03c6f2f/diff,workdir=/var/lib/containers/storage/overlay/39c8d35054f10f6166506ffa72bb99e88d663bd42485b9935572a008c03c6f2f/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/043be13507077f8baedbe915fbaee702912cb0f9c1f52f7ccafac6363bc75961/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/043be13507077f8baedbe915fbaee702912cb0f9c1f52f7ccafac6363bc75961/diff,workdir=/var/lib/containers/storage/overlay/043be13507077f8baedbe915fbaee702912cb0f9c1f52f7ccafac6363bc75961/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/77d7dc88a6d480337b64f46566f406a6f42b61bd200cf215828aa883cb360763/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/77d7dc88a6d480337b64f46566f406a6f42b61bd200cf215828aa883cb360763/diff,workdir=/var/lib/containers/storage/overlay/77d7dc88a6d480337b64f46566f406a6f42b61bd200cf215828aa883cb360763/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ec3d2a8f67db94d858a444f359b7a03cfe35091ec57bbb597906f03a7f22f638/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/ec3d2a8f67db94d858a444f359b7a03cfe35091ec57bbb597906f03a7f22f638/diff,workdir=/var/lib/containers/storage/overlay/ec3d2a8f67db94d858a444f359b7a03cfe35091ec57bbb597906f03a7f22f638/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2a3ca69562047921d82a707bc1ff2a42690f8b4fa6e07ae56376dfad922b34d5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SV6YMVARMIYXVHCV4FQIX5W3G5:/var/lib/containers/storage/overlay/l/354IQY2VOKKQONLNJ2NSAFYE3K,upperdir=/var/lib/containers/storage/overlay/2a3ca69562047921d82a707bc1ff2a42690f8b4fa6e07ae56376dfad922b34d5/diff,workdir=/var/lib/containers/storage/overlay/2a3ca69562047921d82a707bc1ff2a42690f8b4fa6e07ae56376dfad922b34d5/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 2024-06-13T07:26:46.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:26:46.401 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-06-13T07:26:46.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:26:46.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-06-13T07:26:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-06-13T07:26:46.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:26:46.683 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-13T07:26:46.693 INFO:tasks.cephadm:Teardown begin 2024-06-13T07:26:46.693 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-13T07:26:46.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:26:46.779 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:46 smithi026 ceph-mon[32041]: pgmap v1297: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:26:46.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:46 smithi135 ceph-mon[36276]: pgmap v1297: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:26:46.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:46 smithi136 ceph-mon[37229]: pgmap v1297: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T07:26:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:26:48.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:48 smithi026 ceph-mon[32041]: pgmap v1298: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:26:48.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:48 smithi135 ceph-mon[36276]: pgmap v1298: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:26:48.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:48 smithi136 ceph-mon[37229]: pgmap v1298: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T07:26:49.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:49 smithi026 ceph-mon[32041]: pgmap v1299: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T07:26:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 -- 172.21.15.26:0/745604754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc98102fc0 msgr2=0x7fcc98107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 --2- 172.21.15.26:0/745604754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc98102fc0 0x7fcc98107010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcc840099c0 tx=0x7fcc84009cd0 comp rx=0 tx=0).stop 2024-06-13T07:26:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 -- 172.21.15.26:0/745604754 shutdown_connections 2024-06-13T07:26:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 --2- 172.21.15.26:0/745604754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc981076f0 0x7fcc9810f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 --2- 172.21.15.26:0/745604754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc98102610 0x7fcc981029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 --2- 172.21.15.26:0/745604754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc98102fc0 0x7fcc98107010 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 -- 172.21.15.26:0/745604754 >> 172.21.15.26:0/745604754 conn(0x7fcc980fdea0 msgr2=0x7fcc981002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:26:49.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 -- 172.21.15.26:0/745604754 shutdown_connections 2024-06-13T07:26:49.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.792+0000 7fcc97e00700 1 -- 172.21.15.26:0/745604754 wait complete. 2024-06-13T07:26:49.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.793+0000 7fcc97e00700 1 Processor -- start 2024-06-13T07:26:49.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.793+0000 7fcc97e00700 1 -- start start 2024-06-13T07:26:49.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.794+0000 7fcc97e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc98102610 0x7fcc98074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:49.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.794+0000 7fcc97e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc98102fc0 0x7fcc98074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:49.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.794+0000 7fcc97e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc981076f0 0x7fcc98075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:49.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.794+0000 7fcc97e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98112720 con 0x7fcc98102fc0 2024-06-13T07:26:49.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.794+0000 7fcc97e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981125a0 con 0x7fcc981076f0 2024-06-13T07:26:49.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.794+0000 7fcc97e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981128a0 con 0x7fcc98102610 2024-06-13T07:26:49.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.794+0000 7fcc96a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc98102610 0x7fcc98074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:49.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc96000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc98102fc0 0x7fcc98074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:49.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc981076f0 0x7fcc98075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:49.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc981076f0 0x7fcc98075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:44344/0 (socket says 172.21.15.26:44344) 2024-06-13T07:26:49.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc96a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc98102610 0x7fcc98074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:53004/0 (socket says 172.21.15.26:53004) 2024-06-13T07:26:49.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 -- 172.21.15.26:0/90706838 learned_addr learned my addr 172.21.15.26:0/90706838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:26:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 -- 172.21.15.26:0/90706838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc98102610 msgr2=0x7fcc98074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc98102610 0x7fcc98074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 -- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc98102fc0 msgr2=0x7fcc98074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc98102fc0 0x7fcc98074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 -- 172.21.15.26:0/90706838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc981a7110 con 0x7fcc981076f0 2024-06-13T07:26:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.795+0000 7fcc97400700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc981076f0 0x7fcc98075210 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000ef30 tx=0x7fcc9000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:26:49.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.796+0000 7fcc8f400700 1 -- 172.21.15.26:0/90706838 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc900092e0 con 0x7fcc981076f0 2024-06-13T07:26:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.796+0000 7fcc8f400700 1 -- 172.21.15.26:0/90706838 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc90004bb0 con 0x7fcc981076f0 2024-06-13T07:26:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.796+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc981a73a0 con 0x7fcc981076f0 2024-06-13T07:26:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.796+0000 7fcc8f400700 1 -- 172.21.15.26:0/90706838 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90004280 con 0x7fcc981076f0 2024-06-13T07:26:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.796+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc981a78b0 con 0x7fcc981076f0 2024-06-13T07:26:49.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.798+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc98069180 con 0x7fcc981076f0 2024-06-13T07:26:49.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.799+0000 7fcc8f400700 1 -- 172.21.15.26:0/90706838 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcc9001b030 con 0x7fcc981076f0 2024-06-13T07:26:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.800+0000 7fcc8f400700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcc7c0a2ca0 0x7fcc7c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.801+0000 7fcc8f400700 1 -- 172.21.15.26:0/90706838 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcc900ceaa0 con 0x7fcc981076f0 2024-06-13T07:26:49.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.801+0000 7fcc96a00700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcc7c0a2ca0 0x7fcc7c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:49.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.801+0000 7fcc96a00700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcc7c0a2ca0 0x7fcc7c0a5160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009c60 tx=0x7fcc88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:26:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:49.805+0000 7fcc8f400700 1 -- 172.21.15.26:0/90706838 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcc90010040 con 0x7fcc981076f0 2024-06-13T07:26:49.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:49 smithi136 ceph-mon[37229]: pgmap v1299: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T07:26:50.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:49 smithi135 ceph-mon[36276]: pgmap v1299: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T07:26:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.084+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc9802dca0 con 0x7fcc981076f0 2024-06-13T07:26:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.086+0000 7fcc8f400700 1 -- 172.21.15.26:0/90706838 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11876 (secure 0 0 0) 0x7fcc9001b320 con 0x7fcc981076f0 2024-06-13T07:26:50.086 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:26:50.086 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T07:25:54.620798+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":117,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":119,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24506,24511,24517]},"id":1}]} 2024-06-13T07:26:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcc7c0a2ca0 msgr2=0x7fcc7c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcc7c0a2ca0 0x7fcc7c0a5160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009c60 tx=0x7fcc88009260 comp rx=0 tx=0).stop 2024-06-13T07:26:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc981076f0 msgr2=0x7fcc98075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc981076f0 0x7fcc98075210 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000ef30 tx=0x7fcc9000c3a0 comp rx=0 tx=0).stop 2024-06-13T07:26:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 shutdown_connections 2024-06-13T07:26:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fcc7c0a2ca0 0x7fcc7c0a5160 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:50.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc981076f0 0x7fcc98075210 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:50.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc98102610 0x7fcc98074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:50.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 --2- 172.21.15.26:0/90706838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc98102fc0 0x7fcc98074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:50.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 >> 172.21.15.26:0/90706838 conn(0x7fcc980fdea0 msgr2=0x7fcc981001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:26:50.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.089+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 shutdown_connections 2024-06-13T07:26:50.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:50.090+0000 7fcc97e00700 1 -- 172.21.15.26:0/90706838 wait complete. 2024-06-13T07:26:50.090 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 54 2024-06-13T07:26:50.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:50 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/90706838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:26:51.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:50 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/90706838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:26:51.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:50 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/90706838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:26:51.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:26:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:26:51.801 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:51 smithi026 ceph-mon[32041]: pgmap v1300: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:26:51.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:51 smithi136 ceph-mon[37229]: pgmap v1300: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:26:52.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:51 smithi135 ceph-mon[36276]: pgmap v1300: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:26:54.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.492+0000 7faf02200700 1 -- 172.21.15.26:0/2682503040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc1076f0 msgr2=0x7faefc10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:54.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.492+0000 7faf02200700 1 --2- 172.21.15.26:0/2682503040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc1076f0 0x7faefc10f2c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faef4009f90 tx=0x7faef400b330 comp rx=0 tx=0).stop 2024-06-13T07:26:54.494 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:54 smithi026 ceph-mon[32041]: pgmap v1301: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-13T07:26:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.493+0000 7faf02200700 1 -- 172.21.15.26:0/2682503040 shutdown_connections 2024-06-13T07:26:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.493+0000 7faf02200700 1 --2- 172.21.15.26:0/2682503040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc1076f0 0x7faefc10f2c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.493+0000 7faf02200700 1 --2- 172.21.15.26:0/2682503040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc102610 0x7faefc1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.493+0000 7faf02200700 1 --2- 172.21.15.26:0/2682503040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faefc102fc0 0x7faefc107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.493+0000 7faf02200700 1 -- 172.21.15.26:0/2682503040 >> 172.21.15.26:0/2682503040 conn(0x7faefc0fdea0 msgr2=0x7faefc1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.493+0000 7faf02200700 1 -- 172.21.15.26:0/2682503040 shutdown_connections 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.493+0000 7faf02200700 1 -- 172.21.15.26:0/2682503040 wait complete. 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.494+0000 7faf02200700 1 Processor -- start 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.494+0000 7faf02200700 1 -- start start 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf02200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc102610 0x7faefc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf02200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faefc102fc0 0x7faefc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf02200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1076f0 0x7faefc1a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf02200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faefc1105b0 con 0x7faefc1076f0 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf02200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faefc110430 con 0x7faefc102610 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf02200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faefc110730 con 0x7faefc102fc0 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf01800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1076f0 0x7faefc1a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf01800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1076f0 0x7faefc1a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51704/0 (socket says 172.21.15.26:51704) 2024-06-13T07:26:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf01800700 1 -- 172.21.15.26:0/3208933874 learned_addr learned my addr 172.21.15.26:0/3208933874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:26:54.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faefbe00700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faefc102fc0 0x7faefc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:54.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.495+0000 7faf00e00700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc102610 0x7faefc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:54.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faefbe00700 1 -- 172.21.15.26:0/3208933874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc102610 msgr2=0x7faefc19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:54.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faefbe00700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc102610 0x7faefc19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faefbe00700 1 -- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1076f0 msgr2=0x7faefc1a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:54.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faefbe00700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1076f0 0x7faefc1a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faefbe00700 1 -- 172.21.15.26:0/3208933874 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc1a4130 con 0x7faefc102fc0 2024-06-13T07:26:54.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faefbe00700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faefc102fc0 0x7faefc19f6b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faee800dfd0 tx=0x7faee800bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:26:54.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faef9600700 1 -- 172.21.15.26:0/3208933874 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee801a070 con 0x7faefc102fc0 2024-06-13T07:26:54.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faef9600700 1 -- 172.21.15.26:0/3208933874 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee8004500 con 0x7faefc102fc0 2024-06-13T07:26:54.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc1a4420 con 0x7faefc102fc0 2024-06-13T07:26:54.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faef9600700 1 -- 172.21.15.26:0/3208933874 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8005310 con 0x7faefc102fc0 2024-06-13T07:26:54.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.496+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc1abca0 con 0x7faefc102fc0 2024-06-13T07:26:54.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.498+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc069180 con 0x7faefc102fc0 2024-06-13T07:26:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.500+0000 7faef9600700 1 -- 172.21.15.26:0/3208933874 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faee801d070 con 0x7faefc102fc0 2024-06-13T07:26:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.501+0000 7faef9600700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faee40a2ca0 0x7faee40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.501+0000 7faef9600700 1 -- 172.21.15.26:0/3208933874 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faee80ce670 con 0x7faefc102fc0 2024-06-13T07:26:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.501+0000 7faf00e00700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faee40a2ca0 0x7faee40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.502+0000 7faf00e00700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faee40a2ca0 0x7faee40a5160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faef0009750 tx=0x7faef0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:26:54.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.506+0000 7faef9600700 1 -- 172.21.15.26:0/3208933874 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faee8004750 con 0x7faefc102fc0 2024-06-13T07:26:54.580 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:54 smithi136 ceph-mon[37229]: pgmap v1301: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-13T07:26:54.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:54 smithi135 ceph-mon[36276]: pgmap v1301: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-13T07:26:54.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.798+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faefc1a0480 con 0x7faefc102fc0 2024-06-13T07:26:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.799+0000 7faef9600700 1 -- 172.21.15.26:0/3208933874 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11876 (secure 0 0 0) 0x7faee809b180 con 0x7faefc102fc0 2024-06-13T07:26:54.799 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:26:54.799 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/2065578555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":2065578555},{"type":"v1","addr":"172.21.15.135:6833","nonce":2065578555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2047884215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2047884215},{"type":"v1","addr":"172.21.15.136:6833","nonce":2047884215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1212410310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1212410310},{"type":"v1","addr":"172.21.15.135:6835","nonce":1212410310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/153334101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":153334101},{"type":"v1","addr":"172.21.15.26:6835","nonce":153334101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/363278001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":363278001},{"type":"v1","addr":"172.21.15.135:6837","nonce":363278001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/69365853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":69365853},{"type":"v1","addr":"172.21.15.26:6837","nonce":69365853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:55:01.881338+0000","modified":"2024-06-13T07:25:54.620798+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24517,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/582335478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":582335478},{"type":"v1","addr":"172.21.15.136:6835","nonce":582335478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"a","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6841/3560957238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":3560957238},{"type":"v1","addr":"172.21.15.26:6841","nonce":3560957238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/1512917649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":1512917649},{"type":"v1","addr":"172.21.15.136:6837","nonce":1512917649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":117,"addr":"172.21.15.135:6839/1905980530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1905980530},{"type":"v1","addr":"172.21.15.135:6839","nonce":1905980530}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":119,"addr":"172.21.15.136:6839/2531149732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2531149732},{"type":"v1","addr":"172.21.15.136:6839","nonce":2531149732}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6839/173910348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":173910348},{"type":"v1","addr":"172.21.15.26:6839","nonce":173910348}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24506,24511,24517]},"id":1}]} 2024-06-13T07:26:54.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faee40a2ca0 msgr2=0x7faee40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:54.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faee40a2ca0 0x7faee40a5160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faef0009750 tx=0x7faef0009310 comp rx=0 tx=0).stop 2024-06-13T07:26:54.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faefc102fc0 msgr2=0x7faefc19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:54.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faefc102fc0 0x7faefc19f6b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faee800dfd0 tx=0x7faee800bdd0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 shutdown_connections 2024-06-13T07:26:54.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faee40a2ca0 0x7faee40a5160 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc102610 0x7faefc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1076f0 0x7faefc1a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 --2- 172.21.15.26:0/3208933874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faefc102fc0 0x7faefc19f6b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:54.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.802+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 >> 172.21.15.26:0/3208933874 conn(0x7faefc0fdea0 msgr2=0x7faefc0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:26:54.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.803+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 shutdown_connections 2024-06-13T07:26:54.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:54.803+0000 7faf02200700 1 -- 172.21.15.26:0/3208933874 wait complete. 2024-06-13T07:26:54.804 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 54 2024-06-13T07:26:55.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:55 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/3208933874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:26:55.592 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:55 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/3208933874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:26:55.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T07:26:55.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:55 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/3208933874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:26:56.208 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:26:56.599 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:56 smithi026 ceph-mon[32041]: pgmap v1302: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-13T07:26:56.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:56 smithi136 ceph-mon[37229]: pgmap v1302: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-13T07:26:56.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:56 smithi135 ceph-mon[36276]: pgmap v1302: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-13T07:26:58.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:58 smithi026 ceph-mon[32041]: pgmap v1303: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-13T07:26:58.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:58 smithi136 ceph-mon[37229]: pgmap v1303: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-13T07:26:58.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:58 smithi135 ceph-mon[36276]: pgmap v1303: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-13T07:26:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 -- 172.21.15.26:0/3904733306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 msgr2=0x7fa8a8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 --2- 172.21.15.26:0/3904733306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 0x7fa8a8101810 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa894009990 tx=0x7fa894009ca0 comp rx=0 tx=0).stop 2024-06-13T07:26:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 -- 172.21.15.26:0/3904733306 shutdown_connections 2024-06-13T07:26:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 --2- 172.21.15.26:0/3904733306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8a8101d50 0x7fa8a810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 --2- 172.21.15.26:0/3904733306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8a8107400 0x7fa8a81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 --2- 172.21.15.26:0/3904733306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 0x7fa8a8101810 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 -- 172.21.15.26:0/3904733306 >> 172.21.15.26:0/3904733306 conn(0x7fa8a8078340 msgr2=0x7fa8a8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:26:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 -- 172.21.15.26:0/3904733306 shutdown_connections 2024-06-13T07:26:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.884+0000 7fa8ad200700 1 -- 172.21.15.26:0/3904733306 wait complete. 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.885+0000 7fa8ad200700 1 Processor -- start 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.885+0000 7fa8ad200700 1 -- start start 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8ad200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 0x7fa8a819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8ad200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8a8101d50 0x7fa8a819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8ad200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8a8107400 0x7fa8a8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8ad200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81116c0 con 0x7fa8a8107400 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8ad200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8111540 con 0x7fa8a8101390 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8ad200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8111840 con 0x7fa8a8101d50 2024-06-13T07:26:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8a8101d50 0x7fa8a819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:58.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 0x7fa8a819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:58.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8a8101d50 0x7fa8a819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.26:50272/0 (socket says 172.21.15.26:50272) 2024-06-13T07:26:58.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5400700 1 -- 172.21.15.26:0/1451961187 learned_addr learned my addr 172.21.15.26:0/1451961187 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:26:58.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 0x7fa8a819e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:50062/0 (socket says 172.21.15.26:50062) 2024-06-13T07:26:58.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a6800700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8a8107400 0x7fa8a8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:58.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5400700 1 -- 172.21.15.26:0/1451961187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 msgr2=0x7fa8a819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:58.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5400700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 0x7fa8a819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:58.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5400700 1 -- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8a8107400 msgr2=0x7fa8a8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:58.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.886+0000 7fa8a5400700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8a8107400 0x7fa8a8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:58.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.887+0000 7fa8a5400700 1 -- 172.21.15.26:0/1451961187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a8199200 con 0x7fa8a8101d50 2024-06-13T07:26:58.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.887+0000 7fa8a5400700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8a8101d50 0x7fa8a819ee20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa894005910 tx=0x7fa89400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:26:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.887+0000 7fa89ea00700 1 -- 172.21.15.26:0/1451961187 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894027070 con 0x7fa8a8101d50 2024-06-13T07:26:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.887+0000 7fa89ea00700 1 -- 172.21.15.26:0/1451961187 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa894004650 con 0x7fa8a8101d50 2024-06-13T07:26:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.887+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a8199490 con 0x7fa8a8101d50 2024-06-13T07:26:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.887+0000 7fa89ea00700 1 -- 172.21.15.26:0/1451961187 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89400f460 con 0x7fa8a8101d50 2024-06-13T07:26:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.887+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a57b0 con 0x7fa8a8101d50 2024-06-13T07:26:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.888+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a8069180 con 0x7fa8a8101d50 2024-06-13T07:26:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.891+0000 7fa89ea00700 1 -- 172.21.15.26:0/1451961187 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa89402a070 con 0x7fa8a8101d50 2024-06-13T07:26:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.892+0000 7fa89ea00700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa88c0a2ca0 0x7fa88c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:26:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.892+0000 7fa89ea00700 1 -- 172.21.15.26:0/1451961187 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa8940d0700 con 0x7fa8a8101d50 2024-06-13T07:26:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.892+0000 7fa8a5e00700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa88c0a2ca0 0x7fa88c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:26:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.893+0000 7fa8a5e00700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa88c0a2ca0 0x7fa88c0a5160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8199be0 tx=0x7fa898009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:26:58.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:58.895+0000 7fa89ea00700 1 -- 172.21.15.26:0/1451961187 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa894031070 con 0x7fa8a8101d50 2024-06-13T07:26:59.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.134+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8a81a5980 con 0x7fa8a8101d50 2024-06-13T07:26:59.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.136+0000 7fa89ea00700 1 -- 172.21.15.26:0/1451961187 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24891 (secure 0 0 0) 0x7fa894013070 con 0x7fa8a8101d50 2024-06-13T07:26:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-06-13T07:26:59.137 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"559474f0-2950-11ef-bc9c-c7b262605968","created":"2024-06-13T06:45:08.671358+0000","modified":"2024-06-13T06:56:29.509131+0000","last_up_change":"2024-06-13T06:53:39.340353+0000","last_in_change":"2024-06-13T06:53:19.244878+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:49:18.857067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:54:52.893467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:54:57.814451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:55:10.534099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a6b984b-52fb-4d69-9e00-ff59d760e29e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6803","nonce":4252497348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6805","nonce":4252497348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6809","nonce":4252497348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4252497348},{"type":"v1","addr":"172.21.15.26:6807","nonce":4252497348}]},"public_addr":"172.21.15.26:6803/4252497348","cluster_addr":"172.21.15.26:6805/4252497348","heartbeat_back_addr":"172.21.15.26:6809/4252497348","heartbeat_front_addr":"172.21.15.26:6807/4252497348","state":["exists","up"]},{"osd":1,"uuid":"270b9d62-0c93-48d4-b822-eb2e41ea3bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6801","nonce":2669544474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6803","nonce":2669544474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6807","nonce":2669544474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2669544474},{"type":"v1","addr":"172.21.15.135:6805","nonce":2669544474}]},"public_addr":"172.21.15.135:6801/2669544474","cluster_addr":"172.21.15.135:6803/2669544474","heartbeat_back_addr":"172.21.15.135:6807/2669544474","heartbeat_front_addr":"172.21.15.135:6805/2669544474","state":["exists","up"]},{"osd":2,"uuid":"4f1bc547-0cb8-434f-9467-57da190ce910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6801","nonce":4076851589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6803","nonce":4076851589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6807","nonce":4076851589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":4076851589},{"type":"v1","addr":"172.21.15.136:6805","nonce":4076851589}]},"public_addr":"172.21.15.136:6801/4076851589","cluster_addr":"172.21.15.136:6803/4076851589","heartbeat_back_addr":"172.21.15.136:6807/4076851589","heartbeat_front_addr":"172.21.15.136:6805/4076851589","state":["exists","up"]},{"osd":3,"uuid":"0a77d3db-9cde-4c90-86e2-9030466c4753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6811","nonce":2922553278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6813","nonce":2922553278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6817","nonce":2922553278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2922553278},{"type":"v1","addr":"172.21.15.26:6815","nonce":2922553278}]},"public_addr":"172.21.15.26:6811/2922553278","cluster_addr":"172.21.15.26:6813/2922553278","heartbeat_back_addr":"172.21.15.26:6817/2922553278","heartbeat_front_addr":"172.21.15.26:6815/2922553278","state":["exists","up"]},{"osd":4,"uuid":"6cd55a9b-cc9a-4e67-b486-f1a420368bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6809","nonce":2020131902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6811","nonce":2020131902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6815","nonce":2020131902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2020131902},{"type":"v1","addr":"172.21.15.135:6813","nonce":2020131902}]},"public_addr":"172.21.15.135:6809/2020131902","cluster_addr":"172.21.15.135:6811/2020131902","heartbeat_back_addr":"172.21.15.135:6815/2020131902","heartbeat_front_addr":"172.21.15.135:6813/2020131902","state":["exists","up"]},{"osd":5,"uuid":"5ca38a30-516e-4466-bbdf-4873adf0779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6809","nonce":3459833212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6811","nonce":3459833212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6815","nonce":3459833212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3459833212},{"type":"v1","addr":"172.21.15.136:6813","nonce":3459833212}]},"public_addr":"172.21.15.136:6809/3459833212","cluster_addr":"172.21.15.136:6811/3459833212","heartbeat_back_addr":"172.21.15.136:6815/3459833212","heartbeat_front_addr":"172.21.15.136:6813/3459833212","state":["exists","up"]},{"osd":6,"uuid":"ad8f82b9-822e-4b98-8629-b0bd8723c4c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6819","nonce":2705956031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6821","nonce":2705956031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6825","nonce":2705956031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2705956031},{"type":"v1","addr":"172.21.15.26:6823","nonce":2705956031}]},"public_addr":"172.21.15.26:6819/2705956031","cluster_addr":"172.21.15.26:6821/2705956031","heartbeat_back_addr":"172.21.15.26:6825/2705956031","heartbeat_front_addr":"172.21.15.26:6823/2705956031","state":["exists","up"]},{"osd":7,"uuid":"9756cfaa-3ffb-42d6-9154-98f4d4aceb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6817","nonce":2969819310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6819","nonce":2969819310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6823","nonce":2969819310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2969819310},{"type":"v1","addr":"172.21.15.135:6821","nonce":2969819310}]},"public_addr":"172.21.15.135:6817/2969819310","cluster_addr":"172.21.15.135:6819/2969819310","heartbeat_back_addr":"172.21.15.135:6823/2969819310","heartbeat_front_addr":"172.21.15.135:6821/2969819310","state":["exists","up"]},{"osd":8,"uuid":"9ef2c367-8eae-4f63-9ea8-5af0a4129fd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6817","nonce":2605237371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6819","nonce":2605237371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6823","nonce":2605237371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2605237371},{"type":"v1","addr":"172.21.15.136:6821","nonce":2605237371}]},"public_addr":"172.21.15.136:6817/2605237371","cluster_addr":"172.21.15.136:6819/2605237371","heartbeat_back_addr":"172.21.15.136:6823/2605237371","heartbeat_front_addr":"172.21.15.136:6821/2605237371","state":["exists","up"]},{"osd":9,"uuid":"47d5b8cf-ab62-44a6-b713-84d52661c442","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6827","nonce":1456557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6829","nonce":1456557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6833","nonce":1456557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1456557},{"type":"v1","addr":"172.21.15.26:6831","nonce":1456557}]},"public_addr":"172.21.15.26:6827/1456557","cluster_addr":"172.21.15.26:6829/1456557","heartbeat_back_addr":"172.21.15.26:6833/1456557","heartbeat_front_addr":"172.21.15.26:6831/1456557","state":["exists","up"]},{"osd":10,"uuid":"aed06586-2329-4eb7-8d42-e8c06e20313f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6825","nonce":1309174884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6827","nonce":1309174884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6831","nonce":1309174884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1309174884},{"type":"v1","addr":"172.21.15.135:6829","nonce":1309174884}]},"public_addr":"172.21.15.135:6825/1309174884","cluster_addr":"172.21.15.135:6827/1309174884","heartbeat_back_addr":"172.21.15.135:6831/1309174884","heartbeat_front_addr":"172.21.15.135:6829/1309174884","state":["exists","up"]},{"osd":11,"uuid":"ca783667-d631-4e49-bf3a-5baf754f95fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6825","nonce":2902066544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6827","nonce":2902066544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6831","nonce":2902066544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2902066544},{"type":"v1","addr":"172.21.15.136:6829","nonce":2902066544}]},"public_addr":"172.21.15.136:6825/2902066544","cluster_addr":"172.21.15.136:6827/2902066544","heartbeat_back_addr":"172.21.15.136:6831/2902066544","heartbeat_front_addr":"172.21.15.136:6829/2902066544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:24.807543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:50.564516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:14.570808+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:49:48.376423+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:15.235612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:50:40.262620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:15.231475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:51:42.277428+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:08.390669+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:52:43.408993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:11.642100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:53:37.486251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3363905776":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/2900186702":"2024-06-14T06:46:28.109797+0000","172.21.15.26:6801/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/2776242437":"2024-06-14T06:45:51.299898+0000","172.21.15.26:6800/3558618334":"2024-06-14T06:46:28.109797+0000","172.21.15.26:0/106059480":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2539477633":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/3002519462":"2024-06-14T06:45:51.299898+0000","172.21.15.26:0/2082657826":"2024-06-14T06:46:28.109797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T07:26:59.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.139+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa88c0a2ca0 msgr2=0x7fa88c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:59.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.139+0000 7fa8ad200700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa88c0a2ca0 0x7fa88c0a5160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8199be0 tx=0x7fa898009260 comp rx=0 tx=0).stop 2024-06-13T07:26:59.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.139+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8a8101d50 msgr2=0x7fa8a819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.139+0000 7fa8ad200700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8a8101d50 0x7fa8a819ee20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa894005910 tx=0x7fa89400bab0 comp rx=0 tx=0).stop 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.140+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 shutdown_connections 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.140+0000 7fa8ad200700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fa88c0a2ca0 0x7fa88c0a5160 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.140+0000 7fa8ad200700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8a8101390 0x7fa8a819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.140+0000 7fa8ad200700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8a8107400 0x7fa8a8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.140+0000 7fa8ad200700 1 --2- 172.21.15.26:0/1451961187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8a8101d50 0x7fa8a819ee20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.140+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 >> 172.21.15.26:0/1451961187 conn(0x7fa8a8078340 msgr2=0x7fa8a8100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.140+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 shutdown_connections 2024-06-13T07:26:59.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:26:59.140+0000 7fa8ad200700 1 -- 172.21.15.26:0/1451961187 wait complete. 2024-06-13T07:26:59.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:26:59 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1451961187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:26:59.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:26:59 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1451961187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:26:59.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:26:59 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1451961187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:27:00.178 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs fail cephfs 2024-06-13T07:27:00.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:00 smithi136 ceph-mon[37229]: pgmap v1304: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:27:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:27:00.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:00 smithi026 ceph-mon[32041]: pgmap v1304: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:27:00.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:00 smithi135 ceph-mon[36276]: pgmap v1304: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-13T07:27:02.609 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:02 smithi026 ceph-mon[32041]: pgmap v1305: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T07:27:02.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:02 smithi136 ceph-mon[37229]: pgmap v1305: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T07:27:02.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:02 smithi135 ceph-mon[36276]: pgmap v1305: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T07:27:03.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 -- 172.21.15.26:0/1549771282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f260410ee90 msgr2=0x7f2604111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:03.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 --2- 172.21.15.26:0/1549771282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f260410ee90 0x7f2604111280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00b0a0 tx=0x7f25fc00b3b0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 -- 172.21.15.26:0/1549771282 shutdown_connections 2024-06-13T07:27:03.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 --2- 172.21.15.26:0/1549771282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26040746c0 0x7f2604074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 --2- 172.21.15.26:0/1549771282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f260410ee90 0x7f2604111280 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 --2- 172.21.15.26:0/1549771282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604074fe0 0x7f260410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 -- 172.21.15.26:0/1549771282 >> 172.21.15.26:0/1549771282 conn(0x7f26040795f0 msgr2=0x7f2604079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 -- 172.21.15.26:0/1549771282 shutdown_connections 2024-06-13T07:27:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.164+0000 7f260c200700 1 -- 172.21.15.26:0/1549771282 wait complete. 2024-06-13T07:27:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.166+0000 7f260c200700 1 Processor -- start 2024-06-13T07:27:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.166+0000 7f260c200700 1 -- start start 2024-06-13T07:27:03.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.166+0000 7f260c200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26040746c0 0x7f260419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:03.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.166+0000 7f260c200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604074fe0 0x7f260419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.166+0000 7f260c200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260410ee90 0x7f260419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f260c200700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2604114970 con 0x7f26040746c0 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f260c200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26041147f0 con 0x7f260410ee90 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f260c200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2604114af0 con 0x7f2604074fe0 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2609800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26040746c0 0x7f260419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2609800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26040746c0 0x7f260419ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43266/0 (socket says 172.21.15.26:43266) 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2609800700 1 -- 172.21.15.26:0/1234765395 learned_addr learned my addr 172.21.15.26:0/1234765395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2608e00700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604074fe0 0x7f260419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f260a200700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260410ee90 0x7f260419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:03.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2609800700 1 -- 172.21.15.26:0/1234765395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604074fe0 msgr2=0x7f260419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:03.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2609800700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604074fe0 0x7f260419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2609800700 1 -- 172.21.15.26:0/1234765395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260410ee90 msgr2=0x7f260419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:03.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2609800700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260410ee90 0x7f260419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2609800700 1 -- 172.21.15.26:0/1234765395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260419fd40 con 0x7f26040746c0 2024-06-13T07:27:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.167+0000 7f2608e00700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604074fe0 0x7f260419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:27:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.168+0000 7f2609800700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26040746c0 0x7f260419ad80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e9a0 tx=0x7f25f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.168+0000 7f2602000700 1 -- 172.21.15.26:0/1234765395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f800ca10 con 0x7f26040746c0 2024-06-13T07:27:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.168+0000 7f2602000700 1 -- 172.21.15.26:0/1234765395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f800cbb0 con 0x7f26040746c0 2024-06-13T07:27:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.168+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26041a0030 con 0x7f26040746c0 2024-06-13T07:27:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.168+0000 7f2602000700 1 -- 172.21.15.26:0/1234765395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8018680 con 0x7f26040746c0 2024-06-13T07:27:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.168+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a78b0 con 0x7f26040746c0 2024-06-13T07:27:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.169+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2604069180 con 0x7f26040746c0 2024-06-13T07:27:03.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.172+0000 7f2602000700 1 -- 172.21.15.26:0/1234765395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f25f8010430 con 0x7f26040746c0 2024-06-13T07:27:03.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.174+0000 7f2602000700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25ec0a2c20 0x7f25ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:03.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.174+0000 7f2602000700 1 -- 172.21.15.26:0/1234765395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f25f80cfb80 con 0x7f26040746c0 2024-06-13T07:27:03.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.175+0000 7f2608e00700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25ec0a2c20 0x7f25ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:03.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.175+0000 7f2608e00700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25ec0a2c20 0x7f25ec0a50e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2604110ac0 tx=0x7f25f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:03.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.177+0000 7f2602000700 1 -- 172.21.15.26:0/1234765395 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f25f809c690 con 0x7f26040746c0 2024-06-13T07:27:03.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.458+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f26041a7ab0 con 0x7f26040746c0 2024-06-13T07:27:03.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.462+0000 7f2602000700 1 -- 172.21.15.26:0/1234765395 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) ==== 161+0+0 (secure 0 0 0) 0x7f25f809bf90 con 0x7f26040746c0 2024-06-13T07:27:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25ec0a2c20 msgr2=0x7f25ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25ec0a2c20 0x7f25ec0a50e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2604110ac0 tx=0x7f25f0009260 comp rx=0 tx=0).stop 2024-06-13T07:27:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26040746c0 msgr2=0x7f260419ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26040746c0 0x7f260419ad80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e9a0 tx=0x7f25f800ecb0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 shutdown_connections 2024-06-13T07:27:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25ec0a2c20 0x7f25ec0a50e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260410ee90 0x7f260419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26040746c0 0x7f260419ad80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 --2- 172.21.15.26:0/1234765395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604074fe0 0x7f260419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 >> 172.21.15.26:0/1234765395 conn(0x7f26040795f0 msgr2=0x7f26040ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 shutdown_connections 2024-06-13T07:27:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:03.466+0000 7f260c200700 1 -- 172.21.15.26:0/1234765395 wait complete. 2024-06-13T07:27:03.468 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-13T07:27:03.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:27:03.459+0000 7fc606600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-13T07:27:03.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c[68243]: ignoring --setuser ceph since I am not root 2024-06-13T07:27:03.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c[68243]: ignoring --setgroup ceph since I am not root 2024-06-13T07:27:03.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-mds[68247]: -- 172.21.15.136:0/3707664028 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618739dda40 con 0x5618739b1000 2024-06-13T07:27:03.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c[68243]: starting mds.c at 2024-06-13T07:27:03.877 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f[67939]: ignoring --setuser ceph since I am not root 2024-06-13T07:27:03.877 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f[67939]: ignoring --setgroup ceph since I am not root 2024-06-13T07:27:03.877 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f[67939]: starting mds.f at 2024-06-13T07:27:03.878 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: ignoring --setuser ceph since I am not root 2024-06-13T07:27:03.878 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: ignoring --setgroup ceph since I am not root 2024-06-13T07:27:03.878 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-mds[68551]: -- 172.21.15.136:0/1586249353 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b106a73a40 con 0x55b106a47000 2024-06-13T07:27:03.878 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:27:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: starting mds.i at 2024-06-13T07:27:04.033 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:27:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: ignoring --setuser ceph since I am not root 2024-06-13T07:27:04.033 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:27:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: ignoring --setgroup ceph since I am not root 2024-06-13T07:27:04.033 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:27:03 smithi135 ceph-mds[66562]: -- 172.21.15.135:0/1679731794 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd91be9a40 con 0x55bd91bbcc00 2024-06-13T07:27:04.033 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:27:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: starting mds.b at 2024-06-13T07:27:04.253 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a[74382]: ignoring --setuser ceph since I am not root 2024-06-13T07:27:04.253 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a[74382]: ignoring --setgroup ceph since I am not root 2024-06-13T07:27:04.254 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-mds[74386]: -- 172.21.15.26:0/3142771178 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555da85e7a40 con 0x555da85bac00 2024-06-13T07:27:04.254 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a[74382]: starting mds.a at 2024-06-13T07:27:04.254 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: ignoring --setuser ceph since I am not root 2024-06-13T07:27:04.254 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: ignoring --setgroup ceph since I am not root 2024-06-13T07:27:04.254 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-mds[73989]: -- 172.21.15.26:0/1058977512 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a78941a40 con 0x563a78915000 2024-06-13T07:27:04.254 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:27:03 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: starting mds.g at 2024-06-13T07:27:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:04 smithi136 ceph-mon[37229]: pgmap v1306: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:27:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:04 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1234765395' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-13T07:27:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:04 smithi136 ceph-mon[37229]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-13T07:27:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:04 smithi136 ceph-mon[37229]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-13T07:27:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:04 smithi136 ceph-mon[37229]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-06-13T07:27:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:04 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1234765395' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-13T07:27:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:04 smithi136 ceph-mon[37229]: osdmap e100: 12 total, 12 up, 12 in 2024-06-13T07:27:04.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:04 smithi136 ceph-mon[37229]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-13T07:27:04.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:04 smithi026 ceph-mon[32041]: pgmap v1306: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:27:04.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:04 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1234765395' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-13T07:27:04.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:04 smithi026 ceph-mon[32041]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-13T07:27:04.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:04 smithi026 ceph-mon[32041]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-13T07:27:04.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:04 smithi026 ceph-mon[32041]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-06-13T07:27:04.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:04 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1234765395' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-13T07:27:04.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:04 smithi026 ceph-mon[32041]: osdmap e100: 12 total, 12 up, 12 in 2024-06-13T07:27:04.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:04 smithi026 ceph-mon[32041]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-13T07:27:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:04 smithi135 ceph-mon[36276]: pgmap v1306: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T07:27:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:04 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1234765395' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-13T07:27:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:04 smithi135 ceph-mon[36276]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-13T07:27:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:04 smithi135 ceph-mon[36276]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-13T07:27:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:04 smithi135 ceph-mon[36276]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-06-13T07:27:04.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:04 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1234765395' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-13T07:27:04.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:04 smithi135 ceph-mon[36276]: osdmap e100: 12 total, 12 up, 12 in 2024-06-13T07:27:04.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:04 smithi135 ceph-mon[36276]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-13T07:27:04.867 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-13T07:27:05.338 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:27:05.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.135:6838/203909243,v1:172.21.15.135:6839/203909243] up:boot 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6834/2407647350,v1:172.21.15.136:6835/2407647350] up:boot 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6837/3695099893,v1:172.21.15.136:6839/3695099893] up:boot 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.136:6836/997989082,v1:172.21.15.136:6838/997989082] up:boot 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.26:6839/1376202066,v1:172.21.15.26:6841/1376202066] up:boot 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: mds.? [v2:172.21.15.26:6838/2681599052,v1:172.21.15.26:6840/2681599052] up:boot 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T07:27:05.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:05 smithi026 ceph-mon[32041]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T07:27:05.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.135:6838/203909243,v1:172.21.15.135:6839/203909243] up:boot 2024-06-13T07:27:05.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6834/2407647350,v1:172.21.15.136:6835/2407647350] up:boot 2024-06-13T07:27:05.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6837/3695099893,v1:172.21.15.136:6839/3695099893] up:boot 2024-06-13T07:27:05.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.136:6836/997989082,v1:172.21.15.136:6838/997989082] up:boot 2024-06-13T07:27:05.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.26:6839/1376202066,v1:172.21.15.26:6841/1376202066] up:boot 2024-06-13T07:27:05.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: mds.? [v2:172.21.15.26:6838/2681599052,v1:172.21.15.26:6840/2681599052] up:boot 2024-06-13T07:27:05.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-13T07:27:05.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T07:27:05.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T07:27:05.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T07:27:05.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T07:27:05.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T07:27:05.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:05 smithi135 ceph-mon[36276]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.135:6838/203909243,v1:172.21.15.135:6839/203909243] up:boot 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6834/2407647350,v1:172.21.15.136:6835/2407647350] up:boot 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6837/3695099893,v1:172.21.15.136:6839/3695099893] up:boot 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.136:6836/997989082,v1:172.21.15.136:6838/997989082] up:boot 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.26:6839/1376202066,v1:172.21.15.26:6841/1376202066] up:boot 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: mds.? [v2:172.21.15.26:6838/2681599052,v1:172.21.15.26:6840/2681599052] up:boot 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T07:27:05.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T07:27:05.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T07:27:05.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T07:27:05.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:05 smithi136 ceph-mon[37229]: from='mgr.14152 172.21.15.26:0/4015203814' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T07:27:06.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:06 smithi026 ceph-mon[32041]: pgmap v1308: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T07:27:06.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:06 smithi135 ceph-mon[36276]: pgmap v1308: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T07:27:06.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:06 smithi136 ceph-mon[37229]: pgmap v1308: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T07:27:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 -- 172.21.15.26:0/2268519645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 msgr2=0x7f2574100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 --2- 172.21.15.26:0/2268519645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 0x7f2574100f00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f255c009990 tx=0x7f255c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:27:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 -- 172.21.15.26:0/2268519645 shutdown_connections 2024-06-13T07:27:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 --2- 172.21.15.26:0/2268519645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25741086f0 0x7f2574108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 --2- 172.21.15.26:0/2268519645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2574101440 0x7f257410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 --2- 172.21.15.26:0/2268519645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 0x7f2574100f00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 -- 172.21.15.26:0/2268519645 >> 172.21.15.26:0/2268519645 conn(0x7f2574077a00 msgr2=0x7f25740fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 -- 172.21.15.26:0/2268519645 shutdown_connections 2024-06-13T07:27:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.738+0000 7f2579e00700 1 -- 172.21.15.26:0/2268519645 wait complete. 2024-06-13T07:27:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.740+0000 7f2579e00700 1 Processor -- start 2024-06-13T07:27:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.740+0000 7f2579e00700 1 -- start start 2024-06-13T07:27:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.740+0000 7f2579e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 0x7f257419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.740+0000 7f2579e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2574101440 0x7f257419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.740+0000 7f2579e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25741086f0 0x7f257419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.740+0000 7f2579e00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2574112a00 con 0x7f2574101440 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.740+0000 7f2579e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2574112880 con 0x7f25741086f0 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.740+0000 7f2579e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2574112b80 con 0x7f2574100a80 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f256a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2574101440 0x7f257419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f2573400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25741086f0 0x7f257419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f256a000700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2574101440 0x7f257419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43338/0 (socket says 172.21.15.26:43338) 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f256a000700 1 -- 172.21.15.26:0/2097554967 learned_addr learned my addr 172.21.15.26:0/2097554967 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:27:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f2572a00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 0x7f257419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f2572a00700 1 -- 172.21.15.26:0/2097554967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25741086f0 msgr2=0x7f257419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f2572a00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25741086f0 0x7f257419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f2572a00700 1 -- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2574101440 msgr2=0x7f257419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f2572a00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2574101440 0x7f257419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.741+0000 7f2572a00700 1 -- 172.21.15.26:0/2097554967 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257419fff0 con 0x7f2574100a80 2024-06-13T07:27:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.742+0000 7f2572a00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 0x7f257419afe0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9c0 tx=0x7f256400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.742+0000 7f2578a00700 1 -- 172.21.15.26:0/2097554967 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564004230 con 0x7f2574100a80 2024-06-13T07:27:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.742+0000 7f2578a00700 1 -- 172.21.15.26:0/2097554967 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25640043d0 con 0x7f2574100a80 2024-06-13T07:27:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.742+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25741a02e0 con 0x7f2574100a80 2024-06-13T07:27:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.742+0000 7f2578a00700 1 -- 172.21.15.26:0/2097554967 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564003e30 con 0x7f2574100a80 2024-06-13T07:27:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.742+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25741a7b60 con 0x7f2574100a80 2024-06-13T07:27:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.743+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2574069180 con 0x7f2574100a80 2024-06-13T07:27:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.746+0000 7f2578a00700 1 -- 172.21.15.26:0/2097554967 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f256400c7c0 con 0x7f2574100a80 2024-06-13T07:27:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.747+0000 7f2578a00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25580a2ca0 0x7f25580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.747+0000 7f2578a00700 1 -- 172.21.15.26:0/2097554967 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f25640cf770 con 0x7f2574100a80 2024-06-13T07:27:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.747+0000 7f256a000700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25580a2ca0 0x7f25580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.748+0000 7f256a000700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25580a2ca0 0x7f25580a5160 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f255c009750 tx=0x7f255c00b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:07.752+0000 7f2578a00700 1 -- 172.21.15.26:0/2097554967 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f256409c180 con 0x7f2574100a80 2024-06-13T07:27:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.032+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2574069f50 con 0x7f2574100a80 2024-06-13T07:27:08.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.490+0000 7f2578a00700 1 -- 172.21.15.26:0/2097554967 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) ==== 104+0+0 (secure 0 0 0) 0x7f256409ba80 con 0x7f2574100a80 2024-06-13T07:27:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.494+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25580a2ca0 msgr2=0x7f25580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.494+0000 7f2579e00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25580a2ca0 0x7f25580a5160 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f255c009750 tx=0x7f255c00b2a0 comp rx=0 tx=0).stop 2024-06-13T07:27:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.494+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 msgr2=0x7f257419afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.494+0000 7f2579e00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 0x7f257419afe0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9c0 tx=0x7f256400ecd0 comp rx=0 tx=0).stop 2024-06-13T07:27:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.494+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 shutdown_connections 2024-06-13T07:27:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.494+0000 7f2579e00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f25580a2ca0 0x7f25580a5160 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.495+0000 7f2579e00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25741086f0 0x7f257419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.495+0000 7f2579e00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2574100a80 0x7f257419afe0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.495+0000 7f2579e00700 1 --2- 172.21.15.26:0/2097554967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2574101440 0x7f257419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.495+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 >> 172.21.15.26:0/2097554967 conn(0x7f2574077a00 msgr2=0x7f25740fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.495+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 shutdown_connections 2024-06-13T07:27:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:08.495+0000 7f2579e00700 1 -- 172.21.15.26:0/2097554967 wait complete. 2024-06-13T07:27:08.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:08 smithi026 ceph-mon[32041]: pgmap v1309: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T07:27:08.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:08 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2097554967' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:27:08.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:08 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:27:08.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:08 smithi135 ceph-mon[36276]: pgmap v1309: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T07:27:08.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:08 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2097554967' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:27:08.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:08 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:27:08.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:08 smithi136 ceph-mon[37229]: pgmap v1309: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T07:27:08.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:08 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2097554967' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:27:08.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:08 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:27:09.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-13T07:27:09.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:09 smithi026 ceph-mon[32041]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-13T07:27:09.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:09 smithi026 ceph-mon[32041]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-13T07:27:09.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:09 smithi026 ceph-mon[32041]: Cluster is now healthy 2024-06-13T07:27:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:09 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-13T07:27:09.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:09 smithi026 ceph-mon[32041]: fsmap 12 up:standby 2024-06-13T07:27:09.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:09 smithi135 ceph-mon[36276]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-13T07:27:09.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:09 smithi135 ceph-mon[36276]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-13T07:27:09.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:09 smithi135 ceph-mon[36276]: Cluster is now healthy 2024-06-13T07:27:09.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:09 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-13T07:27:09.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:09 smithi135 ceph-mon[36276]: fsmap 12 up:standby 2024-06-13T07:27:09.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:09 smithi136 ceph-mon[37229]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-13T07:27:09.892 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:09 smithi136 ceph-mon[37229]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-13T07:27:09.892 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:09 smithi136 ceph-mon[37229]: Cluster is now healthy 2024-06-13T07:27:09.892 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:09 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-13T07:27:09.892 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:09 smithi136 ceph-mon[37229]: fsmap 12 up:standby 2024-06-13T07:27:10.149 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:27:10.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:10 smithi026 ceph-mon[32041]: pgmap v1310: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-13T07:27:10.784 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:10 smithi135 ceph-mon[36276]: pgmap v1310: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-13T07:27:10.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:10 smithi136 ceph-mon[37229]: pgmap v1310: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-13T07:27:12.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:12 smithi026 ceph-mon[32041]: pgmap v1311: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-06-13T07:27:12.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:12 smithi135 ceph-mon[36276]: pgmap v1311: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-06-13T07:27:12.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:12 smithi136 ceph-mon[37229]: pgmap v1311: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 -- 172.21.15.26:0/1392358205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf8105090 msgr2=0x7faaf81115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 --2- 172.21.15.26:0/1392358205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf8105090 0x7faaf81115a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faaf000b0a0 tx=0x7faaf000b3b0 comp rx=0 tx=0).stop 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 -- 172.21.15.26:0/1392358205 shutdown_connections 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 --2- 172.21.15.26:0/1392358205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf8105090 0x7faaf81115a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 --2- 172.21.15.26:0/1392358205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf810a6d0 0x7faaf810aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 --2- 172.21.15.26:0/1392358205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf81046d0 0x7faaf8104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 -- 172.21.15.26:0/1392358205 >> 172.21.15.26:0/1392358205 conn(0x7faaf81001c0 msgr2=0x7faaf81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 -- 172.21.15.26:0/1392358205 shutdown_connections 2024-06-13T07:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.879+0000 7faafd400700 1 -- 172.21.15.26:0/1392358205 wait complete. 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.880+0000 7faafd400700 1 Processor -- start 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.880+0000 7faafd400700 1 -- start start 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faafd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf81046d0 0x7faaf819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faafd400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf8105090 0x7faaf819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faafd400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf810a6d0 0x7faaf819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faafd400700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faaf8114ab0 con 0x7faaf810a6d0 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faafd400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faaf8114930 con 0x7faaf81046d0 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faafd400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faaf8114c30 con 0x7faaf8105090 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf7200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf810a6d0 0x7faaf819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaefe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf8105090 0x7faaf819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf7200700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf810a6d0 0x7faaf819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56530/0 (socket says 172.21.15.26:56530) 2024-06-13T07:27:12.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf7200700 1 -- 172.21.15.26:0/2742452447 learned_addr learned my addr 172.21.15.26:0/2742452447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf6800700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf81046d0 0x7faaf819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf7200700 1 -- 172.21.15.26:0/2742452447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf8105090 msgr2=0x7faaf819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf7200700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf8105090 0x7faaf819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf7200700 1 -- 172.21.15.26:0/2742452447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf81046d0 msgr2=0x7faaf819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf7200700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf81046d0 0x7faaf819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf7200700 1 -- 172.21.15.26:0/2742452447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf819fe70 con 0x7faaf810a6d0 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaefe00700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf8105090 0x7faaf819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.881+0000 7faaf6800700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf81046d0 0x7faaf819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.882+0000 7faaf7200700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf810a6d0 0x7faaf819f730 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faaf000b070 tx=0x7faaf0004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.882+0000 7faaef400700 1 -- 172.21.15.26:0/2742452447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0021070 con 0x7faaf810a6d0 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.882+0000 7faaef400700 1 -- 172.21.15.26:0/2742452447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf0003c00 con 0x7faaf810a6d0 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.882+0000 7faaef400700 1 -- 172.21.15.26:0/2742452447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0004740 con 0x7faaf810a6d0 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.882+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf81a0100 con 0x7faaf810a6d0 2024-06-13T07:27:12.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.882+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf81a79e0 con 0x7faaf810a6d0 2024-06-13T07:27:12.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.884+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf8069180 con 0x7faaf810a6d0 2024-06-13T07:27:12.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.886+0000 7faaef400700 1 -- 172.21.15.26:0/2742452447 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faaf0007980 con 0x7faaf810a6d0 2024-06-13T07:27:12.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.888+0000 7faaef400700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faadc0a2ca0 0x7faadc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:12.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.888+0000 7faaef400700 1 -- 172.21.15.26:0/2742452447 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7faaf000f020 con 0x7faaf810a6d0 2024-06-13T07:27:12.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.888+0000 7faaf6800700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faadc0a2ca0 0x7faadc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:12.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.889+0000 7faaf6800700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faadc0a2ca0 0x7faadc0a5160 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faae8009fd0 tx=0x7faae8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:12.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:12.891+0000 7faaef400700 1 -- 172.21.15.26:0/2742452447 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faaf002a020 con 0x7faaf810a6d0 2024-06-13T07:27:13.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.119+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faaf819c1c0 con 0x7faaf810a6d0 2024-06-13T07:27:13.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.531+0000 7faaef400700 1 -- 172.21.15.26:0/2742452447 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7faaf819c1c0 con 0x7faaf810a6d0 2024-06-13T07:27:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faadc0a2ca0 msgr2=0x7faadc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faadc0a2ca0 0x7faadc0a5160 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faae8009fd0 tx=0x7faae8009260 comp rx=0 tx=0).stop 2024-06-13T07:27:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf810a6d0 msgr2=0x7faaf819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf810a6d0 0x7faaf819f730 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faaf000b070 tx=0x7faaf0004c60 comp rx=0 tx=0).stop 2024-06-13T07:27:13.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 shutdown_connections 2024-06-13T07:27:13.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7faadc0a2ca0 0x7faadc0a5160 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf81046d0 0x7faaf819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf810a6d0 0x7faaf819f730 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 --2- 172.21.15.26:0/2742452447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faaf8105090 0x7faaf819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.538+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 >> 172.21.15.26:0/2742452447 conn(0x7faaf81001c0 msgr2=0x7faaf8100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.539+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 shutdown_connections 2024-06-13T07:27:13.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:13.540+0000 7faafd400700 1 -- 172.21.15.26:0/2742452447 wait complete. 2024-06-13T07:27:13.548 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-06-13T07:27:13.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:13 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2742452447' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:13.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:13 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2742452447' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:14.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:13 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2742452447' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:14.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-13T07:27:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:14 smithi026 ceph-mon[32041]: pgmap v1312: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-06-13T07:27:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:14 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/2742452447' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:14.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:14 smithi026 ceph-mon[32041]: osdmap e101: 12 total, 12 up, 12 in 2024-06-13T07:27:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:14 smithi135 ceph-mon[36276]: pgmap v1312: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-06-13T07:27:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:14 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/2742452447' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:14.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:14 smithi135 ceph-mon[36276]: osdmap e101: 12 total, 12 up, 12 in 2024-06-13T07:27:14.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:14 smithi136 ceph-mon[37229]: pgmap v1312: 129 pgs: 129 active+clean; 465 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-06-13T07:27:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:14 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/2742452447' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:14.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:14 smithi136 ceph-mon[37229]: osdmap e101: 12 total, 12 up, 12 in 2024-06-13T07:27:14.988 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:27:15.737 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:15 smithi026 ceph-mon[32041]: pgmap v1314: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:15.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:15 smithi136 ceph-mon[37229]: pgmap v1314: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:16.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:15 smithi135 ceph-mon[36276]: pgmap v1314: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 -- 172.21.15.26:0/3532570140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c0746c0 msgr2=0x7f013c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 --2- 172.21.15.26:0/3532570140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c0746c0 0x7f013c074aa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f012c009990 tx=0x7f012c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:27:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 -- 172.21.15.26:0/3532570140 shutdown_connections 2024-06-13T07:27:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 --2- 172.21.15.26:0/3532570140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013c10ee90 0x7f013c111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 --2- 172.21.15.26:0/3532570140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c0746c0 0x7f013c074aa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 --2- 172.21.15.26:0/3532570140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f013c074fe0 0x7f013c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 -- 172.21.15.26:0/3532570140 >> 172.21.15.26:0/3532570140 conn(0x7f013c0795f0 msgr2=0x7f013c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 -- 172.21.15.26:0/3532570140 shutdown_connections 2024-06-13T07:27:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.628+0000 7f0140c00700 1 -- 172.21.15.26:0/3532570140 wait complete. 2024-06-13T07:27:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.629+0000 7f0140c00700 1 Processor -- start 2024-06-13T07:27:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0140c00700 1 -- start start 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0140c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f013c0746c0 0x7f013c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0140c00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c074fe0 0x7f013c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0140c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013c10ee90 0x7f013c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0140c00700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f013c114a10 con 0x7f013c074fe0 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0140c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f013c114890 con 0x7f013c0746c0 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0140c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f013c114b90 con 0x7f013c10ee90 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0131400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c074fe0 0x7f013c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0131400700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c074fe0 0x7f013c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56534/0 (socket says 172.21.15.26:56534) 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0131400700 1 -- 172.21.15.26:0/1024722939 learned_addr learned my addr 172.21.15.26:0/1024722939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f0139e00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f013c0746c0 0x7f013c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.630+0000 7f013a800700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013c10ee90 0x7f013c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.631+0000 7f0131400700 1 -- 172.21.15.26:0/1024722939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013c10ee90 msgr2=0x7f013c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.631+0000 7f0131400700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013c10ee90 0x7f013c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.631+0000 7f0131400700 1 -- 172.21.15.26:0/1024722939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f013c0746c0 msgr2=0x7f013c19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.631+0000 7f0131400700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f013c0746c0 0x7f013c19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.631+0000 7f0131400700 1 -- 172.21.15.26:0/1024722939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c19fe40 con 0x7f013c074fe0 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.631+0000 7f0139e00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f013c0746c0 0x7f013c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.631+0000 7f013a800700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013c10ee90 0x7f013c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.631+0000 7f0131400700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c074fe0 0x7f013c19b3c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0124007f10 tx=0x7f012400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.632+0000 7f0133400700 1 -- 172.21.15.26:0/1024722939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124004db0 con 0x7f013c074fe0 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.632+0000 7f0133400700 1 -- 172.21.15.26:0/1024722939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0124003a00 con 0x7f013c074fe0 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.632+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c1a0130 con 0x7f013c074fe0 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.632+0000 7f0133400700 1 -- 172.21.15.26:0/1024722939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124004650 con 0x7f013c074fe0 2024-06-13T07:27:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.632+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a79b0 con 0x7f013c074fe0 2024-06-13T07:27:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.633+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c069180 con 0x7f013c074fe0 2024-06-13T07:27:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.635+0000 7f0133400700 1 -- 172.21.15.26:0/1024722939 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f012401c020 con 0x7f013c074fe0 2024-06-13T07:27:17.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.636+0000 7f0133400700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f01200a2c20 0x7f01200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:17.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.637+0000 7f0133400700 1 -- 172.21.15.26:0/1024722939 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8941+0+0 (secure 0 0 0) 0x7f01240cebb0 con 0x7f013c074fe0 2024-06-13T07:27:17.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.637+0000 7f0139e00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f01200a2c20 0x7f01200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:17.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.637+0000 7f0139e00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f01200a2c20 0x7f01200a50e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f012c006010 tx=0x7f012c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:17.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.641+0000 7f0133400700 1 -- 172.21.15.26:0/1024722939 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0124010070 con 0x7f013c074fe0 2024-06-13T07:27:17.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:17.867+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f013c02dca0 con 0x7f013c074fe0 2024-06-13T07:27:18.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.334+0000 7f0133400700 1 -- 172.21.15.26:0/1024722939 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f012409b7f0 con 0x7f013c074fe0 2024-06-13T07:27:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.340+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f01200a2c20 msgr2=0x7f01200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.340+0000 7f0140c00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f01200a2c20 0x7f01200a50e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f012c006010 tx=0x7f012c00b540 comp rx=0 tx=0).stop 2024-06-13T07:27:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.340+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c074fe0 msgr2=0x7f013c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.340+0000 7f0140c00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c074fe0 0x7f013c19b3c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0124007f10 tx=0x7f012400d3b0 comp rx=0 tx=0).stop 2024-06-13T07:27:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.341+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 shutdown_connections 2024-06-13T07:27:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.341+0000 7f0140c00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7f01200a2c20 0x7f01200a50e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.341+0000 7f0140c00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f013c0746c0 0x7f013c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.341+0000 7f0140c00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013c10ee90 0x7f013c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.341+0000 7f0140c00700 1 --2- 172.21.15.26:0/1024722939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f013c074fe0 0x7f013c19b3c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.341+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 >> 172.21.15.26:0/1024722939 conn(0x7f013c0795f0 msgr2=0x7f013c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.342+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 shutdown_connections 2024-06-13T07:27:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:18.342+0000 7f0140c00700 1 -- 172.21.15.26:0/1024722939 wait complete. 2024-06-13T07:27:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-06-13T07:27:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:18 smithi136 ceph-mon[37229]: pgmap v1315: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:18.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:18 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1024722939' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:18.736 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:18 smithi135 ceph-mon[36276]: pgmap v1315: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:18.736 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:18 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1024722939' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:18 smithi026 ceph-mon[32041]: pgmap v1315: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:18.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:18 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1024722939' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:19.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 559474f0-2950-11ef-bc9c-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-13T07:27:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:19 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1024722939' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:19.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:19 smithi026 ceph-mon[32041]: osdmap e102: 12 total, 12 up, 12 in 2024-06-13T07:27:19.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:19 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1024722939' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:19.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:19 smithi136 ceph-mon[37229]: osdmap e102: 12 total, 12 up, 12 in 2024-06-13T07:27:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:27:19.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:19 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1024722939' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:19.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:19 smithi135 ceph-mon[36276]: osdmap e102: 12 total, 12 up, 12 in 2024-06-13T07:27:20.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:20 smithi026 ceph-mon[32041]: pgmap v1317: 33 pgs: 33 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:20.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:20 smithi136 ceph-mon[37229]: pgmap v1317: 33 pgs: 33 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:20.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:20 smithi135 ceph-mon[36276]: pgmap v1317: 33 pgs: 33 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:22.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 -- 172.21.15.26:0/1503420935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944108ac0 msgr2=0x7fd944108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:22.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 --2- 172.21.15.26:0/1503420935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944108ac0 0x7fd944108f40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd9300098f0 tx=0x7fd930009c00 comp rx=0 tx=0).stop 2024-06-13T07:27:22.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 -- 172.21.15.26:0/1503420935 shutdown_connections 2024-06-13T07:27:22.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 --2- 172.21.15.26:0/1503420935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd944109480 0x7fd944115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:22.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 --2- 172.21.15.26:0/1503420935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94410eac0 0x7fd94410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:22.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 --2- 172.21.15.26:0/1503420935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944108ac0 0x7fd944108f40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:22.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 -- 172.21.15.26:0/1503420935 >> 172.21.15.26:0/1503420935 conn(0x7fd944075c60 msgr2=0x7fd9441072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:22.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 -- 172.21.15.26:0/1503420935 shutdown_connections 2024-06-13T07:27:22.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.333+0000 7fd94b800700 1 -- 172.21.15.26:0/1503420935 wait complete. 2024-06-13T07:27:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.335+0000 7fd94b800700 1 Processor -- start 2024-06-13T07:27:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.335+0000 7fd94b800700 1 -- start start 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd94b800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd944108ac0 0x7fd94419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd94b800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944109480 0x7fd94419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd94b800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd94b800700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd944118e60 con 0x7fd944108ac0 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd94b800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd944118ce0 con 0x7fd944109480 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd94b800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd944118fe0 con 0x7fd94410eac0 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd948e00700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd944108ac0 0x7fd94419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd943e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944109480 0x7fd94419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd943e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944109480 0x7fd94419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.26:34524/0 (socket says 172.21.15.26:34524) 2024-06-13T07:27:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd949800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.336+0000 7fd943e00700 1 -- 172.21.15.26:0/1018384997 learned_addr learned my addr 172.21.15.26:0/1018384997 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-06-13T07:27:22.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd943e00700 1 -- 172.21.15.26:0/1018384997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd94410eac0 msgr2=0x7fd9441a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:22.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd943e00700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:22.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd943e00700 1 -- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd944108ac0 msgr2=0x7fd94419f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd943e00700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd944108ac0 0x7fd94419f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd943e00700 1 -- 172.21.15.26:0/1018384997 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9441a42b0 con 0x7fd944109480 2024-06-13T07:27:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd948e00700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd944108ac0 0x7fd94419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:27:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd943e00700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944109480 0x7fd94419f830 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd930005950 tx=0x7fd93000fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd941600700 1 -- 172.21.15.26:0/1018384997 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930005140 con 0x7fd944109480 2024-06-13T07:27:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.337+0000 7fd941600700 1 -- 172.21.15.26:0/1018384997 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93000b560 con 0x7fd944109480 2024-06-13T07:27:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.338+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9441a4540 con 0x7fd944109480 2024-06-13T07:27:22.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.338+0000 7fd941600700 1 -- 172.21.15.26:0/1018384997 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930003ae0 con 0x7fd944109480 2024-06-13T07:27:22.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.338+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9441abe20 con 0x7fd944109480 2024-06-13T07:27:22.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.339+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd944069180 con 0x7fd944109480 2024-06-13T07:27:22.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.341+0000 7fd941600700 1 -- 172.21.15.26:0/1018384997 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd93000ba80 con 0x7fd944109480 2024-06-13T07:27:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.343+0000 7fd941600700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd92c0a2ca0 0x7fd92c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:27:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.343+0000 7fd941600700 1 -- 172.21.15.26:0/1018384997 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8324+0+0 (secure 0 0 0) 0x7fd9300d0160 con 0x7fd944109480 2024-06-13T07:27:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.343+0000 7fd948e00700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd92c0a2ca0 0x7fd92c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:27:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.344+0000 7fd948e00700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd92c0a2ca0 0x7fd92c0a5160 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd938005fd0 tx=0x7fd938009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:27:22.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.345+0000 7fd941600700 1 -- 172.21.15.26:0/1018384997 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd93005e850 con 0x7fd944109480 2024-06-13T07:27:22.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:22 smithi026 ceph-mon[32041]: pgmap v1318: 33 pgs: 33 active+clean; 577 KiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:22.580+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd94402dca0 con 0x7fd944109480 2024-06-13T07:27:22.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:22 smithi136 ceph-mon[37229]: pgmap v1318: 33 pgs: 33 active+clean; 577 KiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:22.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:22 smithi135 ceph-mon[36276]: pgmap v1318: 33 pgs: 33 active+clean; 577 KiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.365+0000 7fd941600700 1 -- 172.21.15.26:0/1018384997 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v103) ==== 178+0+0 (secure 0 0 0) 0x7fd930013030 con 0x7fd944109480 2024-06-13T07:27:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd92c0a2ca0 msgr2=0x7fd92c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd92c0a2ca0 0x7fd92c0a5160 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd938005fd0 tx=0x7fd938009310 comp rx=0 tx=0).stop 2024-06-13T07:27:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944109480 msgr2=0x7fd94419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:27:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944109480 0x7fd94419f830 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd930005950 tx=0x7fd93000fb20 comp rx=0 tx=0).stop 2024-06-13T07:27:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 shutdown_connections 2024-06-13T07:27:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:6800/662324914,v1:172.21.15.26:6801/662324914] conn(0x7fd92c0a2ca0 0x7fd92c0a5160 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd944108ac0 0x7fd94419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 --2- 172.21.15.26:0/1018384997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd944109480 0x7fd94419f830 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:27:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 >> 172.21.15.26:0/1018384997 conn(0x7fd944075c60 msgr2=0x7fd944076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:27:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.369+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 shutdown_connections 2024-06-13T07:27:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-06-13T07:27:23.370+0000 7fd94b800700 1 -- 172.21.15.26:0/1018384997 wait complete. 2024-06-13T07:27:23.371 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data_ec' removed 2024-06-13T07:27:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:23 smithi136 ceph-mon[37229]: from='client.? 172.21.15.26:0/1018384997' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:23.627 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:23 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:23 smithi026 ceph-mon[32041]: from='client.? 172.21.15.26:0/1018384997' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:23.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:23 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:23.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:23 smithi135 ceph-mon[36276]: from='client.? 172.21.15.26:0/1018384997' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:23.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:23 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:27:24.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:27:24.433 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:27:24.475 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:27:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:24 smithi026 ceph-mon[32041]: pgmap v1319: 33 pgs: 33 active+clean; 577 KiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:24 smithi026 ceph-mon[32041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:24.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:24 smithi026 ceph-mon[32041]: osdmap e103: 12 total, 12 up, 12 in 2024-06-13T07:27:24.535 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-13T07:27:24.535 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-13T07:27:24.601 DEBUG:teuthology.orchestra.run.smithi135:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-13T07:27:24.616 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-13T07:27:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:24 smithi136 ceph-mon[37229]: pgmap v1319: 33 pgs: 33 active+clean; 577 KiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:24 smithi136 ceph-mon[37229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:24.626 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:24 smithi136 ceph-mon[37229]: osdmap e103: 12 total, 12 up, 12 in 2024-06-13T07:27:24.630 INFO:tasks.cephadm:Stopping all daemons... 2024-06-13T07:27:24.631 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-13T07:27:24.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a 2024-06-13T07:27:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:24 smithi135 ceph-mon[36276]: pgmap v1319: 33 pgs: 33 active+clean; 577 KiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:27:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:24 smithi135 ceph-mon[36276]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:27:24.783 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:24 smithi135 ceph-mon[36276]: osdmap e103: 12 total, 12 up, 12 in 2024-06-13T07:27:25.002 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:24 smithi026 systemd[1]: Stopping Ceph mon.a for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:25.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:27:25.192+0000 7fc60d400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:27:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a[32037]: 2024-06-13T07:27:25.192+0000 7fc60d400700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-13T07:27:25.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:25 smithi026 podman[111729]: 2024-06-13 07:27:25.265507362 +0000 UTC m=+0.223794517 container died 9dcd7ed340094569bf40d4ad44fec9babd9ea284548e291cd24d575e35cd80ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-13T07:27:26.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:25 smithi026 podman[111729]: 2024-06-13 07:27:25.803260209 +0000 UTC m=+0.761547357 container cleanup 9dcd7ed340094569bf40d4ad44fec9babd9ea284548e291cd24d575e35cd80ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-13T07:27:26.003 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:25 smithi026 bash[111729]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a 2024-06-13T07:27:26.752 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jun 13 07:27:26 smithi026 podman[111743]: 2024-06-13 07:27:26.338511486 +0000 UTC m=+1.053122329 container remove 9dcd7ed340094569bf40d4ad44fec9babd9ea284548e291cd24d575e35cd80ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:27:26.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.a.service' 2024-06-13T07:27:27.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:27:27.150 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-13T07:27:27.150 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-13T07:27:27.151 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.b 2024-06-13T07:27:27.533 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:27 smithi135 systemd[1]: Stopping Ceph mon.b for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:27.950 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:27 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-b[36272]: 2024-06-13T07:27:27.558+0000 7feb20d31700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:27:27.950 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:27 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-b[36272]: 2024-06-13T07:27:27.558+0000 7feb20d31700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-13T07:27:27.950 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:27 smithi135 podman[76570]: 2024-06-13 07:27:27.661262423 +0000 UTC m=+0.261211949 container died 6bf89dff8b95e2cf1ef94da3d56886a371b810f43d1cdb1b4dc9ac461e82da12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-13T07:27:28.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:28 smithi135 podman[76570]: 2024-06-13 07:27:28.059386505 +0000 UTC m=+0.659336034 container cleanup 6bf89dff8b95e2cf1ef94da3d56886a371b810f43d1cdb1b4dc9ac461e82da12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-13T07:27:28.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Jun 13 07:27:28 smithi135 bash[76570]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-b 2024-06-13T07:27:28.519 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.b.service' 2024-06-13T07:27:28.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:27:28.564 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-13T07:27:28.565 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-13T07:27:28.565 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.c 2024-06-13T07:27:28.876 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:28 smithi136 systemd[1]: Stopping Ceph mon.c for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:29.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:28 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-c[37212]: 2024-06-13T07:27:28.966+0000 7f892a47d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:27:29.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:28 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-c[37212]: 2024-06-13T07:27:28.966+0000 7f892a47d700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-13T07:27:29.377 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:29 smithi136 podman[80806]: 2024-06-13 07:27:29.0833454 +0000 UTC m=+0.266041611 container died 6cc9ad3d952baf2177e5d3f3d0ffaca4e825a355eda952df9da78998e763e2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-13T07:27:29.727 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:29 smithi136 podman[80806]: 2024-06-13 07:27:29.484278375 +0000 UTC m=+0.666974627 container cleanup 6cc9ad3d952baf2177e5d3f3d0ffaca4e825a355eda952df9da78998e763e2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-13T07:27:29.727 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:29 smithi136 bash[80806]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-c 2024-06-13T07:27:29.972 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.c.service' 2024-06-13T07:27:29.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:29 smithi136 podman[80818]: 2024-06-13 07:27:29.728237931 +0000 UTC m=+0.637233472 container remove 6cc9ad3d952baf2177e5d3f3d0ffaca4e825a355eda952df9da78998e763e2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-13T07:27:29.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:29 smithi136 systemd[1]: ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.c.service: Deactivated successfully. 2024-06-13T07:27:29.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:29 smithi136 systemd[1]: Stopped Ceph mon.c for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T07:27:29.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Jun 13 07:27:29 smithi136 systemd[1]: ceph-559474f0-2950-11ef-bc9c-c7b262605968@mon.c.service: Consumed 42.248s CPU time. 2024-06-13T07:27:30.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:27:30.016 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-13T07:27:30.016 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-13T07:27:30.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.x 2024-06-13T07:27:30.502 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 07:27:30 smithi026 systemd[1]: Stopping Ceph mgr.x for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:31.003 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 07:27:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T07:27:30.681+0000 7f6875800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:27:31.003 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 07:27:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x[32280]: 2024-06-13T07:27:30.681+0000 7f6875800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-13T07:27:31.003 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 07:27:30 smithi026 podman[111850]: 2024-06-13 07:27:30.7752271 +0000 UTC m=+0.289377082 container died 0d1fbd0ba318f8bd48a0378be6dd95467d6e1c779fdd2291d0f959e7a4eab050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:27:31.728 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 07:27:31 smithi026 podman[111850]: 2024-06-13 07:27:31.434919328 +0000 UTC m=+0.949069349 container cleanup 0d1fbd0ba318f8bd48a0378be6dd95467d6e1c779fdd2291d0f959e7a4eab050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:27:31.728 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 07:27:31 smithi026 bash[111850]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x 2024-06-13T07:27:32.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jun 13 07:27:31 smithi026 podman[111862]: 2024-06-13 07:27:31.728350243 +0000 UTC m=+0.917599655 container remove 0d1fbd0ba318f8bd48a0378be6dd95467d6e1c779fdd2291d0f959e7a4eab050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-13T07:27:32.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.x.service' 2024-06-13T07:27:32.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:27:32.401 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-13T07:27:32.401 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-13T07:27:32.401 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.y 2024-06-13T07:27:32.783 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 07:27:32 smithi135 systemd[1]: Stopping Ceph mgr.y for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:33.165 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 07:27:32 smithi135 podman[76691]: 2024-06-13 07:27:32.852177334 +0000 UTC m=+0.254064664 container died 463b2b5d80ed59d4cd6661e6f1152654c1c58423c45b2de42e519c7ad904caec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-13T07:27:33.532 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 07:27:33 smithi135 podman[76691]: 2024-06-13 07:27:33.265380059 +0000 UTC m=+0.667267386 container cleanup 463b2b5d80ed59d4cd6661e6f1152654c1c58423c45b2de42e519c7ad904caec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-13T07:27:33.532 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Jun 13 07:27:33 smithi135 bash[76691]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-y 2024-06-13T07:27:33.690 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.y.service' 2024-06-13T07:27:33.724 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:27:33.724 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-13T07:27:33.724 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-13T07:27:33.724 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.z 2024-06-13T07:27:34.060 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 07:27:33 smithi136 systemd[1]: Stopping Ceph mgr.z for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:34.376 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 07:27:34 smithi136 podman[80930]: 2024-06-13 07:27:34.060777373 +0000 UTC m=+0.139862303 container died e4d4b276fcb420feb2fb7e7ce6b46d2ca92ab2eaeaa21756527491336b7558f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:27:34.877 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 07:27:34 smithi136 podman[80930]: 2024-06-13 07:27:34.615944551 +0000 UTC m=+0.695029530 container cleanup e4d4b276fcb420feb2fb7e7ce6b46d2ca92ab2eaeaa21756527491336b7558f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-13T07:27:34.877 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Jun 13 07:27:34 smithi136 bash[80930]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mgr-z 2024-06-13T07:27:35.060 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mgr.z.service' 2024-06-13T07:27:35.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:27:35.095 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-13T07:27:35.095 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-13T07:27:35.095 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.0 2024-06-13T07:27:35.457 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:35 smithi026 systemd[1]: Stopping Ceph osd.0 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:36.002 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0[46694]: 2024-06-13T07:27:35.723+0000 7f81fe600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:27:36.002 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0[46694]: 2024-06-13T07:27:35.723+0000 7f81fe600700 -1 osd.0 103 *** Got signal Terminated *** 2024-06-13T07:27:36.002 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0[46694]: 2024-06-13T07:27:35.723+0000 7f81fe600700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:27:41.924 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:41 smithi026 podman[111968]: 2024-06-13 07:27:41.546245012 +0000 UTC m=+6.006676225 container died 78c35aa21b67d43e82b0df5a4d7765ef375212d7aaff2e802422477360cdd5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-13T07:27:42.252 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:42 smithi026 podman[111968]: 2024-06-13 07:27:42.090780154 +0000 UTC m=+6.551211343 container cleanup 78c35aa21b67d43e82b0df5a4d7765ef375212d7aaff2e802422477360cdd5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-13T07:27:42.252 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:42 smithi026 bash[111968]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0 2024-06-13T07:27:42.752 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:42 smithi026 podman[111999]: 2024-06-13 07:27:42.400888623 +0000 UTC m=+0.806016499 container remove 78c35aa21b67d43e82b0df5a4d7765ef375212d7aaff2e802422477360cdd5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-13T07:27:43.531 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:43 smithi026 podman[112084]: 2024-06-13 07:27:43.160026502 +0000 UTC m=+0.033814094 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:27:43.531 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:43 smithi026 podman[112084]: 2024-06-13 07:27:43.335221361 +0000 UTC m=+0.209008993 container create ce47f8d703db5d78ba60d5e3bb039e2fa93ffd2fa918dd922e5481d52fbd6183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-13T07:27:44.002 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:43 smithi026 podman[112084]: 2024-06-13 07:27:43.664488901 +0000 UTC m=+0.538276577 container init ce47f8d703db5d78ba60d5e3bb039e2fa93ffd2fa918dd922e5481d52fbd6183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-13T07:27:44.002 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:43 smithi026 podman[112084]: 2024-06-13 07:27:43.677520521 +0000 UTC m=+0.551308122 container start ce47f8d703db5d78ba60d5e3bb039e2fa93ffd2fa918dd922e5481d52fbd6183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-13T07:27:44.003 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:43 smithi026 podman[112084]: 2024-06-13 07:27:43.77990785 +0000 UTC m=+0.653695522 container attach ce47f8d703db5d78ba60d5e3bb039e2fa93ffd2fa918dd922e5481d52fbd6183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-13T07:27:44.954 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jun 13 07:27:44 smithi026 podman[112084]: 2024-06-13 07:27:44.536784396 +0000 UTC m=+1.410572051 container died ce47f8d703db5d78ba60d5e3bb039e2fa93ffd2fa918dd922e5481d52fbd6183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-13T07:27:45.422 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.0.service' 2024-06-13T07:27:45.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:27:45.540 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-13T07:27:45.540 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-13T07:27:45.541 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.1 2024-06-13T07:27:45.960 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:45 smithi135 systemd[1]: Stopping Ceph osd.1 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:46.283 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:45 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1[40842]: 2024-06-13T07:27:45.960+0000 7f5c539a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:27:46.283 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:45 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1[40842]: 2024-06-13T07:27:45.960+0000 7f5c539a8700 -1 osd.1 103 *** Got signal Terminated *** 2024-06-13T07:27:46.283 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:45 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1[40842]: 2024-06-13T07:27:45.960+0000 7f5c539a8700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:27:51.689 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:51 smithi135 podman[76814]: 2024-06-13 07:27:51.41079901 +0000 UTC m=+5.670655892 container died c6b241fd141c1e6b99cbd914bc6e110fcd49d01621db7e175bce77409ce45929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-13T07:27:52.033 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:51 smithi135 podman[76814]: 2024-06-13 07:27:51.820217685 +0000 UTC m=+6.080074566 container cleanup c6b241fd141c1e6b99cbd914bc6e110fcd49d01621db7e175bce77409ce45929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-13T07:27:52.033 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:51 smithi135 bash[76814]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1 2024-06-13T07:27:52.505 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:52 smithi135 podman[76831]: 2024-06-13 07:27:52.189068041 +0000 UTC m=+0.768147049 container remove c6b241fd141c1e6b99cbd914bc6e110fcd49d01621db7e175bce77409ce45929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-13T07:27:52.783 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:52 smithi135 podman[76921]: 2024-06-13 07:27:52.409056852 +0000 UTC m=+0.017080004 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:27:52.783 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:52 smithi135 podman[76921]: 2024-06-13 07:27:52.567374707 +0000 UTC m=+0.175397888 container create ca0897a9300775b58adb3f91e9fc6ae5b93fcb64152dd856b552f4dbbab7d6a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1-deactivate, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-13T07:27:53.199 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:52 smithi135 podman[76921]: 2024-06-13 07:27:52.843355196 +0000 UTC m=+0.451378349 container init ca0897a9300775b58adb3f91e9fc6ae5b93fcb64152dd856b552f4dbbab7d6a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-13T07:27:53.199 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:52 smithi135 podman[76921]: 2024-06-13 07:27:52.846699184 +0000 UTC m=+0.454722331 container start ca0897a9300775b58adb3f91e9fc6ae5b93fcb64152dd856b552f4dbbab7d6a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-13T07:27:53.199 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:52 smithi135 podman[76921]: 2024-06-13 07:27:52.951502183 +0000 UTC m=+0.559525337 container attach ca0897a9300775b58adb3f91e9fc6ae5b93fcb64152dd856b552f4dbbab7d6a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-13T07:27:53.477 INFO:journalctl@ceph.osd.1.smithi135.stdout:Jun 13 07:27:53 smithi135 podman[76921]: 2024-06-13 07:27:53.200598741 +0000 UTC m=+0.808621881 container died ca0897a9300775b58adb3f91e9fc6ae5b93fcb64152dd856b552f4dbbab7d6a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-1-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-13T07:27:53.915 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.1.service' 2024-06-13T07:27:53.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:27:53.954 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-13T07:27:53.954 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-13T07:27:53.954 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.2 2024-06-13T07:27:54.268 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:27:54 smithi136 systemd[1]: Stopping Ceph osd.2 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:27:54.527 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:27:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2[42323]: 2024-06-13T07:27:54.267+0000 7f1752dad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:27:54.527 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:27:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2[42323]: 2024-06-13T07:27:54.267+0000 7f1752dad700 -1 osd.2 103 *** Got signal Terminated *** 2024-06-13T07:27:54.527 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:27:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2[42323]: 2024-06-13T07:27:54.267+0000 7f1752dad700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:28:00.119 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:27:59 smithi136 podman[81053]: 2024-06-13 07:27:59.813014142 +0000 UTC m=+5.661871795 container died b700336d35d688d0b5c3f7c7c6f2bfdb99fc88b2ef9b35518075238c114864c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-13T07:28:00.376 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:00 smithi136 podman[81053]: 2024-06-13 07:28:00.21905009 +0000 UTC m=+6.067907784 container cleanup b700336d35d688d0b5c3f7c7c6f2bfdb99fc88b2ef9b35518075238c114864c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-06-13T07:28:00.376 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:00 smithi136 bash[81053]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2 2024-06-13T07:28:00.835 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:00 smithi136 podman[81080]: 2024-06-13 07:28:00.490913459 +0000 UTC m=+0.679501146 container remove b700336d35d688d0b5c3f7c7c6f2bfdb99fc88b2ef9b35518075238c114864c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:01.123 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:00 smithi136 podman[81169]: 2024-06-13 07:28:00.740075526 +0000 UTC m=+0.017550686 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:28:01.123 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:00 smithi136 podman[81169]: 2024-06-13 07:28:00.847050891 +0000 UTC m=+0.124526082 container create c61e5c7935bbb12e2e94ff47f1e3da75d463e8fe8b507e425f92e8cfba3baf4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-13T07:28:01.124 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:01 smithi136 podman[81169]: 2024-06-13 07:28:01.125007742 +0000 UTC m=+0.402482892 container init c61e5c7935bbb12e2e94ff47f1e3da75d463e8fe8b507e425f92e8cfba3baf4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:28:01.377 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:01 smithi136 podman[81169]: 2024-06-13 07:28:01.128324288 +0000 UTC m=+0.405799434 container start c61e5c7935bbb12e2e94ff47f1e3da75d463e8fe8b507e425f92e8cfba3baf4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-13T07:28:01.377 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:01 smithi136 podman[81169]: 2024-06-13 07:28:01.225132121 +0000 UTC m=+0.502607280 container attach c61e5c7935bbb12e2e94ff47f1e3da75d463e8fe8b507e425f92e8cfba3baf4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-13T07:28:01.877 INFO:journalctl@ceph.osd.2.smithi136.stdout:Jun 13 07:28:01 smithi136 podman[81169]: 2024-06-13 07:28:01.480435238 +0000 UTC m=+0.757910390 container died c61e5c7935bbb12e2e94ff47f1e3da75d463e8fe8b507e425f92e8cfba3baf4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:28:02.206 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.2.service' 2024-06-13T07:28:02.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:28:02.248 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-13T07:28:02.248 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-13T07:28:02.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.3 2024-06-13T07:28:02.752 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:02 smithi026 systemd[1]: Stopping Ceph osd.3 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:28:03.252 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:02 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3[53605]: 2024-06-13T07:28:02.893+0000 7fc799e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:28:03.252 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:02 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3[53605]: 2024-06-13T07:28:02.893+0000 7fc799e00700 -1 osd.3 103 *** Got signal Terminated *** 2024-06-13T07:28:03.252 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:02 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3[53605]: 2024-06-13T07:28:02.893+0000 7fc799e00700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:28:03.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:02 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:02.948+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:04.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:03.906+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:04.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:04 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:04.039+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:04.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:04.421+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:05.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:04.923+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:05.404 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:05 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:05.027+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:05.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:05 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:05.453+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:06.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:05 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:05.901+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:06.437 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:06 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:06.032+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:06.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:06.492+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:06.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:06.641+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:07.126 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:07 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:07.024+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:07.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:06 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:06.877+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:07.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:07.541+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:07.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:07.610+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:08.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:07 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:07.916+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:08.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:08 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:08.031+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:08.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:08.513+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:08.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:08.637+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:09.206 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:09 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:09.038+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:09.206 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:08 smithi026 podman[112187]: 2024-06-13 07:28:08.817514088 +0000 UTC m=+6.079132691 container died d01020853db1af90198e2b5795609bde5c636204efff0e79d732f0c76cf97f23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-13T07:28:09.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:08 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:08.879+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:09.469 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:09 smithi026 podman[112187]: 2024-06-13 07:28:09.437657746 +0000 UTC m=+6.699276342 container cleanup d01020853db1af90198e2b5795609bde5c636204efff0e79d732f0c76cf97f23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-13T07:28:09.469 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:09 smithi026 bash[112187]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3 2024-06-13T07:28:09.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:09.507+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:09.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:09.676+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:10.028 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:09 smithi026 podman[112218]: 2024-06-13 07:28:09.764431909 +0000 UTC m=+0.925635655 container remove d01020853db1af90198e2b5795609bde5c636204efff0e79d732f0c76cf97f23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-13T07:28:10.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:09 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:09.893+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:10.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:10 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:10.027+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:10.861 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:10 smithi026 podman[112308]: 2024-06-13 07:28:10.513610921 +0000 UTC m=+0.033249522 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:28:10.861 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:10 smithi026 podman[112308]: 2024-06-13 07:28:10.662300131 +0000 UTC m=+0.181938792 container create 3fb13800f2fd2d15d1a0aaa4522c3c5e20f08f890c5cb7101f2d1c99095c185d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3-deactivate, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-13T07:28:10.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:10.457+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:10.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:10.699+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:11.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:11 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:11.026+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:11.252 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:10 smithi026 podman[112308]: 2024-06-13 07:28:10.994302518 +0000 UTC m=+0.513941162 container init 3fb13800f2fd2d15d1a0aaa4522c3c5e20f08f890c5cb7101f2d1c99095c185d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-13T07:28:11.253 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:11 smithi026 podman[112308]: 2024-06-13 07:28:11.007243856 +0000 UTC m=+0.526882467 container start 3fb13800f2fd2d15d1a0aaa4522c3c5e20f08f890c5cb7101f2d1c99095c185d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:28:11.253 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:11 smithi026 podman[112308]: 2024-06-13 07:28:11.101613362 +0000 UTC m=+0.621252023 container attach 3fb13800f2fd2d15d1a0aaa4522c3c5e20f08f890c5cb7101f2d1c99095c185d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:11.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:10 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:10.894+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:11.752 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jun 13 07:28:11 smithi026 podman[112308]: 2024-06-13 07:28:11.476435753 +0000 UTC m=+0.996074385 container died 3fb13800f2fd2d15d1a0aaa4522c3c5e20f08f890c5cb7101f2d1c99095c185d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-13T07:28:11.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:11.697+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:11.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:11.483+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:11.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:11.483+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:12.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:12 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:12.045+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:12.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:11 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:11.883+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:12.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.3.service' 2024-06-13T07:28:12.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:12 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:12.562+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:12.783 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:12 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:12.530+0000 7f602af41700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.052057+0000 front 2024-06-13T07:27:47.052038+0000 (oldest deadline 2024-06-13T07:28:12.252911+0000) 2024-06-13T07:28:12.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:28:12.793 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-13T07:28:12.793 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-13T07:28:12.793 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.4 2024-06-13T07:28:12.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:12.525+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:12.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:12.525+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:12.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:12.655+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:12.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:12.655+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:13.116 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:12 smithi135 systemd[1]: Stopping Ceph osd.4 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:28:13.116 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:12 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:12.903+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:13.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:13 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:13.006+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:13.523 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:13 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:13.116+0000 7f6032955700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:28:13.524 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:13 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:13.116+0000 7f6032955700 -1 osd.4 103 *** Got signal Terminated *** 2024-06-13T07:28:13.524 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:13 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:13.116+0000 7f6032955700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:28:13.783 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:13 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:13.565+0000 7f602af41700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.052057+0000 front 2024-06-13T07:27:47.052038+0000 (oldest deadline 2024-06-13T07:28:12.252911+0000) 2024-06-13T07:28:13.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:13 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:13.524+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:13.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:13.675+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:13.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:13.675+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:13.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:13.760+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:13.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:13.520+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:13.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:13.520+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:14.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:13 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:13.859+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:14.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:14 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:14.030+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:14.808 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:14.551+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:14.808 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:14.574+0000 7f602af41700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.052057+0000 front 2024-06-13T07:27:47.052038+0000 (oldest deadline 2024-06-13T07:28:12.252911+0000) 2024-06-13T07:28:14.876 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:14.730+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:14.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:14.640+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:14.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:14.640+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:14.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:14.538+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:14.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:14.538+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:15.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:14 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:14.808+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:15.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:15 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:15.067+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:15.770 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:15.490+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:15.770 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:15.490+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:15.770 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:15.654+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:15.770 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:15.654+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:15.836 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:15 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:15.579+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:15.837 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:15 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:15.606+0000 7f602af41700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.052057+0000 front 2024-06-13T07:27:47.052038+0000 (oldest deadline 2024-06-13T07:28:12.252911+0000) 2024-06-13T07:28:16.126 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:15.770+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:16.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:16 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:16.089+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:16.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:16 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:16.089+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:16.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:15 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:15.979+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:16.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:15 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:15.836+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:16.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:15 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:15.836+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:16.876 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:16.736+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:16.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:16.493+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:16.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:16.493+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:16.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:16.613+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:16.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:16.613+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:17.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:16 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:16.615+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:17.033 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:16 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:16.633+0000 7f602af41700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.052057+0000 front 2024-06-13T07:27:47.052038+0000 (oldest deadline 2024-06-13T07:28:12.252911+0000) 2024-06-13T07:28:17.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:16 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:16.828+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:17.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:16 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:16.828+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:17.252 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:16 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:16.936+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:17.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:17 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:17.135+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:17.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:17 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:17.135+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:17.858 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:17 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:17.567+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:17.859 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:17 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4[47773]: 2024-06-13T07:28:17.632+0000 7f602af41700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.052057+0000 front 2024-06-13T07:27:47.052038+0000 (oldest deadline 2024-06-13T07:28:12.252911+0000) 2024-06-13T07:28:17.876 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:17.700+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:17.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:17.566+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:17.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:17.566+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:17.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:17.540+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:17.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:17.540+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:18.176 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:17 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:17.916+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:18.176 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:18 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:18.109+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:18.176 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:18 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:18.109+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:18.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:17 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:17.860+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:18.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:17 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:17.860+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:18.850 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:18 smithi135 podman[77042]: 2024-06-13 07:28:18.750911365 +0000 UTC m=+5.762642827 container died e8ed13ea01457f9cfb079e4ee7c9fb99ad7b62afd719e7611c1d7576a5537c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-13T07:28:18.851 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:18 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:18.540+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:18.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:18 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:18.746+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:18.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:18 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:18.532+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:18.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:18 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:18.532+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:18.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:18 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:18.590+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:18.878 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:18 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:18.590+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:19.116 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:18 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:18.850+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:19.116 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:18 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:18.850+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:19.252 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:18 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:18.899+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:19.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:19 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:19.133+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:19.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:19 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:19.133+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:19.376 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:19 smithi135 podman[77042]: 2024-06-13 07:28:19.117222564 +0000 UTC m=+6.128954024 container cleanup e8ed13ea01457f9cfb079e4ee7c9fb99ad7b62afd719e7611c1d7576a5537c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:19.376 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:19 smithi135 bash[77042]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4 2024-06-13T07:28:19.770 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:19 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:19.510+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:19.770 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:19 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:19.510+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:19.770 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:19 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:19.583+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:19.770 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:19 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:19.583+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:19.774 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:19 smithi135 podman[77058]: 2024-06-13 07:28:19.377377975 +0000 UTC m=+0.626180278 container remove e8ed13ea01457f9cfb079e4ee7c9fb99ad7b62afd719e7611c1d7576a5537c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:19.774 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:19 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:19.530+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:20.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:19 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:19.811+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:20.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:19 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:19.811+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:20.033 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:19 smithi135 podman[77146]: 2024-06-13 07:28:19.678418053 +0000 UTC m=+0.017146245 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:28:20.033 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:19 smithi135 podman[77146]: 2024-06-13 07:28:19.778036018 +0000 UTC m=+0.116764196 container create dfdeecf3ef32326bdcf910a6016a504825e594fe3dce4b659cd8a7665d1b5aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:28:20.033 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:20 smithi135 podman[77146]: 2024-06-13 07:28:20.022271063 +0000 UTC m=+0.360999242 container init dfdeecf3ef32326bdcf910a6016a504825e594fe3dce4b659cd8a7665d1b5aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-13T07:28:20.034 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:20 smithi135 podman[77146]: 2024-06-13 07:28:20.025233751 +0000 UTC m=+0.363961932 container start dfdeecf3ef32326bdcf910a6016a504825e594fe3dce4b659cd8a7665d1b5aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:28:20.126 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:19 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:19.770+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:20.168 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:19 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:19.880+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:20.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:20 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:20.167+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:20.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:20 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:20.167+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:20.533 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:20 smithi135 podman[77146]: 2024-06-13 07:28:20.13942421 +0000 UTC m=+0.478152389 container attach dfdeecf3ef32326bdcf910a6016a504825e594fe3dce4b659cd8a7665d1b5aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-13T07:28:20.533 INFO:journalctl@ceph.osd.4.smithi135.stdout:Jun 13 07:28:20 smithi135 podman[77146]: 2024-06-13 07:28:20.316156326 +0000 UTC m=+0.654884505 container died dfdeecf3ef32326bdcf910a6016a504825e594fe3dce4b659cd8a7665d1b5aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-13T07:28:20.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:20 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:20.558+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:20.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:20 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:20.558+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:20.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:20 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:20.544+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:20.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:20 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:20.544+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:20.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:20 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:20.754+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:20.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:20 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:20.754+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.284432+0000 front 2024-06-13T07:27:58.284463+0000 (oldest deadline 2024-06-13T07:28:20.584612+0000) 2024-06-13T07:28:20.958 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:20 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:20.548+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:20.959 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:20 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:20.764+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:20.959 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:20 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:20.764+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:20.981 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.4.service' 2024-06-13T07:28:21.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:28:21.019 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-13T07:28:21.019 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-13T07:28:21.019 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.5 2024-06-13T07:28:21.154 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:20 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:20.849+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:21.339 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:21 smithi136 systemd[1]: Stopping Ceph osd.5 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:28:21.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:21 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:21.155+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:21.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:21 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:21.155+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:21.608 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:21.506+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:21.608 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:21.506+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:21.608 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:21.338+0000 7f2fe44d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:28:21.608 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:21.339+0000 7f2fe44d9700 -1 osd.5 103 *** Got signal Terminated *** 2024-06-13T07:28:21.608 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:21.339+0000 7f2fe44d9700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:28:21.746 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:21 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:21.510+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:21.876 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:21.716+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:21.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:21.716+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.284432+0000 front 2024-06-13T07:27:58.284463+0000 (oldest deadline 2024-06-13T07:28:20.584612+0000) 2024-06-13T07:28:21.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:21.608+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:21.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:21 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:21.608+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:22.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:21 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:21.746+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:22.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:21 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:21.746+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:22.105 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:21 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:21.820+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:22.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:22 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:22.105+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:22.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:22 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:22.105+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:22.559 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:22 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:22.465+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:22.559 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:22 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:22.465+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:22.876 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:22 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:22.497+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:22.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:22 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:22.497+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:22.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:22 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:22.692+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:22.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:22 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:22.692+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.284432+0000 front 2024-06-13T07:27:58.284463+0000 (oldest deadline 2024-06-13T07:28:20.584612+0000) 2024-06-13T07:28:22.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:22 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:22.572+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:22.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:22 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:22.572+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:22.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:22 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:22.572+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:23.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:22 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:22.744+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:23.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:22 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:22.744+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:23.126 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:22 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:22.795+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:23.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:23 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:23.127+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:23.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:23 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:23.127+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:23.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:23 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:23.127+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:23.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:23 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:23.463+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:23.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:23 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:23.463+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:23.784 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:23 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:23.694+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:23.784 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:23 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:23.694+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:23.784 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:23 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:23.694+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:23.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:23 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:23.499+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:23.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:23 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:23.499+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:23.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:23 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:23.674+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:23.878 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:23 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:23.674+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.284432+0000 front 2024-06-13T07:27:58.284463+0000 (oldest deadline 2024-06-13T07:28:20.584612+0000) 2024-06-13T07:28:23.878 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:23 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:23.600+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:23.878 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:23 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:23.600+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:23.878 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:23 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:23.600+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:24.102 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:23 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:23.838+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:24.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:24 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:24.102+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:24.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:24 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:24.102+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:24.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:24 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:24.102+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:24.778 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:24 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:24.481+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:24.778 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:24 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:24.481+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:24.779 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:24 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:24.594+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:24.779 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:24 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:24.594+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:24.779 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:24 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:24.594+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:24.779 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:24 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:24.625+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:24.779 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:24 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:24.625+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.284432+0000 front 2024-06-13T07:27:58.284463+0000 (oldest deadline 2024-06-13T07:28:20.584612+0000) 2024-06-13T07:28:24.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:24 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:24.507+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:24.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:24 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:24.507+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:24.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:24 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:24.714+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:24.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:24 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:24.714+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:24.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:24 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:24.714+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:25.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:25.053+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:25.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:25.053+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:25.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:25.053+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:25.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:24 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:24.808+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:25.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:24 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:24.808+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:25.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:25 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:25.491+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:25.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:25 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:25.491+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:25.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:25 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:25.735+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:25.784 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:25 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:25.735+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:25.784 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:25 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:25.735+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:25.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:25 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:25.594+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:25.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:25 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:25.594+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:25.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:25 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:25.594+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:25.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:25 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:25.669+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.681431+0000 front 2024-06-13T07:27:49.681456+0000 (oldest deadline 2024-06-13T07:28:13.682318+0000) 2024-06-13T07:28:25.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:25 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5[49261]: 2024-06-13T07:28:25.669+0000 7f2fe0acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.284432+0000 front 2024-06-13T07:27:58.284463+0000 (oldest deadline 2024-06-13T07:28:20.584612+0000) 2024-06-13T07:28:25.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:25 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:25.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:25.878 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:25 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:25.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:25.878 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:25 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:25.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:26.033 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:25.768+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:26.033 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:25 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:25.768+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:26.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:26.033+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:26.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:26.033+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:26.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:26.033+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:26.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:26 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:26.495+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:26.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:26 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:26.495+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:26.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:26 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:26.701+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:26.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:26 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:26.701+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:26.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:26 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:26.701+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:26.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:26 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:26.630+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:26.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:26 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:26.630+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:26.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:26 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:26.630+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:26.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:26 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:26.419+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:26.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:26 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:26.419+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:26.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:26 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:26.419+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:27.040 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:26.780+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:27.040 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:26.780+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:27.231 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:26 smithi136 podman[81278]: 2024-06-13 07:28:26.946725733 +0000 UTC m=+5.730594324 container died e4560e200a47f854309059e3351e03b10321b352846585500f6c2f2dd346e793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-13T07:28:27.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:27.040+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:27.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:27.040+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:27.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:27.040+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:27.590 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:27 smithi136 podman[81278]: 2024-06-13 07:28:27.331091233 +0000 UTC m=+6.114959823 container cleanup e4560e200a47f854309059e3351e03b10321b352846585500f6c2f2dd346e793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-13T07:28:27.590 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:27 smithi136 bash[81278]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5 2024-06-13T07:28:27.590 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:27 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:27.391+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:27.590 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:27 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:27.391+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:27.591 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:27 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:27.391+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:27.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:27 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:27.508+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:27.783 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:27 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:27.508+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:27.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:27 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:27.726+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:27.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:27 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:27.726+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:27.783 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:27 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:27.726+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:27.877 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:27 smithi136 podman[81293]: 2024-06-13 07:28:27.591289549 +0000 UTC m=+0.646904229 container remove e4560e200a47f854309059e3351e03b10321b352846585500f6c2f2dd346e793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-13T07:28:27.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:27 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:27.605+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:27.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:27 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:27.605+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:27.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:27 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:27.605+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:28.045 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:27.778+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:28.045 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:27 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:27.778+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:28.203 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:27 smithi136 podman[81386]: 2024-06-13 07:28:27.85301159 +0000 UTC m=+0.017054636 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:28:28.203 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:27 smithi136 podman[81386]: 2024-06-13 07:28:27.975780844 +0000 UTC m=+0.139823890 container create a9e24530991ba6e7375ce6ce6541d17c84731e1b20a8b4a00286d7482d0f8f09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:28.203 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:28 smithi136 podman[81386]: 2024-06-13 07:28:28.203701113 +0000 UTC m=+0.367744178 container init a9e24530991ba6e7375ce6ce6541d17c84731e1b20a8b4a00286d7482d0f8f09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-06-13T07:28:28.491 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:28 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:28.405+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:28.491 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:28 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:28.405+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:28.491 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:28 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:28.405+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:28.492 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:28 smithi136 podman[81386]: 2024-06-13 07:28:28.207081285 +0000 UTC m=+0.371124339 container start a9e24530991ba6e7375ce6ce6541d17c84731e1b20a8b4a00286d7482d0f8f09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-13T07:28:28.492 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:28 smithi136 podman[81386]: 2024-06-13 07:28:28.311859724 +0000 UTC m=+0.475902788 container attach a9e24530991ba6e7375ce6ce6541d17c84731e1b20a8b4a00286d7482d0f8f09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS) 2024-06-13T07:28:28.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:28 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:28.045+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:28.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:28 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:28.045+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:28.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:28 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:28.045+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:28.797 INFO:journalctl@ceph.osd.5.smithi136.stdout:Jun 13 07:28:28 smithi136 podman[81386]: 2024-06-13 07:28:28.493280851 +0000 UTC m=+0.657323899 container died a9e24530991ba6e7375ce6ce6541d17c84731e1b20a8b4a00286d7482d0f8f09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-13T07:28:28.798 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:28 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:28.566+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:28.798 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:28 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:28.566+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:28.798 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:28 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:28.566+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:29.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:28 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:28.535+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:29.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:28 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:28.535+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:29.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:28 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:28.708+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:29.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:28 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:28.708+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:29.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:28 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:28.708+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:29.094 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:28 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:28.828+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:29.095 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:28 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:28.828+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:29.177 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.5.service' 2024-06-13T07:28:29.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:28:29.221 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-13T07:28:29.221 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-13T07:28:29.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.6 2024-06-13T07:28:29.352 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:29.095+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:29.352 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:29.095+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:29.353 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:29.095+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:29.752 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:29 smithi026 systemd[1]: Stopping Ceph osd.6 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:28:29.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:29 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:29.537+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:29.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:29 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:29.537+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:29.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:29 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:29.537+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:29.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:29 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:29.446+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:29.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:29 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:29.446+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:29.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:29 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:29.446+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:30.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:29 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:29.560+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:30.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:29 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:29.560+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:30.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:29 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:29.560+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:30.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:29 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:29.750+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:30.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:29 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:29.750+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:30.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:29 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:29.750+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:30.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:30.070+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:30.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:30.070+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:30.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:30.070+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:30.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:29.878+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:30.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:29.878+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:30.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:29.878+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.850523+0000 front 2024-06-13T07:28:03.850613+0000 (oldest deadline 2024-06-13T07:28:29.150913+0000) 2024-06-13T07:28:30.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:29.924+0000 7f266bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:28:30.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:29.924+0000 7f266bc00700 -1 osd.6 103 *** Got signal Terminated *** 2024-06-13T07:28:30.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:29 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:29.924+0000 7f266bc00700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:28:30.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:30 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:30.541+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:30.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:30 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:30.541+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:30.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:30 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:30.541+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:30.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:30 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:30.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:30.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:30 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:30.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:30.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:30 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:30.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:31.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:30 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:30.600+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:31.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:30 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:30.600+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:31.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:30 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:30.600+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:31.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:30 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:30.791+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:31.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:30 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:30.791+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:31.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:30 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:30.791+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:31.252 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:30.898+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:31.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:30.898+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:31.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:30 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:30.898+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.850523+0000 front 2024-06-13T07:28:03.850613+0000 (oldest deadline 2024-06-13T07:28:29.150913+0000) 2024-06-13T07:28:31.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:31.029+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:31.254 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:31.029+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:31.254 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:31.029+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:31.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:31 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:31.509+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:31.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:31 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:31.509+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:31.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:31 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:31.509+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:31.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:31 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:31.442+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:31.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:31 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:31.442+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:31.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:31 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:31.442+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:31.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:31 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:31.442+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:32.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:31 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:31.622+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:32.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:31 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:31.622+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:32.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:31 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:31.622+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:32.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:31 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:31.834+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:32.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:31 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:31.834+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:32.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:31 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:31.834+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:32.252 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:31.853+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:32.252 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:31.853+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:32.252 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:31.853+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.850523+0000 front 2024-06-13T07:28:03.850613+0000 (oldest deadline 2024-06-13T07:28:29.150913+0000) 2024-06-13T07:28:32.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:32.040+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:32.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:32.040+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:32.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:32.040+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:32.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:32 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:32.552+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:32.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:32 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:32.552+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:32.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:32 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:32.552+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:32.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:32 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:32.420+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:32.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:32 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:32.420+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:32.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:32 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:32.420+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:32.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:32 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:32.420+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:32.993 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:32 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:32.801+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:32.993 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:32 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:32.801+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:32.993 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:32 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:32.801+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:32.993 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:32 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:32.643+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:32.993 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:32 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:32.643+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:32.993 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:32 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:32.643+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:33.252 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:32.837+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:33.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:32.837+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:33.253 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:32 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:32.837+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.850523+0000 front 2024-06-13T07:28:03.850613+0000 (oldest deadline 2024-06-13T07:28:29.150913+0000) 2024-06-13T07:28:33.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:33.025+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:33.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:33.025+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:33.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:33.025+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:33.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:33.025+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:33.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:33 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:33.508+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:33.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:33 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:33.508+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:33.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:33 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:33.508+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:33.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:33 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:33.401+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:33.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:33 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:33.401+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:33.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:33 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:33.401+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:33.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:33 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:33.401+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:34.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:33 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:33.693+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:34.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:33 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:33.693+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:34.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:33 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:33.693+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:34.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:33 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:33.756+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:34.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:33 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:33.756+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:34.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:33 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:33.756+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:34.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:33 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:33.756+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:34.053 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:33.791+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:34.053 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:33.791+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:34.053 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:33 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:33.791+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.850523+0000 front 2024-06-13T07:28:03.850613+0000 (oldest deadline 2024-06-13T07:28:29.150913+0000) 2024-06-13T07:28:34.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:34.052+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:34.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:34.052+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:34.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:34.052+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:34.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:34.052+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:34.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:34 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:34.539+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:34.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:34 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:34.539+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:34.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:34 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:34.539+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:34.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:34 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:34.377+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:34.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:34 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:34.377+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:34.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:34 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:34.377+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:34.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:34 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:34.377+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:35.002 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:34.744+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.442723+0000 front 2024-06-13T07:27:50.442618+0000 (oldest deadline 2024-06-13T07:28:15.142867+0000) 2024-06-13T07:28:35.002 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:34.744+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:59.148904+0000 front 2024-06-13T07:27:59.148812+0000 (oldest deadline 2024-06-13T07:28:23.849109+0000) 2024-06-13T07:28:35.003 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:34 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6[60263]: 2024-06-13T07:28:34.744+0000 7f2662400700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.850523+0000 front 2024-06-13T07:28:03.850613+0000 (oldest deadline 2024-06-13T07:28:29.150913+0000) 2024-06-13T07:28:35.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:34 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:34.725+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:35.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:34 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:34.725+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:35.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:34 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:34.725+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:35.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:34 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:34.761+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:35.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:34 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:34.761+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:35.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:34 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:34.761+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:35.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:34 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:34.761+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:35.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:35.026+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:35.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:35.026+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:35.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:35.026+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:35.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:35 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:35.026+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:35.857 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:35 smithi026 podman[112411]: 2024-06-13 07:28:35.556376063 +0000 UTC m=+5.817662812 container died 1059948290d68a2c3b089990c0db33bb825314068a7ceb736ff4c111c4fc8c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-13T07:28:35.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:35 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:35.584+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:35.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:35 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:35.584+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:35.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:35 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:35.584+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:35.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:35 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:35.391+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:35.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:35 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:35.391+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:35.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:35 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:35.391+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:35.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:35 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:35.391+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:36.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:35 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:35.754+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:36.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:35 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:35.754+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:36.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:35 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:35.754+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:36.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:35 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:35.773+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:36.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:35 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:35.773+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:36.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:35 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:35.773+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:36.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:35 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:35.773+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:36.172 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:36 smithi026 podman[112411]: 2024-06-13 07:28:36.026026281 +0000 UTC m=+6.287313034 container cleanup 1059948290d68a2c3b089990c0db33bb825314068a7ceb736ff4c111c4fc8c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-13T07:28:36.172 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:36 smithi026 bash[112411]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6 2024-06-13T07:28:36.172 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:36 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:36.062+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:36.172 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:36 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:36.062+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:36.172 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:36 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:36.062+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:36.172 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:36 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:36.062+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:36.502 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:36 smithi026 podman[112430]: 2024-06-13 07:28:36.294308787 +0000 UTC m=+0.719669087 container remove 1059948290d68a2c3b089990c0db33bb825314068a7ceb736ff4c111c4fc8c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-13T07:28:36.626 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:36 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:36.604+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:36.627 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:36 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:36.604+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:36.627 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:36 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:36.604+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:36.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:36 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:36.361+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:36.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:36 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:36.361+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:36.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:36 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:36.361+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:36.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:36 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:36.361+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:37.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:36 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:36.732+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:37.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:36 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:36.732+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:37.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:36 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:36.732+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:37.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:36 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:36.779+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:37.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:36 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:36.779+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:37.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:36 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:36.779+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:37.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:36 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:36.779+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:37.385 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:37 smithi026 podman[112526]: 2024-06-13 07:28:36.974813979 +0000 UTC m=+0.045819572 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:28:37.385 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:37 smithi026 podman[112526]: 2024-06-13 07:28:37.112178641 +0000 UTC m=+0.183184268 container create ff44a9559b20d91357f6e4105263da107abd31ed42e1086092721d8f62d5404b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-06-13T07:28:37.385 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:37 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:37.088+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:37.385 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:37 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:37.088+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:37.386 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:37 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:37.088+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:37.386 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:37 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:37.088+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:37.613 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:37 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:37.356+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:37.613 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:37 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:37.356+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:37.613 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:37 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:37.356+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:37.614 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:37 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:37.356+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:37.752 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:37 smithi026 podman[112526]: 2024-06-13 07:28:37.385759151 +0000 UTC m=+0.456764801 container init ff44a9559b20d91357f6e4105263da107abd31ed42e1086092721d8f62d5404b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:37.752 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:37 smithi026 podman[112526]: 2024-06-13 07:28:37.396805231 +0000 UTC m=+0.467810888 container start ff44a9559b20d91357f6e4105263da107abd31ed42e1086092721d8f62d5404b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:28:37.752 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:37 smithi026 podman[112526]: 2024-06-13 07:28:37.548522977 +0000 UTC m=+0.619528654 container attach ff44a9559b20d91357f6e4105263da107abd31ed42e1086092721d8f62d5404b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:28:37.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:37 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:37.614+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:37.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:37 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:37.614+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:37.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:37 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:37.614+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:38.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:37 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:37.747+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:38.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:37 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:37.747+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:38.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:37 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:37.747+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:38.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:37 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:37.822+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:38.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:37 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:37.822+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:38.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:37 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:37.822+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:38.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:37 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:37.822+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:38.214 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jun 13 07:28:37 smithi026 podman[112526]: 2024-06-13 07:28:37.871062264 +0000 UTC m=+0.942067901 container died ff44a9559b20d91357f6e4105263da107abd31ed42e1086092721d8f62d5404b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-13T07:28:38.214 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:38 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:38.079+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:38.215 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:38 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:38.079+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:38.215 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:38 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:38.079+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:38.215 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:38 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:38.079+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:38.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:38.312+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:38.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:38.312+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:38.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:38.312+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:38.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:38.312+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:38.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.6.service' 2024-06-13T07:28:38.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:28:38.919 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-13T07:28:38.919 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-13T07:28:38.919 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.7 2024-06-13T07:28:38.969 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:38 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:38.718+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:38.970 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:38 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:38.718+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:38.970 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:38 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:38.718+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:38.970 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:38 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:38.853+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:38.970 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:38 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:38.853+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:38.970 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:38 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:38.853+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:38.970 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:38 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:38.853+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:39.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:38.659+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:39.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:38.659+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:39.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:38 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:38.659+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:39.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:38 smithi135 systemd[1]: Stopping Ceph osd.7 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:28:39.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:39 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:39.029+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:39.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:39 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:39.029+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:39.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:39 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:39.029+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:39.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:39 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:39.029+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:39.533 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:39.228+0000 7f0be5217700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:28:39.534 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:39.228+0000 7f0be5217700 -1 osd.7 103 *** Got signal Terminated *** 2024-06-13T07:28:39.534 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:39.228+0000 7f0be5217700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:28:39.615 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:39 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:39.350+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:39.615 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:39 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:39.350+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:39.615 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:39 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:39.350+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:39.615 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:39 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:39.350+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:39.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:39 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:39.615+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:39.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:39 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:39.615+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:39.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:39 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:39.615+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:40.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:39.679+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:40.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:39.679+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:40.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:39.679+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:40.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:39.679+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:15.364903+0000 front 2024-06-13T07:28:15.364930+0000 (oldest deadline 2024-06-13T07:28:38.865113+0000) 2024-06-13T07:28:40.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:39.824+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:40.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:39.824+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:40.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:39.824+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:40.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:39 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:39.824+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:40.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.015+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:40.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.015+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:40.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.015+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:40.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.015+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:40.653 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:40.378+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:40.653 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:40.378+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:40.653 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:40.378+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:40.653 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:40.378+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:41.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:40 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:40.691+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:41.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:40 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:40.691+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:41.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:40 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:40.691+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:41.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:40 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:40.691+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:15.364903+0000 front 2024-06-13T07:28:15.364930+0000 (oldest deadline 2024-06-13T07:28:38.865113+0000) 2024-06-13T07:28:41.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:40 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:40.861+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:41.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:40 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:40.861+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:41.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:40 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:40.861+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:41.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:40 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:40.861+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:41.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:40.654+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:41.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:40.654+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:41.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:40 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:40.654+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:41.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.993+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:41.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.993+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:41.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.993+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:41.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.993+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:41.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:40 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:40.993+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:41.700 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:41 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:41.428+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:41.700 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:41 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:41.428+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:41.700 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:41 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:41.428+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:41.700 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:41 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:41.428+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:42.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:41.673+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:42.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:41.673+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:42.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:41.673+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:42.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:41.673+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:15.364903+0000 front 2024-06-13T07:28:15.364930+0000 (oldest deadline 2024-06-13T07:28:38.865113+0000) 2024-06-13T07:28:42.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:41.817+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:42.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:41.817+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:42.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:41.817+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:42.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:41.817+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:42.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:41 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:41.817+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:42.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:41 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:41.700+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:42.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:41 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:41.700+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:42.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:41 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:41.700+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:42.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:41 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:41.700+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:42.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.018+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:42.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.018+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:42.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.018+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:42.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.018+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:42.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.018+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:42.876 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:42 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:42.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:42.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:42 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:42.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:42.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:42 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:42.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:42.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:42 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:42.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:42.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:42 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:42.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:42.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:42 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:42.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:42.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:42 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:42.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:42.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:42 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:42.444+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:43.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:42.687+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:43.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:42.687+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:43.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:42.687+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:43.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:42.687+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:15.364903+0000 front 2024-06-13T07:28:15.364930+0000 (oldest deadline 2024-06-13T07:28:38.865113+0000) 2024-06-13T07:28:43.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:42.809+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:43.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:42.809+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:43.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:42.809+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:43.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:42.809+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:43.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:42.809+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:43.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.981+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:43.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.981+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:43.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.981+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:43.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.981+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:43.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:42 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:42.981+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:43.698 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:43 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:43.426+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:43.699 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:43 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:43.426+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:43.699 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:43 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:43.426+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:43.699 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:43 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:43.426+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:44.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:43.638+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:47.860921+0000 front 2024-06-13T07:27:47.861001+0000 (oldest deadline 2024-06-13T07:28:12.561112+0000) 2024-06-13T07:28:44.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:43.638+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.162626+0000 front 2024-06-13T07:27:57.162651+0000 (oldest deadline 2024-06-13T07:28:22.462651+0000) 2024-06-13T07:28:44.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:43.638+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:04.764008+0000 front 2024-06-13T07:28:04.763979+0000 (oldest deadline 2024-06-13T07:28:29.463867+0000) 2024-06-13T07:28:44.033 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7[54314]: 2024-06-13T07:28:43.638+0000 7f0be180b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:15.364903+0000 front 2024-06-13T07:28:15.364930+0000 (oldest deadline 2024-06-13T07:28:38.865113+0000) 2024-06-13T07:28:44.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:43.856+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:44.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:43.856+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:44.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:43.856+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:44.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:43.856+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:44.034 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:43 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:43.856+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:44.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:43 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:43.699+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:44.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:43 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:43.699+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:44.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:43 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:43.699+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:44.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:43 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:43.699+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:44.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:43 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:43.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:44.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:43 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:43.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:44.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:43 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:43.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:44.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:43 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:43.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:44.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:43 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:43.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:44.711 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:44 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:44.392+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:44.711 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:44 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:44.392+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:44.711 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:44 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:44.392+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:44.711 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:44 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:44.392+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:44.920 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:44 smithi135 podman[77264]: 2024-06-13 07:28:44.630251837 +0000 UTC m=+5.509423930 container died c4f7b4f6e676f43184bd6e3ce0c1aa437515867218b3b04d514161cab09f870d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-13T07:28:44.921 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:44 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:44.849+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:44.921 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:44 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:44.849+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:44.921 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:44 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:44.849+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:44.921 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:44 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:44.849+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:44.921 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:44 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:44.849+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:45.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:44 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:44.711+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:45.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:44 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:44.711+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:45.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:44 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:44.711+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:45.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:44 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:44.711+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:45.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:44 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:44.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:45.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:44 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:44.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:45.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:44 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:44.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:45.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:44 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:44.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:45.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:44 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:44.970+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:45.283 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:45 smithi135 podman[77264]: 2024-06-13 07:28:45.031679001 +0000 UTC m=+5.910851093 container cleanup c4f7b4f6e676f43184bd6e3ce0c1aa437515867218b3b04d514161cab09f870d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:28:45.283 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:45 smithi135 bash[77264]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7 2024-06-13T07:28:45.626 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:45 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:45.373+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:45.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:45 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:45.373+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:45.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:45 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:45.373+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:45.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:45 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:45.373+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:45.647 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:45 smithi135 podman[77281]: 2024-06-13 07:28:45.299720733 +0000 UTC m=+0.662888631 container remove c4f7b4f6e676f43184bd6e3ce0c1aa437515867218b3b04d514161cab09f870d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-13T07:28:45.910 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:45 smithi135 podman[77370]: 2024-06-13 07:28:45.551717491 +0000 UTC m=+0.017081795 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:28:45.910 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:45 smithi135 podman[77370]: 2024-06-13 07:28:45.683811468 +0000 UTC m=+0.149175763 container create 9ac4aaf568e1dd65f01ce1c32dc69a8b390a27b8c596d51c021db6487d69e18b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-13T07:28:45.910 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:45 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:45.803+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:45.911 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:45 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:45.803+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:45.911 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:45 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:45.803+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:45.911 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:45 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:45.803+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:45.911 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:45 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:45.803+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:46.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:45 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:45.664+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:46.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:45 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:45.664+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:46.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:45 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:45.664+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:46.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:45 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:45.664+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:46.198 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:45 smithi135 podman[77370]: 2024-06-13 07:28:45.911137656 +0000 UTC m=+0.376501960 container init 9ac4aaf568e1dd65f01ce1c32dc69a8b390a27b8c596d51c021db6487d69e18b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:46.198 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:45 smithi135 podman[77370]: 2024-06-13 07:28:45.915295066 +0000 UTC m=+0.380659365 container start 9ac4aaf568e1dd65f01ce1c32dc69a8b390a27b8c596d51c021db6487d69e18b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-13T07:28:46.198 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:45 smithi135 podman[77370]: 2024-06-13 07:28:45.994805796 +0000 UTC m=+0.460170108 container attach 9ac4aaf568e1dd65f01ce1c32dc69a8b390a27b8c596d51c021db6487d69e18b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-13T07:28:46.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:45.992+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:46.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:45.992+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:46.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:45.992+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:46.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:45.992+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:46.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:45 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:45.992+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:46.472 INFO:journalctl@ceph.osd.7.smithi135.stdout:Jun 13 07:28:46 smithi135 podman[77370]: 2024-06-13 07:28:46.199679613 +0000 UTC m=+0.665043914 container died 9ac4aaf568e1dd65f01ce1c32dc69a8b390a27b8c596d51c021db6487d69e18b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:28:46.652 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:46 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:46.402+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:46.652 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:46 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:46.402+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:46.652 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:46 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:46.402+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:46.652 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:46 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:46.402+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:46.825 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.7.service' 2024-06-13T07:28:46.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:28:46.863 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-13T07:28:46.864 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-13T07:28:46.864 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.8 2024-06-13T07:28:46.919 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:46 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:46.653+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:46.919 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:46 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:46.653+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:46.919 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:46 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:46.653+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:46.919 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:46 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:46.653+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:47.218 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:46 smithi136 systemd[1]: Stopping Ceph osd.8 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:28:47.218 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:47.218+0000 7fb8f1622700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:28:47.218 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:47.218+0000 7fb8f1622700 -1 osd.8 103 *** Got signal Terminated *** 2024-06-13T07:28:47.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:47.218+0000 7fb8f1622700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:28:47.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:46 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:46.977+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:47.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:46 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:46.977+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:47.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:46 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:46.977+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:47.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:46 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:46.977+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:47.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:46 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:46.977+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:47.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:46 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:46.825+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:47.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:46 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:46.825+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:47.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:46 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:46.825+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:47.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:46 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:46.825+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:47.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:46 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:46.825+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:47.626 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:47.412+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:47.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:47.412+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:47.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:47.412+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:47.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:47.412+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:48.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:47.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:48.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:47.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:48.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:47.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:48.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:47 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:47.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:48.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:47 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:47.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:48.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:47 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:47.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:48.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:47 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:47.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:48.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:47 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:47.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:48.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:47 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:47.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:48.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:47 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:47.784+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:48.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:47 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:47.784+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:48.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:47 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:47.784+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:48.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:47 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:47.784+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:48.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:47 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:47.784+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:48.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:47 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:47.784+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:48.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:48.367+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:48.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:48.367+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:48.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:48.367+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:48.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:48.367+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:48.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:48.367+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:24.362893+0000 front 2024-06-13T07:28:24.362940+0000 (oldest deadline 2024-06-13T07:28:47.863002+0000) 2024-06-13T07:28:49.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:48.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:49.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:48.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:49.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:48.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:49.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:48.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:49.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:48 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:48.666+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:49.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:48 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:48.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:49.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:48 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:48.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:49.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:48 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:48.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:49.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:48 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:48.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:49.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:48 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:48.983+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:49.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:48 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:48.821+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:49.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:48 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:48.821+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:49.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:48 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:48.821+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:49.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:48 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:48.821+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:49.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:48 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:48.821+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:49.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:48 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:48.821+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:49.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:49.363+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:49.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:49.363+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:49.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:49.363+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:49.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:49.363+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:49.627 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:49.363+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:24.362893+0000 front 2024-06-13T07:28:24.362940+0000 (oldest deadline 2024-06-13T07:28:47.863002+0000) 2024-06-13T07:28:50.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:49.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:50.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:49.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:50.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:49.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:50.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:49.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:50.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:49 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:49.681+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:50.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:49 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:49.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:50.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:49 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:49.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:50.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:49 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:49.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:50.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:49 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:49.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:50.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:49 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:49.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:50.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:49 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:49.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:50.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:50 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:50.021+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:50.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:50 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:50.021+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:50.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:50 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:50.021+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:50.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:50 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:50.021+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:50.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:50 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:50.021+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:50.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:50 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:50.021+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:50.718 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:50.408+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:50.718 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:50.408+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:50.718 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:50.408+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:50.718 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:50.408+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:50.718 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:50.408+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:24.362893+0000 front 2024-06-13T07:28:24.362940+0000 (oldest deadline 2024-06-13T07:28:47.863002+0000) 2024-06-13T07:28:51.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:50.718+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:51.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:50.718+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:51.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:50.718+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:51.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:50.718+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:51.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:50 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:50.718+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:51.184 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:50 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:50.894+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:51.184 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:50 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:50.894+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:51.184 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:50 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:50.894+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:51.184 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:50 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:50.894+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:51.184 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:50 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:50.894+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:51.185 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:50 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:50.894+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:51.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:51 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:51.000+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:51.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:51 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:51.000+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:51.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:51 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:51.000+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:51.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:51 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:51.000+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:51.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:51 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:51.000+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:51.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:51 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:51.000+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:51.692 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:51.429+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:39.851979+0000 front 2024-06-13T07:27:39.852007+0000 (oldest deadline 2024-06-13T07:28:03.951851+0000) 2024-06-13T07:28:51.693 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:51.429+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:49.154300+0000 front 2024-06-13T07:27:49.154284+0000 (oldest deadline 2024-06-13T07:28:11.454358+0000) 2024-06-13T07:28:51.693 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:51.429+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.657666+0000 front 2024-06-13T07:27:57.657573+0000 (oldest deadline 2024-06-13T07:28:24.658295+0000) 2024-06-13T07:28:51.693 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:51.429+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:06.359586+0000 front 2024-06-13T07:28:06.359831+0000 (oldest deadline 2024-06-13T07:28:31.059413+0000) 2024-06-13T07:28:51.693 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8[56070]: 2024-06-13T07:28:51.429+0000 7fb8edc16700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:24.362893+0000 front 2024-06-13T07:28:24.362940+0000 (oldest deadline 2024-06-13T07:28:47.863002+0000) 2024-06-13T07:28:52.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:51.693+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:52.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:51.693+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:52.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:51.693+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:52.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:51.693+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:52.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:51 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:51.693+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:52.196 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:51 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:51.913+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:52.197 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:51 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:51.913+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:52.197 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:51 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:51.913+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:52.197 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:51 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:51.913+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:52.197 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:51 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:51.913+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:52.197 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:51 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:51.913+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:52.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:52 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:52.047+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:52.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:52 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:52.047+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:52.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:52 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:52.047+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:52.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:52 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:52.048+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:52.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:52 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:52.048+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:52.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:52 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:52.048+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:52.877 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:52 smithi136 podman[81492]: 2024-06-13 07:28:52.622400329 +0000 UTC m=+5.514406072 container died 9f0bcfbc7e18112ebf6899b5c2cbbc10ad01ce297cd93449a667941adb229f26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-13T07:28:52.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:52 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:52.698+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:52.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:52 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:52.698+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:52.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:52 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:52.698+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:52.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:52 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:52.698+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:52.877 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:52 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:52.698+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:53.210 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:52 smithi136 podman[81492]: 2024-06-13 07:28:52.984953403 +0000 UTC m=+5.876959144 container cleanup 9f0bcfbc7e18112ebf6899b5c2cbbc10ad01ce297cd93449a667941adb229f26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-13T07:28:53.210 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:52 smithi136 bash[81492]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8 2024-06-13T07:28:53.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:52 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:52.958+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:53.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:52 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:52.958+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:53.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:52 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:52.958+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:53.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:52 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:52.958+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:53.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:52 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:52.958+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:53.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:52 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:52.958+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:53.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:53 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:53.097+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:53.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:53 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:53.097+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:53.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:53 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:53.097+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:53.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:53 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:53.097+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:53.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:53 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:53.097+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:53.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:53 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:53.097+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:53.626 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:53 smithi136 podman[81508]: 2024-06-13 07:28:53.21135077 +0000 UTC m=+0.588392881 container remove 9f0bcfbc7e18112ebf6899b5c2cbbc10ad01ce297cd93449a667941adb229f26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-13T07:28:53.929 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:53.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:53.930 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:53.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:53.930 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:53.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:53.930 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:53.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:53.930 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:53 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:53.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:53.930 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:53 smithi136 podman[81597]: 2024-06-13 07:28:53.63723435 +0000 UTC m=+0.103847918 container create 1ef63c17271cf31efc091fccec0ccfd0bf83cf01902f09d8f31f7ff377697910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8-deactivate, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:53.930 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:53 smithi136 podman[81597]: 2024-06-13 07:28:53.550329625 +0000 UTC m=+0.016943201 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:28:53.931 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:53 smithi136 podman[81597]: 2024-06-13 07:28:53.847897014 +0000 UTC m=+0.314510597 container init 1ef63c17271cf31efc091fccec0ccfd0bf83cf01902f09d8f31f7ff377697910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-13T07:28:53.931 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:53 smithi136 podman[81597]: 2024-06-13 07:28:53.850943323 +0000 UTC m=+0.317556901 container start 1ef63c17271cf31efc091fccec0ccfd0bf83cf01902f09d8f31f7ff377697910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-13T07:28:53.931 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:53 smithi136 podman[81597]: 2024-06-13 07:28:53.930950381 +0000 UTC m=+0.397563964 container attach 1ef63c17271cf31efc091fccec0ccfd0bf83cf01902f09d8f31f7ff377697910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-13T07:28:54.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:53 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:53.932+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:54.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:53 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:53.932+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:54.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:53 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:53.932+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:54.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:53 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:53.932+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:54.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:53 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:53.932+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:54.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:53 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:53.932+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:54.377 INFO:journalctl@ceph.osd.8.smithi136.stdout:Jun 13 07:28:54 smithi136 podman[81597]: 2024-06-13 07:28:54.139664817 +0000 UTC m=+0.606278385 container died 1ef63c17271cf31efc091fccec0ccfd0bf83cf01902f09d8f31f7ff377697910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-13T07:28:54.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:54 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:54.086+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:54.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:54 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:54.086+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:54.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:54 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:54.086+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:54.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:54 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:54.086+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:54.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:54 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:54.086+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:54.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:54 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:54.086+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:54.777 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.8.service' 2024-06-13T07:28:54.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:28:54.815 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-13T07:28:54.815 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-13T07:28:54.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.9 2024-06-13T07:28:55.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:54.715+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:55.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:54.715+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:55.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:54.715+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:55.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:54.715+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:55.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:54 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:54.715+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:55.252 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:54 smithi026 systemd[1]: Stopping Ceph osd.9 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:28:55.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.074+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:55.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.074+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:55.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.074+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:55.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.074+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:55.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.074+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:55.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.074+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:55.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:54.906+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:55.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:54.906+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:55.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:54.906+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:55.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:54.906+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:55.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:54.906+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:55.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:54 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:54.906+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:55.752 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.386+0000 7f1df5600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:28:55.752 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.386+0000 7f1df5600700 -1 osd.9 103 *** Got signal Terminated *** 2024-06-13T07:28:55.753 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:55 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:55.386+0000 7f1df5600700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:28:56.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:55.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:56.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:55.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:56.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:55.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:56.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:55.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:56.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:55.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:56.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:55 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:55.740+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:28:56.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:55.912+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:56.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:55.912+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:56.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:55.912+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:56.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:55.912+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:56.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:55.912+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:56.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:55.912+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:56.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:56 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:56.068+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:56.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:56 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:56.068+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:56.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:56 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:56.068+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:56.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:56 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:56.068+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:56.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:56 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:56.068+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:56.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:56 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:56.068+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:57.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:56.707+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:57.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:56.707+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:57.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:56.707+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:57.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:56.707+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:57.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:56.707+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:57.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:56 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:56.707+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:28:57.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:56.925+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:57.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:56.925+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:57.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:56.925+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:57.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:56.925+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:57.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:56.925+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:57.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:56 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:56.925+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:57.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:57 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.043+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:57.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:57 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.043+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:57.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:57 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.043+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:57.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:57 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.043+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:57.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:57 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.043+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:57.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:57 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.043+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:58.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:57 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:57.688+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:58.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:57 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:57.688+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:58.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:57 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:57.688+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:58.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:57 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:57.688+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:58.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:57 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:57.688+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:58.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:57 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:57.688+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:28:58.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.999+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:58.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.999+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:58.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.999+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:58.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.999+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:58.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.999+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:58.253 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:58 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:57.999+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:28:58.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:57 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:57.970+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:58.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:57 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:57.970+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:58.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:57 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:57.970+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:58.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:57 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:57.970+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:58.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:57 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:57.970+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:58.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:57 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:57.970+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:59.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:58 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:58.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:28:59.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:58 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:58.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:28:59.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:58 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:58.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:28:59.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:58 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:58.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:28:59.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:58 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:58.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:28:59.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:58 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:58.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:28:59.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:58.945+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:28:59.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:58.945+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:28:59.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:58.945+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:28:59.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:58.945+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:28:59.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:58.945+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:28:59.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:58.945+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:28:59.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:58 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:58.945+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:28:59.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:59 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:59.046+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:28:59.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:59 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:59.046+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:28:59.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:59 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:59.046+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:28:59.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:59 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:59.046+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:28:59.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:59 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:59.046+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:28:59.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:28:59 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:28:59.046+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:29:00.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:59 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:59.732+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:00.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:59 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:59.732+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:00.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:59 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:59.732+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:00.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:59 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:59.732+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:00.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:59 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:59.732+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:00.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:28:59 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:28:59.732+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:00.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:59.937+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:00.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:59.937+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:00.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:59.937+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:00.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:59.937+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:00.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:59.937+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:00.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:59.937+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:00.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:28:59 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:28:59.937+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:00.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:29:00.038+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:37.722160+0000 front 2024-06-13T07:27:37.721635+0000 (oldest deadline 2024-06-13T07:28:03.621762+0000) 2024-06-13T07:29:00.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:29:00.038+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:48.324767+0000 front 2024-06-13T07:27:48.324595+0000 (oldest deadline 2024-06-13T07:28:15.326480+0000) 2024-06-13T07:29:00.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:29:00.038+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:57.029428+0000 front 2024-06-13T07:27:57.029576+0000 (oldest deadline 2024-06-13T07:28:22.231148+0000) 2024-06-13T07:29:00.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:29:00.038+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:07.533686+0000 front 2024-06-13T07:28:07.533743+0000 (oldest deadline 2024-06-13T07:28:32.833895+0000) 2024-06-13T07:29:00.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:29:00.038+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:14.536483+0000 front 2024-06-13T07:28:14.536287+0000 (oldest deadline 2024-06-13T07:28:40.436385+0000) 2024-06-13T07:29:00.503 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:00 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9[67131]: 2024-06-13T07:29:00.038+0000 7f1df0e00700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:23.938082+0000 front 2024-06-13T07:28:23.938138+0000 (oldest deadline 2024-06-13T07:28:49.237924+0000) 2024-06-13T07:29:01.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:00 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:00.779+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:01.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:00 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:00.779+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:01.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:00 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:00.779+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:01.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:00 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:00.779+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:01.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:00 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:00.779+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:01.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:00 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:00.779+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:01.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:00.979+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:01.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:00.979+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:01.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:00.979+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:01.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:00.979+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:01.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:00.979+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:01.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:00.979+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:01.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:00 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:00.979+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:01.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:01 smithi026 podman[112630]: 2024-06-13 07:29:01.155547323 +0000 UTC m=+5.912441676 container died 4f218e7c2e4279a98d9d4bfde6e8c74596e99ffb25f1efaa894d2a05318cda19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:29:02.002 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:01 smithi026 podman[112630]: 2024-06-13 07:29:01.78765648 +0000 UTC m=+6.544550840 container cleanup 4f218e7c2e4279a98d9d4bfde6e8c74596e99ffb25f1efaa894d2a05318cda19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-06-13T07:29:02.002 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:01 smithi026 bash[112630]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9 2024-06-13T07:29:02.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:01 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:01.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:02.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:01 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:01.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:02.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:01 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:01.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:02.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:01 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:01.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:02.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:01 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:01.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:02.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:01 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:01.739+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:02.237 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:01 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:01.955+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:02.237 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:01 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:01.955+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:02.237 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:01 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:01.955+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:02.237 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:01 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:01.955+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:02.238 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:01 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:01.955+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:02.238 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:01 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:01.955+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:02.238 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:01 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:01.955+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:02.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:02 smithi026 podman[112649]: 2024-06-13 07:29:02.013775667 +0000 UTC m=+0.837889157 container remove 4f218e7c2e4279a98d9d4bfde6e8c74596e99ffb25f1efaa894d2a05318cda19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-13T07:29:03.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:02 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:02.755+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:03.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:02 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:02.755+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:03.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:02 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:02.755+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:03.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:02 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:02.755+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:03.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:02 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:02.755+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:03.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:02 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:02.755+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:03.176 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:02 smithi026 podman[112736]: 2024-06-13 07:29:02.795502388 +0000 UTC m=+0.033823856 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:29:03.176 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:02 smithi026 podman[112736]: 2024-06-13 07:29:02.923079077 +0000 UTC m=+0.161400585 container create d3cd64bec0c60abc2aba3fde868aa4bb111851b1d5d24c1220bedb0ef0479fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-06-13T07:29:03.176 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:03 smithi026 podman[112736]: 2024-06-13 07:29:03.176622493 +0000 UTC m=+0.414944002 container init d3cd64bec0c60abc2aba3fde868aa4bb111851b1d5d24c1220bedb0ef0479fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-13T07:29:03.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:02 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:02.944+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:03.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:02 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:02.944+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:03.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:02 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:02.944+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:03.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:02 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:02.944+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:03.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:02 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:02.944+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:03.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:02 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:02.944+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:03.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:02 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:02.944+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:03.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:03 smithi026 podman[112736]: 2024-06-13 07:29:03.188645969 +0000 UTC m=+0.426967469 container start d3cd64bec0c60abc2aba3fde868aa4bb111851b1d5d24c1220bedb0ef0479fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:29:03.502 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:03 smithi026 podman[112736]: 2024-06-13 07:29:03.333627136 +0000 UTC m=+0.571948634 container attach d3cd64bec0c60abc2aba3fde868aa4bb111851b1d5d24c1220bedb0ef0479fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-13T07:29:04.003 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jun 13 07:29:03 smithi026 podman[112736]: 2024-06-13 07:29:03.658780361 +0000 UTC m=+0.897101870 container died d3cd64bec0c60abc2aba3fde868aa4bb111851b1d5d24c1220bedb0ef0479fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:29:04.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:03.773+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:04.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:03.773+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:04.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:03.773+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:04.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:03.773+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:04.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:03.773+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:04.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:03 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:03.773+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:04.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:03.908+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:04.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:03.908+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:04.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:03.908+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:04.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:03.908+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:04.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:03.908+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:04.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:03.908+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:04.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:03 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:03.908+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:04.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.9.service' 2024-06-13T07:29:04.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:04.563 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-13T07:29:04.563 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-13T07:29:04.564 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.10 2024-06-13T07:29:04.870 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 systemd[1]: Stopping Ceph osd.10 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:05.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:04.809+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:05.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:04.809+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:05.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:04.809+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:05.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:04.809+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:05.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:04.809+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:05.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:04.809+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:05.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:05.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:05.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:05.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:05.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:05.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:05.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.871+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:05.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.981+0000 7f0b3d877700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:05.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.981+0000 7f0b3d877700 -1 osd.10 103 *** Got signal Terminated *** 2024-06-13T07:29:05.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:04 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:04.981+0000 7f0b3d877700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:29:06.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:05 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:05.762+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:06.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:05 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:05.762+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:06.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:05 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:05.762+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:06.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:05 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:05.762+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:06.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:05 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:05.762+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:06.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:05 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:05.762+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:06.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:05 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:05.762+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:06.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:05 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:05.859+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:06.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:05 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:05.859+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:06.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:05 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:05.859+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:06.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:05 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:05.859+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:06.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:05 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:05.859+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:06.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:05 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:05.859+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:06.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:05 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:05.859+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:07.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:06.757+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:07.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:06.757+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:07.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:06.757+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:07.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:06.757+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:07.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:06.757+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:07.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:06.757+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:07.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:06 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:06.757+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:07.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:06 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:06.895+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:07.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:06 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:06.895+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:07.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:06 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:06.895+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:07.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:06 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:06.895+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:07.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:06 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:06.895+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:07.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:06 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:06.895+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:07.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:06 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:06.895+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:08.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:07.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:08.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:07.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:08.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:07.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:08.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:07.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:08.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:07.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:08.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:07.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:08.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:07 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:07.737+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:08.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:07 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:07.920+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:08.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:07 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:07.920+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:08.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:07 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:07.920+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:08.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:07 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:07.920+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:08.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:07 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:07.920+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:08.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:07 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:07.920+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:08.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:07 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:07.920+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:09.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:08.780+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:09.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:08.780+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:09.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:08.780+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:09.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:08.780+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:09.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:08.780+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:09.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:08.780+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:09.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:08.780+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:09.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:08 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:08.947+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:09.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:08 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:08.947+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:09.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:08 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:08.947+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:09.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:08 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:08.947+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:09.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:08 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:08.947+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:09.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:08 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:08.947+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:09.284 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:08 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:08.947+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:10.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:09.744+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:10.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:09.744+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:10.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:09.745+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:10.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:09.745+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:10.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:09.745+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:10.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:09.745+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:10.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:09 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:09.745+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:10.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:09 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:09.978+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.826304+0000 front 2024-06-13T07:27:36.826355+0000 (oldest deadline 2024-06-13T07:28:02.726346+0000) 2024-06-13T07:29:10.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:09 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:09.978+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.328367+0000 front 2024-06-13T07:27:50.328391+0000 (oldest deadline 2024-06-13T07:28:15.028491+0000) 2024-06-13T07:29:10.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:09 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:09.978+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.430100+0000 front 2024-06-13T07:27:58.429947+0000 (oldest deadline 2024-06-13T07:28:23.130117+0000) 2024-06-13T07:29:10.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:09 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:09.978+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-06-13T07:28:03.131188+0000 front 2024-06-13T07:28:03.130987+0000 (oldest deadline 2024-06-13T07:28:33.731526+0000) 2024-06-13T07:29:10.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:09 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:09.978+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:16.632528+0000 front 2024-06-13T07:28:16.632584+0000 (oldest deadline 2024-06-13T07:28:41.332703+0000) 2024-06-13T07:29:10.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:09 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:09.978+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.333124+0000 front 2024-06-13T07:28:21.333167+0000 (oldest deadline 2024-06-13T07:28:47.233065+0000) 2024-06-13T07:29:10.283 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:09 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10[61184]: 2024-06-13T07:29:09.978+0000 7f0b35e63700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:32.434879+0000 front 2024-06-13T07:28:32.434698+0000 (oldest deadline 2024-06-13T07:28:58.334424+0000) 2024-06-13T07:29:10.757 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:10 smithi135 podman[77487]: 2024-06-13 07:29:10.490445149 +0000 UTC m=+5.719678129 container died 03fbffabc1d80aafb255754e5a116359781c7cc4cc03252f7fa5891f501c3f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:29:11.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:10 smithi135 podman[77487]: 2024-06-13 07:29:10.839717394 +0000 UTC m=+6.068950372 container cleanup 03fbffabc1d80aafb255754e5a116359781c7cc4cc03252f7fa5891f501c3f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:29:11.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:10 smithi135 bash[77487]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10 2024-06-13T07:29:11.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:10.774+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:11.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:10.774+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:11.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:10.774+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:11.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:10.774+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:11.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:10.774+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:11.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:10.774+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:11.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:10 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:10.774+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:11.511 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:11 smithi135 podman[77504]: 2024-06-13 07:29:11.149209248 +0000 UTC m=+0.656302966 container remove 03fbffabc1d80aafb255754e5a116359781c7cc4cc03252f7fa5891f501c3f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:29:11.768 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:11 smithi135 podman[77597]: 2024-06-13 07:29:11.415358456 +0000 UTC m=+0.017537155 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:29:11.768 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:11 smithi135 podman[77597]: 2024-06-13 07:29:11.533690546 +0000 UTC m=+0.135869236 container create 4c5327b67e65a5ec41076d8f0facdd65ff8dcc2b847f96b388ccc8bd5b58238c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-13T07:29:11.768 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:11 smithi135 podman[77597]: 2024-06-13 07:29:11.7694638 +0000 UTC m=+0.371642498 container init 4c5327b67e65a5ec41076d8f0facdd65ff8dcc2b847f96b388ccc8bd5b58238c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-06-13T07:29:12.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:11 smithi135 podman[77597]: 2024-06-13 07:29:11.7727988 +0000 UTC m=+0.374977492 container start 4c5327b67e65a5ec41076d8f0facdd65ff8dcc2b847f96b388ccc8bd5b58238c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-13T07:29:12.033 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:11 smithi135 podman[77597]: 2024-06-13 07:29:11.852592966 +0000 UTC m=+0.454771654 container attach 4c5327b67e65a5ec41076d8f0facdd65ff8dcc2b847f96b388ccc8bd5b58238c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-13T07:29:12.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:11.747+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:12.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:11.747+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:12.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:11.747+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:12.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:11.747+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:12.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:11.747+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:12.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:11.747+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:12.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:11 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:11.747+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:12.354 INFO:journalctl@ceph.osd.10.smithi135.stdout:Jun 13 07:29:12 smithi135 podman[77597]: 2024-06-13 07:29:12.062029419 +0000 UTC m=+0.664208107 container died 4c5327b67e65a5ec41076d8f0facdd65ff8dcc2b847f96b388ccc8bd5b58238c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-13T07:29:12.673 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.10.service' 2024-06-13T07:29:12.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:12.712 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-13T07:29:12.712 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-13T07:29:12.712 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.11 2024-06-13T07:29:13.034 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:12 smithi136 systemd[1]: Stopping Ceph osd.11 for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:12.781+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:12.781+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:12.781+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:12.781+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:12.781+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:12.781+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:12.781+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.034+0000 7f4fc52b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.034+0000 7f4fc52b7700 -1 osd.11 103 *** Got signal Terminated *** 2024-06-13T07:29:13.035 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.034+0000 7f4fc52b7700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:29:14.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:14.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:14.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:14.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:14.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:14.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:14.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:13 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:13.742+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:15.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:14.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:15.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:14.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:15.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:14.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:15.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:14.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:15.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:14.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:15.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:14.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:15.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:14 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:14.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:16.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:15.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:16.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:15.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:16.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:15.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:16.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:15.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:16.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:15.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:16.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:15.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:16.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:15 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:15.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:17.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:16.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:17.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:16.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:17.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:16.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:17.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:16.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:17.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:16.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:17.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:16.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:17.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:16.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:17.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:16 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:16.770+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-06-13T07:28:50.227619+0000 front 2024-06-13T07:28:50.227656+0000 (oldest deadline 2024-06-13T07:29:16.127631+0000) 2024-06-13T07:29:18.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:17.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-06-13T07:27:36.112513+0000 front 2024-06-13T07:27:36.112452+0000 (oldest deadline 2024-06-13T07:28:06.113113+0000) 2024-06-13T07:29:18.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:17.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-06-13T07:27:50.715007+0000 front 2024-06-13T07:27:50.714937+0000 (oldest deadline 2024-06-13T07:28:11.815081+0000) 2024-06-13T07:29:18.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:17.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-06-13T07:27:58.717615+0000 front 2024-06-13T07:27:58.717640+0000 (oldest deadline 2024-06-13T07:28:21.617648+0000) 2024-06-13T07:29:18.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:17.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-06-13T07:28:17.221778+0000 front 2024-06-13T07:28:17.221794+0000 (oldest deadline 2024-06-13T07:28:41.321687+0000) 2024-06-13T07:29:18.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:17.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-06-13T07:28:21.322073+0000 front 2024-06-13T07:28:21.322016+0000 (oldest deadline 2024-06-13T07:28:47.722697+0000) 2024-06-13T07:29:18.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:17.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-06-13T07:28:34.525315+0000 front 2024-06-13T07:28:34.525187+0000 (oldest deadline 2024-06-13T07:28:55.624984+0000) 2024-06-13T07:29:18.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:17.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-06-13T07:28:39.026739+0000 front 2024-06-13T07:28:39.026757+0000 (oldest deadline 2024-06-13T07:29:04.926699+0000) 2024-06-13T07:29:18.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:17 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11[62808]: 2024-06-13T07:29:17.743+0000 7f4fbd8a3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-06-13T07:28:50.227619+0000 front 2024-06-13T07:28:50.227656+0000 (oldest deadline 2024-06-13T07:29:16.127631+0000) 2024-06-13T07:29:18.784 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:18 smithi136 podman[81716]: 2024-06-13 07:29:18.513744388 +0000 UTC m=+5.581006202 container died 90c99be6c2027fc9a8c08e648d97d96fb133948845ce1567cf1594187b90541b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-06-13T07:29:19.126 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:18 smithi136 podman[81716]: 2024-06-13 07:29:18.959370172 +0000 UTC m=+6.026631984 container cleanup 90c99be6c2027fc9a8c08e648d97d96fb133948845ce1567cf1594187b90541b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-13T07:29:19.127 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:18 smithi136 bash[81716]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11 2024-06-13T07:29:19.626 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:19 smithi136 podman[81731]: 2024-06-13 07:29:19.194300502 +0000 UTC m=+0.677360628 container remove 90c99be6c2027fc9a8c08e648d97d96fb133948845ce1567cf1594187b90541b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:29:19.912 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:19 smithi136 podman[81820]: 2024-06-13 07:29:19.531448772 +0000 UTC m=+0.017089290 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:29:19.912 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:19 smithi136 podman[81820]: 2024-06-13 07:29:19.636482663 +0000 UTC m=+0.122123170 container create 04403f43b9982c633456fbe35583716dbfa52fc143c09adfe4a7e2d643db9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-13T07:29:19.912 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:19 smithi136 podman[81820]: 2024-06-13 07:29:19.813774764 +0000 UTC m=+0.299415285 container init 04403f43b9982c633456fbe35583716dbfa52fc143c09adfe4a7e2d643db9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-13T07:29:19.912 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:19 smithi136 podman[81820]: 2024-06-13 07:29:19.817026419 +0000 UTC m=+0.302666925 container start 04403f43b9982c633456fbe35583716dbfa52fc143c09adfe4a7e2d643db9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-13T07:29:19.912 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:19 smithi136 podman[81820]: 2024-06-13 07:29:19.913667013 +0000 UTC m=+0.399307522 container attach 04403f43b9982c633456fbe35583716dbfa52fc143c09adfe4a7e2d643db9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-13T07:29:20.376 INFO:journalctl@ceph.osd.11.smithi136.stdout:Jun 13 07:29:20 smithi136 podman[81820]: 2024-06-13 07:29:20.103708636 +0000 UTC m=+0.589349144 container died 04403f43b9982c633456fbe35583716dbfa52fc143c09adfe4a7e2d643db9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-13T07:29:20.753 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@osd.11.service' 2024-06-13T07:29:20.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:20.789 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-13T07:29:20.790 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-13T07:29:20.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.a 2024-06-13T07:29:21.252 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:29:20 smithi026 systemd[1]: Stopping Ceph mds.a for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:21.671 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:29:21 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a[74382]: 2024-06-13T07:29:21.352+0000 7f7782200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:21.671 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:29:21 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a[74382]: 2024-06-13T07:29:21.352+0000 7f7782200700 -1 mds.a *** got signal Terminated *** 2024-06-13T07:29:24.319 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:29:24 smithi026 podman[112840]: 2024-06-13 07:29:24.024215468 +0000 UTC m=+2.838528248 container died 80963fba2169601c989395b0ee9ea8f6d8280df5b9393ab9454fd9ae8089e52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-13T07:29:24.713 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:29:24 smithi026 podman[112840]: 2024-06-13 07:29:24.507164618 +0000 UTC m=+3.321477391 container cleanup 80963fba2169601c989395b0ee9ea8f6d8280df5b9393ab9454fd9ae8089e52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:29:24.714 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:29:24 smithi026 bash[112840]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a 2024-06-13T07:29:25.002 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jun 13 07:29:24 smithi026 podman[112863]: 2024-06-13 07:29:24.714409795 +0000 UTC m=+0.668818122 container remove 80963fba2169601c989395b0ee9ea8f6d8280df5b9393ab9454fd9ae8089e52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-a, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-13T07:29:25.265 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.a.service' 2024-06-13T07:29:25.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:25.377 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-13T07:29:25.378 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-13T07:29:25.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.d 2024-06-13T07:29:26.003 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 07:29:25 smithi026 systemd[1]: Stopping Ceph mds.d for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:26.502 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 07:29:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-d[73212]: 2024-06-13T07:29:26.029+0000 7fc06e600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:26.503 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 07:29:26 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-d[73212]: 2024-06-13T07:29:26.029+0000 7fc06e600700 -1 mds.d *** got signal Terminated *** 2024-06-13T07:29:30.252 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 07:29:29 smithi026 podman[112971]: 2024-06-13 07:29:29.985864548 +0000 UTC m=+4.117697734 container died 7b56f2264536c6f3d099c6aaf6bcca20e7b2049226f691d528b04b96f0d3091f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-13T07:29:30.727 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 07:29:30 smithi026 podman[112971]: 2024-06-13 07:29:30.45115157 +0000 UTC m=+4.582984760 container cleanup 7b56f2264536c6f3d099c6aaf6bcca20e7b2049226f691d528b04b96f0d3091f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-d, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-13T07:29:30.727 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 07:29:30 smithi026 bash[112971]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-d 2024-06-13T07:29:31.002 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jun 13 07:29:30 smithi026 podman[112984]: 2024-06-13 07:29:30.727293637 +0000 UTC m=+0.699196557 container remove 7b56f2264536c6f3d099c6aaf6bcca20e7b2049226f691d528b04b96f0d3091f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-13T07:29:31.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.d.service' 2024-06-13T07:29:31.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:31.409 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-13T07:29:31.409 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-13T07:29:31.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.g 2024-06-13T07:29:31.987 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:29:31 smithi026 systemd[1]: Stopping Ceph mds.g for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:32.252 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:29:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: 2024-06-13T07:29:31.986+0000 7fe3ade00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:32.252 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:29:31 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g[73985]: 2024-06-13T07:29:31.986+0000 7fe3ade00700 -1 mds.g *** got signal Terminated *** 2024-06-13T07:29:36.325 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:29:36 smithi026 podman[113091]: 2024-06-13 07:29:36.01023115 +0000 UTC m=+4.144580055 container died e8a84a9ef3c5701505ced762c37272798e029bb437ce1eb21f29a6b964d77326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-13T07:29:36.578 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:29:36 smithi026 podman[113091]: 2024-06-13 07:29:36.446149199 +0000 UTC m=+4.580498098 container cleanup e8a84a9ef3c5701505ced762c37272798e029bb437ce1eb21f29a6b964d77326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-13T07:29:36.578 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:29:36 smithi026 bash[113091]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g 2024-06-13T07:29:37.002 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jun 13 07:29:36 smithi026 podman[113103]: 2024-06-13 07:29:36.764301493 +0000 UTC m=+0.740225153 container remove e8a84a9ef3c5701505ced762c37272798e029bb437ce1eb21f29a6b964d77326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-g, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-13T07:29:37.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.g.service' 2024-06-13T07:29:37.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:37.459 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-13T07:29:37.459 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-13T07:29:37.459 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.j 2024-06-13T07:29:38.002 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jun 13 07:29:37 smithi026 systemd[1]: Stopping Ceph mds.j for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:38.502 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jun 13 07:29:38 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-j[72796]: 2024-06-13T07:29:38.106+0000 7f1ac4600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:38.503 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jun 13 07:29:38 smithi026 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-j[72796]: 2024-06-13T07:29:38.106+0000 7f1ac4600700 -1 mds.j *** got signal Terminated *** 2024-06-13T07:29:40.752 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jun 13 07:29:40 smithi026 podman[113212]: 2024-06-13 07:29:40.489729573 +0000 UTC m=+2.530233793 container died d470254b40177869308865bf959217a4ffe78f958c70b05df3e62588567feef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-j, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-13T07:29:41.101 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jun 13 07:29:40 smithi026 podman[113212]: 2024-06-13 07:29:40.917048647 +0000 UTC m=+2.957552868 container cleanup d470254b40177869308865bf959217a4ffe78f958c70b05df3e62588567feef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-j, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-13T07:29:41.101 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jun 13 07:29:40 smithi026 bash[113212]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-j 2024-06-13T07:29:41.502 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jun 13 07:29:41 smithi026 podman[113225]: 2024-06-13 07:29:41.101306884 +0000 UTC m=+0.592515115 container remove d470254b40177869308865bf959217a4ffe78f958c70b05df3e62588567feef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-j, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-13T07:29:41.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.j.service' 2024-06-13T07:29:41.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:41.765 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-13T07:29:41.765 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-13T07:29:41.765 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.b 2024-06-13T07:29:42.097 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:41 smithi135 systemd[1]: Stopping Ceph mds.b for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:42.533 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:29:42.097+0000 7f2e7b8ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:42.533 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:42 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b[66558]: 2024-06-13T07:29:42.097+0000 7f2e7b8ab700 -1 mds.b *** got signal Terminated *** 2024-06-13T07:29:43.914 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:43 smithi135 podman[77703]: 2024-06-13 07:29:43.657204277 +0000 UTC m=+1.672660953 container died 9e7a3d12d3b61765aee5df85890a8287a8bec8e45ed03d3dbd1be7878b57ecf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-13T07:29:44.283 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:44 smithi135 podman[77703]: 2024-06-13 07:29:44.089822816 +0000 UTC m=+2.105279533 container cleanup 9e7a3d12d3b61765aee5df85890a8287a8bec8e45ed03d3dbd1be7878b57ecf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-13T07:29:44.283 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:44 smithi135 bash[77703]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b 2024-06-13T07:29:44.528 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.b.service' 2024-06-13T07:29:44.549 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:44 smithi135 podman[77729]: 2024-06-13 07:29:44.299452349 +0000 UTC m=+0.643521246 container remove 9e7a3d12d3b61765aee5df85890a8287a8bec8e45ed03d3dbd1be7878b57ecf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-13T07:29:44.549 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:44 smithi135 systemd[1]: ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.b.service: Deactivated successfully. 2024-06-13T07:29:44.549 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:44 smithi135 systemd[1]: Stopped Ceph mds.b for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T07:29:44.549 INFO:journalctl@ceph.mds.b.smithi135.stdout:Jun 13 07:29:44 smithi135 systemd[1]: ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.b.service: Consumed 14min 41.608s CPU time. 2024-06-13T07:29:44.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:44.562 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-13T07:29:44.563 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-13T07:29:44.563 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.e 2024-06-13T07:29:44.885 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 07:29:44 smithi135 systemd[1]: Stopping Ceph mds.e for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:45.283 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 07:29:44 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e[65937]: 2024-06-13T07:29:44.885+0000 7f902c52a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:45.283 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 07:29:44 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e[65937]: 2024-06-13T07:29:44.885+0000 7f902c52a700 -1 mds.e *** got signal Terminated *** 2024-06-13T07:29:49.151 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 07:29:48 smithi135 podman[77836]: 2024-06-13 07:29:48.870475432 +0000 UTC m=+4.084439052 container died 35b31f9a25c97c33667d50fb342ce1d5f6ab344d4fcdcdbb6a6893ace5f2aa51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:29:49.518 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 07:29:49 smithi135 podman[77836]: 2024-06-13 07:29:49.235134107 +0000 UTC m=+4.449097737 container cleanup 35b31f9a25c97c33667d50fb342ce1d5f6ab344d4fcdcdbb6a6893ace5f2aa51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:29:49.519 INFO:journalctl@ceph.mds.e.smithi135.stdout:Jun 13 07:29:49 smithi135 bash[77836]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-e 2024-06-13T07:29:49.660 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.e.service' 2024-06-13T07:29:49.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:49.692 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-13T07:29:49.692 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-13T07:29:49.692 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.h 2024-06-13T07:29:49.783 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 07:29:49 smithi135 systemd[1]: Stopping Ceph mds.h for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:50.283 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 07:29:50 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h[66243]: 2024-06-13T07:29:50.013+0000 7f476d843700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:50.283 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 07:29:50 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h[66243]: 2024-06-13T07:29:50.013+0000 7f476d843700 -1 mds.h *** got signal Terminated *** 2024-06-13T07:29:54.131 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 07:29:53 smithi135 podman[77957]: 2024-06-13 07:29:53.855556164 +0000 UTC m=+3.947003563 container died f27642e54dd87fd11822fa7e1ed127ca749b157af35961a7314eb44fe4fa3343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-13T07:29:54.533 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 07:29:54 smithi135 podman[77957]: 2024-06-13 07:29:54.222213205 +0000 UTC m=+4.313660646 container cleanup f27642e54dd87fd11822fa7e1ed127ca749b157af35961a7314eb44fe4fa3343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-06-13T07:29:54.533 INFO:journalctl@ceph.mds.h.smithi135.stdout:Jun 13 07:29:54 smithi135 bash[77957]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-h 2024-06-13T07:29:54.802 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.h.service' 2024-06-13T07:29:54.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:54.833 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-13T07:29:54.833 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-13T07:29:54.834 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.k 2024-06-13T07:29:55.166 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 07:29:54 smithi135 systemd[1]: Stopping Ceph mds.k for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:29:55.533 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 07:29:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k[65623]: 2024-06-13T07:29:55.167+0000 7f4949c20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:29:55.533 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 07:29:55 smithi135 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k[65623]: 2024-06-13T07:29:55.167+0000 7f4949c20700 -1 mds.k *** got signal Terminated *** 2024-06-13T07:29:59.249 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 07:29:58 smithi135 podman[78078]: 2024-06-13 07:29:58.914151764 +0000 UTC m=+3.863211172 container died 2bd02c9f1caa7660cabadd093600c4ef2a116e5a0e85c104b61e6801e531c86f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-13T07:29:59.533 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 07:29:59 smithi135 podman[78078]: 2024-06-13 07:29:59.34245562 +0000 UTC m=+4.291515028 container cleanup 2bd02c9f1caa7660cabadd093600c4ef2a116e5a0e85c104b61e6801e531c86f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:29:59.533 INFO:journalctl@ceph.mds.k.smithi135.stdout:Jun 13 07:29:59 smithi135 bash[78078]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-k 2024-06-13T07:29:59.760 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.k.service' 2024-06-13T07:29:59.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:29:59.791 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-13T07:29:59.791 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-13T07:29:59.792 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.c 2024-06-13T07:30:00.087 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:29:59 smithi136 systemd[1]: Stopping Ceph mds.c for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:30:00.377 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:00 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c[68243]: 2024-06-13T07:30:00.087+0000 7f0468caf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:30:00.377 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:00 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c[68243]: 2024-06-13T07:30:00.087+0000 7f0468caf700 -1 mds.c *** got signal Terminated *** 2024-06-13T07:30:03.877 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:03 smithi136 podman[81927]: 2024-06-13 07:30:03.581245135 +0000 UTC m=+3.587282164 container died 0a3f5b1f765a2fdd694d5d3c21f0801443cb3954da3a94819d665ec16a41b9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-13T07:30:04.196 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:03 smithi136 podman[81927]: 2024-06-13 07:30:03.99620245 +0000 UTC m=+4.002239468 container cleanup 0a3f5b1f765a2fdd694d5d3c21f0801443cb3954da3a94819d665ec16a41b9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:30:04.196 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:03 smithi136 bash[81927]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c 2024-06-13T07:30:04.428 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.c.service' 2024-06-13T07:30:04.450 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:04 smithi136 podman[81954]: 2024-06-13 07:30:04.197488673 +0000 UTC m=+0.611963047 container remove 0a3f5b1f765a2fdd694d5d3c21f0801443cb3954da3a94819d665ec16a41b9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-06-13T07:30:04.450 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:04 smithi136 systemd[1]: ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.c.service: Deactivated successfully. 2024-06-13T07:30:04.450 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:04 smithi136 systemd[1]: Stopped Ceph mds.c for 559474f0-2950-11ef-bc9c-c7b262605968. 2024-06-13T07:30:04.450 INFO:journalctl@ceph.mds.c.smithi136.stdout:Jun 13 07:30:04 smithi136 systemd[1]: ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.c.service: Consumed 47.359s CPU time. 2024-06-13T07:30:04.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:30:04.464 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-13T07:30:04.464 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-13T07:30:04.464 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.f 2024-06-13T07:30:04.783 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:30:04 smithi136 systemd[1]: Stopping Ceph mds.f for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:30:05.126 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:30:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f[67939]: 2024-06-13T07:30:04.783+0000 7f3ee1219700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:30:05.127 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:30:04 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f[67939]: 2024-06-13T07:30:04.783+0000 7f3ee1219700 -1 mds.f *** got signal Terminated *** 2024-06-13T07:30:07.859 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:30:07 smithi136 podman[82062]: 2024-06-13 07:30:07.586458807 +0000 UTC m=+2.903508754 container died 39599f86b7a559e58f6ef46d5f71bf2f4e83cd20f7aa8940a2c3d1f44ef6a1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-13T07:30:08.126 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:30:07 smithi136 podman[82062]: 2024-06-13 07:30:07.941323883 +0000 UTC m=+3.258373826 container cleanup 39599f86b7a559e58f6ef46d5f71bf2f4e83cd20f7aa8940a2c3d1f44ef6a1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-13T07:30:08.126 INFO:journalctl@ceph.mds.f.smithi136.stdout:Jun 13 07:30:07 smithi136 bash[82062]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-f 2024-06-13T07:30:08.355 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.f.service' 2024-06-13T07:30:08.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:30:08.387 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-13T07:30:08.387 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-13T07:30:08.387 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.i 2024-06-13T07:30:08.626 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:30:08 smithi136 systemd[1]: Stopping Ceph mds.i for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:30:09.126 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:30:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: 2024-06-13T07:30:08.710+0000 7f16b83a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:30:09.126 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:30:08 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i[68547]: 2024-06-13T07:30:08.710+0000 7f16b83a8700 -1 mds.i *** got signal Terminated *** 2024-06-13T07:30:11.827 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:30:11 smithi136 podman[82183]: 2024-06-13 07:30:11.573773033 +0000 UTC m=+2.971342246 container died 2de7eedbed6cce50fd5e48ff87fe52cc0bb5daf1168e3ea0cd5782f84954ac55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:30:12.126 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:30:11 smithi136 podman[82183]: 2024-06-13 07:30:11.919638622 +0000 UTC m=+3.317207902 container cleanup 2de7eedbed6cce50fd5e48ff87fe52cc0bb5daf1168e3ea0cd5782f84954ac55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-13T07:30:12.126 INFO:journalctl@ceph.mds.i.smithi136.stdout:Jun 13 07:30:11 smithi136 bash[82183]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-i 2024-06-13T07:30:12.391 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.i.service' 2024-06-13T07:30:12.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:30:12.422 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-13T07:30:12.422 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-13T07:30:12.423 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.l 2024-06-13T07:30:12.876 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 07:30:12 smithi136 systemd[1]: Stopping Ceph mds.l for 559474f0-2950-11ef-bc9c-c7b262605968... 2024-06-13T07:30:12.877 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 07:30:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l[67624]: 2024-06-13T07:30:12.730+0000 7fe2c8a14700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:30:12.877 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 07:30:12 smithi136 ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l[67624]: 2024-06-13T07:30:12.730+0000 7fe2c8a14700 -1 mds.l *** got signal Terminated *** 2024-06-13T07:30:16.195 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 07:30:15 smithi136 podman[82305]: 2024-06-13 07:30:15.924720176 +0000 UTC m=+3.286320684 container died f687d66e3dc9527555db7e91e063559ebf69dfa15a6b41c04970da68a43f564e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:30:16.564 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 07:30:16 smithi136 podman[82305]: 2024-06-13 07:30:16.297843314 +0000 UTC m=+3.659443820 container cleanup f687d66e3dc9527555db7e91e063559ebf69dfa15a6b41c04970da68a43f564e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-13T07:30:16.564 INFO:journalctl@ceph.mds.l.smithi136.stdout:Jun 13 07:30:16 smithi136 bash[82305]: ceph-559474f0-2950-11ef-bc9c-c7b262605968-mds-l 2024-06-13T07:30:16.723 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-559474f0-2950-11ef-bc9c-c7b262605968@mds.l.service' 2024-06-13T07:30:16.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:30:16.753 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-13T07:30:16.754 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 559474f0-2950-11ef-bc9c-c7b262605968 --force --keep-logs 2024-06-13T07:30:17.086 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T07:30:22.318 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 559474f0-2950-11ef-bc9c-c7b262605968 --force --keep-logs 2024-06-13T07:30:22.507 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T07:30:25.061 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 559474f0-2950-11ef-bc9c-c7b262605968 --force --keep-logs 2024-06-13T07:30:25.250 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T07:30:27.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:30:27.869 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:30:27.900 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:30:27.926 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-13T07:30:27.928 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi026/crash 2024-06-13T07:30:27.929 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash -- . 2024-06-13T07:30:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash: Cannot open: No such file or directory 2024-06-13T07:30:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2024-06-13T07:30:28.083 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi135/crash 2024-06-13T07:30:28.084 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash -- . 2024-06-13T07:30:28.119 INFO:teuthology.orchestra.run.smithi135.stderr:tar: /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash: Cannot open: No such file or directory 2024-06-13T07:30:28.119 INFO:teuthology.orchestra.run.smithi135.stderr:tar: Error is not recoverable: exiting now 2024-06-13T07:30:28.121 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi136/crash 2024-06-13T07:30:28.122 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash -- . 2024-06-13T07:30:28.158 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/559474f0-2950-11ef-bc9c-c7b262605968/crash: Cannot open: No such file or directory 2024-06-13T07:30:28.158 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-06-13T07:30:28.159 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-13T07:30:28.159 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-13T07:30:28.322 INFO:teuthology.orchestra.run.smithi026.stdout:2024-06-13T07:02:14.077654+0000 mds.b (mds.2) 2 : cluster [WRN] Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/1d7d31dc-0547-4061-aea7-beb40a9acb25/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.b log and `damage ls` output for details 2024-06-13T07:30:28.323 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-13T07:30:28.323 INFO:tasks.cephadm:Compressing logs... 2024-06-13T07:30:28.324 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-13T07:30:28.328 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-13T07:30:28.331 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-13T07:30:28.363 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-13T07:30:28.363 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log 2024-06-13T07:30:28.364 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.2.log 2024-06-13T07:30:28.365 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.f.log 2024-06-13T07:30:28.365 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log 2024-06-13T07:30:28.367 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.5.log 2024-06-13T07:30:28.367 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.c.log 2024-06-13T07:30:28.367 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.c.log 2024-06-13T07:30:28.368 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.z.log 2024-06-13T07:30:28.368 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.i.log 2024-06-13T07:30:28.368 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.z.log: -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log 2024-06-13T07:30:28.368 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.i.log: --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.11.log 2024-06-13T07:30:28.368 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.8.log 2024-06-13T07:30:28.368 INFO:teuthology.orchestra.run.smithi136.stderr:gzip/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.11.log: -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log 2024-06-13T07:30:28.368 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.l.log 2024-06-13T07:30:28.368 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-13T07:30:28.370 INFO:teuthology.orchestra.run.smithi135.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-13T07:30:28.371 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log 2024-06-13T07:30:28.372 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log 2024-06-13T07:30:28.372 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.7.log 2024-06-13T07:30:28.372 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.b.log 2024-06-13T07:30:28.374 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.4.log 2024-06-13T07:30:28.375 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.1.log 2024-06-13T07:30:28.375 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.y.log 2024-06-13T07:30:28.375 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.h.log 2024-06-13T07:30:28.375 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.10.log 2024-06-13T07:30:28.375 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log 2024-06-13T07:30:28.375 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.e.log 2024-06-13T07:30:28.375 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log 2024-06-13T07:30:28.376 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.b.log 2024-06-13T07:30:28.376 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.k.log 2024-06-13T07:30:28.376 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-13T07:30:28.404 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-06-13T07:30:28.404 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log 2024-06-13T07:30:28.405 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log 2024-06-13T07:30:28.406 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.g.log 2024-06-13T07:30:28.407 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.j.log 2024-06-13T07:30:28.409 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.3.log 2024-06-13T07:30:28.410 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.0.log 2024-06-13T07:30:28.410 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log.gz 2024-06-13T07:30:28.411 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.x.log 2024-06-13T07:30:28.412 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.a.log 2024-06-13T07:30:28.413 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.6.log 2024-06-13T07:30:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log 2024-06-13T07:30:28.415 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.a.log 2024-06-13T07:30:28.416 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log 2024-06-13T07:30:28.417 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.a.log: 94.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.j.log.gz 2024-06-13T07:30:28.417 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.d.log 2024-06-13T07:30:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.9.log 2024-06-13T07:30:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-13T07:30:28.419 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-13T07:30:28.423 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-13T07:30:28.426 INFO:teuthology.orchestra.run.smithi026.stderr: 94.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.d.log.gz 2024-06-13T07:30:28.452 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 84.1% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log.gz 2024-06-13T07:30:28.454 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 84.0% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log.gz 2024-06-13T07:30:28.472 INFO:teuthology.orchestra.run.smithi135.stderr: 90.5% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log.gz 2024-06-13T07:30:28.516 INFO:teuthology.orchestra.run.smithi135.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-13T07:30:28.525 INFO:teuthology.orchestra.run.smithi026.stderr: 83.7% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.cephadm.log.gz 2024-06-13T07:30:28.561 INFO:teuthology.orchestra.run.smithi135.stderr: 94.4% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.h.log.gz 2024-06-13T07:30:28.568 INFO:teuthology.orchestra.run.smithi136.stderr: 91.1% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log.gz 2024-06-13T07:30:28.593 INFO:teuthology.orchestra.run.smithi136.stderr: 87.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log.gz 2024-06-13T07:30:28.652 INFO:teuthology.orchestra.run.smithi135.stderr: 93.6% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.y.log.gz 2024-06-13T07:30:28.659 INFO:teuthology.orchestra.run.smithi135.stderr: 87.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.log.gz 2024-06-13T07:30:28.688 INFO:teuthology.orchestra.run.smithi135.stderr: 94.5% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.e.log.gz 2024-06-13T07:30:28.698 INFO:teuthology.orchestra.run.smithi136.stderr: 94.5% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.l.log.gz 2024-06-13T07:30:28.745 INFO:teuthology.orchestra.run.smithi136.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-13T07:30:28.767 INFO:teuthology.orchestra.run.smithi026.stderr: 90.4% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log.gz 2024-06-13T07:30:28.797 INFO:teuthology.orchestra.run.smithi135.stderr: 91.1% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log.gz 2024-06-13T07:30:28.827 INFO:teuthology.orchestra.run.smithi026.stderr: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-13T07:30:28.848 INFO:teuthology.orchestra.run.smithi136.stderr: 93.7% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.z.log.gz 2024-06-13T07:30:28.867 INFO:teuthology.orchestra.run.smithi136.stderr: 90.6% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph.audit.log.gz 2024-06-13T07:30:29.037 INFO:teuthology.orchestra.run.smithi135.stderr: 94.5% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.k.log.gz 2024-06-13T07:30:29.046 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-volume.log.gz 2024-06-13T07:30:30.318 INFO:teuthology.orchestra.run.smithi026.stderr: 90.5% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mgr.x.log.gz 2024-06-13T07:30:38.130 INFO:teuthology.orchestra.run.smithi135.stderr: 91.8% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.b.log.gz 2024-06-13T07:30:47.790 INFO:teuthology.orchestra.run.smithi026.stderr: 90.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.a.log.gz 2024-06-13T07:32:19.625 INFO:teuthology.orchestra.run.smithi136.stderr: 93.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mon.c.log.gz 2024-06-13T07:34:11.521 INFO:teuthology.orchestra.run.smithi026.stderr: 95.0% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.a.log.gz 2024-06-13T07:35:57.557 INFO:teuthology.orchestra.run.smithi136.stderr: 94.9% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.c.log.gz 2024-06-13T07:36:40.272 INFO:teuthology.orchestra.run.smithi136.stderr: 94.8% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.f.log.gz 2024-06-13T07:38:32.716 INFO:teuthology.orchestra.run.smithi026.stderr: 91.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.0.log.gz 2024-06-13T07:39:01.381 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.6.log.gz 2024-06-13T07:39:08.839 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.3.log.gz 2024-06-13T07:39:41.616 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.9.log.gz 2024-06-13T07:40:20.028 INFO:teuthology.orchestra.run.smithi135.stderr: 91.4% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.1.log.gz 2024-06-13T07:40:36.260 INFO:teuthology.orchestra.run.smithi135.stderr: 91.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.10.log.gz 2024-06-13T07:41:03.168 INFO:teuthology.orchestra.run.smithi136.stderr: 91.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.8.log.gz 2024-06-13T07:42:14.758 INFO:teuthology.orchestra.run.smithi136.stderr: 91.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.11.log.gz 2024-06-13T07:42:35.933 INFO:teuthology.orchestra.run.smithi135.stderr: 91.1% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.7.log.gz 2024-06-13T07:43:56.869 INFO:teuthology.orchestra.run.smithi026.stderr: 93.1% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.g.log.gz 2024-06-13T07:43:56.892 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-06-13T07:43:56.892 INFO:teuthology.orchestra.run.smithi026.stderr:real 13m28.505s 2024-06-13T07:43:56.892 INFO:teuthology.orchestra.run.smithi026.stderr:user 16m30.844s 2024-06-13T07:43:56.892 INFO:teuthology.orchestra.run.smithi026.stderr:sys 2m3.511s 2024-06-13T07:44:08.918 INFO:teuthology.orchestra.run.smithi135.stderr: 91.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.4.log.gz 2024-06-13T07:46:17.671 INFO:teuthology.orchestra.run.smithi136.stderr: 91.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.2.log.gz 2024-06-13T07:47:39.188 INFO:teuthology.orchestra.run.smithi135.stderr: 93.1% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.b.log.gz 2024-06-13T07:47:39.190 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-06-13T07:47:39.190 INFO:teuthology.orchestra.run.smithi135.stderr:real 17m10.846s 2024-06-13T07:47:39.190 INFO:teuthology.orchestra.run.smithi135.stderr:user 7m56.538s 2024-06-13T07:47:39.190 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m21.551s 2024-06-13T07:48:32.065 INFO:teuthology.orchestra.run.smithi136.stderr: 90.9% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-osd.5.log.gz 2024-06-13T07:51:35.797 INFO:teuthology.orchestra.run.smithi136.stderr: 93.2% -- replaced with /var/log/ceph/559474f0-2950-11ef-bc9c-c7b262605968/ceph-mds.i.log.gz 2024-06-13T07:51:35.798 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-06-13T07:51:35.799 INFO:teuthology.orchestra.run.smithi136.stderr:real 21m7.455s 2024-06-13T07:51:35.799 INFO:teuthology.orchestra.run.smithi136.stderr:user 8m26.186s 2024-06-13T07:51:35.799 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m23.379s 2024-06-13T07:51:35.799 INFO:tasks.cephadm:Archiving logs... 2024-06-13T07:51:35.800 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi026/log 2024-06-13T07:51:35.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-13T07:52:58.896 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi135/log 2024-06-13T07:52:58.898 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-13T07:54:15.824 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi136/log 2024-06-13T07:54:15.825 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-13T07:55:39.747 INFO:tasks.cephadm:Removing cluster... 2024-06-13T07:55:39.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 559474f0-2950-11ef-bc9c-c7b262605968 --force 2024-06-13T07:55:40.600 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T07:55:42.760 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 559474f0-2950-11ef-bc9c-c7b262605968 --force 2024-06-13T07:55:43.336 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T07:55:44.156 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 559474f0-2950-11ef-bc9c-c7b262605968 --force 2024-06-13T07:55:44.649 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: 559474f0-2950-11ef-bc9c-c7b262605968 2024-06-13T07:55:45.542 INFO:tasks.cephadm:Removing cephadm ... 2024-06-13T07:55:45.542 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-13T07:55:45.609 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-13T07:55:45.629 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-13T07:55:45.644 INFO:tasks.cephadm:Teardown complete 2024-06-13T07:55:45.644 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-13T07:55:45.656 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-13T07:55:45.656 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-13T07:55:45.664 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-13T07:55:45.674 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-13T07:55:45.813 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-13T07:55:45.823 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-13T07:55:45.824 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T07:55:45.831 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T07:55:45.834 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T07:55:45.847 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-06-13T07:55:45.848 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-06-13T07:55:45.872 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T07:55:45.872 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-06-13T07:55:45.872 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +1184us[+1084us] +/- 43ms 2024-06-13T07:55:45.873 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 25 -517us[ -617us] +/- 62ms 2024-06-13T07:55:45.873 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -1163us[-1263us] +/- 62ms 2024-06-13T07:55:45.873 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T07:55:45.882 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-06-13T07:55:45.899 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T07:55:45.899 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-06-13T07:55:45.899 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1145us[+1163us] +/- 43ms 2024-06-13T07:55:45.899 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 90 -569us[ -553us] +/- 61ms 2024-06-13T07:55:45.899 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -1179us[-1179us] +/- 62ms 2024-06-13T07:55:45.899 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T07:55:46.001 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T07:55:46.002 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-06-13T07:55:46.002 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1034us[+1040us] +/- 43ms 2024-06-13T07:55:46.002 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -663us[ -663us] +/- 62ms 2024-06-13T07:55:46.002 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -1261us[-1261us] +/- 62ms 2024-06-13T07:55:46.002 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T07:55:46.005 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-13T07:55:46.016 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-13T07:55:46.017 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-13T07:55:46.027 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-13T07:55:46.356 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-13T07:55:46.500 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-13T07:55:46.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T07:55:46.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:55:46.736 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T07:55:46.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:55:46.769 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T07:55:46.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:55:46.799 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-13T07:55:46.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-13T07:55:46.837 INFO:teuthology.task.internal:Duration was 4810.509682 seconds 2024-06-13T07:55:46.838 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-13T07:55:46.847 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-13T07:55:46.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-13T07:55:46.855 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-13T07:55:46.858 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-13T07:55:46.923 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T07:55:46.932 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T07:55:47.037 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T07:55:47.379 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-13T07:55:47.379 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-06-13T07:55:47.380 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-13T07:55:47.448 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-06-13T07:55:47.449 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-13T07:55:47.483 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-06-13T07:55:47.483 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-13T07:55:47.511 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-13T07:55:47.511 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-13T07:55:47.519 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-13T07:55:47.526 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-13T07:55:47.607 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-13T07:55:47.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-13T07:55:47.614 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-13T07:55:47.617 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-13T07:55:48.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-13T07:55:48.501 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-13T07:55:48.502 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-13T07:55:48.598 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-13T07:55:48.628 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-13T07:55:48.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-13T07:55:48.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:55:48.703 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:55:48.708 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:55:48.747 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-06-13T07:55:48.747 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-06-13T07:55:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-06-13T07:55:48.885 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:55:48.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:55:48.942 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:55:48.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:55:48.962 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:55:48.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:55:48.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-13T07:55:48.992 INFO:teuthology.task.internal:Transferring archived files... 2024-06-13T07:55:48.994 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi026 2024-06-13T07:55:48.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-13T07:55:49.112 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi135 2024-06-13T07:55:49.112 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-13T07:55:49.150 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752750/remote/smithi136 2024-06-13T07:55:49.150 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-13T07:55:49.188 INFO:teuthology.task.internal:Removing archive directory... 2024-06-13T07:55:49.188 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-13T07:55:49.196 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-13T07:55:49.199 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-13T07:55:49.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-13T07:55:49.260 INFO:teuthology.task.internal:Not uploading archives. 2024-06-13T07:55:49.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-13T07:55:49.271 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-13T07:55:49.271 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-13T07:55:49.279 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-13T07:55:49.282 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-13T07:55:49.295 INFO:teuthology.orchestra.run.smithi135.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 13 07:55 /home/ubuntu/cephtest 2024-06-13T07:55:49.297 INFO:teuthology.orchestra.run.smithi136.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 13 07:55 /home/ubuntu/cephtest 2024-06-13T07:55:49.331 INFO:teuthology.orchestra.run.smithi026.stdout: 393243 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 13 07:55 /home/ubuntu/cephtest 2024-06-13T07:55:49.335 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-13T07:55:49.347 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-13T07:55:49.376 INFO:teuthology.run:Summary data: client.0-kernel-sha1: ff88c41504f3e4c4268de18904a6f347f1b3f2d6 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 4810.50968170166 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-06-13T07:55:49.377 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-13T07:55:49.454 INFO:teuthology.run:FAIL